2024-09-12T03:09:19.916 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-12T03:09:19.944 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T03:09:20.043 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901679 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/fs/test_o_trunc}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7901679' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 481 sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc sleep_before_teardown: 0 subset: 57/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi064.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFghseZc2C1bMX29GrHF5gPNnuenvnyhM+nzyNTE/ndkS+fDAd7LmlFo9vkaMmSg1bXvdLz3yY9Wh93C0EDcYl4= smithi115.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDn6ttBk4tOE9OKUhkT9xCv2YnqVteUgECyXp9FBe9mAO6sah0qbHB3Ieevy0sJsrI4OniCNRx3aJ5XuI3iePuE= smithi195.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKpzX+3uruwgVrcQEU7IDF7LL05KY6mdespWkdLeb37Kstyab5H0FbiGtk7WcM+Ukj9sku232I0oJUpIihd6Gd8= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - fs/test_o_trunc.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-11_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-12T03:09:20.043 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa; will attempt to use it 2024-09-12T03:09:20.044 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks 2024-09-12T03:09:20.044 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-12T03:09:20.045 INFO:teuthology.task.internal:Checking packages... 2024-09-12T03:09:20.066 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc' 2024-09-12T03:09:20.066 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-12T03:09:20.066 INFO:teuthology.packaging:ref: None 2024-09-12T03:09:20.066 INFO:teuthology.packaging:tag: None 2024-09-12T03:09:20.066 INFO:teuthology.packaging:branch: squid 2024-09-12T03:09:20.066 INFO:teuthology.packaging:sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:09:20.066 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-12T03:09:20.291 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-314.g086c737f 2024-09-12T03:09:20.293 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-12T03:09:20.301 INFO:teuthology.task.internal:no buildpackages task found 2024-09-12T03:09:20.301 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-12T03:09:20.321 INFO:teuthology.task.internal:Saving configuration 2024-09-12T03:09:20.341 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-12T03:09:20.367 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-12T03:09:20.394 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi064.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901679', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 03:04:10.913220', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFghseZc2C1bMX29GrHF5gPNnuenvnyhM+nzyNTE/ndkS+fDAd7LmlFo9vkaMmSg1bXvdLz3yY9Wh93C0EDcYl4='} 2024-09-12T03:09:20.419 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi115.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901679', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 03:04:10.916775', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDn6ttBk4tOE9OKUhkT9xCv2YnqVteUgECyXp9FBe9mAO6sah0qbHB3Ieevy0sJsrI4OniCNRx3aJ5XuI3iePuE='} 2024-09-12T03:09:20.441 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi195.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901679', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 03:04:10.915096', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKpzX+3uruwgVrcQEU7IDF7LL05KY6mdespWkdLeb37Kstyab5H0FbiGtk7WcM+Ukj9sku232I0oJUpIihd6Gd8='} 2024-09-12T03:09:20.441 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-12T03:09:20.458 INFO:teuthology.task.internal:roles: ubuntu@smithi064.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-12T03:09:20.458 INFO:teuthology.task.internal:roles: ubuntu@smithi115.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-12T03:09:20.458 INFO:teuthology.task.internal:roles: ubuntu@smithi195.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-12T03:09:20.458 INFO:teuthology.run_tasks:Running task console_log... 2024-09-12T03:09:20.662 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f79be91bbe0>, signals=[15]) 2024-09-12T03:09:20.662 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-12T03:09:20.672 INFO:teuthology.task.internal:Opening connections... 2024-09-12T03:09:20.672 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi064.front.sepia.ceph.com 2024-09-12T03:09:20.673 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:09:20.761 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi115.front.sepia.ceph.com 2024-09-12T03:09:20.762 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:09:20.837 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi195.front.sepia.ceph.com 2024-09-12T03:09:20.837 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:09:20.914 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-12T03:09:20.922 DEBUG:teuthology.orchestra.run.smithi064:> uname -m 2024-09-12T03:09:20.941 INFO:teuthology.orchestra.run.smithi064.stdout:x86_64 2024-09-12T03:09:20.941 DEBUG:teuthology.orchestra.run.smithi064:> cat /etc/os-release 2024-09-12T03:09:20.999 INFO:teuthology.orchestra.run.smithi064.stdout:NAME="CentOS Stream" 2024-09-12T03:09:20.999 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION="9" 2024-09-12T03:09:20.999 INFO:teuthology.orchestra.run.smithi064.stdout:ID="centos" 2024-09-12T03:09:20.999 INFO:teuthology.orchestra.run.smithi064.stdout:ID_LIKE="rhel fedora" 2024-09-12T03:09:20.999 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_ID="9" 2024-09-12T03:09:21.000 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM_ID="platform:el9" 2024-09-12T03:09:21.000 INFO:teuthology.orchestra.run.smithi064.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T03:09:21.000 INFO:teuthology.orchestra.run.smithi064.stdout:ANSI_COLOR="0;31" 2024-09-12T03:09:21.000 INFO:teuthology.orchestra.run.smithi064.stdout:LOGO="fedora-logo-icon" 2024-09-12T03:09:21.000 INFO:teuthology.orchestra.run.smithi064.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T03:09:21.000 INFO:teuthology.orchestra.run.smithi064.stdout:HOME_URL="https://centos.org/" 2024-09-12T03:09:21.000 INFO:teuthology.orchestra.run.smithi064.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T03:09:21.000 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T03:09:21.000 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T03:09:21.001 INFO:teuthology.lock.ops:Updating smithi064.front.sepia.ceph.com on lock server 2024-09-12T03:09:21.027 DEBUG:teuthology.orchestra.run.smithi115:> uname -m 2024-09-12T03:09:21.046 INFO:teuthology.orchestra.run.smithi115.stdout:x86_64 2024-09-12T03:09:21.046 DEBUG:teuthology.orchestra.run.smithi115:> cat /etc/os-release 2024-09-12T03:09:21.104 INFO:teuthology.orchestra.run.smithi115.stdout:NAME="CentOS Stream" 2024-09-12T03:09:21.104 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION="9" 2024-09-12T03:09:21.104 INFO:teuthology.orchestra.run.smithi115.stdout:ID="centos" 2024-09-12T03:09:21.104 INFO:teuthology.orchestra.run.smithi115.stdout:ID_LIKE="rhel fedora" 2024-09-12T03:09:21.105 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION_ID="9" 2024-09-12T03:09:21.105 INFO:teuthology.orchestra.run.smithi115.stdout:PLATFORM_ID="platform:el9" 2024-09-12T03:09:21.105 INFO:teuthology.orchestra.run.smithi115.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T03:09:21.105 INFO:teuthology.orchestra.run.smithi115.stdout:ANSI_COLOR="0;31" 2024-09-12T03:09:21.105 INFO:teuthology.orchestra.run.smithi115.stdout:LOGO="fedora-logo-icon" 2024-09-12T03:09:21.105 INFO:teuthology.orchestra.run.smithi115.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T03:09:21.105 INFO:teuthology.orchestra.run.smithi115.stdout:HOME_URL="https://centos.org/" 2024-09-12T03:09:21.105 INFO:teuthology.orchestra.run.smithi115.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T03:09:21.105 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T03:09:21.105 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T03:09:21.106 INFO:teuthology.lock.ops:Updating smithi115.front.sepia.ceph.com on lock server 2024-09-12T03:09:21.128 DEBUG:teuthology.orchestra.run.smithi195:> uname -m 2024-09-12T03:09:21.146 INFO:teuthology.orchestra.run.smithi195.stdout:x86_64 2024-09-12T03:09:21.146 DEBUG:teuthology.orchestra.run.smithi195:> cat /etc/os-release 2024-09-12T03:09:21.204 INFO:teuthology.orchestra.run.smithi195.stdout:NAME="CentOS Stream" 2024-09-12T03:09:21.204 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION="9" 2024-09-12T03:09:21.204 INFO:teuthology.orchestra.run.smithi195.stdout:ID="centos" 2024-09-12T03:09:21.204 INFO:teuthology.orchestra.run.smithi195.stdout:ID_LIKE="rhel fedora" 2024-09-12T03:09:21.204 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION_ID="9" 2024-09-12T03:09:21.204 INFO:teuthology.orchestra.run.smithi195.stdout:PLATFORM_ID="platform:el9" 2024-09-12T03:09:21.204 INFO:teuthology.orchestra.run.smithi195.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T03:09:21.204 INFO:teuthology.orchestra.run.smithi195.stdout:ANSI_COLOR="0;31" 2024-09-12T03:09:21.204 INFO:teuthology.orchestra.run.smithi195.stdout:LOGO="fedora-logo-icon" 2024-09-12T03:09:21.205 INFO:teuthology.orchestra.run.smithi195.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T03:09:21.205 INFO:teuthology.orchestra.run.smithi195.stdout:HOME_URL="https://centos.org/" 2024-09-12T03:09:21.205 INFO:teuthology.orchestra.run.smithi195.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T03:09:21.205 INFO:teuthology.orchestra.run.smithi195.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T03:09:21.205 INFO:teuthology.orchestra.run.smithi195.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T03:09:21.205 INFO:teuthology.lock.ops:Updating smithi195.front.sepia.ceph.com on lock server 2024-09-12T03:09:21.227 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-12T03:09:21.239 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-12T03:09:21.301 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-12T03:09:21.301 DEBUG:teuthology.orchestra.run.smithi064:> test '!' -e /home/ubuntu/cephtest 2024-09-12T03:09:21.303 DEBUG:teuthology.orchestra.run.smithi115:> test '!' -e /home/ubuntu/cephtest 2024-09-12T03:09:21.306 DEBUG:teuthology.orchestra.run.smithi195:> test '!' -e /home/ubuntu/cephtest 2024-09-12T03:09:21.323 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-12T03:09:21.438 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-12T03:09:21.438 DEBUG:teuthology.orchestra.run.smithi064:> test -z $(ls -A /var/lib/ceph) 2024-09-12T03:09:21.440 DEBUG:teuthology.orchestra.run.smithi115:> test -z $(ls -A /var/lib/ceph) 2024-09-12T03:09:21.443 DEBUG:teuthology.orchestra.run.smithi195:> test -z $(ls -A /var/lib/ceph) 2024-09-12T03:09:21.534 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-12T03:09:21.624 INFO:teuthology.run_tasks:Running task kernel... 2024-09-12T03:09:21.666 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-12T03:09:21.667 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-12T03:09:21.667 DEBUG:teuthology.orchestra.run.smithi064:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T03:09:21.668 DEBUG:teuthology.orchestra.run.smithi115:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T03:09:21.668 DEBUG:teuthology.orchestra.run.smithi195:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T03:09:21.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:09:21.685 DEBUG:teuthology.orchestra.run.smithi064:> uname -r 2024-09-12T03:09:21.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:09:21.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:09:21.687 DEBUG:teuthology.orchestra.run.smithi195:> uname -r 2024-09-12T03:09:21.688 DEBUG:teuthology.orchestra.run.smithi115:> uname -r 2024-09-12T03:09:21.742 INFO:teuthology.orchestra.run.smithi064.stdout:5.14.0-503.el9.x86_64 2024-09-12T03:09:21.743 INFO:teuthology.task.kernel:Running kernel on smithi064: 5.14.0-503.el9.x86_64 2024-09-12T03:09:21.743 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum install -y kernel 2024-09-12T03:09:21.746 INFO:teuthology.orchestra.run.smithi115.stdout:5.14.0-503.el9.x86_64 2024-09-12T03:09:21.746 INFO:teuthology.orchestra.run.smithi195.stdout:5.14.0-503.el9.x86_64 2024-09-12T03:09:21.746 INFO:teuthology.task.kernel:Running kernel on smithi115: 5.14.0-503.el9.x86_64 2024-09-12T03:09:21.746 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum install -y kernel 2024-09-12T03:09:21.746 INFO:teuthology.task.kernel:Running kernel on smithi195: 5.14.0-503.el9.x86_64 2024-09-12T03:09:21.747 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum install -y kernel 2024-09-12T03:09:24.097 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.2 MB 00:00 2024-09-12T03:09:24.483 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - BaseOS 8.9 MB/s | 8.2 MB 00:00 2024-09-12T03:09:26.226 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - BaseOS 3.2 MB/s | 8.2 MB 00:02 2024-09-12T03:09:27.173 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - AppStream 11 MB/s | 20 MB 00:01 2024-09-12T03:09:29.452 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - AppStream 9.5 MB/s | 20 MB 00:02 2024-09-12T03:09:31.515 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.5 MB 00:00 2024-09-12T03:09:32.991 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - Extras packages 81 kB/s | 19 kB 00:00 2024-09-12T03:09:33.652 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.5 MB 00:00 2024-09-12T03:09:33.879 INFO:teuthology.orchestra.run.smithi064.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-12T03:09:35.036 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - Extras packages 102 kB/s | 19 kB 00:00 2024-09-12T03:09:35.765 INFO:teuthology.orchestra.run.smithi115.stdout:Extra Packages for Enterprise Linux 37 MB/s | 23 MB 00:00 2024-09-12T03:09:36.565 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - AppStream 1.9 MB/s | 20 MB 00:10 2024-09-12T03:09:39.492 INFO:teuthology.orchestra.run.smithi064.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-09-12T03:09:41.279 INFO:teuthology.orchestra.run.smithi115.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-09-12T03:09:41.591 INFO:teuthology.orchestra.run.smithi064.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-12T03:09:41.591 INFO:teuthology.orchestra.run.smithi064.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-12T03:09:41.685 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-09-12T03:09:41.693 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T03:09:41.693 INFO:teuthology.orchestra.run.smithi064.stdout: Package Architecture Version Repository Size 2024-09-12T03:09:41.693 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T03:09:41.693 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-09-12T03:09:41.693 INFO:teuthology.orchestra.run.smithi064.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-12T03:09:41.694 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-09-12T03:09:41.694 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-12T03:09:41.694 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-12T03:09:41.694 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-12T03:09:41.694 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:09:41.694 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-09-12T03:09:41.694 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-12T03:09:41.694 INFO:teuthology.orchestra.run.smithi064.stdout:Install 4 Packages 2024-09-12T03:09:41.694 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:09:41.695 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 79 M 2024-09-12T03:09:41.695 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 126 M 2024-09-12T03:09:41.695 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-09-12T03:09:41.944 INFO:teuthology.orchestra.run.smithi064.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 178 kB/s | 23 kB 00:00 2024-09-12T03:09:42.319 INFO:teuthology.orchestra.run.smithi064.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 31 MB/s | 16 MB 00:00 2024-09-12T03:09:42.483 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - CRB 3.1 MB/s | 6.5 MB 00:02 2024-09-12T03:09:42.644 INFO:teuthology.orchestra.run.smithi064.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 42 MB/s | 35 MB 00:00 2024-09-12T03:09:43.020 INFO:teuthology.orchestra.run.smithi064.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 27 MB/s | 29 MB 00:01 2024-09-12T03:09:43.020 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-09-12T03:09:43.020 INFO:teuthology.orchestra.run.smithi064.stdout:Total 59 MB/s | 79 MB 00:01 2024-09-12T03:09:43.202 INFO:teuthology.orchestra.run.smithi115.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-12T03:09:43.202 INFO:teuthology.orchestra.run.smithi115.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-12T03:09:43.292 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-12T03:09:43.296 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-12T03:09:43.296 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2024-09-12T03:09:43.296 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-12T03:09:43.296 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-12T03:09:43.296 INFO:teuthology.orchestra.run.smithi115.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-12T03:09:43.296 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-12T03:09:43.296 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-12T03:09:43.296 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-12T03:09:43.296 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-12T03:09:43.296 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T03:09:43.297 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-12T03:09:43.297 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-12T03:09:43.297 INFO:teuthology.orchestra.run.smithi115.stdout:Install 4 Packages 2024-09-12T03:09:43.297 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T03:09:43.297 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 79 M 2024-09-12T03:09:43.297 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 126 M 2024-09-12T03:09:43.298 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-12T03:09:43.505 INFO:teuthology.orchestra.run.smithi115.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 281 kB/s | 23 kB 00:00 2024-09-12T03:09:43.726 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-09-12T03:09:43.889 INFO:teuthology.orchestra.run.smithi115.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 34 MB/s | 16 MB 00:00 2024-09-12T03:09:44.078 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-09-12T03:09:44.079 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-09-12T03:09:44.133 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 19 kB 00:00 2024-09-12T03:09:44.156 INFO:teuthology.orchestra.run.smithi115.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 44 MB/s | 29 MB 00:00 2024-09-12T03:09:44.464 INFO:teuthology.orchestra.run.smithi115.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 33 MB/s | 35 MB 00:01 2024-09-12T03:09:44.465 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-12T03:09:44.465 INFO:teuthology.orchestra.run.smithi115.stdout:Total 67 MB/s | 79 MB 00:01 2024-09-12T03:09:45.012 INFO:teuthology.orchestra.run.smithi195.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-12T03:09:45.082 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-09-12T03:09:45.082 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-09-12T03:09:45.145 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-12T03:09:45.480 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-12T03:09:45.480 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-12T03:09:46.455 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-12T03:09:46.456 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-12T03:09:47.649 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-09-12T03:09:48.312 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-12T03:09:48.394 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T03:09:48.877 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-12T03:09:49.580 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-12T03:09:49.665 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T03:09:50.493 INFO:teuthology.orchestra.run.smithi195.stdout:lab-extras 21 kB/s | 1.7 kB 00:00 2024-09-12T03:09:51.180 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T03:09:51.304 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T03:09:52.464 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T03:09:52.474 INFO:teuthology.orchestra.run.smithi195.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-12T03:09:52.474 INFO:teuthology.orchestra.run.smithi195.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-12T03:09:52.572 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-09-12T03:09:52.582 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-09-12T03:09:52.582 INFO:teuthology.orchestra.run.smithi195.stdout: Package Architecture Version Repository Size 2024-09-12T03:09:52.582 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-09-12T03:09:52.582 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-09-12T03:09:52.583 INFO:teuthology.orchestra.run.smithi195.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-12T03:09:52.583 INFO:teuthology.orchestra.run.smithi195.stdout:Installing dependencies: 2024-09-12T03:09:52.583 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-12T03:09:52.583 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-12T03:09:52.583 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-12T03:09:52.583 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-12T03:09:52.583 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-09-12T03:09:52.583 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-09-12T03:09:52.583 INFO:teuthology.orchestra.run.smithi195.stdout:Install 4 Packages 2024-09-12T03:09:52.583 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-12T03:09:52.584 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 79 M 2024-09-12T03:09:52.584 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 126 M 2024-09-12T03:09:52.584 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-09-12T03:09:52.593 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T03:09:52.944 INFO:teuthology.orchestra.run.smithi195.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 73 kB/s | 23 kB 00:00 2024-09-12T03:09:56.061 INFO:teuthology.orchestra.run.smithi195.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 4.6 MB/s | 16 MB 00:03 2024-09-12T03:09:56.426 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T03:09:56.470 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T03:09:57.443 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T03:09:57.701 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T03:09:59.753 INFO:teuthology.orchestra.run.smithi195.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 4.9 MB/s | 35 MB 00:07 2024-09-12T03:10:01.614 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T03:10:02.543 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T03:10:02.852 INFO:teuthology.orchestra.run.smithi195.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 2.9 MB/s | 29 MB 00:09 2024-09-12T03:10:02.853 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-09-12T03:10:02.853 INFO:teuthology.orchestra.run.smithi195.stdout:Total 7.7 MB/s | 79 MB 00:10 2024-09-12T03:10:03.537 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-09-12T03:10:03.883 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-09-12T03:10:03.883 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-09-12T03:10:04.856 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-09-12T03:10:04.856 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-09-12T03:10:07.311 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-09-12T03:10:08.209 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-12T03:10:08.303 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T03:10:11.106 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T03:10:11.240 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T03:10:16.071 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T03:10:16.182 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T03:10:21.027 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T03:10:40.385 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T03:10:40.426 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-12T03:10:40.771 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T03:10:40.798 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T03:10:40.798 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-12T03:10:40.798 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T03:10:40.798 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T03:10:40.811 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-12T03:10:41.143 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T03:10:41.143 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-12T03:10:41.143 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T03:10:41.143 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T03:10:41.642 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T03:10:41.643 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:10:41.643 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-09-12T03:10:41.643 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-12T03:10:41.643 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-12T03:10:41.643 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-12T03:10:41.643 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-12T03:10:41.643 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:10:41.643 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-09-12T03:10:41.993 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T03:10:41.993 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T03:10:41.993 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-12T03:10:41.993 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-12T03:10:41.993 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-12T03:10:41.994 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-12T03:10:41.994 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-12T03:10:41.994 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T03:10:41.994 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-12T03:10:42.001 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T03:10:42.094 INFO:teuthology.orchestra.run.smithi064.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T03:10:42.094 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T03:10:42.095 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-12T03:10:42.095 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-12T03:10:42.095 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-12T03:10:42.095 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-12T03:10:42.095 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi064.front.sepia.ceph.com, path=None, version=distro) 2024-09-12T03:10:42.095 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T03:10:42.154 INFO:teuthology.orchestra.run.smithi064.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T03:10:42.154 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T03:10:42.154 DEBUG:teuthology.orchestra.run.smithi064:> sudo rpm -qi grub2-tools 2024-09-12T03:10:42.186 INFO:teuthology.orchestra.run.smithi064.stdout:Name : grub2-tools 2024-09-12T03:10:42.186 INFO:teuthology.orchestra.run.smithi064.stdout:Epoch : 1 2024-09-12T03:10:42.186 INFO:teuthology.orchestra.run.smithi064.stdout:Version : 2.06 2024-09-12T03:10:42.186 INFO:teuthology.orchestra.run.smithi064.stdout:Release : 82.el9 2024-09-12T03:10:42.186 INFO:teuthology.orchestra.run.smithi064.stdout:Architecture: x86_64 2024-09-12T03:10:42.186 INFO:teuthology.orchestra.run.smithi064.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-12T03:10:42.186 INFO:teuthology.orchestra.run.smithi064.stdout:Group : Unspecified 2024-09-12T03:10:42.186 INFO:teuthology.orchestra.run.smithi064.stdout:Size : 8274670 2024-09-12T03:10:42.187 INFO:teuthology.orchestra.run.smithi064.stdout:License : GPLv3+ 2024-09-12T03:10:42.187 INFO:teuthology.orchestra.run.smithi064.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-12T03:10:42.187 INFO:teuthology.orchestra.run.smithi064.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-12T03:10:42.187 INFO:teuthology.orchestra.run.smithi064.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-12T03:10:42.187 INFO:teuthology.orchestra.run.smithi064.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-12T03:10:42.187 INFO:teuthology.orchestra.run.smithi064.stdout:Packager : builder@centos.org 2024-09-12T03:10:42.187 INFO:teuthology.orchestra.run.smithi064.stdout:Vendor : CentOS 2024-09-12T03:10:42.187 INFO:teuthology.orchestra.run.smithi064.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-12T03:10:42.187 INFO:teuthology.orchestra.run.smithi064.stdout:Summary : Support tools for GRUB. 2024-09-12T03:10:42.187 INFO:teuthology.orchestra.run.smithi064.stdout:Description : 2024-09-12T03:10:42.187 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:10:42.187 INFO:teuthology.orchestra.run.smithi064.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-12T03:10:42.187 INFO:teuthology.orchestra.run.smithi064.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-12T03:10:42.188 INFO:teuthology.orchestra.run.smithi064.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-12T03:10:42.188 INFO:teuthology.orchestra.run.smithi064.stdout:hardware devices. 2024-09-12T03:10:42.188 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:10:42.188 INFO:teuthology.orchestra.run.smithi064.stdout:This subpackage provides tools for support of all platforms. 2024-09-12T03:10:42.189 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-12T03:10:42.189 INFO:teuthology.task.kernel:Updating grub on smithi064 to boot 5.14.0-505.el9.x86_64 2024-09-12T03:10:42.189 DEBUG:teuthology.orchestra.run.smithi064:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-12T03:10:42.342 DEBUG:teuthology.orchestra.run.smithi115:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T03:10:42.422 INFO:teuthology.orchestra.run.smithi115.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T03:10:42.423 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T03:10:42.423 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-12T03:10:42.423 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-12T03:10:42.423 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-12T03:10:42.424 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-12T03:10:42.424 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi115.front.sepia.ceph.com, path=None, version=distro) 2024-09-12T03:10:42.424 DEBUG:teuthology.orchestra.run.smithi115:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T03:10:42.484 INFO:teuthology.orchestra.run.smithi115.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T03:10:42.485 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T03:10:42.485 DEBUG:teuthology.orchestra.run.smithi115:> sudo rpm -qi grub2-tools 2024-09-12T03:10:42.525 INFO:teuthology.orchestra.run.smithi115.stdout:Name : grub2-tools 2024-09-12T03:10:42.526 INFO:teuthology.orchestra.run.smithi115.stdout:Epoch : 1 2024-09-12T03:10:42.526 INFO:teuthology.orchestra.run.smithi115.stdout:Version : 2.06 2024-09-12T03:10:42.526 INFO:teuthology.orchestra.run.smithi115.stdout:Release : 82.el9 2024-09-12T03:10:42.526 INFO:teuthology.orchestra.run.smithi115.stdout:Architecture: x86_64 2024-09-12T03:10:42.526 INFO:teuthology.orchestra.run.smithi115.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-12T03:10:42.526 INFO:teuthology.orchestra.run.smithi115.stdout:Group : Unspecified 2024-09-12T03:10:42.526 INFO:teuthology.orchestra.run.smithi115.stdout:Size : 8274670 2024-09-12T03:10:42.526 INFO:teuthology.orchestra.run.smithi115.stdout:License : GPLv3+ 2024-09-12T03:10:42.526 INFO:teuthology.orchestra.run.smithi115.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-12T03:10:42.526 INFO:teuthology.orchestra.run.smithi115.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-12T03:10:42.526 INFO:teuthology.orchestra.run.smithi115.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-12T03:10:42.526 INFO:teuthology.orchestra.run.smithi115.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-12T03:10:42.526 INFO:teuthology.orchestra.run.smithi115.stdout:Packager : builder@centos.org 2024-09-12T03:10:42.527 INFO:teuthology.orchestra.run.smithi115.stdout:Vendor : CentOS 2024-09-12T03:10:42.527 INFO:teuthology.orchestra.run.smithi115.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-12T03:10:42.527 INFO:teuthology.orchestra.run.smithi115.stdout:Summary : Support tools for GRUB. 2024-09-12T03:10:42.527 INFO:teuthology.orchestra.run.smithi115.stdout:Description : 2024-09-12T03:10:42.527 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T03:10:42.527 INFO:teuthology.orchestra.run.smithi115.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-12T03:10:42.527 INFO:teuthology.orchestra.run.smithi115.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-12T03:10:42.527 INFO:teuthology.orchestra.run.smithi115.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-12T03:10:42.527 INFO:teuthology.orchestra.run.smithi115.stdout:hardware devices. 2024-09-12T03:10:42.527 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T03:10:42.527 INFO:teuthology.orchestra.run.smithi115.stdout:This subpackage provides tools for support of all platforms. 2024-09-12T03:10:42.528 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-12T03:10:42.528 INFO:teuthology.task.kernel:Updating grub on smithi115 to boot 5.14.0-505.el9.x86_64 2024-09-12T03:10:42.528 DEBUG:teuthology.orchestra.run.smithi115:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-12T03:10:42.825 INFO:teuthology.orchestra.run.smithi064.stderr:Generating grub configuration file ... 2024-09-12T03:10:43.196 INFO:teuthology.orchestra.run.smithi115.stderr:Generating grub configuration file ... 2024-09-12T03:10:44.156 INFO:teuthology.orchestra.run.smithi064.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-12T03:10:44.196 INFO:teuthology.orchestra.run.smithi064.stderr:done 2024-09-12T03:10:44.198 DEBUG:teuthology.orchestra.run.smithi064:> mktemp 2024-09-12T03:10:44.235 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.lGeboOOOpW 2024-09-12T03:10:44.236 DEBUG:teuthology.orchestra.run.smithi064:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.lGeboOOOpW 2024-09-12T03:10:44.302 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0666 /tmp/tmp.lGeboOOOpW 2024-09-12T03:10:44.446 DEBUG:teuthology.orchestra.remote:smithi064:/tmp/tmp.lGeboOOOpW is 6KB 2024-09-12T03:10:44.495 DEBUG:teuthology.orchestra.run.smithi064:> rm -fr /tmp/tmp.lGeboOOOpW 2024-09-12T03:10:44.509 DEBUG:teuthology.orchestra.run.smithi064:> sudo /bin/ls /boot/loader/entries || true 2024-09-12T03:10:44.567 INFO:teuthology.orchestra.run.smithi115.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-12T03:10:44.571 INFO:teuthology.orchestra.run.smithi064.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-12T03:10:44.572 INFO:teuthology.orchestra.run.smithi064.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-12T03:10:44.572 INFO:teuthology.orchestra.run.smithi064.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-12T03:10:44.572 INFO:teuthology.orchestra.run.smithi064.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-12T03:10:44.573 DEBUG:teuthology.orchestra.run.smithi064:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-12T03:10:44.606 INFO:teuthology.orchestra.run.smithi115.stderr:done 2024-09-12T03:10:44.608 DEBUG:teuthology.orchestra.run.smithi115:> mktemp 2024-09-12T03:10:44.624 INFO:teuthology.orchestra.run.smithi115.stdout:/tmp/tmp.WUyKv6Yene 2024-09-12T03:10:44.624 DEBUG:teuthology.orchestra.run.smithi115:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.WUyKv6Yene 2024-09-12T03:10:44.700 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0666 /tmp/tmp.WUyKv6Yene 2024-09-12T03:10:44.723 DEBUG:teuthology.orchestra.run.smithi064:> sudo shutdown -r now 2024-09-12T03:10:44.835 DEBUG:teuthology.orchestra.remote:smithi115:/tmp/tmp.WUyKv6Yene is 6KB 2024-09-12T03:10:44.884 DEBUG:teuthology.orchestra.run.smithi115:> rm -fr /tmp/tmp.WUyKv6Yene 2024-09-12T03:10:44.899 DEBUG:teuthology.orchestra.run.smithi115:> sudo /bin/ls /boot/loader/entries || true 2024-09-12T03:10:44.962 INFO:teuthology.orchestra.run.smithi115.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-12T03:10:44.962 INFO:teuthology.orchestra.run.smithi115.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-12T03:10:44.962 INFO:teuthology.orchestra.run.smithi115.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-12T03:10:44.962 INFO:teuthology.orchestra.run.smithi115.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-12T03:10:44.963 DEBUG:teuthology.orchestra.run.smithi115:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-12T03:10:45.106 DEBUG:teuthology.orchestra.run.smithi115:> sudo shutdown -r now 2024-09-12T03:10:59.133 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T03:10:59.172 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-12T03:10:59.548 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T03:10:59.548 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-12T03:10:59.548 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T03:10:59.548 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T03:11:00.411 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T03:11:00.411 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-12T03:11:00.411 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-09-12T03:11:00.411 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-12T03:11:00.411 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-12T03:11:00.411 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-12T03:11:00.411 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-12T03:11:00.411 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-12T03:11:00.411 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-09-12T03:11:00.771 DEBUG:teuthology.orchestra.run.smithi195:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T03:11:00.857 INFO:teuthology.orchestra.run.smithi195.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T03:11:00.858 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T03:11:00.858 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-12T03:11:00.858 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-12T03:11:00.858 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-12T03:11:00.858 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-09-12T03:11:00.858 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi195.front.sepia.ceph.com, path=None, version=distro) 2024-09-12T03:11:00.858 DEBUG:teuthology.orchestra.run.smithi195:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T03:11:00.917 INFO:teuthology.orchestra.run.smithi195.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T03:11:00.918 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T03:11:00.918 DEBUG:teuthology.orchestra.run.smithi195:> sudo rpm -qi grub2-tools 2024-09-12T03:11:00.954 INFO:teuthology.orchestra.run.smithi195.stdout:Name : grub2-tools 2024-09-12T03:11:00.955 INFO:teuthology.orchestra.run.smithi195.stdout:Epoch : 1 2024-09-12T03:11:00.955 INFO:teuthology.orchestra.run.smithi195.stdout:Version : 2.06 2024-09-12T03:11:00.955 INFO:teuthology.orchestra.run.smithi195.stdout:Release : 82.el9 2024-09-12T03:11:00.955 INFO:teuthology.orchestra.run.smithi195.stdout:Architecture: x86_64 2024-09-12T03:11:00.955 INFO:teuthology.orchestra.run.smithi195.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-12T03:11:00.955 INFO:teuthology.orchestra.run.smithi195.stdout:Group : Unspecified 2024-09-12T03:11:00.955 INFO:teuthology.orchestra.run.smithi195.stdout:Size : 8274670 2024-09-12T03:11:00.955 INFO:teuthology.orchestra.run.smithi195.stdout:License : GPLv3+ 2024-09-12T03:11:00.955 INFO:teuthology.orchestra.run.smithi195.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-12T03:11:00.955 INFO:teuthology.orchestra.run.smithi195.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-12T03:11:00.955 INFO:teuthology.orchestra.run.smithi195.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-12T03:11:00.955 INFO:teuthology.orchestra.run.smithi195.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-12T03:11:00.956 INFO:teuthology.orchestra.run.smithi195.stdout:Packager : builder@centos.org 2024-09-12T03:11:00.956 INFO:teuthology.orchestra.run.smithi195.stdout:Vendor : CentOS 2024-09-12T03:11:00.956 INFO:teuthology.orchestra.run.smithi195.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-12T03:11:00.956 INFO:teuthology.orchestra.run.smithi195.stdout:Summary : Support tools for GRUB. 2024-09-12T03:11:00.956 INFO:teuthology.orchestra.run.smithi195.stdout:Description : 2024-09-12T03:11:00.956 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-12T03:11:00.956 INFO:teuthology.orchestra.run.smithi195.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-12T03:11:00.956 INFO:teuthology.orchestra.run.smithi195.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-12T03:11:00.956 INFO:teuthology.orchestra.run.smithi195.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-12T03:11:00.956 INFO:teuthology.orchestra.run.smithi195.stdout:hardware devices. 2024-09-12T03:11:00.956 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-12T03:11:00.956 INFO:teuthology.orchestra.run.smithi195.stdout:This subpackage provides tools for support of all platforms. 2024-09-12T03:11:00.957 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-12T03:11:00.957 INFO:teuthology.task.kernel:Updating grub on smithi195 to boot 5.14.0-505.el9.x86_64 2024-09-12T03:11:00.957 DEBUG:teuthology.orchestra.run.smithi195:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-12T03:11:01.567 INFO:teuthology.orchestra.run.smithi195.stderr:Generating grub configuration file ... 2024-09-12T03:11:02.858 INFO:teuthology.orchestra.run.smithi195.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-12T03:11:02.901 INFO:teuthology.orchestra.run.smithi195.stderr:done 2024-09-12T03:11:02.904 DEBUG:teuthology.orchestra.run.smithi195:> mktemp 2024-09-12T03:11:02.920 INFO:teuthology.orchestra.run.smithi195.stdout:/tmp/tmp.ZCoFaMxIPS 2024-09-12T03:11:02.921 DEBUG:teuthology.orchestra.run.smithi195:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ZCoFaMxIPS 2024-09-12T03:11:02.987 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 0666 /tmp/tmp.ZCoFaMxIPS 2024-09-12T03:11:03.127 DEBUG:teuthology.orchestra.remote:smithi195:/tmp/tmp.ZCoFaMxIPS is 6KB 2024-09-12T03:11:03.174 DEBUG:teuthology.orchestra.run.smithi195:> rm -fr /tmp/tmp.ZCoFaMxIPS 2024-09-12T03:11:03.189 DEBUG:teuthology.orchestra.run.smithi195:> sudo /bin/ls /boot/loader/entries || true 2024-09-12T03:11:03.251 INFO:teuthology.orchestra.run.smithi195.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-12T03:11:03.251 INFO:teuthology.orchestra.run.smithi195.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-12T03:11:03.251 INFO:teuthology.orchestra.run.smithi195.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-12T03:11:03.252 INFO:teuthology.orchestra.run.smithi195.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-12T03:11:03.253 DEBUG:teuthology.orchestra.run.smithi195:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-12T03:11:03.408 DEBUG:teuthology.orchestra.run.smithi195:> sudo shutdown -r now 2024-09-12T03:11:14.737 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-12T03:11:14.737 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-09-12T03:11:14.738 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:11:15.109 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-12T03:11:15.109 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-12T03:11:15.110 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:11:33.181 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.64 2024-09-12T03:11:33.411 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-09-12T03:11:33.411 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-09-12T03:11:33.412 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:11:33.437 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.115 2024-09-12T03:11:42.185 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-09-12T03:11:42.186 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:11:42.438 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-12T03:11:42.439 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:11:45.245 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.64 2024-09-12T03:11:45.501 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.115 2024-09-12T03:11:51.869 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.195 2024-09-12T03:11:57.249 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-09-12T03:11:57.250 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:11:57.502 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-12T03:11:57.502 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:12:00.873 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-09-12T03:12:00.873 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:12:03.933 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.195 2024-09-12T03:12:15.937 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-09-12T03:12:15.938 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:12:15.940 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.115 2024-09-12T03:12:29.096 DEBUG:teuthology.orchestra.run.smithi064:> true 2024-09-12T03:12:29.471 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-09-12T03:12:29.471 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-12T03:12:29.471 DEBUG:teuthology.orchestra.run.smithi064:> uname -r 2024-09-12T03:12:29.487 INFO:teuthology.orchestra.run.smithi064.stdout:5.14.0-505.el9.x86_64 2024-09-12T03:12:29.487 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-12T03:12:29.487 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-12T03:12:29.487 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-12T03:12:30.488 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-12T03:12:30.488 DEBUG:teuthology.orchestra.run.smithi064:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-12T03:12:30.591 INFO:teuthology.orchestra.run.smithi064.stdout:ttyS1 2024-09-12T03:12:30.623 DEBUG:teuthology.parallel:result is None 2024-09-12T03:12:30.941 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-12T03:12:30.941 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:12:31.258 DEBUG:teuthology.orchestra.run.smithi115:> true 2024-09-12T03:12:31.784 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-12T03:12:31.784 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-12T03:12:31.784 DEBUG:teuthology.orchestra.run.smithi115:> uname -r 2024-09-12T03:12:31.799 INFO:teuthology.orchestra.run.smithi115.stdout:5.14.0-505.el9.x86_64 2024-09-12T03:12:31.799 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-12T03:12:31.800 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-12T03:12:31.800 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-12T03:12:32.800 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-12T03:12:32.801 DEBUG:teuthology.orchestra.run.smithi115:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-12T03:12:32.940 INFO:teuthology.orchestra.run.smithi115.stdout:ttyS1 2024-09-12T03:12:32.963 DEBUG:teuthology.parallel:result is None 2024-09-12T03:12:47.703 DEBUG:teuthology.orchestra.run.smithi195:> true 2024-09-12T03:12:48.149 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-09-12T03:12:48.150 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-505.el9.x86_64"... 2024-09-12T03:12:48.150 DEBUG:teuthology.orchestra.run.smithi195:> uname -r 2024-09-12T03:12:48.164 INFO:teuthology.orchestra.run.smithi195.stdout:5.14.0-505.el9.x86_64 2024-09-12T03:12:48.164 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-12T03:12:48.165 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-12T03:12:48.165 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-12T03:12:49.166 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-09-12T03:12:49.166 DEBUG:teuthology.orchestra.run.smithi195:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-12T03:12:49.279 INFO:teuthology.orchestra.run.smithi195.stdout:ttyS1 2024-09-12T03:12:49.313 DEBUG:teuthology.parallel:result is None 2024-09-12T03:12:49.313 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-12T03:12:49.322 INFO:teuthology.task.internal:Creating test directory... 2024-09-12T03:12:49.322 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T03:12:49.327 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T03:12:49.330 DEBUG:teuthology.orchestra.run.smithi195:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T03:12:49.348 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-12T03:12:49.456 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-12T03:12:49.542 INFO:teuthology.task.internal:Creating archive directory... 2024-09-12T03:12:49.542 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T03:12:49.546 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T03:12:49.549 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T03:12:49.681 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-12T03:12:49.688 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-12T03:12:49.689 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T03:12:49.693 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T03:12:49.695 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T03:12:49.722 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T03:12:49.734 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T03:12:49.734 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T03:12:49.745 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T03:12:49.943 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T03:12:49.964 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T03:12:49.966 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-12T03:12:49.973 INFO:teuthology.task.internal:Configuring sudo... 2024-09-12T03:12:49.973 DEBUG:teuthology.orchestra.run.smithi064:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T03:12:49.976 DEBUG:teuthology.orchestra.run.smithi115:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T03:12:49.979 DEBUG:teuthology.orchestra.run.smithi195:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T03:12:50.018 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-12T03:12:50.027 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-12T03:12:50.027 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T03:12:50.048 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T03:12:50.050 DEBUG:teuthology.orchestra.run.smithi195:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T03:12:50.106 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T03:12:50.154 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T03:12:50.207 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T03:12:50.294 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T03:12:50.337 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-12T03:12:50.337 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T03:12:50.404 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T03:12:50.452 DEBUG:teuthology.orchestra.run.smithi115:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T03:12:50.514 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T03:12:50.559 DEBUG:teuthology.orchestra.run.smithi115:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T03:12:50.606 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-12T03:12:50.606 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T03:12:50.675 DEBUG:teuthology.orchestra.run.smithi195:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T03:12:50.720 DEBUG:teuthology.orchestra.run.smithi195:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T03:12:50.769 DEBUG:teuthology.orchestra.run.smithi195:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T03:12:50.854 DEBUG:teuthology.orchestra.run.smithi195:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T03:12:50.901 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-12T03:12:50.901 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T03:12:50.968 DEBUG:teuthology.orchestra.run.smithi064:> sudo service rsyslog restart 2024-09-12T03:12:50.971 DEBUG:teuthology.orchestra.run.smithi115:> sudo service rsyslog restart 2024-09-12T03:12:50.973 DEBUG:teuthology.orchestra.run.smithi195:> sudo service rsyslog restart 2024-09-12T03:12:51.019 INFO:teuthology.orchestra.run.smithi115.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T03:12:51.021 INFO:teuthology.orchestra.run.smithi064.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T03:12:51.148 INFO:teuthology.orchestra.run.smithi195.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T03:12:51.710 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-12T03:12:51.718 INFO:teuthology.task.internal:Starting timer... 2024-09-12T03:12:51.718 INFO:teuthology.run_tasks:Running task pcp... 2024-09-12T03:12:51.856 INFO:teuthology.run_tasks:Running task selinux... 2024-09-12T03:12:51.978 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-12T03:12:51.979 DEBUG:teuthology.orchestra.run.smithi064:> sudo service auditd rotate 2024-09-12T03:12:52.065 INFO:teuthology.orchestra.run.smithi064.stdout:Rotating logs: 2024-09-12T03:12:52.067 DEBUG:teuthology.orchestra.run.smithi115:> sudo service auditd rotate 2024-09-12T03:12:52.122 INFO:teuthology.orchestra.run.smithi115.stdout:Rotating logs: 2024-09-12T03:12:52.124 DEBUG:teuthology.orchestra.run.smithi195:> sudo service auditd rotate 2024-09-12T03:12:52.194 INFO:teuthology.orchestra.run.smithi195.stdout:Rotating logs: 2024-09-12T03:12:52.196 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-12T03:12:52.196 DEBUG:teuthology.orchestra.run.smithi064:> /usr/sbin/getenforce 2024-09-12T03:12:52.240 INFO:teuthology.orchestra.run.smithi064.stdout:Permissive 2024-09-12T03:12:52.240 DEBUG:teuthology.orchestra.run.smithi115:> /usr/sbin/getenforce 2024-09-12T03:12:52.269 INFO:teuthology.orchestra.run.smithi115.stdout:Permissive 2024-09-12T03:12:52.269 DEBUG:teuthology.orchestra.run.smithi195:> /usr/sbin/getenforce 2024-09-12T03:12:52.305 INFO:teuthology.orchestra.run.smithi195.stdout:Permissive 2024-09-12T03:12:52.305 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi064.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi115.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi195.front.sepia.ceph.com': 'permissive'} 2024-09-12T03:12:52.306 DEBUG:teuthology.orchestra.run.smithi064:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T03:12:52.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:12:52.335 DEBUG:teuthology.orchestra.run.smithi115:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T03:12:52.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:12:52.365 DEBUG:teuthology.orchestra.run.smithi195:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T03:12:52.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:12:52.391 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-12T03:12:52.391 DEBUG:teuthology.orchestra.run.smithi064:> sudo /usr/sbin/setenforce permissive 2024-09-12T03:12:52.421 DEBUG:teuthology.orchestra.run.smithi115:> sudo /usr/sbin/setenforce permissive 2024-09-12T03:12:52.454 DEBUG:teuthology.orchestra.run.smithi195:> sudo /usr/sbin/setenforce permissive 2024-09-12T03:12:52.488 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-12T03:12:52.497 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-12T03:12:52.509 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-12T03:12:52.510 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi064.front.sepia.ceph.com,smithi115.front.sepia.ceph.com,smithi195.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-12T03:16:39.577 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi064.front.sepia.ceph.com'), Remote(name='ubuntu@smithi115.front.sepia.ceph.com'), Remote(name='ubuntu@smithi195.front.sepia.ceph.com')] 2024-09-12T03:16:39.578 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-09-12T03:16:39.580 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:16:39.718 DEBUG:teuthology.orchestra.run.smithi064:> true 2024-09-12T03:16:39.815 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-09-12T03:16:39.815 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-12T03:16:39.816 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:16:39.938 DEBUG:teuthology.orchestra.run.smithi115:> true 2024-09-12T03:16:40.020 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-12T03:16:40.021 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-09-12T03:16:40.021 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:16:40.101 DEBUG:teuthology.orchestra.run.smithi195:> true 2024-09-12T03:16:40.183 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-09-12T03:16:40.183 INFO:teuthology.run_tasks:Running task clock... 2024-09-12T03:16:40.200 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-12T03:16:40.200 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T03:16:40.201 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T03:16:40.203 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T03:16:40.203 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T03:16:40.206 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T03:16:40.206 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T03:16:40.232 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T03:16:40.237 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T03:16:40.247 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T03:16:40.253 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T03:16:40.255 INFO:teuthology.orchestra.run.smithi195.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T03:16:40.271 INFO:teuthology.orchestra.run.smithi195.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T03:16:40.272 INFO:teuthology.orchestra.run.smithi115.stderr:sudo: ntpd: command not found 2024-09-12T03:16:40.281 INFO:teuthology.orchestra.run.smithi064.stderr:sudo: ntpd: command not found 2024-09-12T03:16:40.283 INFO:teuthology.orchestra.run.smithi115.stdout:506 Cannot talk to daemon 2024-09-12T03:16:40.294 INFO:teuthology.orchestra.run.smithi064.stdout:506 Cannot talk to daemon 2024-09-12T03:16:40.297 INFO:teuthology.orchestra.run.smithi195.stderr:sudo: ntpd: command not found 2024-09-12T03:16:40.297 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T03:16:40.308 INFO:teuthology.orchestra.run.smithi195.stdout:506 Cannot talk to daemon 2024-09-12T03:16:40.310 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T03:16:40.310 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T03:16:40.323 INFO:teuthology.orchestra.run.smithi195.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T03:16:40.325 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T03:16:40.337 INFO:teuthology.orchestra.run.smithi195.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T03:16:40.373 INFO:teuthology.orchestra.run.smithi115.stderr:bash: line 1: ntpq: command not found 2024-09-12T03:16:40.376 INFO:teuthology.orchestra.run.smithi115.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T03:16:40.376 INFO:teuthology.orchestra.run.smithi115.stdout:=============================================================================== 2024-09-12T03:16:40.376 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:16:40.376 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:16:40.376 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:16:40.376 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:16:40.385 INFO:teuthology.orchestra.run.smithi064.stderr:bash: line 1: ntpq: command not found 2024-09-12T03:16:40.389 INFO:teuthology.orchestra.run.smithi064.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T03:16:40.389 INFO:teuthology.orchestra.run.smithi064.stdout:=============================================================================== 2024-09-12T03:16:40.389 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:16:40.389 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:16:40.389 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:16:40.389 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:16:40.402 INFO:teuthology.orchestra.run.smithi195.stderr:bash: line 1: ntpq: command not found 2024-09-12T03:16:40.405 INFO:teuthology.orchestra.run.smithi195.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T03:16:40.405 INFO:teuthology.orchestra.run.smithi195.stdout:=============================================================================== 2024-09-12T03:16:40.405 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:16:40.405 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:16:40.405 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:16:40.405 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:16:40.406 INFO:teuthology.run_tasks:Running task install... 2024-09-12T03:16:40.415 DEBUG:teuthology.task.install:project ceph 2024-09-12T03:16:40.415 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}} 2024-09-12T03:16:40.415 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'} 2024-09-12T03:16:40.415 INFO:teuthology.task.install:Using flavor: default 2024-09-12T03:16:40.421 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-12T03:16:40.421 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-12T03:16:40.421 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T03:16:40.422 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:16:40.424 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T03:16:40.424 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:16:40.425 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T03:16:40.425 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:16:40.572 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T03:16:40.573 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T03:16:40.577 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T03:16:40.577 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T03:16:40.586 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T03:16:40.586 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T03:16:40.756 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T03:16:40.757 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-12T03:16:40.757 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T03:16:40.781 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T03:16:40.782 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-12T03:16:40.782 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T03:16:40.785 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T03:16:40.785 DEBUG:teuthology.orchestra.run.smithi064:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T03:16:40.787 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T03:16:40.787 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-12T03:16:40.787 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T03:16:40.807 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T03:16:40.807 DEBUG:teuthology.orchestra.run.smithi195:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T03:16:40.813 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T03:16:40.814 DEBUG:teuthology.orchestra.run.smithi115:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T03:16:40.852 DEBUG:teuthology.orchestra.run.smithi064:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T03:16:40.873 DEBUG:teuthology.orchestra.run.smithi195:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T03:16:40.880 DEBUG:teuthology.orchestra.run.smithi115:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T03:16:40.930 DEBUG:teuthology.orchestra.run.smithi064:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T03:16:40.947 DEBUG:teuthology.orchestra.run.smithi195:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T03:16:40.955 DEBUG:teuthology.orchestra.run.smithi115:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T03:16:40.998 INFO:teuthology.orchestra.run.smithi064.stdout:check_obsoletes = 1 2024-09-12T03:16:40.999 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum clean all 2024-09-12T03:16:41.011 INFO:teuthology.orchestra.run.smithi195.stdout:check_obsoletes = 1 2024-09-12T03:16:41.012 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum clean all 2024-09-12T03:16:41.019 INFO:teuthology.orchestra.run.smithi115.stdout:check_obsoletes = 1 2024-09-12T03:16:41.021 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum clean all 2024-09-12T03:16:41.238 INFO:teuthology.orchestra.run.smithi195.stdout:45 files removed 2024-09-12T03:16:41.250 INFO:teuthology.orchestra.run.smithi115.stdout:45 files removed 2024-09-12T03:16:41.256 INFO:teuthology.orchestra.run.smithi064.stdout:45 files removed 2024-09-12T03:16:41.265 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T03:16:41.277 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T03:16:41.284 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T03:16:41.919 INFO:teuthology.orchestra.run.smithi195.stdout:ceph packages for x86_64 226 kB/s | 84 kB 00:00 2024-09-12T03:16:41.931 INFO:teuthology.orchestra.run.smithi115.stdout:ceph packages for x86_64 227 kB/s | 84 kB 00:00 2024-09-12T03:16:41.935 INFO:teuthology.orchestra.run.smithi064.stdout:ceph packages for x86_64 238 kB/s | 84 kB 00:00 2024-09-12T03:16:42.361 INFO:teuthology.orchestra.run.smithi195.stdout:ceph noarch packages 29 kB/s | 12 kB 00:00 2024-09-12T03:16:42.382 INFO:teuthology.orchestra.run.smithi115.stdout:ceph noarch packages 29 kB/s | 12 kB 00:00 2024-09-12T03:16:42.386 INFO:teuthology.orchestra.run.smithi064.stdout:ceph noarch packages 29 kB/s | 12 kB 00:00 2024-09-12T03:16:42.737 INFO:teuthology.orchestra.run.smithi195.stdout:ceph source packages 5.6 kB/s | 1.9 kB 00:00 2024-09-12T03:16:42.765 INFO:teuthology.orchestra.run.smithi115.stdout:ceph source packages 5.5 kB/s | 1.9 kB 00:00 2024-09-12T03:16:42.786 INFO:teuthology.orchestra.run.smithi064.stdout:ceph source packages 5.2 kB/s | 1.9 kB 00:00 2024-09-12T03:16:43.328 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.2 MB 00:00 2024-09-12T03:16:43.990 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - BaseOS 6.9 MB/s | 8.2 MB 00:01 2024-09-12T03:16:44.444 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - BaseOS 5.1 MB/s | 8.2 MB 00:01 2024-09-12T03:16:46.638 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - AppStream 9.2 MB/s | 20 MB 00:02 2024-09-12T03:16:48.541 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - AppStream 5.9 MB/s | 20 MB 00:03 2024-09-12T03:16:48.762 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - AppStream 6.5 MB/s | 20 MB 00:03 2024-09-12T03:16:51.055 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-12T03:16:52.564 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - Extras packages 65 kB/s | 19 kB 00:00 2024-09-12T03:16:52.741 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - CRB 17 MB/s | 6.5 MB 00:00 2024-09-12T03:16:53.188 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-12T03:16:53.337 INFO:teuthology.orchestra.run.smithi195.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-12T03:16:54.492 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 19 kB 00:00 2024-09-12T03:16:54.839 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - Extras packages 65 kB/s | 19 kB 00:00 2024-09-12T03:16:55.764 INFO:teuthology.orchestra.run.smithi115.stdout:Extra Packages for Enterprise Linux 20 MB/s | 23 MB 00:01 2024-09-12T03:16:55.851 INFO:teuthology.orchestra.run.smithi064.stdout:Extra Packages for Enterprise Linux 26 MB/s | 23 MB 00:00 2024-09-12T03:16:58.760 INFO:teuthology.orchestra.run.smithi195.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-12T03:17:01.102 INFO:teuthology.orchestra.run.smithi195.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T03:17:01.103 INFO:teuthology.orchestra.run.smithi195.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T03:17:01.116 INFO:teuthology.orchestra.run.smithi195.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T03:17:01.117 INFO:teuthology.orchestra.run.smithi195.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T03:17:01.118 INFO:teuthology.orchestra.run.smithi195.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T03:17:01.119 INFO:teuthology.orchestra.run.smithi195.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T03:17:01.121 INFO:teuthology.orchestra.run.smithi195.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T03:17:01.124 INFO:teuthology.orchestra.run.smithi195.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T03:17:01.125 INFO:teuthology.orchestra.run.smithi195.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T03:17:01.178 INFO:teuthology.orchestra.run.smithi115.stdout:lab-extras 45 kB/s | 1.7 kB 00:00 2024-09-12T03:17:01.189 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-09-12T03:17:01.202 INFO:teuthology.orchestra.run.smithi195.stdout:====================================================================================== 2024-09-12T03:17:01.202 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repository Size 2024-09-12T03:17:01.202 INFO:teuthology.orchestra.run.smithi195.stdout:====================================================================================== 2024-09-12T03:17:01.202 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-09-12T03:17:01.203 INFO:teuthology.orchestra.run.smithi195.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T03:17:01.203 INFO:teuthology.orchestra.run.smithi195.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T03:17:01.203 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T03:17:01.203 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T03:17:01.203 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T03:17:01.203 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T03:17:01.203 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T03:17:01.203 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T03:17:01.203 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T03:17:01.203 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T03:17:01.203 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T03:17:01.203 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T03:17:01.203 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T03:17:01.204 INFO:teuthology.orchestra.run.smithi195.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T03:17:01.204 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T03:17:01.204 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T03:17:01.204 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T03:17:01.204 INFO:teuthology.orchestra.run.smithi195.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T03:17:01.204 INFO:teuthology.orchestra.run.smithi195.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T03:17:01.204 INFO:teuthology.orchestra.run.smithi195.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T03:17:01.204 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T03:17:01.204 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T03:17:01.204 INFO:teuthology.orchestra.run.smithi195.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T03:17:01.205 INFO:teuthology.orchestra.run.smithi195.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T03:17:01.205 INFO:teuthology.orchestra.run.smithi195.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T03:17:01.205 INFO:teuthology.orchestra.run.smithi195.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T03:17:01.205 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T03:17:01.205 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T03:17:01.205 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T03:17:01.205 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T03:17:01.205 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T03:17:01.205 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T03:17:01.205 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T03:17:01.205 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T03:17:01.205 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T03:17:01.206 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T03:17:01.206 INFO:teuthology.orchestra.run.smithi195.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T03:17:01.206 INFO:teuthology.orchestra.run.smithi195.stdout:Upgrading: 2024-09-12T03:17:01.206 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T03:17:01.206 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T03:17:01.206 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T03:17:01.206 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T03:17:01.206 INFO:teuthology.orchestra.run.smithi195.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T03:17:01.206 INFO:teuthology.orchestra.run.smithi195.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T03:17:01.206 INFO:teuthology.orchestra.run.smithi195.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T03:17:01.206 INFO:teuthology.orchestra.run.smithi195.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T03:17:01.207 INFO:teuthology.orchestra.run.smithi195.stdout:Installing dependencies: 2024-09-12T03:17:01.207 INFO:teuthology.orchestra.run.smithi195.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T03:17:01.207 INFO:teuthology.orchestra.run.smithi195.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T03:17:01.207 INFO:teuthology.orchestra.run.smithi195.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T03:17:01.207 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T03:17:01.207 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T03:17:01.207 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T03:17:01.207 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T03:17:01.207 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T03:17:01.207 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T03:17:01.207 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T03:17:01.207 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T03:17:01.208 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T03:17:01.208 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T03:17:01.208 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T03:17:01.208 INFO:teuthology.orchestra.run.smithi195.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T03:17:01.208 INFO:teuthology.orchestra.run.smithi195.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-12T03:17:01.208 INFO:teuthology.orchestra.run.smithi195.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T03:17:01.208 INFO:teuthology.orchestra.run.smithi195.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T03:17:01.208 INFO:teuthology.orchestra.run.smithi195.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T03:17:01.208 INFO:teuthology.orchestra.run.smithi195.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T03:17:01.208 INFO:teuthology.orchestra.run.smithi195.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T03:17:01.209 INFO:teuthology.orchestra.run.smithi195.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T03:17:01.209 INFO:teuthology.orchestra.run.smithi195.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T03:17:01.209 INFO:teuthology.orchestra.run.smithi195.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T03:17:01.209 INFO:teuthology.orchestra.run.smithi195.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T03:17:01.209 INFO:teuthology.orchestra.run.smithi195.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T03:17:01.209 INFO:teuthology.orchestra.run.smithi195.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T03:17:01.209 INFO:teuthology.orchestra.run.smithi195.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T03:17:01.209 INFO:teuthology.orchestra.run.smithi195.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T03:17:01.209 INFO:teuthology.orchestra.run.smithi195.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T03:17:01.209 INFO:teuthology.orchestra.run.smithi195.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T03:17:01.209 INFO:teuthology.orchestra.run.smithi195.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T03:17:01.209 INFO:teuthology.orchestra.run.smithi195.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T03:17:01.209 INFO:teuthology.orchestra.run.smithi195.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T03:17:01.209 INFO:teuthology.orchestra.run.smithi195.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T03:17:01.210 INFO:teuthology.orchestra.run.smithi195.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T03:17:01.210 INFO:teuthology.orchestra.run.smithi195.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T03:17:01.210 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T03:17:01.210 INFO:teuthology.orchestra.run.smithi195.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T03:17:01.210 INFO:teuthology.orchestra.run.smithi195.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T03:17:01.210 INFO:teuthology.orchestra.run.smithi195.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T03:17:01.210 INFO:teuthology.orchestra.run.smithi195.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T03:17:01.210 INFO:teuthology.orchestra.run.smithi195.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T03:17:01.210 INFO:teuthology.orchestra.run.smithi195.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T03:17:01.210 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T03:17:01.210 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T03:17:01.210 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T03:17:01.211 INFO:teuthology.orchestra.run.smithi195.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T03:17:01.211 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T03:17:01.211 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T03:17:01.211 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T03:17:01.211 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T03:17:01.211 INFO:teuthology.orchestra.run.smithi195.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T03:17:01.211 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T03:17:01.211 INFO:teuthology.orchestra.run.smithi195.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T03:17:01.211 INFO:teuthology.orchestra.run.smithi195.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T03:17:01.212 INFO:teuthology.orchestra.run.smithi195.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T03:17:01.212 INFO:teuthology.orchestra.run.smithi195.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T03:17:01.212 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T03:17:01.212 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T03:17:01.212 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T03:17:01.212 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T03:17:01.212 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T03:17:01.212 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T03:17:01.212 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T03:17:01.212 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T03:17:01.212 INFO:teuthology.orchestra.run.smithi195.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T03:17:01.212 INFO:teuthology.orchestra.run.smithi195.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T03:17:01.212 INFO:teuthology.orchestra.run.smithi195.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T03:17:01.213 INFO:teuthology.orchestra.run.smithi195.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T03:17:01.213 INFO:teuthology.orchestra.run.smithi195.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T03:17:01.213 INFO:teuthology.orchestra.run.smithi195.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T03:17:01.213 INFO:teuthology.orchestra.run.smithi195.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T03:17:01.213 INFO:teuthology.orchestra.run.smithi195.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T03:17:01.213 INFO:teuthology.orchestra.run.smithi195.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T03:17:01.213 INFO:teuthology.orchestra.run.smithi195.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T03:17:01.213 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T03:17:01.213 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T03:17:01.213 INFO:teuthology.orchestra.run.smithi195.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T03:17:01.213 INFO:teuthology.orchestra.run.smithi195.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T03:17:01.213 INFO:teuthology.orchestra.run.smithi195.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T03:17:01.214 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T03:17:01.214 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T03:17:01.214 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T03:17:01.214 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T03:17:01.214 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T03:17:01.214 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T03:17:01.214 INFO:teuthology.orchestra.run.smithi195.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T03:17:01.214 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T03:17:01.214 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T03:17:01.214 INFO:teuthology.orchestra.run.smithi195.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T03:17:01.214 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T03:17:01.214 INFO:teuthology.orchestra.run.smithi195.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T03:17:01.214 INFO:teuthology.orchestra.run.smithi195.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T03:17:01.214 INFO:teuthology.orchestra.run.smithi195.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T03:17:01.215 INFO:teuthology.orchestra.run.smithi195.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T03:17:01.215 INFO:teuthology.orchestra.run.smithi195.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T03:17:01.215 INFO:teuthology.orchestra.run.smithi195.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T03:17:01.215 INFO:teuthology.orchestra.run.smithi195.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T03:17:01.215 INFO:teuthology.orchestra.run.smithi195.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T03:17:01.215 INFO:teuthology.orchestra.run.smithi195.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T03:17:01.215 INFO:teuthology.orchestra.run.smithi195.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T03:17:01.215 INFO:teuthology.orchestra.run.smithi195.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T03:17:01.215 INFO:teuthology.orchestra.run.smithi195.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T03:17:01.215 INFO:teuthology.orchestra.run.smithi195.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T03:17:01.215 INFO:teuthology.orchestra.run.smithi195.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T03:17:01.215 INFO:teuthology.orchestra.run.smithi195.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T03:17:01.215 INFO:teuthology.orchestra.run.smithi195.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T03:17:01.215 INFO:teuthology.orchestra.run.smithi195.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T03:17:01.216 INFO:teuthology.orchestra.run.smithi195.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T03:17:01.216 INFO:teuthology.orchestra.run.smithi195.stdout:Installing weak dependencies: 2024-09-12T03:17:01.216 INFO:teuthology.orchestra.run.smithi195.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T03:17:01.216 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-12T03:17:01.216 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-09-12T03:17:01.216 INFO:teuthology.orchestra.run.smithi195.stdout:====================================================================================== 2024-09-12T03:17:01.216 INFO:teuthology.orchestra.run.smithi195.stdout:Install 148 Packages 2024-09-12T03:17:01.216 INFO:teuthology.orchestra.run.smithi195.stdout:Upgrade 8 Packages 2024-09-12T03:17:01.216 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-12T03:17:01.217 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 226 M 2024-09-12T03:17:01.217 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-09-12T03:17:01.633 INFO:teuthology.orchestra.run.smithi064.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-09-12T03:17:02.003 INFO:teuthology.orchestra.run.smithi195.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 44 kB/s | 6.5 kB 00:00 2024-09-12T03:17:02.328 INFO:teuthology.orchestra.run.smithi195.stdout:(2/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 11 MB/s | 5.4 MB 00:00 2024-09-12T03:17:02.361 INFO:teuthology.orchestra.run.smithi195.stdout:(3/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 2.4 MB/s | 897 kB 00:00 2024-09-12T03:17:02.386 INFO:teuthology.orchestra.run.smithi195.stdout:(4/156): ceph-immutable-object-cache-19.1.1-298 2.4 MB/s | 145 kB 00:00 2024-09-12T03:17:02.479 INFO:teuthology.orchestra.run.smithi195.stdout:(5/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 17 MB/s | 1.5 MB 00:00 2024-09-12T03:17:02.737 INFO:teuthology.orchestra.run.smithi195.stdout:(6/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 18 MB/s | 4.7 MB 00:00 2024-09-12T03:17:02.954 INFO:teuthology.orchestra.run.smithi195.stdout:(7/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 19 MB/s | 21 MB 00:01 2024-09-12T03:17:03.004 INFO:teuthology.orchestra.run.smithi195.stdout:(8/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 3.7 MB/s | 2.4 MB 00:00 2024-09-12T03:17:03.080 INFO:teuthology.orchestra.run.smithi195.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 336 kB/s | 25 kB 00:00 2024-09-12T03:17:03.430 INFO:teuthology.orchestra.run.smithi195.stdout:(10/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 22 MB/s | 11 MB 00:00 2024-09-12T03:17:03.469 INFO:teuthology.orchestra.run.smithi115.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T03:17:03.469 INFO:teuthology.orchestra.run.smithi115.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T03:17:03.479 INFO:teuthology.orchestra.run.smithi115.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T03:17:03.480 INFO:teuthology.orchestra.run.smithi115.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T03:17:03.481 INFO:teuthology.orchestra.run.smithi115.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T03:17:03.482 INFO:teuthology.orchestra.run.smithi115.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T03:17:03.484 INFO:teuthology.orchestra.run.smithi115.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T03:17:03.486 INFO:teuthology.orchestra.run.smithi115.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T03:17:03.487 INFO:teuthology.orchestra.run.smithi115.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T03:17:03.488 INFO:teuthology.orchestra.run.smithi195.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 3.8 MB/s | 228 kB 00:00 2024-09-12T03:17:03.550 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-12T03:17:03.576 INFO:teuthology.orchestra.run.smithi115.stdout:====================================================================================== 2024-09-12T03:17:03.576 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-12T03:17:03.576 INFO:teuthology.orchestra.run.smithi115.stdout:====================================================================================== 2024-09-12T03:17:03.576 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-12T03:17:03.576 INFO:teuthology.orchestra.run.smithi115.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T03:17:03.576 INFO:teuthology.orchestra.run.smithi115.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T03:17:03.576 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T03:17:03.576 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T03:17:03.576 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T03:17:03.576 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T03:17:03.577 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T03:17:03.577 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T03:17:03.577 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T03:17:03.577 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T03:17:03.577 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T03:17:03.577 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T03:17:03.577 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T03:17:03.577 INFO:teuthology.orchestra.run.smithi115.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T03:17:03.577 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T03:17:03.577 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T03:17:03.577 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T03:17:03.577 INFO:teuthology.orchestra.run.smithi115.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T03:17:03.577 INFO:teuthology.orchestra.run.smithi115.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T03:17:03.578 INFO:teuthology.orchestra.run.smithi115.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T03:17:03.578 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T03:17:03.578 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T03:17:03.578 INFO:teuthology.orchestra.run.smithi115.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T03:17:03.578 INFO:teuthology.orchestra.run.smithi115.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T03:17:03.578 INFO:teuthology.orchestra.run.smithi115.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T03:17:03.578 INFO:teuthology.orchestra.run.smithi115.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T03:17:03.578 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T03:17:03.578 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T03:17:03.578 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T03:17:03.578 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T03:17:03.578 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T03:17:03.578 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T03:17:03.578 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T03:17:03.579 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T03:17:03.579 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T03:17:03.579 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T03:17:03.579 INFO:teuthology.orchestra.run.smithi115.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T03:17:03.579 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrading: 2024-09-12T03:17:03.579 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T03:17:03.579 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T03:17:03.579 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T03:17:03.579 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T03:17:03.579 INFO:teuthology.orchestra.run.smithi115.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T03:17:03.579 INFO:teuthology.orchestra.run.smithi115.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T03:17:03.579 INFO:teuthology.orchestra.run.smithi115.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T03:17:03.579 INFO:teuthology.orchestra.run.smithi115.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T03:17:03.580 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-12T03:17:03.580 INFO:teuthology.orchestra.run.smithi115.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T03:17:03.580 INFO:teuthology.orchestra.run.smithi115.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T03:17:03.580 INFO:teuthology.orchestra.run.smithi115.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T03:17:03.580 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T03:17:03.580 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T03:17:03.580 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T03:17:03.580 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T03:17:03.580 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T03:17:03.580 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T03:17:03.580 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T03:17:03.580 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T03:17:03.580 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T03:17:03.580 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T03:17:03.581 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T03:17:03.581 INFO:teuthology.orchestra.run.smithi115.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T03:17:03.581 INFO:teuthology.orchestra.run.smithi115.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-12T03:17:03.581 INFO:teuthology.orchestra.run.smithi115.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T03:17:03.581 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T03:17:03.581 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T03:17:03.581 INFO:teuthology.orchestra.run.smithi115.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T03:17:03.581 INFO:teuthology.orchestra.run.smithi115.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T03:17:03.581 INFO:teuthology.orchestra.run.smithi115.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T03:17:03.581 INFO:teuthology.orchestra.run.smithi115.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T03:17:03.581 INFO:teuthology.orchestra.run.smithi115.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T03:17:03.581 INFO:teuthology.orchestra.run.smithi115.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T03:17:03.581 INFO:teuthology.orchestra.run.smithi115.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T03:17:03.582 INFO:teuthology.orchestra.run.smithi115.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T03:17:03.582 INFO:teuthology.orchestra.run.smithi115.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T03:17:03.582 INFO:teuthology.orchestra.run.smithi115.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T03:17:03.582 INFO:teuthology.orchestra.run.smithi115.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T03:17:03.582 INFO:teuthology.orchestra.run.smithi115.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T03:17:03.582 INFO:teuthology.orchestra.run.smithi115.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T03:17:03.582 INFO:teuthology.orchestra.run.smithi115.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T03:17:03.582 INFO:teuthology.orchestra.run.smithi115.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T03:17:03.582 INFO:teuthology.orchestra.run.smithi115.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T03:17:03.582 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T03:17:03.582 INFO:teuthology.orchestra.run.smithi115.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T03:17:03.582 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T03:17:03.582 INFO:teuthology.orchestra.run.smithi115.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T03:17:03.582 INFO:teuthology.orchestra.run.smithi115.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T03:17:03.583 INFO:teuthology.orchestra.run.smithi115.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T03:17:03.583 INFO:teuthology.orchestra.run.smithi115.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T03:17:03.583 INFO:teuthology.orchestra.run.smithi115.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T03:17:03.583 INFO:teuthology.orchestra.run.smithi115.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T03:17:03.583 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T03:17:03.583 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T03:17:03.583 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T03:17:03.583 INFO:teuthology.orchestra.run.smithi115.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T03:17:03.583 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T03:17:03.583 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T03:17:03.583 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T03:17:03.583 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T03:17:03.583 INFO:teuthology.orchestra.run.smithi115.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T03:17:03.584 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T03:17:03.584 INFO:teuthology.orchestra.run.smithi115.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T03:17:03.584 INFO:teuthology.orchestra.run.smithi115.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T03:17:03.584 INFO:teuthology.orchestra.run.smithi115.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T03:17:03.584 INFO:teuthology.orchestra.run.smithi115.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T03:17:03.584 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T03:17:03.584 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T03:17:03.584 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T03:17:03.584 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T03:17:03.584 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T03:17:03.584 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T03:17:03.584 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T03:17:03.584 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T03:17:03.584 INFO:teuthology.orchestra.run.smithi115.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T03:17:03.585 INFO:teuthology.orchestra.run.smithi115.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T03:17:03.585 INFO:teuthology.orchestra.run.smithi115.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T03:17:03.585 INFO:teuthology.orchestra.run.smithi115.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T03:17:03.585 INFO:teuthology.orchestra.run.smithi115.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T03:17:03.585 INFO:teuthology.orchestra.run.smithi115.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T03:17:03.585 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T03:17:03.585 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T03:17:03.585 INFO:teuthology.orchestra.run.smithi115.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T03:17:03.585 INFO:teuthology.orchestra.run.smithi115.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T03:17:03.585 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T03:17:03.586 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T03:17:03.586 INFO:teuthology.orchestra.run.smithi115.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T03:17:03.586 INFO:teuthology.orchestra.run.smithi115.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T03:17:03.586 INFO:teuthology.orchestra.run.smithi115.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T03:17:03.586 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T03:17:03.586 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T03:17:03.586 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T03:17:03.586 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T03:17:03.586 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T03:17:03.586 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T03:17:03.586 INFO:teuthology.orchestra.run.smithi115.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T03:17:03.586 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T03:17:03.586 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T03:17:03.587 INFO:teuthology.orchestra.run.smithi115.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T03:17:03.587 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T03:17:03.587 INFO:teuthology.orchestra.run.smithi115.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T03:17:03.587 INFO:teuthology.orchestra.run.smithi115.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T03:17:03.587 INFO:teuthology.orchestra.run.smithi115.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T03:17:03.587 INFO:teuthology.orchestra.run.smithi115.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T03:17:03.587 INFO:teuthology.orchestra.run.smithi115.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T03:17:03.587 INFO:teuthology.orchestra.run.smithi115.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T03:17:03.587 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T03:17:03.587 INFO:teuthology.orchestra.run.smithi115.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T03:17:03.587 INFO:teuthology.orchestra.run.smithi115.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T03:17:03.587 INFO:teuthology.orchestra.run.smithi115.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T03:17:03.587 INFO:teuthology.orchestra.run.smithi115.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T03:17:03.587 INFO:teuthology.orchestra.run.smithi115.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T03:17:03.587 INFO:teuthology.orchestra.run.smithi115.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T03:17:03.588 INFO:teuthology.orchestra.run.smithi115.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T03:17:03.588 INFO:teuthology.orchestra.run.smithi115.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T03:17:03.588 INFO:teuthology.orchestra.run.smithi115.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T03:17:03.588 INFO:teuthology.orchestra.run.smithi115.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T03:17:03.588 INFO:teuthology.orchestra.run.smithi115.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T03:17:03.588 INFO:teuthology.orchestra.run.smithi115.stdout:Installing weak dependencies: 2024-09-12T03:17:03.588 INFO:teuthology.orchestra.run.smithi115.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T03:17:03.588 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T03:17:03.588 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-12T03:17:03.588 INFO:teuthology.orchestra.run.smithi115.stdout:====================================================================================== 2024-09-12T03:17:03.588 INFO:teuthology.orchestra.run.smithi115.stdout:Install 148 Packages 2024-09-12T03:17:03.588 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrade 8 Packages 2024-09-12T03:17:03.588 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T03:17:03.589 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 226 M 2024-09-12T03:17:03.589 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-12T03:17:03.673 INFO:teuthology.orchestra.run.smithi195.stdout:(12/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 18 MB/s | 17 MB 00:00 2024-09-12T03:17:03.705 INFO:teuthology.orchestra.run.smithi195.stdout:(13/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 124 kB/s | 27 kB 00:00 2024-09-12T03:17:03.731 INFO:teuthology.orchestra.run.smithi195.stdout:(14/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 544 kB/s | 32 kB 00:00 2024-09-12T03:17:03.765 INFO:teuthology.orchestra.run.smithi195.stdout:(15/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 13 MB/s | 752 kB 00:00 2024-09-12T03:17:03.790 INFO:teuthology.orchestra.run.smithi195.stdout:(16/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 2.7 MB/s | 163 kB 00:00 2024-09-12T03:17:03.817 INFO:teuthology.orchestra.run.smithi195.stdout:(17/156): librados-devel-19.1.1-298.gb53b6d3e.e 2.5 MB/s | 127 kB 00:00 2024-09-12T03:17:03.849 INFO:teuthology.orchestra.run.smithi195.stdout:(18/156): libradosstriper1-19.1.1-298.gb53b6d3e 8.4 MB/s | 506 kB 00:00 2024-09-12T03:17:03.908 INFO:teuthology.orchestra.run.smithi195.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 772 kB/s | 45 kB 00:00 2024-09-12T03:17:03.967 INFO:teuthology.orchestra.run.smithi195.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 2.3 MB/s | 136 kB 00:00 2024-09-12T03:17:04.011 INFO:teuthology.orchestra.run.smithi064.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T03:17:04.011 INFO:teuthology.orchestra.run.smithi064.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T03:17:04.024 INFO:teuthology.orchestra.run.smithi064.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T03:17:04.025 INFO:teuthology.orchestra.run.smithi064.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T03:17:04.026 INFO:teuthology.orchestra.run.smithi064.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T03:17:04.027 INFO:teuthology.orchestra.run.smithi064.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T03:17:04.029 INFO:teuthology.orchestra.run.smithi064.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T03:17:04.031 INFO:teuthology.orchestra.run.smithi064.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T03:17:04.033 INFO:teuthology.orchestra.run.smithi064.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T03:17:04.066 INFO:teuthology.orchestra.run.smithi195.stdout:(21/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 21 MB/s | 5.3 MB 00:00 2024-09-12T03:17:04.092 INFO:teuthology.orchestra.run.smithi195.stdout:(22/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 1.3 MB/s | 162 kB 00:00 2024-09-12T03:17:04.100 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-09-12T03:17:04.121 INFO:teuthology.orchestra.run.smithi064.stdout:====================================================================================== 2024-09-12T03:17:04.121 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-09-12T03:17:04.121 INFO:teuthology.orchestra.run.smithi064.stdout:====================================================================================== 2024-09-12T03:17:04.121 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-09-12T03:17:04.121 INFO:teuthology.orchestra.run.smithi064.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T03:17:04.121 INFO:teuthology.orchestra.run.smithi064.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T03:17:04.121 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T03:17:04.121 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T03:17:04.121 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T03:17:04.121 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T03:17:04.121 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T03:17:04.122 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T03:17:04.122 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T03:17:04.122 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T03:17:04.122 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T03:17:04.122 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T03:17:04.122 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T03:17:04.122 INFO:teuthology.orchestra.run.smithi064.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T03:17:04.122 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T03:17:04.122 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T03:17:04.122 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T03:17:04.122 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T03:17:04.122 INFO:teuthology.orchestra.run.smithi064.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T03:17:04.122 INFO:teuthology.orchestra.run.smithi064.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T03:17:04.122 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T03:17:04.123 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T03:17:04.123 INFO:teuthology.orchestra.run.smithi064.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T03:17:04.123 INFO:teuthology.orchestra.run.smithi064.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T03:17:04.123 INFO:teuthology.orchestra.run.smithi064.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T03:17:04.123 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T03:17:04.123 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T03:17:04.123 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T03:17:04.123 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T03:17:04.123 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T03:17:04.123 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T03:17:04.123 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T03:17:04.123 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T03:17:04.124 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T03:17:04.124 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T03:17:04.124 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T03:17:04.124 INFO:teuthology.orchestra.run.smithi064.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T03:17:04.124 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrading: 2024-09-12T03:17:04.124 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T03:17:04.124 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T03:17:04.124 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T03:17:04.124 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T03:17:04.124 INFO:teuthology.orchestra.run.smithi064.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T03:17:04.124 INFO:teuthology.orchestra.run.smithi064.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T03:17:04.124 INFO:teuthology.orchestra.run.smithi064.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T03:17:04.124 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T03:17:04.124 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-09-12T03:17:04.125 INFO:teuthology.orchestra.run.smithi064.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T03:17:04.125 INFO:teuthology.orchestra.run.smithi064.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T03:17:04.125 INFO:teuthology.orchestra.run.smithi064.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T03:17:04.125 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T03:17:04.125 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T03:17:04.125 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T03:17:04.125 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T03:17:04.125 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T03:17:04.125 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T03:17:04.125 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T03:17:04.125 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T03:17:04.125 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T03:17:04.125 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T03:17:04.126 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T03:17:04.126 INFO:teuthology.orchestra.run.smithi064.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T03:17:04.126 INFO:teuthology.orchestra.run.smithi064.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-12T03:17:04.126 INFO:teuthology.orchestra.run.smithi064.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T03:17:04.126 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T03:17:04.126 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T03:17:04.126 INFO:teuthology.orchestra.run.smithi064.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T03:17:04.126 INFO:teuthology.orchestra.run.smithi064.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T03:17:04.126 INFO:teuthology.orchestra.run.smithi064.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T03:17:04.126 INFO:teuthology.orchestra.run.smithi064.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T03:17:04.126 INFO:teuthology.orchestra.run.smithi064.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T03:17:04.126 INFO:teuthology.orchestra.run.smithi064.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T03:17:04.126 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T03:17:04.127 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T03:17:04.127 INFO:teuthology.orchestra.run.smithi064.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T03:17:04.127 INFO:teuthology.orchestra.run.smithi064.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T03:17:04.127 INFO:teuthology.orchestra.run.smithi064.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T03:17:04.127 INFO:teuthology.orchestra.run.smithi064.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T03:17:04.127 INFO:teuthology.orchestra.run.smithi064.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T03:17:04.127 INFO:teuthology.orchestra.run.smithi064.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T03:17:04.127 INFO:teuthology.orchestra.run.smithi064.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T03:17:04.127 INFO:teuthology.orchestra.run.smithi064.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T03:17:04.127 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T03:17:04.127 INFO:teuthology.orchestra.run.smithi064.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T03:17:04.127 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T03:17:04.127 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T03:17:04.127 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T03:17:04.128 INFO:teuthology.orchestra.run.smithi064.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T03:17:04.128 INFO:teuthology.orchestra.run.smithi064.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T03:17:04.128 INFO:teuthology.orchestra.run.smithi064.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T03:17:04.128 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T03:17:04.128 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T03:17:04.128 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T03:17:04.128 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T03:17:04.128 INFO:teuthology.orchestra.run.smithi064.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T03:17:04.128 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T03:17:04.128 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T03:17:04.128 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T03:17:04.128 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T03:17:04.129 INFO:teuthology.orchestra.run.smithi064.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T03:17:04.129 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T03:17:04.129 INFO:teuthology.orchestra.run.smithi064.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T03:17:04.129 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T03:17:04.129 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T03:17:04.129 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T03:17:04.129 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T03:17:04.129 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T03:17:04.129 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T03:17:04.129 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T03:17:04.129 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T03:17:04.130 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T03:17:04.130 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T03:17:04.130 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T03:17:04.130 INFO:teuthology.orchestra.run.smithi064.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T03:17:04.130 INFO:teuthology.orchestra.run.smithi064.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T03:17:04.130 INFO:teuthology.orchestra.run.smithi064.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T03:17:04.130 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T03:17:04.130 INFO:teuthology.orchestra.run.smithi064.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T03:17:04.130 INFO:teuthology.orchestra.run.smithi064.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T03:17:04.130 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T03:17:04.130 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T03:17:04.130 INFO:teuthology.orchestra.run.smithi064.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T03:17:04.130 INFO:teuthology.orchestra.run.smithi064.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T03:17:04.130 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T03:17:04.131 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T03:17:04.131 INFO:teuthology.orchestra.run.smithi064.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T03:17:04.131 INFO:teuthology.orchestra.run.smithi064.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T03:17:04.131 INFO:teuthology.orchestra.run.smithi064.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T03:17:04.131 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T03:17:04.131 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T03:17:04.131 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T03:17:04.131 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T03:17:04.131 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T03:17:04.131 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T03:17:04.131 INFO:teuthology.orchestra.run.smithi064.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T03:17:04.131 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T03:17:04.131 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T03:17:04.131 INFO:teuthology.orchestra.run.smithi064.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T03:17:04.132 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T03:17:04.132 INFO:teuthology.orchestra.run.smithi064.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T03:17:04.132 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T03:17:04.132 INFO:teuthology.orchestra.run.smithi064.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T03:17:04.132 INFO:teuthology.orchestra.run.smithi064.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T03:17:04.132 INFO:teuthology.orchestra.run.smithi064.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T03:17:04.132 INFO:teuthology.orchestra.run.smithi064.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T03:17:04.132 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T03:17:04.132 INFO:teuthology.orchestra.run.smithi064.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T03:17:04.132 INFO:teuthology.orchestra.run.smithi064.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T03:17:04.133 INFO:teuthology.orchestra.run.smithi064.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T03:17:04.133 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T03:17:04.133 INFO:teuthology.orchestra.run.smithi064.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T03:17:04.133 INFO:teuthology.orchestra.run.smithi064.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T03:17:04.133 INFO:teuthology.orchestra.run.smithi064.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T03:17:04.133 INFO:teuthology.orchestra.run.smithi064.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T03:17:04.133 INFO:teuthology.orchestra.run.smithi064.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T03:17:04.133 INFO:teuthology.orchestra.run.smithi064.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T03:17:04.133 INFO:teuthology.orchestra.run.smithi064.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T03:17:04.133 INFO:teuthology.orchestra.run.smithi064.stdout:Installing weak dependencies: 2024-09-12T03:17:04.133 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T03:17:04.133 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:17:04.133 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-09-12T03:17:04.133 INFO:teuthology.orchestra.run.smithi064.stdout:====================================================================================== 2024-09-12T03:17:04.134 INFO:teuthology.orchestra.run.smithi064.stdout:Install 148 Packages 2024-09-12T03:17:04.134 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrade 8 Packages 2024-09-12T03:17:04.134 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:17:04.134 INFO:teuthology.orchestra.run.smithi195.stdout:(23/156): python3-rados-19.1.1-298.gb53b6d3e.el 5.4 MB/s | 321 kB 00:00 2024-09-12T03:17:04.135 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 226 M 2024-09-12T03:17:04.135 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-09-12T03:17:04.150 INFO:teuthology.orchestra.run.smithi195.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 5.0 MB/s | 301 kB 00:00 2024-09-12T03:17:04.176 INFO:teuthology.orchestra.run.smithi195.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 2.0 MB/s | 100 kB 00:00 2024-09-12T03:17:04.201 INFO:teuthology.orchestra.run.smithi195.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-12T03:17:04.276 INFO:teuthology.orchestra.run.smithi195.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 2.2 MB/s | 171 kB 00:00 2024-09-12T03:17:04.362 INFO:teuthology.orchestra.run.smithi115.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 35 kB/s | 6.5 kB 00:00 2024-09-12T03:17:04.368 INFO:teuthology.orchestra.run.smithi195.stdout:(28/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 16 MB/s | 3.1 MB 00:00 2024-09-12T03:17:04.385 INFO:teuthology.orchestra.run.smithi195.stdout:(29/156): ceph-grafana-dashboards-19.1.1-298.gb 293 kB/s | 32 kB 00:00 2024-09-12T03:17:04.419 INFO:teuthology.orchestra.run.smithi195.stdout:(30/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 2.8 MB/s | 146 kB 00:00 2024-09-12T03:17:04.703 INFO:teuthology.orchestra.run.smithi195.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 11 MB/s | 3.6 MB 00:00 2024-09-12T03:17:04.794 INFO:teuthology.orchestra.run.smithi195.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 2.7 MB/s | 249 kB 00:00 2024-09-12T03:17:04.920 INFO:teuthology.orchestra.run.smithi195.stdout:(33/156): ceph-mgr-diskprediction-local-19.1.1- 15 MB/s | 7.4 MB 00:00 2024-09-12T03:17:04.934 INFO:teuthology.orchestra.run.smithi064.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 27 kB/s | 6.5 kB 00:00 2024-09-12T03:17:04.945 INFO:teuthology.orchestra.run.smithi195.stdout:(34/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 328 kB/s | 49 kB 00:00 2024-09-12T03:17:04.970 INFO:teuthology.orchestra.run.smithi195.stdout:(35/156): ceph-prometheus-alerts-19.1.1-298.gb5 329 kB/s | 16 kB 00:00 2024-09-12T03:17:04.979 INFO:teuthology.orchestra.run.smithi115.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.4 MB/s | 897 kB 00:00 2024-09-12T03:17:04.995 INFO:teuthology.orchestra.run.smithi195.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 5.5 MB/s | 285 kB 00:00 2024-09-12T03:17:05.037 INFO:teuthology.orchestra.run.smithi195.stdout:(37/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 11 MB/s | 765 kB 00:00 2024-09-12T03:17:05.063 INFO:teuthology.orchestra.run.smithi195.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 272 kB/s | 18 kB 00:00 2024-09-12T03:17:05.063 INFO:teuthology.orchestra.run.smithi115.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 1.7 MB/s | 145 kB 00:00 2024-09-12T03:17:05.142 INFO:teuthology.orchestra.run.smithi064.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 4.2 MB/s | 897 kB 00:00 2024-09-12T03:17:05.226 INFO:teuthology.orchestra.run.smithi064.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 1.7 MB/s | 145 kB 00:00 2024-09-12T03:17:05.380 INFO:teuthology.orchestra.run.smithi115.stdout:(4/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 4.5 MB/s | 5.4 MB 00:01 2024-09-12T03:17:05.384 INFO:teuthology.orchestra.run.smithi064.stdout:(4/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 15 MB/s | 2.4 MB 00:00 2024-09-12T03:17:05.501 INFO:teuthology.orchestra.run.smithi064.stdout:(5/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 13 MB/s | 1.5 MB 00:00 2024-09-12T03:17:05.538 INFO:teuthology.orchestra.run.smithi115.stdout:(5/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 9.6 MB/s | 1.5 MB 00:00 2024-09-12T03:17:05.722 INFO:teuthology.orchestra.run.smithi195.stdout:(39/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 18 MB/s | 49 MB 00:02 2024-09-12T03:17:05.738 INFO:teuthology.orchestra.run.smithi195.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 114 kB/s | 80 kB 00:00 2024-09-12T03:17:05.764 INFO:teuthology.orchestra.run.smithi115.stdout:(6/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 3.4 MB/s | 2.4 MB 00:00 2024-09-12T03:17:05.777 INFO:teuthology.orchestra.run.smithi064.stdout:(6/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 17 MB/s | 4.7 MB 00:00 2024-09-12T03:17:05.952 INFO:teuthology.orchestra.run.smithi064.stdout:(7/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 4.3 MB/s | 5.4 MB 00:01 2024-09-12T03:17:06.006 INFO:teuthology.orchestra.run.smithi115.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 10 MB/s | 4.7 MB 00:00 2024-09-12T03:17:06.205 INFO:teuthology.orchestra.run.smithi195.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 543 kB/s | 253 kB 00:00 2024-09-12T03:17:06.264 INFO:teuthology.orchestra.run.smithi195.stdout:(42/156): libquadmath-11.5.0-2.el9.x86_64.rpm 353 kB/s | 191 kB 00:00 2024-09-12T03:17:06.289 INFO:teuthology.orchestra.run.smithi195.stdout:(43/156): libgfortran-11.5.0-2.el9.x86_64.rpm 653 kB/s | 801 kB 00:01 2024-09-12T03:17:06.381 INFO:teuthology.orchestra.run.smithi195.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 912 kB/s | 106 kB 00:00 2024-09-12T03:17:06.406 INFO:teuthology.orchestra.run.smithi195.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-09-12T03:17:06.490 INFO:teuthology.orchestra.run.smithi195.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-09-12T03:17:06.523 INFO:teuthology.orchestra.run.smithi195.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-09-12T03:17:06.707 INFO:teuthology.orchestra.run.smithi195.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 2.4 MB/s | 1.2 MB 00:00 2024-09-12T03:17:06.743 INFO:teuthology.orchestra.run.smithi064.stdout:(8/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86_ 17 MB/s | 17 MB 00:00 2024-09-12T03:17:06.782 INFO:teuthology.orchestra.run.smithi195.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 2.3 MB/s | 689 kB 00:00 2024-09-12T03:17:06.810 INFO:teuthology.orchestra.run.smithi064.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 378 kB/s | 25 kB 00:00 2024-09-12T03:17:06.824 INFO:teuthology.orchestra.run.smithi195.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 3.1 MB/s | 943 kB 00:00 2024-09-12T03:17:06.850 INFO:teuthology.orchestra.run.smithi195.stdout:(51/156): boost-program-options-1.75.0-8.el9.x8 750 kB/s | 107 kB 00:00 2024-09-12T03:17:06.892 INFO:teuthology.orchestra.run.smithi195.stdout:(52/156): elfutils-libelf-devel-0.191-4.el9.x86 199 kB/s | 22 kB 00:00 2024-09-12T03:17:06.942 INFO:teuthology.orchestra.run.smithi195.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-12T03:17:06.956 INFO:teuthology.orchestra.run.smithi115.stdout:(8/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9. 11 MB/s | 11 MB 00:00 2024-09-12T03:17:06.967 INFO:teuthology.orchestra.run.smithi195.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 279 kB/s | 33 kB 00:00 2024-09-12T03:17:07.051 INFO:teuthology.orchestra.run.smithi195.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 18 kB 00:00 2024-09-12T03:17:07.056 INFO:teuthology.orchestra.run.smithi115.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 252 kB/s | 25 kB 00:00 2024-09-12T03:17:07.076 INFO:teuthology.orchestra.run.smithi195.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 103 kB/s | 11 kB 00:00 2024-09-12T03:17:07.168 INFO:teuthology.orchestra.run.smithi195.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-09-12T03:17:07.202 INFO:teuthology.orchestra.run.smithi195.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 361 kB/s | 45 kB 00:00 2024-09-12T03:17:07.293 INFO:teuthology.orchestra.run.smithi195.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 5.2 MB/s | 662 kB 00:00 2024-09-12T03:17:07.327 INFO:teuthology.orchestra.run.smithi195.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 4.5 MB/s | 581 kB 00:00 2024-09-12T03:17:07.419 INFO:teuthology.orchestra.run.smithi064.stdout:(10/156): ceph-common-19.1.1-298.gb53b6d3e.el9. 7.8 MB/s | 21 MB 00:02 2024-09-12T03:17:07.452 INFO:teuthology.orchestra.run.smithi195.stdout:(61/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.3 MB/s | 3.0 MB 00:00 2024-09-12T03:17:07.477 INFO:teuthology.orchestra.run.smithi064.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 3.8 MB/s | 228 kB 00:00 2024-09-12T03:17:07.478 INFO:teuthology.orchestra.run.smithi195.stdout:(62/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 133 kB/s | 24 kB 00:00 2024-09-12T03:17:07.503 INFO:teuthology.orchestra.run.smithi195.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 270 kB/s | 47 kB 00:00 2024-09-12T03:17:07.595 INFO:teuthology.orchestra.run.smithi195.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-12T03:17:07.612 INFO:teuthology.orchestra.run.smithi195.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 355 kB/s | 38 kB 00:00 2024-09-12T03:17:07.619 INFO:teuthology.orchestra.run.smithi064.stdout:(12/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 6.4 MB/s | 11 MB 00:01 2024-09-12T03:17:07.645 INFO:teuthology.orchestra.run.smithi064.stdout:(13/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 162 kB/s | 27 kB 00:00 2024-09-12T03:17:07.664 INFO:teuthology.orchestra.run.smithi115.stdout:(10/156): ceph-common-19.1.1-298.gb53b6d3e.el9. 6.1 MB/s | 21 MB 00:03 2024-09-12T03:17:07.670 INFO:teuthology.orchestra.run.smithi064.stdout:(14/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 633 kB/s | 32 kB 00:00 2024-09-12T03:17:07.723 INFO:teuthology.orchestra.run.smithi115.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 3.8 MB/s | 228 kB 00:00 2024-09-12T03:17:07.737 INFO:teuthology.orchestra.run.smithi064.stdout:(15/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 8.0 MB/s | 752 kB 00:00 2024-09-12T03:17:07.762 INFO:teuthology.orchestra.run.smithi064.stdout:(16/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 1.7 MB/s | 163 kB 00:00 2024-09-12T03:17:07.773 INFO:teuthology.orchestra.run.smithi115.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 542 kB/s | 27 kB 00:00 2024-09-12T03:17:07.813 INFO:teuthology.orchestra.run.smithi064.stdout:(17/156): librados-devel-19.1.1-298.gb53b6d3e.e 1.6 MB/s | 127 kB 00:00 2024-09-12T03:17:07.857 INFO:teuthology.orchestra.run.smithi115.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 380 kB/s | 32 kB 00:00 2024-09-12T03:17:07.879 INFO:teuthology.orchestra.run.smithi064.stdout:(18/156): libradosstriper1-19.1.1-298.gb53b6d3e 4.2 MB/s | 506 kB 00:00 2024-09-12T03:17:07.920 INFO:teuthology.orchestra.run.smithi195.stdout:(66/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 625 kB/s | 292 kB 00:00 2024-09-12T03:17:07.963 INFO:teuthology.orchestra.run.smithi064.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 541 kB/s | 45 kB 00:00 2024-09-12T03:17:08.007 INFO:teuthology.orchestra.run.smithi115.stdout:(14/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 4.9 MB/s | 752 kB 00:00 2024-09-12T03:17:08.054 INFO:teuthology.orchestra.run.smithi195.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 11 MB/s | 4.9 MB 00:00 2024-09-12T03:17:08.055 INFO:teuthology.orchestra.run.smithi064.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 1.4 MB/s | 136 kB 00:00 2024-09-12T03:17:08.091 INFO:teuthology.orchestra.run.smithi115.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 1.9 MB/s | 163 kB 00:00 2024-09-12T03:17:08.129 INFO:teuthology.orchestra.run.smithi195.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 8.6 MB/s | 4.4 MB 00:00 2024-09-12T03:17:08.147 INFO:teuthology.orchestra.run.smithi064.stdout:(21/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 1.7 MB/s | 162 kB 00:00 2024-09-12T03:17:08.182 INFO:teuthology.orchestra.run.smithi115.stdout:(16/156): librados-devel-19.1.1-298.gb53b6d3e.e 1.4 MB/s | 127 kB 00:00 2024-09-12T03:17:08.188 INFO:teuthology.orchestra.run.smithi195.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 6.7 MB/s | 910 kB 00:00 2024-09-12T03:17:08.239 INFO:teuthology.orchestra.run.smithi064.stdout:(22/156): python3-rados-19.1.1-298.gb53b6d3e.el 3.4 MB/s | 321 kB 00:00 2024-09-12T03:17:08.255 INFO:teuthology.orchestra.run.smithi195.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-12T03:17:08.296 INFO:teuthology.orchestra.run.smithi195.stdout:(71/156): postgresql-13.16-1.el9.x86_64.rpm 4.2 MB/s | 1.6 MB 00:00 2024-09-12T03:17:08.330 INFO:teuthology.orchestra.run.smithi064.stdout:(23/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 3.2 MB/s | 301 kB 00:00 2024-09-12T03:17:08.349 INFO:teuthology.orchestra.run.smithi115.stdout:(17/156): libradosstriper1-19.1.1-298.gb53b6d3e 3.0 MB/s | 506 kB 00:00 2024-09-12T03:17:08.397 INFO:teuthology.orchestra.run.smithi195.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 7.1 MB/s | 1.0 MB 00:00 2024-09-12T03:17:08.414 INFO:teuthology.orchestra.run.smithi195.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 748 kB/s | 87 kB 00:00 2024-09-12T03:17:08.423 INFO:teuthology.orchestra.run.smithi064.stdout:(24/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 8.8 MB/s | 5.3 MB 00:00 2024-09-12T03:17:08.456 INFO:teuthology.orchestra.run.smithi064.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 794 kB/s | 100 kB 00:00 2024-09-12T03:17:08.489 INFO:teuthology.orchestra.run.smithi195.stdout:(74/156): postgresql-server-13.16-1.el9.x86_64. 19 MB/s | 5.8 MB 00:00 2024-09-12T03:17:08.506 INFO:teuthology.orchestra.run.smithi064.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.0 MB/s | 85 kB 00:00 2024-09-12T03:17:08.539 INFO:teuthology.orchestra.run.smithi195.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 489 kB/s | 61 kB 00:00 2024-09-12T03:17:08.598 INFO:teuthology.orchestra.run.smithi064.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 1.8 MB/s | 171 kB 00:00 2024-09-12T03:17:08.606 INFO:teuthology.orchestra.run.smithi195.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.0 MB/s | 245 kB 00:00 2024-09-12T03:17:08.682 INFO:teuthology.orchestra.run.smithi064.stdout:(28/156): ceph-grafana-dashboards-19.1.1-298.gb 382 kB/s | 32 kB 00:00 2024-09-12T03:17:08.690 INFO:teuthology.orchestra.run.smithi195.stdout:(77/156): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 6.0 MB 00:00 2024-09-12T03:17:08.715 INFO:teuthology.orchestra.run.smithi195.stdout:(78/156): python3-jinja2-2.11.3-6.el9.noarch.rp 1.4 MB/s | 248 kB 00:00 2024-09-12T03:17:08.740 INFO:teuthology.orchestra.run.smithi195.stdout:(79/156): python3-jsonpatch-1.21-16.el9.noarch. 192 kB/s | 26 kB 00:00 2024-09-12T03:17:08.774 INFO:teuthology.orchestra.run.smithi064.stdout:(29/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 1.6 MB/s | 146 kB 00:00 2024-09-12T03:17:08.799 INFO:teuthology.orchestra.run.smithi195.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 176 kB/s | 19 kB 00:00 2024-09-12T03:17:08.833 INFO:teuthology.orchestra.run.smithi195.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-09-12T03:17:08.858 INFO:teuthology.orchestra.run.smithi195.stdout:(82/156): python3-markupsafe-1.1.1-12.el9.x86_6 296 kB/s | 35 kB 00:00 2024-09-12T03:17:08.975 INFO:teuthology.orchestra.run.smithi195.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.9 MB/s | 222 kB 00:00 2024-09-12T03:17:09.050 INFO:teuthology.orchestra.run.smithi195.stdout:(84/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 5.1 MB 00:00 2024-09-12T03:17:09.091 INFO:teuthology.orchestra.run.smithi115.stdout:(18/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 5.1 MB/s | 17 MB 00:03 2024-09-12T03:17:09.100 INFO:teuthology.orchestra.run.smithi195.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 616 kB/s | 77 kB 00:00 2024-09-12T03:17:09.159 INFO:teuthology.orchestra.run.smithi195.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 389 kB/s | 42 kB 00:00 2024-09-12T03:17:09.166 INFO:teuthology.orchestra.run.smithi115.stdout:(19/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 6.5 MB/s | 5.3 MB 00:00 2024-09-12T03:17:09.192 INFO:teuthology.orchestra.run.smithi115.stdout:(20/156): python3-ceph-argparse-19.1.1-298.gb53 449 kB/s | 45 kB 00:00 2024-09-12T03:17:09.217 INFO:teuthology.orchestra.run.smithi115.stdout:(21/156): python3-ceph-common-19.1.1-298.gb53b6 2.6 MB/s | 136 kB 00:00 2024-09-12T03:17:09.218 INFO:teuthology.orchestra.run.smithi195.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 2.2 MB/s | 269 kB 00:00 2024-09-12T03:17:09.242 INFO:teuthology.orchestra.run.smithi115.stdout:(22/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 3.2 MB/s | 162 kB 00:00 2024-09-12T03:17:09.243 INFO:teuthology.orchestra.run.smithi195.stdout:(88/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 611 kB/s | 250 kB 00:00 2024-09-12T03:17:09.268 INFO:teuthology.orchestra.run.smithi115.stdout:(23/156): python3-rados-19.1.1-298.gb53b6d3e.el 6.2 MB/s | 321 kB 00:00 2024-09-12T03:17:09.268 INFO:teuthology.orchestra.run.smithi195.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-12T03:17:09.293 INFO:teuthology.orchestra.run.smithi115.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 5.9 MB/s | 301 kB 00:00 2024-09-12T03:17:09.310 INFO:teuthology.orchestra.run.smithi115.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 2.3 MB/s | 100 kB 00:00 2024-09-12T03:17:09.335 INFO:teuthology.orchestra.run.smithi195.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-09-12T03:17:09.336 INFO:teuthology.orchestra.run.smithi115.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 2.0 MB/s | 85 kB 00:00 2024-09-12T03:17:09.341 INFO:teuthology.orchestra.run.smithi064.stdout:(30/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 19 MB/s | 49 MB 00:02 2024-09-12T03:17:09.360 INFO:teuthology.orchestra.run.smithi195.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 435 kB/s | 51 kB 00:00 2024-09-12T03:17:09.386 INFO:teuthology.orchestra.run.smithi195.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 458 kB/s | 54 kB 00:00 2024-09-12T03:17:09.389 INFO:teuthology.orchestra.run.smithi115.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 3.3 MB/s | 171 kB 00:00 2024-09-12T03:17:09.393 INFO:teuthology.orchestra.run.smithi064.stdout:(31/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 3.3 MB/s | 3.1 MB 00:00 2024-09-12T03:17:09.427 INFO:teuthology.orchestra.run.smithi115.stdout:(28/156): ceph-grafana-dashboards-19.1.1-298.gb 767 kB/s | 32 kB 00:00 2024-09-12T03:17:09.469 INFO:teuthology.orchestra.run.smithi064.stdout:(32/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 5.1 MB/s | 3.6 MB 00:00 2024-09-12T03:17:09.486 INFO:teuthology.orchestra.run.smithi195.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 333 kB/s | 42 kB 00:00 2024-09-12T03:17:09.494 INFO:teuthology.orchestra.run.smithi115.stdout:(29/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 2.1 MB/s | 146 kB 00:00 2024-09-12T03:17:09.502 INFO:teuthology.orchestra.run.smithi064.stdout:(33/156): ceph-mgr-modules-core-19.1.1-298.gb53 2.3 MB/s | 249 kB 00:00 2024-09-12T03:17:09.511 INFO:teuthology.orchestra.run.smithi195.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 353 kB/s | 44 kB 00:00 2024-09-12T03:17:09.544 INFO:teuthology.orchestra.run.smithi064.stdout:(34/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 660 kB/s | 49 kB 00:00 2024-09-12T03:17:09.578 INFO:teuthology.orchestra.run.smithi115.stdout:(30/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 12 MB/s | 3.1 MB 00:00 2024-09-12T03:17:09.586 INFO:teuthology.orchestra.run.smithi064.stdout:(35/156): ceph-prometheus-alerts-19.1.1-298.gb5 197 kB/s | 16 kB 00:00 2024-09-12T03:17:09.611 INFO:teuthology.orchestra.run.smithi195.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 1.7 MB/s | 221 kB 00:00 2024-09-12T03:17:09.620 INFO:teuthology.orchestra.run.smithi064.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 3.7 MB/s | 285 kB 00:00 2024-09-12T03:17:09.637 INFO:teuthology.orchestra.run.smithi195.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 286 kB/s | 36 kB 00:00 2024-09-12T03:17:09.728 INFO:teuthology.orchestra.run.smithi064.stdout:(37/156): ceph-mgr-diskprediction-local-19.1.1- 19 MB/s | 7.4 MB 00:00 2024-09-12T03:17:09.729 INFO:teuthology.orchestra.run.smithi195.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 405 kB/s | 47 kB 00:00 2024-09-12T03:17:09.754 INFO:teuthology.orchestra.run.smithi064.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 137 kB/s | 18 kB 00:00 2024-09-12T03:17:09.754 INFO:teuthology.orchestra.run.smithi195.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 2.5 MB/s | 304 kB 00:00 2024-09-12T03:17:09.787 INFO:teuthology.orchestra.run.smithi064.stdout:(39/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 3.7 MB/s | 765 kB 00:00 2024-09-12T03:17:09.846 INFO:teuthology.orchestra.run.smithi195.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 495 kB/s | 58 kB 00:00 2024-09-12T03:17:09.871 INFO:teuthology.orchestra.run.smithi195.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 818 kB/s | 96 kB 00:00 2024-09-12T03:17:09.879 INFO:teuthology.orchestra.run.smithi064.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 532 kB/s | 80 kB 00:00 2024-09-12T03:17:09.913 INFO:teuthology.orchestra.run.smithi064.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.5 MB/s | 191 kB 00:00 2024-09-12T03:17:09.955 INFO:teuthology.orchestra.run.smithi064.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.3 MB/s | 253 kB 00:00 2024-09-12T03:17:09.970 INFO:teuthology.orchestra.run.smithi115.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 7.5 MB/s | 3.6 MB 00:00 2024-09-12T03:17:09.980 INFO:teuthology.orchestra.run.smithi064.stdout:(43/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.5 MB/s | 801 kB 00:00 2024-09-12T03:17:10.005 INFO:teuthology.orchestra.run.smithi064.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-12T03:17:10.020 INFO:teuthology.orchestra.run.smithi115.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 4.9 MB/s | 249 kB 00:00 2024-09-12T03:17:10.030 INFO:teuthology.orchestra.run.smithi195.stdout:(101/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-09-12T03:17:10.030 INFO:teuthology.orchestra.run.smithi064.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-12T03:17:10.055 INFO:teuthology.orchestra.run.smithi195.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 244 kB/s | 45 kB 00:00 2024-09-12T03:17:10.055 INFO:teuthology.orchestra.run.smithi064.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-12T03:17:10.080 INFO:teuthology.orchestra.run.smithi195.stdout:(103/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 272 kB/s | 64 kB 00:00 2024-09-12T03:17:10.081 INFO:teuthology.orchestra.run.smithi064.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-12T03:17:10.123 INFO:teuthology.orchestra.run.smithi064.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 5.8 MB/s | 1.2 MB 00:00 2024-09-12T03:17:10.130 INFO:teuthology.orchestra.run.smithi195.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-12T03:17:10.146 INFO:teuthology.orchestra.run.smithi115.stdout:(33/156): ceph-mgr-diskprediction-local-19.1.1- 13 MB/s | 7.4 MB 00:00 2024-09-12T03:17:10.156 INFO:teuthology.orchestra.run.smithi195.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 12 MB/s | 308 kB 00:00 2024-09-12T03:17:10.171 INFO:teuthology.orchestra.run.smithi115.stdout:(34/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 328 kB/s | 49 kB 00:00 2024-09-12T03:17:10.181 INFO:teuthology.orchestra.run.smithi195.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 779 kB/s | 19 kB 00:00 2024-09-12T03:17:10.196 INFO:teuthology.orchestra.run.smithi115.stdout:(35/156): ceph-prometheus-alerts-19.1.1-298.gb5 328 kB/s | 16 kB 00:00 2024-09-12T03:17:10.221 INFO:teuthology.orchestra.run.smithi115.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 5.5 MB/s | 285 kB 00:00 2024-09-12T03:17:10.265 INFO:teuthology.orchestra.run.smithi064.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 755 kB/s | 107 kB 00:00 2024-09-12T03:17:10.281 INFO:teuthology.orchestra.run.smithi195.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 44 MB/s | 4.4 MB 00:00 2024-09-12T03:17:10.307 INFO:teuthology.orchestra.run.smithi064.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 519 kB/s | 22 kB 00:00 2024-09-12T03:17:10.315 INFO:teuthology.orchestra.run.smithi195.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.3 MB/s | 26 kB 00:00 2024-09-12T03:17:10.332 INFO:teuthology.orchestra.run.smithi195.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 2.8 MB/s | 49 kB 00:00 2024-09-12T03:17:10.357 INFO:teuthology.orchestra.run.smithi064.stdout:(51/156): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 689 kB 00:00 2024-09-12T03:17:10.357 INFO:teuthology.orchestra.run.smithi195.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-12T03:17:10.383 INFO:teuthology.orchestra.run.smithi195.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-09-12T03:17:10.416 INFO:teuthology.orchestra.run.smithi195.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 26 MB/s | 839 kB 00:00 2024-09-12T03:17:10.424 INFO:teuthology.orchestra.run.smithi064.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 2.7 MB/s | 943 kB 00:00 2024-09-12T03:17:10.441 INFO:teuthology.orchestra.run.smithi064.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 392 kB/s | 33 kB 00:00 2024-09-12T03:17:10.450 INFO:teuthology.orchestra.run.smithi195.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 16 MB/s | 548 kB 00:00 2024-09-12T03:17:10.475 INFO:teuthology.orchestra.run.smithi064.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 1.8 MB/s | 313 kB 00:00 2024-09-12T03:17:10.475 INFO:teuthology.orchestra.run.smithi195.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-12T03:17:10.491 INFO:teuthology.orchestra.run.smithi064.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-09-12T03:17:10.500 INFO:teuthology.orchestra.run.smithi195.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-12T03:17:10.525 INFO:teuthology.orchestra.run.smithi064.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-09-12T03:17:10.525 INFO:teuthology.orchestra.run.smithi195.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 562 kB/s | 14 kB 00:00 2024-09-12T03:17:10.550 INFO:teuthology.orchestra.run.smithi064.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-12T03:17:10.551 INFO:teuthology.orchestra.run.smithi195.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-09-12T03:17:10.576 INFO:teuthology.orchestra.run.smithi064.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 898 kB/s | 45 kB 00:00 2024-09-12T03:17:10.585 INFO:teuthology.orchestra.run.smithi195.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 11 MB/s | 358 kB 00:00 2024-09-12T03:17:10.610 INFO:teuthology.orchestra.run.smithi195.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 12 MB/s | 303 kB 00:00 2024-09-12T03:17:10.635 INFO:teuthology.orchestra.run.smithi195.stdout:(120/156): python3-google-auth-2.34.0-1.el9.noa 8.7 MB/s | 220 kB 00:00 2024-09-12T03:17:10.647 INFO:teuthology.orchestra.run.smithi115.stdout:(37/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 14 MB/s | 49 MB 00:03 2024-09-12T03:17:10.660 INFO:teuthology.orchestra.run.smithi195.stdout:(121/156): lua-devel-5.4.4-4.el9.x86_64.rpm 35 kB/s | 22 kB 00:00 2024-09-12T03:17:10.672 INFO:teuthology.orchestra.run.smithi115.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 40 kB/s | 18 kB 00:00 2024-09-12T03:17:10.697 INFO:teuthology.orchestra.run.smithi115.stdout:(39/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 1.5 MB/s | 765 kB 00:00 2024-09-12T03:17:10.701 INFO:teuthology.orchestra.run.smithi064.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-09-12T03:17:10.734 INFO:teuthology.orchestra.run.smithi064.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 3.6 MB/s | 581 kB 00:00 2024-09-12T03:17:10.735 INFO:teuthology.orchestra.run.smithi195.stdout:(122/156): python3-grpcio-1.46.7-10.el9.x86_64. 21 MB/s | 2.0 MB 00:00 2024-09-12T03:17:10.760 INFO:teuthology.orchestra.run.smithi064.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 418 kB/s | 24 kB 00:00 2024-09-12T03:17:10.761 INFO:teuthology.orchestra.run.smithi195.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-09-12T03:17:10.785 INFO:teuthology.orchestra.run.smithi064.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 941 kB/s | 47 kB 00:00 2024-09-12T03:17:10.786 INFO:teuthology.orchestra.run.smithi195.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 212 kB/s | 11 kB 00:00 2024-09-12T03:17:10.806 INFO:teuthology.orchestra.run.smithi115.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 504 kB/s | 80 kB 00:00 2024-09-12T03:17:10.811 INFO:teuthology.orchestra.run.smithi195.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 351 kB/s | 18 kB 00:00 2024-09-12T03:17:10.837 INFO:teuthology.orchestra.run.smithi195.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-09-12T03:17:10.844 INFO:teuthology.orchestra.run.smithi064.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-09-12T03:17:10.848 INFO:teuthology.orchestra.run.smithi115.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.2 MB/s | 191 kB 00:00 2024-09-12T03:17:10.862 INFO:teuthology.orchestra.run.smithi195.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-09-12T03:17:10.869 INFO:teuthology.orchestra.run.smithi064.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-12T03:17:10.887 INFO:teuthology.orchestra.run.smithi195.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-09-12T03:17:10.894 INFO:teuthology.orchestra.run.smithi064.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 767 kB/s | 38 kB 00:00 2024-09-12T03:17:10.898 INFO:teuthology.orchestra.run.smithi115.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-09-12T03:17:10.913 INFO:teuthology.orchestra.run.smithi195.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-12T03:17:10.932 INFO:teuthology.orchestra.run.smithi115.stdout:(43/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.0 MB/s | 801 kB 00:00 2024-09-12T03:17:10.938 INFO:teuthology.orchestra.run.smithi195.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-12T03:17:10.957 INFO:teuthology.orchestra.run.smithi115.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-12T03:17:10.963 INFO:teuthology.orchestra.run.smithi195.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-12T03:17:10.982 INFO:teuthology.orchestra.run.smithi115.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-12T03:17:10.997 INFO:teuthology.orchestra.run.smithi195.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-12T03:17:11.007 INFO:teuthology.orchestra.run.smithi115.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-12T03:17:11.014 INFO:teuthology.orchestra.run.smithi195.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 991 kB/s | 16 kB 00:00 2024-09-12T03:17:11.028 INFO:teuthology.orchestra.run.smithi064.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.1 MB/s | 3.0 MB 00:00 2024-09-12T03:17:11.033 INFO:teuthology.orchestra.run.smithi115.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-12T03:17:11.055 INFO:teuthology.orchestra.run.smithi195.stdout:(134/156): python3-kubernetes-26.1.0-2.el9.noar 5.3 MB/s | 1.0 MB 00:00 2024-09-12T03:17:11.066 INFO:teuthology.orchestra.run.smithi115.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 5.6 MB/s | 1.2 MB 00:00 2024-09-12T03:17:11.081 INFO:teuthology.orchestra.run.smithi195.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-09-12T03:17:11.098 INFO:teuthology.orchestra.run.smithi195.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 672 kB/s | 28 kB 00:00 2024-09-12T03:17:11.115 INFO:teuthology.orchestra.run.smithi195.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 920 kB/s | 31 kB 00:00 2024-09-12T03:17:11.140 INFO:teuthology.orchestra.run.smithi195.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-12T03:17:11.165 INFO:teuthology.orchestra.run.smithi195.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.7 MB/s | 188 kB 00:00 2024-09-12T03:17:11.191 INFO:teuthology.orchestra.run.smithi195.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-09-12T03:17:11.216 INFO:teuthology.orchestra.run.smithi195.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-12T03:17:11.241 INFO:teuthology.orchestra.run.smithi195.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-12T03:17:11.267 INFO:teuthology.orchestra.run.smithi195.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-12T03:17:11.287 INFO:teuthology.orchestra.run.smithi064.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 6.1 MB/s | 1.6 MB 00:00 2024-09-12T03:17:11.292 INFO:teuthology.orchestra.run.smithi195.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 802 kB/s | 20 kB 00:00 2024-09-12T03:17:11.367 INFO:teuthology.orchestra.run.smithi195.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-12T03:17:11.392 INFO:teuthology.orchestra.run.smithi195.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 1.9 MB/s | 191 kB 00:00 2024-09-12T03:17:11.443 INFO:teuthology.orchestra.run.smithi195.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-12T03:17:11.454 INFO:teuthology.orchestra.run.smithi064.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 5.3 MB/s | 910 kB 00:00 2024-09-12T03:17:11.521 INFO:teuthology.orchestra.run.smithi064.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-12T03:17:11.652 INFO:teuthology.orchestra.run.smithi195.stdout:(148/156): librados2-19.1.1-298.gb53b6d3e.el9.x 13 MB/s | 3.4 MB 00:00 2024-09-12T03:17:11.662 INFO:teuthology.orchestra.run.smithi064.stdout:(70/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 5.8 MB/s | 4.4 MB 00:00 2024-09-12T03:17:11.735 INFO:teuthology.orchestra.run.smithi195.stdout:(149/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 11 MB/s | 3.2 MB 00:00 2024-09-12T03:17:11.769 INFO:teuthology.orchestra.run.smithi195.stdout:(150/156): protobuf-compiler-3.14.0-13.el9.x86_ 504 kB/s | 863 kB 00:01 2024-09-12T03:17:11.792 INFO:teuthology.orchestra.run.smithi115.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 147 kB/s | 107 kB 00:00 2024-09-12T03:17:11.840 INFO:teuthology.orchestra.run.smithi064.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 5.8 MB/s | 1.0 MB 00:00 2024-09-12T03:17:11.877 INFO:teuthology.orchestra.run.smithi195.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 377 kB/s | 41 kB 00:00 2024-09-12T03:17:11.896 INFO:teuthology.orchestra.run.smithi064.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-12T03:17:12.161 INFO:teuthology.orchestra.run.smithi195.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 878 kB/s | 249 kB 00:00 2024-09-12T03:17:12.203 INFO:teuthology.orchestra.run.smithi195.stdout:(153/156): NetworkManager-1.49.90-1.el9.x86_64. 4.2 MB/s | 2.3 MB 00:00 2024-09-12T03:17:12.237 INFO:teuthology.orchestra.run.smithi195.stdout:(154/156): NetworkManager-libnm-1.49.90-1.el9.x 3.6 MB/s | 1.8 MB 00:00 2024-09-12T03:17:12.246 INFO:teuthology.orchestra.run.smithi064.stdout:(73/156): openblas-openmp-0.3.26-2.el9.x86_64.r 3.6 MB/s | 4.9 MB 00:01 2024-09-12T03:17:12.305 INFO:teuthology.orchestra.run.smithi064.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-09-12T03:17:12.353 INFO:teuthology.orchestra.run.smithi195.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 16 MB/s | 2.4 MB 00:00 2024-09-12T03:17:12.367 INFO:teuthology.orchestra.run.smithi115.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 38 kB/s | 22 kB 00:00 2024-09-12T03:17:12.422 INFO:teuthology.orchestra.run.smithi064.stdout:(75/156): postgresql-server-13.16-1.el9.x86_64. 6.4 MB/s | 5.8 MB 00:00 2024-09-12T03:17:12.442 INFO:teuthology.orchestra.run.smithi115.stdout:(51/156): automake-1.16.2-8.el9.noarch.rpm 480 kB/s | 689 kB 00:01 2024-09-12T03:17:12.447 INFO:teuthology.orchestra.run.smithi064.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 1.7 MB/s | 245 kB 00:00 2024-09-12T03:17:12.481 INFO:teuthology.orchestra.run.smithi064.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-09-12T03:17:12.506 INFO:teuthology.orchestra.run.smithi064.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 437 kB/s | 26 kB 00:00 2024-09-12T03:17:12.531 INFO:teuthology.orchestra.run.smithi064.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-09-12T03:17:12.582 INFO:teuthology.orchestra.run.smithi064.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 694 kB/s | 35 kB 00:00 2024-09-12T03:17:12.607 INFO:teuthology.orchestra.run.smithi064.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-09-12T03:17:12.654 INFO:teuthology.orchestra.run.smithi195.stdout:(156/156): openssl-3.2.2-6.el9.x86_64.rpm 2.8 MB/s | 1.4 MB 00:00 2024-09-12T03:17:12.660 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-09-12T03:17:12.660 INFO:teuthology.orchestra.run.smithi195.stdout:Total 20 MB/s | 226 MB 00:11 2024-09-12T03:17:12.691 INFO:teuthology.orchestra.run.smithi064.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-09-12T03:17:12.734 INFO:teuthology.orchestra.run.smithi115.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 554 kB/s | 943 kB 00:01 2024-09-12T03:17:12.803 INFO:teuthology.orchestra.run.smithi064.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.9 MB/s | 222 kB 00:00 2024-09-12T03:17:12.908 INFO:teuthology.orchestra.run.smithi064.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 740 kB/s | 77 kB 00:00 2024-09-12T03:17:12.983 INFO:teuthology.orchestra.run.smithi064.stdout:(85/156): python3-babel-2.9.1-2.el9.noarch.rpm 5.5 MB/s | 6.0 MB 00:01 2024-09-12T03:17:13.008 INFO:teuthology.orchestra.run.smithi064.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 421 kB/s | 42 kB 00:00 2024-09-12T03:17:13.018 INFO:teuthology.orchestra.run.smithi115.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 57 kB/s | 33 kB 00:00 2024-09-12T03:17:13.067 INFO:teuthology.orchestra.run.smithi064.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 3.2 MB/s | 269 kB 00:00 2024-09-12T03:17:13.092 INFO:teuthology.orchestra.run.smithi064.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-09-12T03:17:13.142 INFO:teuthology.orchestra.run.smithi064.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-12T03:17:13.168 INFO:teuthology.orchestra.run.smithi064.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-09-12T03:17:13.193 INFO:teuthology.orchestra.run.smithi064.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-12T03:17:13.252 INFO:teuthology.orchestra.run.smithi064.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 710 kB/s | 42 kB 00:00 2024-09-12T03:17:13.277 INFO:teuthology.orchestra.run.smithi115.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 345 kB/s | 313 kB 00:00 2024-09-12T03:17:13.302 INFO:teuthology.orchestra.run.smithi064.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 883 kB/s | 44 kB 00:00 2024-09-12T03:17:13.386 INFO:teuthology.orchestra.run.smithi064.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-09-12T03:17:13.436 INFO:teuthology.orchestra.run.smithi064.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 716 kB/s | 36 kB 00:00 2024-09-12T03:17:13.486 INFO:teuthology.orchestra.run.smithi064.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-09-12T03:17:13.493 INFO:teuthology.orchestra.run.smithi115.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 38 kB/s | 18 kB 00:00 2024-09-12T03:17:13.553 INFO:teuthology.orchestra.run.smithi064.stdout:(97/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.2 MB/s | 5.1 MB 00:00 2024-09-12T03:17:13.603 INFO:teuthology.orchestra.run.smithi064.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-12T03:17:13.637 INFO:teuthology.orchestra.run.smithi064.stdout:(99/156): socat-1.7.4.1-6.el9.x86_64.rpm 2.0 MB/s | 304 kB 00:00 2024-09-12T03:17:13.662 INFO:teuthology.orchestra.run.smithi064.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-12T03:17:13.688 INFO:teuthology.orchestra.run.smithi064.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-12T03:17:13.713 INFO:teuthology.orchestra.run.smithi064.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-09-12T03:17:13.743 INFO:teuthology.orchestra.run.smithi115.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 24 kB/s | 11 kB 00:00 2024-09-12T03:17:13.780 INFO:teuthology.orchestra.run.smithi064.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 243 kB/s | 22 kB 00:00 2024-09-12T03:17:13.830 INFO:teuthology.orchestra.run.smithi064.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-12T03:17:13.859 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-09-12T03:17:13.864 INFO:teuthology.orchestra.run.smithi064.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 9.1 MB/s | 308 kB 00:00 2024-09-12T03:17:13.889 INFO:teuthology.orchestra.run.smithi064.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 780 kB/s | 19 kB 00:00 2024-09-12T03:17:13.956 INFO:teuthology.orchestra.run.smithi064.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.5 MB/s | 863 kB 00:00 2024-09-12T03:17:13.998 INFO:teuthology.orchestra.run.smithi064.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 624 kB/s | 26 kB 00:00 2024-09-12T03:17:14.008 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-09-12T03:17:14.008 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-09-12T03:17:14.023 INFO:teuthology.orchestra.run.smithi064.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-12T03:17:14.048 INFO:teuthology.orchestra.run.smithi064.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-12T03:17:14.074 INFO:teuthology.orchestra.run.smithi064.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-09-12T03:17:14.157 INFO:teuthology.orchestra.run.smithi064.stdout:(112/156): libarrow-9.0.0-11.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-09-12T03:17:14.183 INFO:teuthology.orchestra.run.smithi064.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.5 MB/s | 839 kB 00:00 2024-09-12T03:17:14.208 INFO:teuthology.orchestra.run.smithi064.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-12T03:17:14.210 INFO:teuthology.orchestra.run.smithi115.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 226 kB/s | 160 kB 00:00 2024-09-12T03:17:14.225 INFO:teuthology.orchestra.run.smithi064.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.9 MB/s | 32 kB 00:00 2024-09-12T03:17:14.250 INFO:teuthology.orchestra.run.smithi064.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 562 kB/s | 14 kB 00:00 2024-09-12T03:17:14.260 INFO:teuthology.orchestra.run.smithi115.stdout:(58/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.0 MB/s | 3.0 MB 00:01 2024-09-12T03:17:14.275 INFO:teuthology.orchestra.run.smithi064.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-09-12T03:17:14.309 INFO:teuthology.orchestra.run.smithi064.stdout:(118/156): python3-asyncssh-2.13.2-5.el9.noarch 3.5 MB/s | 548 kB 00:00 2024-09-12T03:17:14.311 INFO:teuthology.orchestra.run.smithi115.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 80 kB/s | 45 kB 00:00 2024-09-12T03:17:14.343 INFO:teuthology.orchestra.run.smithi064.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 8.9 MB/s | 303 kB 00:00 2024-09-12T03:17:14.368 INFO:teuthology.orchestra.run.smithi064.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-09-12T03:17:14.402 INFO:teuthology.orchestra.run.smithi064.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.6 MB/s | 220 kB 00:00 2024-09-12T03:17:14.427 INFO:teuthology.orchestra.run.smithi064.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-12T03:17:14.453 INFO:teuthology.orchestra.run.smithi064.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 428 kB/s | 11 kB 00:00 2024-09-12T03:17:14.478 INFO:teuthology.orchestra.run.smithi064.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 713 kB/s | 18 kB 00:00 2024-09-12T03:17:14.503 INFO:teuthology.orchestra.run.smithi064.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 925 kB/s | 23 kB 00:00 2024-09-12T03:17:14.528 INFO:teuthology.orchestra.run.smithi064.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 785 kB/s | 19 kB 00:00 2024-09-12T03:17:14.553 INFO:teuthology.orchestra.run.smithi064.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 792 kB/s | 20 kB 00:00 2024-09-12T03:17:14.604 INFO:teuthology.orchestra.run.smithi064.stdout:(128/156): python3-kubernetes-26.1.0-2.el9.noar 20 MB/s | 1.0 MB 00:00 2024-09-12T03:17:14.629 INFO:teuthology.orchestra.run.smithi064.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-12T03:17:14.655 INFO:teuthology.orchestra.run.smithi064.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-12T03:17:14.697 INFO:teuthology.orchestra.run.smithi064.stdout:(131/156): python3-grpcio-1.46.7-10.el9.x86_64. 6.2 MB/s | 2.0 MB 00:00 2024-09-12T03:17:14.722 INFO:teuthology.orchestra.run.smithi064.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 861 kB/s | 58 kB 00:00 2024-09-12T03:17:14.747 INFO:teuthology.orchestra.run.smithi064.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 662 kB/s | 16 kB 00:00 2024-09-12T03:17:14.772 INFO:teuthology.orchestra.run.smithi064.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.6 MB/s | 90 kB 00:00 2024-09-12T03:17:14.798 INFO:teuthology.orchestra.run.smithi064.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-12T03:17:14.823 INFO:teuthology.orchestra.run.smithi064.stdout:(136/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-09-12T03:17:14.848 INFO:teuthology.orchestra.run.smithi064.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 610 kB/s | 31 kB 00:00 2024-09-12T03:17:14.853 INFO:teuthology.orchestra.run.smithi115.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 45 kB/s | 24 kB 00:00 2024-09-12T03:17:14.874 INFO:teuthology.orchestra.run.smithi064.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-12T03:17:14.899 INFO:teuthology.orchestra.run.smithi064.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-12T03:17:14.924 INFO:teuthology.orchestra.run.smithi064.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-09-12T03:17:14.949 INFO:teuthology.orchestra.run.smithi064.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-12T03:17:14.975 INFO:teuthology.orchestra.run.smithi064.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-12T03:17:15.000 INFO:teuthology.orchestra.run.smithi064.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-12T03:17:15.025 INFO:teuthology.orchestra.run.smithi064.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 808 kB/s | 20 kB 00:00 2024-09-12T03:17:15.050 INFO:teuthology.orchestra.run.smithi064.stdout:(145/156): re2-20211101-20.el9.x86_64.rpm 7.5 MB/s | 191 kB 00:00 2024-09-12T03:17:15.084 INFO:teuthology.orchestra.run.smithi064.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-09-12T03:17:15.134 INFO:teuthology.orchestra.run.smithi064.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-09-12T03:17:15.393 INFO:teuthology.orchestra.run.smithi064.stdout:(148/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 12 MB/s | 3.2 MB 00:00 2024-09-12T03:17:15.428 INFO:teuthology.orchestra.run.smithi115.stdout:(61/156): libtool-2.4.6-46.el9.x86_64.rpm 498 kB/s | 581 kB 00:01 2024-09-12T03:17:15.453 INFO:teuthology.orchestra.run.smithi115.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 79 kB/s | 47 kB 00:00 2024-09-12T03:17:15.468 INFO:teuthology.orchestra.run.smithi064.stdout:(149/156): librados2-19.1.1-298.gb53b6d3e.el9.x 8.9 MB/s | 3.4 MB 00:00 2024-09-12T03:17:15.512 INFO:teuthology.orchestra.run.smithi115.stdout:(63/156): librdkafka-1.6.1-102.el9.x86_64.rpm 509 kB/s | 662 kB 00:01 2024-09-12T03:17:15.577 INFO:teuthology.orchestra.run.smithi064.stdout:(150/156): NetworkManager-1.49.90-1.el9.x86_64. 13 MB/s | 2.3 MB 00:00 2024-09-12T03:17:15.736 INFO:teuthology.orchestra.run.smithi064.stdout:(151/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 6.3 MB/s | 16 MB 00:02 2024-09-12T03:17:15.761 INFO:teuthology.orchestra.run.smithi064.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 222 kB/s | 41 kB 00:00 2024-09-12T03:17:15.803 INFO:teuthology.orchestra.run.smithi064.stdout:(153/156): NetworkManager-libnm-1.49.90-1.el9.x 5.4 MB/s | 1.8 MB 00:00 2024-09-12T03:17:15.845 INFO:teuthology.orchestra.run.smithi064.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 16 MB/s | 1.4 MB 00:00 2024-09-12T03:17:15.870 INFO:teuthology.orchestra.run.smithi064.stdout:(155/156): NetworkManager-tui-1.49.90-1.el9.x86 1.8 MB/s | 249 kB 00:00 2024-09-12T03:17:15.920 INFO:teuthology.orchestra.run.smithi064.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 20 MB/s | 2.4 MB 00:00 2024-09-12T03:17:15.928 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-09-12T03:17:15.928 INFO:teuthology.orchestra.run.smithi064.stdout:Total 19 MB/s | 226 MB 00:11 2024-09-12T03:17:16.070 INFO:teuthology.orchestra.run.smithi115.stdout:(64/156): openblas-0.3.26-2.el9.x86_64.rpm 69 kB/s | 38 kB 00:00 2024-09-12T03:17:16.190 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-09-12T03:17:16.191 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-09-12T03:17:16.204 INFO:teuthology.orchestra.run.smithi115.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 251 kB/s | 188 kB 00:00 2024-09-12T03:17:16.329 INFO:teuthology.orchestra.run.smithi115.stdout:(66/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 325 kB/s | 292 kB 00:00 2024-09-12T03:17:17.209 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-09-12T03:17:17.376 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-09-12T03:17:17.376 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-09-12T03:17:17.787 INFO:teuthology.orchestra.run.smithi115.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 1.1 MB/s | 1.6 MB 00:01 2024-09-12T03:17:17.870 INFO:teuthology.orchestra.run.smithi115.stdout:(68/156): openblas-openmp-0.3.26-2.el9.x86_64.r 2.7 MB/s | 4.9 MB 00:01 2024-09-12T03:17:17.946 INFO:teuthology.orchestra.run.smithi115.stdout:(69/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 2.5 MB/s | 4.4 MB 00:01 2024-09-12T03:17:18.580 INFO:teuthology.orchestra.run.smithi115.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 193 kB/s | 137 kB 00:00 2024-09-12T03:17:19.397 INFO:teuthology.orchestra.run.smithi115.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 566 kB/s | 910 kB 00:01 2024-09-12T03:17:19.672 INFO:teuthology.orchestra.run.smithi115.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 947 kB/s | 1.0 MB 00:01 2024-09-12T03:17:19.705 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-09-12T03:17:19.706 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-09-12T03:17:20.055 INFO:teuthology.orchestra.run.smithi115.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 133 kB/s | 87 kB 00:00 2024-09-12T03:17:20.722 INFO:teuthology.orchestra.run.smithi115.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 92 kB/s | 61 kB 00:00 2024-09-12T03:17:20.881 INFO:teuthology.orchestra.run.smithi115.stdout:(75/156): postgresql-server-13.16-1.el9.x86_64. 2.0 MB/s | 5.8 MB 00:02 2024-09-12T03:17:21.681 INFO:teuthology.orchestra.run.smithi115.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 255 kB/s | 245 kB 00:00 2024-09-12T03:17:21.739 INFO:teuthology.orchestra.run.smithi115.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 289 kB/s | 248 kB 00:00 2024-09-12T03:17:21.840 INFO:teuthology.orchestra.run.smithi115.stdout:(78/156): python3-babel-2.9.1-2.el9.noarch.rpm 2.8 MB/s | 6.0 MB 00:02 2024-09-12T03:17:22.240 INFO:teuthology.orchestra.run.smithi115.stdout:(79/156): python3-jsonpatch-1.21-16.el9.noarch. 46 kB/s | 26 kB 00:00 2024-09-12T03:17:22.257 INFO:teuthology.orchestra.run.smithi115.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 37 kB/s | 19 kB 00:00 2024-09-12T03:17:22.587 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-09-12T03:17:22.682 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T03:17:22.699 INFO:teuthology.orchestra.run.smithi115.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 201 kB/s | 172 kB 00:00 2024-09-12T03:17:22.724 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T03:17:22.782 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T03:17:22.815 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T03:17:22.851 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T03:17:22.882 INFO:teuthology.orchestra.run.smithi115.stdout:(82/156): python3-markupsafe-1.1.1-12.el9.x86_6 54 kB/s | 35 kB 00:00 2024-09-12T03:17:22.900 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T03:17:23.225 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T03:17:23.361 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:17:23.466 INFO:teuthology.orchestra.run.smithi115.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 327 kB/s | 250 kB 00:00 2024-09-12T03:17:23.482 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:17:23.512 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:17:23.603 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:17:23.616 INFO:teuthology.orchestra.run.smithi115.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 303 kB/s | 222 kB 00:00 2024-09-12T03:17:23.633 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T03:17:23.808 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T03:17:23.871 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T03:17:23.891 INFO:teuthology.orchestra.run.smithi115.stdout:(85/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.1 MB/s | 5.1 MB 00:01 2024-09-12T03:17:23.900 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:17:24.003 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:17:24.025 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T03:17:24.089 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T03:17:24.118 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T03:17:24.138 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T03:17:24.158 INFO:teuthology.orchestra.run.smithi115.stdout:(86/156): python3-packaging-20.9-5.el9.noarch.r 112 kB/s | 77 kB 00:00 2024-09-12T03:17:24.267 INFO:teuthology.orchestra.run.smithi115.stdout:(87/156): python3-prettytable-0.7.2-27.el9.noar 65 kB/s | 42 kB 00:00 2024-09-12T03:17:24.328 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T03:17:24.523 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T03:17:24.650 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T03:17:24.771 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:17:24.784 INFO:teuthology.orchestra.run.smithi115.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 305 kB/s | 269 kB 00:00 2024-09-12T03:17:24.852 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:17:24.934 INFO:teuthology.orchestra.run.smithi115.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 205 kB/s | 159 kB 00:00 2024-09-12T03:17:25.088 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:17:25.142 INFO:teuthology.orchestra.run.smithi115.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 319 kB/s | 279 kB 00:00 2024-09-12T03:17:25.194 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T03:17:25.286 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T03:17:25.333 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T03:17:25.383 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T03:17:25.393 INFO:teuthology.orchestra.run.smithi115.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 84 kB/s | 51 kB 00:00 2024-09-12T03:17:25.448 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T03:17:25.500 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T03:17:25.533 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T03:17:25.535 INFO:teuthology.orchestra.run.smithi115.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 89 kB/s | 54 kB 00:00 2024-09-12T03:17:25.576 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T03:17:25.617 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T03:17:25.654 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T03:17:25.721 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T03:17:25.774 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T03:17:25.807 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T03:17:25.850 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T03:17:25.925 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T03:17:25.960 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T03:17:25.968 INFO:teuthology.orchestra.run.smithi115.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 72 kB/s | 42 kB 00:00 2024-09-12T03:17:26.004 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T03:17:26.032 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T03:17:26.092 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T03:17:26.110 INFO:teuthology.orchestra.run.smithi115.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 77 kB/s | 44 kB 00:00 2024-09-12T03:17:26.147 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T03:17:26.220 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T03:17:26.353 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T03:17:26.408 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T03:17:26.496 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-09-12T03:17:26.599 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T03:17:26.642 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T03:17:26.652 INFO:teuthology.orchestra.run.smithi115.stdout:(95/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-12T03:17:26.701 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T03:17:26.733 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T03:17:26.752 INFO:teuthology.orchestra.run.smithi115.stdout:(96/156): qatlib-24.02.0-1.el9.x86_64.rpm 282 kB/s | 221 kB 00:00 2024-09-12T03:17:26.761 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T03:17:26.809 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T03:17:26.861 INFO:teuthology.orchestra.run.smithi115.stdout:(97/156): qatlib-service-24.02.0-1.el9.x86_64.r 48 kB/s | 36 kB 00:00 2024-09-12T03:17:26.921 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T03:17:27.027 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T03:17:27.058 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T03:17:27.100 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T03:17:27.138 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T03:17:27.170 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T03:17:27.180 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T03:17:27.236 INFO:teuthology.orchestra.run.smithi115.stdout:(98/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 81 kB/s | 47 kB 00:00 2024-09-12T03:17:27.269 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:17:27.362 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T03:17:27.386 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:17:27.412 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:17:27.425 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T03:17:27.478 INFO:teuthology.orchestra.run.smithi115.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 94 kB/s | 58 kB 00:00 2024-09-12T03:17:27.478 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T03:17:27.505 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:17:27.520 INFO:teuthology.orchestra.run.smithi115.stdout:(100/156): socat-1.7.4.1-6.el9.x86_64.rpm 396 kB/s | 304 kB 00:00 2024-09-12T03:17:27.525 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T03:17:27.543 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T03:17:27.576 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T03:17:27.612 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T03:17:27.658 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T03:17:27.708 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T03:17:27.745 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T03:17:27.772 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T03:17:27.799 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:17:27.899 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T03:17:27.905 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:17:27.920 INFO:teuthology.orchestra.run.smithi115.stdout:(101/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 140 kB/s | 96 kB 00:00 2024-09-12T03:17:27.933 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T03:17:27.991 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T03:17:28.012 INFO:teuthology.orchestra.run.smithi115.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 245 kB/s | 22 kB 00:00 2024-09-12T03:17:28.018 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T03:17:28.042 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T03:17:28.088 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T03:17:28.095 INFO:teuthology.orchestra.run.smithi115.stdout:(103/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 78 kB/s | 45 kB 00:00 2024-09-12T03:17:28.120 INFO:teuthology.orchestra.run.smithi115.stdout:(104/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 100 kB/s | 64 kB 00:00 2024-09-12T03:17:28.222 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T03:17:28.228 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T03:17:28.288 INFO:teuthology.orchestra.run.smithi115.stdout:(105/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.1 MB/s | 863 kB 00:00 2024-09-12T03:17:28.313 INFO:teuthology.orchestra.run.smithi115.stdout:(106/156): abseil-cpp-20211102.0-3.el9.x86_64.r 2.5 MB/s | 548 kB 00:00 2024-09-12T03:17:28.347 INFO:teuthology.orchestra.run.smithi115.stdout:(107/156): gperftools-libs-2.9.1-3.el9.x86_64.r 1.3 MB/s | 308 kB 00:00 2024-09-12T03:17:28.372 INFO:teuthology.orchestra.run.smithi115.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.1 MB/s | 26 kB 00:00 2024-09-12T03:17:28.385 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T03:17:28.397 INFO:teuthology.orchestra.run.smithi115.stdout:(109/156): grpc-data-1.46.7-10.el9.noarch.rpm 178 kB/s | 19 kB 00:00 2024-09-12T03:17:28.423 INFO:teuthology.orchestra.run.smithi115.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 967 kB/s | 49 kB 00:00 2024-09-12T03:17:28.423 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T03:17:28.448 INFO:teuthology.orchestra.run.smithi115.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-12T03:17:28.465 INFO:teuthology.orchestra.run.smithi115.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 3.5 MB/s | 151 kB 00:00 2024-09-12T03:17:28.498 INFO:teuthology.orchestra.run.smithi115.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 16 MB/s | 548 kB 00:00 2024-09-12T03:17:28.523 INFO:teuthology.orchestra.run.smithi115.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-12T03:17:28.545 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T03:17:28.549 INFO:teuthology.orchestra.run.smithi115.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-12T03:17:28.574 INFO:teuthology.orchestra.run.smithi115.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 565 kB/s | 14 kB 00:00 2024-09-12T03:17:28.598 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T03:17:28.608 INFO:teuthology.orchestra.run.smithi115.stdout:(117/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.1 MB/s | 839 kB 00:00 2024-09-12T03:17:28.633 INFO:teuthology.orchestra.run.smithi115.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-12T03:17:28.655 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:17:28.695 INFO:teuthology.orchestra.run.smithi115.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.8 MB/s | 303 kB 00:00 2024-09-12T03:17:28.704 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T03:17:28.709 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:17:28.725 INFO:teuthology.orchestra.run.smithi115.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-09-12T03:17:28.741 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T03:17:28.750 INFO:teuthology.orchestra.run.smithi115.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.9 MB/s | 220 kB 00:00 2024-09-12T03:17:28.784 INFO:teuthology.orchestra.run.smithi115.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-12T03:17:28.792 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T03:17:28.809 INFO:teuthology.orchestra.run.smithi115.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 431 kB/s | 11 kB 00:00 2024-09-12T03:17:28.822 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-12T03:17:28.835 INFO:teuthology.orchestra.run.smithi115.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 707 kB/s | 18 kB 00:00 2024-09-12T03:17:28.855 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T03:17:28.860 INFO:teuthology.orchestra.run.smithi115.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 929 kB/s | 23 kB 00:00 2024-09-12T03:17:28.885 INFO:teuthology.orchestra.run.smithi115.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 783 kB/s | 19 kB 00:00 2024-09-12T03:17:28.911 INFO:teuthology.orchestra.run.smithi115.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 790 kB/s | 20 kB 00:00 2024-09-12T03:17:28.924 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:17:28.948 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T03:17:28.986 INFO:teuthology.orchestra.run.smithi115.stdout:(128/156): libarrow-9.0.0-11.el9.x86_64.rpm 6.6 MB/s | 4.4 MB 00:00 2024-09-12T03:17:29.036 INFO:teuthology.orchestra.run.smithi115.stdout:(129/156): python3-grpcio-1.46.7-10.el9.x86_64. 6.6 MB/s | 2.0 MB 00:00 2024-09-12T03:17:29.049 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T03:17:29.061 INFO:teuthology.orchestra.run.smithi115.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 617 kB/s | 46 kB 00:00 2024-09-12T03:17:29.073 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T03:17:29.086 INFO:teuthology.orchestra.run.smithi115.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-12T03:17:29.105 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T03:17:29.120 INFO:teuthology.orchestra.run.smithi115.stdout:(132/156): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-09-12T03:17:29.135 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T03:17:29.153 INFO:teuthology.orchestra.run.smithi115.stdout:(133/156): python3-kubernetes-26.1.0-2.el9.noar 4.2 MB/s | 1.0 MB 00:00 2024-09-12T03:17:29.164 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T03:17:29.175 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T03:17:29.204 INFO:teuthology.orchestra.run.smithi115.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 196 kB/s | 16 kB 00:00 2024-09-12T03:17:29.207 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T03:17:29.214 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T03:17:29.229 INFO:teuthology.orchestra.run.smithi115.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-12T03:17:29.242 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T03:17:29.253 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T03:17:29.271 INFO:teuthology.orchestra.run.smithi115.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 768 kB/s | 90 kB 00:00 2024-09-12T03:17:29.284 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T03:17:29.297 INFO:teuthology.orchestra.run.smithi115.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 459 kB/s | 31 kB 00:00 2024-09-12T03:17:29.313 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T03:17:29.318 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T03:17:29.322 INFO:teuthology.orchestra.run.smithi115.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-12T03:17:29.346 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T03:17:29.347 INFO:teuthology.orchestra.run.smithi115.stdout:(139/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.0 MB/s | 272 kB 00:00 2024-09-12T03:17:29.368 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T03:17:29.372 INFO:teuthology.orchestra.run.smithi115.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-09-12T03:17:29.379 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T03:17:29.398 INFO:teuthology.orchestra.run.smithi115.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-09-12T03:17:29.414 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T03:17:29.416 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T03:17:29.423 INFO:teuthology.orchestra.run.smithi115.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 988 kB/s | 74 kB 00:00 2024-09-12T03:17:29.448 INFO:teuthology.orchestra.run.smithi115.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-12T03:17:29.460 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T03:17:29.473 INFO:teuthology.orchestra.run.smithi115.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-12T03:17:29.492 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T03:17:29.499 INFO:teuthology.orchestra.run.smithi115.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 400 kB/s | 20 kB 00:00 2024-09-12T03:17:29.500 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T03:17:29.520 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T03:17:29.524 INFO:teuthology.orchestra.run.smithi115.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-12T03:17:29.545 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T03:17:29.549 INFO:teuthology.orchestra.run.smithi115.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-12T03:17:29.576 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T03:17:29.591 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T03:17:29.599 INFO:teuthology.orchestra.run.smithi115.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-12T03:17:29.643 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T03:17:29.678 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T03:17:29.715 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T03:17:29.726 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T03:17:29.758 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T03:17:29.792 INFO:teuthology.orchestra.run.smithi115.stdout:(149/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 13 MB/s | 3.2 MB 00:00 2024-09-12T03:17:29.847 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T03:17:30.079 INFO:teuthology.orchestra.run.smithi115.stdout:(150/156): NetworkManager-1.49.90-1.el9.x86_64. 7.5 MB/s | 2.3 MB 00:00 2024-09-12T03:17:30.079 INFO:teuthology.orchestra.run.smithi115.stdout:(151/156): librados2-19.1.1-298.gb53b6d3e.el9.x 7.7 MB/s | 3.4 MB 00:00 2024-09-12T03:17:30.079 INFO:teuthology.orchestra.run.smithi115.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 489 kB/s | 41 kB 00:00 2024-09-12T03:17:30.079 INFO:teuthology.orchestra.run.smithi115.stdout:(153/156): openssl-3.2.2-6.el9.x86_64.rpm 20 MB/s | 1.4 MB 00:00 2024-09-12T03:17:30.079 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T03:17:30.080 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T03:17:30.080 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T03:17:30.080 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T03:17:30.080 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T03:17:30.142 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T03:17:30.143 INFO:teuthology.orchestra.run.smithi115.stdout:(154/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 29 MB/s | 2.4 MB 00:00 2024-09-12T03:17:30.168 INFO:teuthology.orchestra.run.smithi115.stdout:(155/156): NetworkManager-tui-1.49.90-1.el9.x86 1.2 MB/s | 249 kB 00:00 2024-09-12T03:17:30.295 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T03:17:30.343 INFO:teuthology.orchestra.run.smithi115.stdout:(156/156): NetworkManager-libnm-1.49.90-1.el9.x 3.3 MB/s | 1.8 MB 00:00 2024-09-12T03:17:30.350 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-12T03:17:30.350 INFO:teuthology.orchestra.run.smithi115.stdout:Total 8.5 MB/s | 226 MB 00:26 2024-09-12T03:17:30.351 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T03:17:30.456 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T03:17:30.499 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T03:17:30.533 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T03:17:30.631 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T03:17:30.688 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T03:17:30.732 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T03:17:30.827 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T03:17:30.860 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T03:17:30.928 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T03:17:31.014 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T03:17:31.044 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T03:17:31.049 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T03:17:31.074 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T03:17:31.094 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T03:17:31.130 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T03:17:31.152 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T03:17:31.172 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T03:17:31.343 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T03:17:31.400 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T03:17:31.456 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T03:17:31.537 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T03:17:31.569 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-12T03:17:31.575 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T03:17:31.612 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T03:17:31.659 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T03:17:31.695 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T03:17:31.719 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-12T03:17:31.719 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-12T03:17:31.738 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T03:17:31.781 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T03:17:31.795 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T03:17:31.907 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T03:17:31.925 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T03:17:31.973 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T03:17:32.212 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T03:17:32.405 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T03:17:32.441 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T03:17:32.490 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T03:17:32.522 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-12T03:17:32.547 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T03:17:32.638 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T03:17:32.770 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T03:17:32.812 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T03:17:32.851 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T03:17:32.893 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T03:17:32.930 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T03:17:32.971 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T03:17:33.000 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T03:17:33.035 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T03:17:33.068 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T03:17:33.110 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T03:17:33.144 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T03:17:33.144 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T03:17:33.229 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T03:17:33.231 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T03:17:33.266 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T03:17:33.279 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T03:17:33.299 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T03:17:33.317 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T03:17:33.425 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T03:17:33.478 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T03:17:33.610 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T03:17:33.654 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T03:17:33.737 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T03:17:33.767 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T03:17:33.809 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T03:17:34.128 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-12T03:17:34.129 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-12T03:17:34.239 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T03:17:34.299 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T03:17:34.332 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T03:17:34.343 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T03:17:34.390 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T03:17:34.436 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T03:17:34.444 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T03:17:34.467 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T03:17:34.488 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T03:17:34.533 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T03:17:34.623 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T03:17:34.660 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T03:17:34.851 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T03:17:34.874 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T03:17:34.954 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T03:17:35.648 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T03:17:35.768 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T03:17:36.763 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:17:36.797 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:17:37.091 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T03:17:37.179 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T03:17:37.207 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T03:17:37.240 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T03:17:37.357 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:17:37.392 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T03:17:37.482 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T03:17:37.482 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T03:17:37.483 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T03:17:37.483 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-12T03:17:37.543 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T03:17:37.568 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T03:17:37.619 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T03:17:37.711 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T03:17:37.741 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T03:17:37.782 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T03:17:38.293 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T03:17:38.322 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T03:17:38.377 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T03:17:38.407 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T03:17:40.423 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-12T03:17:40.590 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T03:17:40.637 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T03:17:40.690 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T03:17:40.732 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T03:17:40.758 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T03:17:40.810 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T03:17:40.847 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:17:40.879 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:17:41.140 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T03:17:41.243 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:17:41.365 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:17:41.393 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:17:41.474 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:17:41.486 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:17:41.507 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T03:17:41.514 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T03:17:41.598 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T03:17:41.598 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T03:17:41.598 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T03:17:41.598 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:17:41.633 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T03:17:41.669 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T03:17:41.720 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T03:17:41.748 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:17:41.852 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:17:41.882 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T03:17:41.928 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T03:17:41.958 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T03:17:41.977 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T03:17:42.195 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T03:17:42.394 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T03:17:42.507 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T03:17:42.612 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:17:42.683 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:17:42.897 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:17:43.013 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T03:17:43.108 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T03:17:43.138 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T03:17:43.172 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T03:17:43.237 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T03:17:43.281 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T03:17:43.314 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T03:17:43.357 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T03:17:43.391 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T03:17:43.418 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T03:17:43.486 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T03:17:43.540 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T03:17:43.571 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T03:17:43.614 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T03:17:43.690 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T03:17:43.725 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T03:17:43.769 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T03:17:43.797 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T03:17:43.865 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T03:17:43.911 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T03:17:43.986 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T03:17:44.127 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T03:17:44.181 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T03:17:44.701 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T03:17:44.783 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T03:17:44.822 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T03:17:44.856 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T03:17:44.894 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T03:17:44.936 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T03:17:45.085 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T03:17:45.156 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T03:17:45.210 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T03:17:45.283 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T03:17:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T03:17:45.360 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T03:17:45.406 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T03:17:45.443 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T03:17:45.486 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T03:17:45.527 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T03:17:45.662 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T03:17:45.705 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T03:17:45.746 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T03:17:45.847 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T03:17:45.889 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T03:17:45.929 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T03:17:45.962 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-12T03:17:45.986 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T03:17:46.077 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T03:17:46.212 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T03:17:46.244 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T03:17:46.272 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T03:17:46.317 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T03:17:46.357 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T03:17:46.388 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T03:17:46.418 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T03:17:46.453 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T03:17:46.486 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T03:17:46.527 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T03:17:46.563 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T03:17:46.640 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T03:17:46.692 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T03:17:46.724 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T03:17:46.834 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T03:17:46.883 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T03:17:47.596 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T03:17:47.647 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T03:17:47.680 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T03:17:47.778 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T03:17:47.827 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T03:17:47.872 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T03:17:47.959 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T03:17:48.008 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T03:17:48.193 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T03:17:48.223 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T03:17:48.301 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T03:17:48.945 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T03:17:49.064 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T03:17:49.477 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T03:17:49.477 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /sys 2024-09-12T03:17:49.477 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /proc 2024-09-12T03:17:49.477 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /mnt 2024-09-12T03:17:49.478 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /var/tmp 2024-09-12T03:17:49.478 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /home 2024-09-12T03:17:49.478 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /root 2024-09-12T03:17:49.478 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /tmp 2024-09-12T03:17:49.478 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-12T03:17:49.649 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T03:17:49.665 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T03:17:49.665 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:17:49.665 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T03:17:49.665 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-12T03:17:50.307 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T03:17:50.395 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T03:17:50.423 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T03:17:50.456 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T03:17:50.764 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T03:17:50.802 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T03:17:50.893 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T03:17:50.924 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T03:17:50.960 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T03:17:51.152 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T03:17:51.178 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T03:17:51.178 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:17:51.178 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T03:17:51.179 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T03:17:51.179 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T03:17:51.179 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-12T03:17:51.402 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T03:17:51.426 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T03:17:51.426 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:17:51.426 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T03:17:51.426 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T03:17:51.426 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T03:17:51.426 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-12T03:17:51.436 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T03:17:51.472 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T03:17:51.526 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T03:17:51.557 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T03:17:51.862 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T03:17:51.885 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T03:17:51.889 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:17:51.889 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T03:17:51.889 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T03:17:51.889 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T03:17:51.889 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-12T03:17:51.992 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T03:17:52.040 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T03:17:52.095 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T03:17:52.280 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T03:17:52.328 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T03:17:52.398 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T03:17:52.486 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T03:17:52.615 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T03:17:52.667 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T03:17:52.705 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T03:17:52.731 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T03:17:53.164 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T03:17:53.196 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T03:17:53.227 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T03:17:53.289 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T03:17:53.319 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T03:17:53.805 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T03:17:53.805 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /sys 2024-09-12T03:17:53.805 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /proc 2024-09-12T03:17:53.805 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /mnt 2024-09-12T03:17:53.806 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /var/tmp 2024-09-12T03:17:53.806 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /home 2024-09-12T03:17:53.806 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /root 2024-09-12T03:17:53.806 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /tmp 2024-09-12T03:17:53.806 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:17:53.873 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:17:53.920 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:17:53.981 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T03:17:53.998 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T03:17:53.998 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:17:53.999 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T03:17:53.999 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:17:54.353 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T03:17:54.396 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T03:17:54.466 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:17:54.507 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T03:17:54.605 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T03:17:54.605 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T03:17:54.606 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T03:17:54.606 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T03:17:54.650 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T03:17:55.356 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T03:17:55.388 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T03:17:55.491 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T03:17:55.533 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T03:17:55.562 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T03:17:55.562 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:17:55.562 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T03:17:55.562 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T03:17:55.562 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T03:17:55.562 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:17:55.658 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T03:17:55.690 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T03:17:55.713 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T03:17:55.714 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:17:55.714 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T03:17:55.714 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T03:17:55.714 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T03:17:55.714 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-12T03:17:55.766 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T03:17:55.777 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T03:17:55.809 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T03:17:55.834 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T03:17:55.835 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:17:55.835 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T03:17:55.835 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T03:17:55.835 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T03:17:55.835 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:17:55.891 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T03:17:55.945 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T03:17:56.293 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T03:17:56.318 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T03:17:56.319 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:17:56.319 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T03:17:56.319 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T03:17:56.319 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T03:17:56.319 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:17:56.424 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T03:17:56.490 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T03:17:56.554 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T03:17:56.750 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T03:17:56.804 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T03:17:56.862 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T03:17:56.884 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T03:17:56.893 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T03:17:56.916 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T03:17:56.916 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:17:56.917 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T03:17:56.917 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T03:17:56.917 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T03:17:56.917 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-12T03:17:56.961 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T03:17:56.981 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T03:17:56.983 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T03:17:56.984 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:17:56.984 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T03:17:56.984 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-12T03:17:57.036 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T03:17:57.060 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T03:17:57.060 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:17:57.060 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T03:17:57.060 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T03:17:57.060 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T03:17:57.060 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-12T03:17:57.125 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T03:17:57.174 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T03:17:57.210 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T03:17:57.237 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T03:17:57.370 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T03:17:57.393 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T03:17:57.394 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:17:57.394 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T03:17:57.394 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T03:17:57.394 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T03:17:57.394 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-12T03:17:57.656 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T03:17:57.694 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T03:17:57.726 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T03:17:57.791 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T03:17:57.817 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T03:17:58.922 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T03:17:58.961 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T03:17:59.980 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T03:18:00.003 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T03:18:00.113 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T03:18:00.290 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T03:18:00.321 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T03:18:00.348 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T03:18:00.348 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:18:00.348 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T03:18:00.348 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T03:18:00.349 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T03:18:00.349 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:18:00.397 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T03:18:00.409 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T03:18:00.511 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T03:18:00.569 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T03:18:01.558 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T03:18:01.591 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T03:18:01.616 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T03:18:01.617 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:18:01.617 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T03:18:01.617 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T03:18:01.617 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T03:18:01.617 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:18:01.659 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T03:18:01.683 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T03:18:01.684 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:18:01.684 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T03:18:01.684 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:18:01.734 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T03:18:01.760 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T03:18:01.760 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:18:01.760 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T03:18:01.760 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T03:18:01.760 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T03:18:01.761 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:18:01.949 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T03:18:01.990 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T03:18:02.031 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T03:18:02.063 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T03:18:02.085 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T03:18:02.090 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T03:18:02.110 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T03:18:02.110 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:18:02.110 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T03:18:02.110 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T03:18:02.110 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T03:18:02.110 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:18:02.284 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T03:18:02.345 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T03:18:02.375 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T03:18:02.405 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T03:18:02.443 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T03:18:02.998 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:18:03.041 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:18:03.061 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:18:03.879 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T03:18:03.962 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T03:18:04.005 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T03:18:04.093 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T03:18:04.132 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T03:18:04.158 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T03:18:04.192 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T03:18:04.192 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T03:18:04.276 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T03:18:04.276 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T03:18:04.352 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T03:18:04.377 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T03:18:04.382 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:18:04.382 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:18:04.570 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:18:04.571 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T03:18:04.654 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T03:18:04.728 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:18:06.554 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T03:18:06.554 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /sys 2024-09-12T03:18:06.554 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /proc 2024-09-12T03:18:06.554 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /mnt 2024-09-12T03:18:06.554 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /var/tmp 2024-09-12T03:18:06.554 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /home 2024-09-12T03:18:06.554 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /root 2024-09-12T03:18:06.555 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /tmp 2024-09-12T03:18:06.555 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T03:18:06.730 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T03:18:06.745 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T03:18:06.746 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:18:06.746 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T03:18:06.746 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T03:18:06.956 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T03:18:06.997 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T03:18:07.038 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T03:18:07.070 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T03:18:07.097 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T03:18:07.300 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T03:18:07.352 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T03:18:07.382 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T03:18:07.412 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T03:18:07.450 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T03:18:08.051 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:18:08.097 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:18:08.119 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:18:08.224 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T03:18:08.251 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T03:18:08.252 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:18:08.252 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T03:18:08.252 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T03:18:08.252 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T03:18:08.252 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T03:18:08.475 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T03:18:08.499 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T03:18:08.499 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:18:08.499 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T03:18:08.499 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T03:18:08.499 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T03:18:08.499 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T03:18:08.926 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T03:18:08.949 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T03:18:08.949 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:18:08.949 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T03:18:08.949 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T03:18:08.949 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T03:18:08.949 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T03:18:09.039 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T03:18:09.064 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T03:18:09.112 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T03:18:09.119 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T03:18:09.167 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T03:18:09.168 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T03:18:09.290 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T03:18:09.322 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T03:18:09.348 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T03:18:09.352 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T03:18:09.398 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T03:18:09.398 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T03:18:09.401 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T03:18:09.478 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T03:18:09.499 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T03:18:09.499 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T03:18:09.567 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T03:18:09.583 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T03:18:09.608 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T03:18:09.614 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:18:09.615 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:18:09.698 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T03:18:09.747 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T03:18:09.783 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T03:18:09.802 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:18:09.802 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T03:18:09.811 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T03:18:09.886 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T03:18:09.954 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:18:10.270 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T03:18:10.301 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T03:18:10.333 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T03:18:10.395 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T03:18:10.424 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T03:18:10.454 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:18:10.454 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T03:18:10.454 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T03:18:10.454 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T03:18:10.455 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T03:18:10.455 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T03:18:10.455 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T03:18:10.455 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T03:18:10.455 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:18:10.455 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:18:10.455 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T03:18:10.455 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T03:18:10.455 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T03:18:10.455 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:18:10.455 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T03:18:10.455 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T03:18:10.456 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T03:18:10.456 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T03:18:10.456 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T03:18:10.456 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T03:18:10.456 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T03:18:10.456 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T03:18:10.456 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T03:18:10.456 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T03:18:10.456 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T03:18:10.456 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T03:18:10.456 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T03:18:10.456 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T03:18:10.457 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T03:18:10.457 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T03:18:10.457 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T03:18:10.457 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T03:18:10.457 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T03:18:10.457 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T03:18:10.457 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T03:18:10.457 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T03:18:10.457 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T03:18:10.457 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T03:18:10.457 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T03:18:10.457 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T03:18:10.457 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T03:18:10.458 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T03:18:10.458 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T03:18:10.458 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T03:18:10.458 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T03:18:10.458 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T03:18:10.458 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T03:18:10.458 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T03:18:10.458 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T03:18:10.458 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T03:18:10.458 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T03:18:10.458 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T03:18:10.458 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T03:18:10.458 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T03:18:10.459 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T03:18:10.459 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T03:18:10.459 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T03:18:10.459 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T03:18:10.459 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T03:18:10.459 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T03:18:10.459 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T03:18:10.459 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T03:18:10.459 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T03:18:10.459 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T03:18:10.459 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T03:18:10.459 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T03:18:10.459 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T03:18:10.460 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T03:18:10.460 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T03:18:10.460 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T03:18:10.460 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T03:18:10.460 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T03:18:10.460 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T03:18:10.460 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T03:18:10.460 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T03:18:10.460 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T03:18:10.460 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T03:18:10.460 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T03:18:10.460 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T03:18:10.460 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T03:18:10.461 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T03:18:10.461 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T03:18:10.461 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T03:18:10.461 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T03:18:10.461 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T03:18:10.461 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T03:18:10.461 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T03:18:10.461 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T03:18:10.461 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T03:18:10.461 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T03:18:10.461 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T03:18:10.461 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T03:18:10.461 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T03:18:10.462 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T03:18:10.462 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T03:18:10.462 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T03:18:10.462 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T03:18:10.462 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T03:18:10.462 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T03:18:10.462 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T03:18:10.462 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T03:18:10.462 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T03:18:10.462 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T03:18:10.462 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T03:18:10.462 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T03:18:10.462 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T03:18:10.463 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T03:18:10.463 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-12T03:18:10.463 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T03:18:10.463 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T03:18:10.463 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T03:18:10.463 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T03:18:10.463 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T03:18:10.463 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T03:18:10.463 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T03:18:10.463 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T03:18:10.463 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T03:18:10.463 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T03:18:10.463 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T03:18:10.463 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T03:18:10.464 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T03:18:10.464 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T03:18:10.464 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T03:18:10.464 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T03:18:10.464 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T03:18:10.464 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T03:18:10.464 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T03:18:10.464 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T03:18:10.464 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T03:18:10.464 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T03:18:10.464 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T03:18:10.464 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T03:18:10.464 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T03:18:10.465 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T03:18:10.465 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T03:18:10.465 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T03:18:10.465 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T03:18:10.465 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T03:18:10.465 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T03:18:10.465 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T03:18:10.465 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T03:18:10.465 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T03:18:10.465 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T03:18:10.466 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T03:18:10.466 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T03:18:10.466 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T03:18:10.466 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T03:18:10.466 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T03:18:10.466 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T03:18:10.466 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T03:18:10.466 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T03:18:10.466 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T03:18:10.466 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T03:18:10.466 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T03:18:10.466 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T03:18:10.466 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T03:18:10.467 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T03:18:10.467 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T03:18:10.467 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T03:18:10.467 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T03:18:10.467 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T03:18:10.467 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T03:18:10.467 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T03:18:10.467 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T03:18:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T03:18:11.492 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T03:18:12.456 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T03:18:12.485 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T03:18:12.585 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T03:18:12.747 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T03:18:12.778 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T03:18:12.802 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T03:18:12.802 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:18:12.802 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T03:18:12.802 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T03:18:12.802 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T03:18:12.802 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T03:18:12.845 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T03:18:12.857 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T03:18:12.954 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T03:18:13.004 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T03:18:13.377 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:18:13.377 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-12T03:18:13.378 INFO:teuthology.orchestra.run.smithi195.stdout:Upgraded: 2024-09-12T03:18:13.378 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T03:18:13.378 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T03:18:13.378 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T03:18:13.378 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T03:18:13.378 INFO:teuthology.orchestra.run.smithi195.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:13.378 INFO:teuthology.orchestra.run.smithi195.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:13.378 INFO:teuthology.orchestra.run.smithi195.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T03:18:13.378 INFO:teuthology.orchestra.run.smithi195.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T03:18:13.378 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-09-12T03:18:13.378 INFO:teuthology.orchestra.run.smithi195.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T03:18:13.378 INFO:teuthology.orchestra.run.smithi195.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T03:18:13.378 INFO:teuthology.orchestra.run.smithi195.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T03:18:13.379 INFO:teuthology.orchestra.run.smithi195.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T03:18:13.379 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:13.379 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:13.379 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:13.379 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:13.379 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:18:13.379 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:13.379 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:13.379 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:13.379 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:18:13.379 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:18:13.379 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:18:13.379 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:18:13.380 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:18:13.380 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:13.380 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:13.380 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:18:13.380 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:13.380 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:13.380 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:13.380 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:18:13.380 INFO:teuthology.orchestra.run.smithi195.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:18:13.380 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:13.380 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:13.380 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:18:13.380 INFO:teuthology.orchestra.run.smithi195.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T03:18:13.381 INFO:teuthology.orchestra.run.smithi195.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T03:18:13.381 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T03:18:13.381 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T03:18:13.381 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T03:18:13.381 INFO:teuthology.orchestra.run.smithi195.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T03:18:13.381 INFO:teuthology.orchestra.run.smithi195.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-12T03:18:13.381 INFO:teuthology.orchestra.run.smithi195.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T03:18:13.381 INFO:teuthology.orchestra.run.smithi195.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T03:18:13.381 INFO:teuthology.orchestra.run.smithi195.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T03:18:13.381 INFO:teuthology.orchestra.run.smithi195.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T03:18:13.381 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:13.381 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:13.381 INFO:teuthology.orchestra.run.smithi195.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:13.382 INFO:teuthology.orchestra.run.smithi195.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T03:18:13.382 INFO:teuthology.orchestra.run.smithi195.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T03:18:13.382 INFO:teuthology.orchestra.run.smithi195.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T03:18:13.382 INFO:teuthology.orchestra.run.smithi195.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T03:18:13.382 INFO:teuthology.orchestra.run.smithi195.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T03:18:13.382 INFO:teuthology.orchestra.run.smithi195.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:13.382 INFO:teuthology.orchestra.run.smithi195.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:13.382 INFO:teuthology.orchestra.run.smithi195.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T03:18:13.382 INFO:teuthology.orchestra.run.smithi195.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:13.382 INFO:teuthology.orchestra.run.smithi195.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T03:18:13.382 INFO:teuthology.orchestra.run.smithi195.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T03:18:13.382 INFO:teuthology.orchestra.run.smithi195.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T03:18:13.382 INFO:teuthology.orchestra.run.smithi195.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T03:18:13.382 INFO:teuthology.orchestra.run.smithi195.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T03:18:13.383 INFO:teuthology.orchestra.run.smithi195.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T03:18:13.383 INFO:teuthology.orchestra.run.smithi195.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T03:18:13.383 INFO:teuthology.orchestra.run.smithi195.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T03:18:13.383 INFO:teuthology.orchestra.run.smithi195.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T03:18:13.383 INFO:teuthology.orchestra.run.smithi195.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T03:18:13.383 INFO:teuthology.orchestra.run.smithi195.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T03:18:13.383 INFO:teuthology.orchestra.run.smithi195.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T03:18:13.383 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T03:18:13.383 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T03:18:13.383 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T03:18:13.383 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T03:18:13.383 INFO:teuthology.orchestra.run.smithi195.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T03:18:13.383 INFO:teuthology.orchestra.run.smithi195.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T03:18:13.384 INFO:teuthology.orchestra.run.smithi195.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T03:18:13.384 INFO:teuthology.orchestra.run.smithi195.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T03:18:13.384 INFO:teuthology.orchestra.run.smithi195.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T03:18:13.384 INFO:teuthology.orchestra.run.smithi195.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T03:18:13.384 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T03:18:13.384 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:13.384 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:13.384 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:13.384 INFO:teuthology.orchestra.run.smithi195.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T03:18:13.384 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T03:18:13.384 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T03:18:13.384 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T03:18:13.384 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T03:18:13.385 INFO:teuthology.orchestra.run.smithi195.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T03:18:13.385 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T03:18:13.385 INFO:teuthology.orchestra.run.smithi195.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T03:18:13.385 INFO:teuthology.orchestra.run.smithi195.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T03:18:13.385 INFO:teuthology.orchestra.run.smithi195.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T03:18:13.385 INFO:teuthology.orchestra.run.smithi195.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T03:18:13.385 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T03:18:13.385 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T03:18:13.385 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T03:18:13.385 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T03:18:13.385 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T03:18:13.385 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T03:18:13.385 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T03:18:13.385 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T03:18:13.386 INFO:teuthology.orchestra.run.smithi195.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T03:18:13.386 INFO:teuthology.orchestra.run.smithi195.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T03:18:13.386 INFO:teuthology.orchestra.run.smithi195.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T03:18:13.386 INFO:teuthology.orchestra.run.smithi195.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T03:18:13.386 INFO:teuthology.orchestra.run.smithi195.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T03:18:13.386 INFO:teuthology.orchestra.run.smithi195.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T03:18:13.386 INFO:teuthology.orchestra.run.smithi195.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T03:18:13.386 INFO:teuthology.orchestra.run.smithi195.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T03:18:13.386 INFO:teuthology.orchestra.run.smithi195.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T03:18:13.386 INFO:teuthology.orchestra.run.smithi195.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T03:18:13.386 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T03:18:13.386 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T03:18:13.386 INFO:teuthology.orchestra.run.smithi195.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T03:18:13.386 INFO:teuthology.orchestra.run.smithi195.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T03:18:13.387 INFO:teuthology.orchestra.run.smithi195.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T03:18:13.387 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T03:18:13.387 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T03:18:13.387 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T03:18:13.387 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T03:18:13.387 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T03:18:13.387 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T03:18:13.387 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:13.387 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:13.387 INFO:teuthology.orchestra.run.smithi195.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T03:18:13.387 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T03:18:13.387 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T03:18:13.387 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:13.387 INFO:teuthology.orchestra.run.smithi195.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T03:18:13.388 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T03:18:13.388 INFO:teuthology.orchestra.run.smithi195.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T03:18:13.388 INFO:teuthology.orchestra.run.smithi195.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T03:18:13.388 INFO:teuthology.orchestra.run.smithi195.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T03:18:13.388 INFO:teuthology.orchestra.run.smithi195.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T03:18:13.388 INFO:teuthology.orchestra.run.smithi195.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T03:18:13.388 INFO:teuthology.orchestra.run.smithi195.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T03:18:13.388 INFO:teuthology.orchestra.run.smithi195.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T03:18:13.388 INFO:teuthology.orchestra.run.smithi195.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T03:18:13.388 INFO:teuthology.orchestra.run.smithi195.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T03:18:13.388 INFO:teuthology.orchestra.run.smithi195.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T03:18:13.388 INFO:teuthology.orchestra.run.smithi195.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T03:18:13.388 INFO:teuthology.orchestra.run.smithi195.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T03:18:13.389 INFO:teuthology.orchestra.run.smithi195.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T03:18:13.389 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:13.389 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:13.389 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:13.389 INFO:teuthology.orchestra.run.smithi195.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T03:18:13.389 INFO:teuthology.orchestra.run.smithi195.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T03:18:13.389 INFO:teuthology.orchestra.run.smithi195.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T03:18:13.389 INFO:teuthology.orchestra.run.smithi195.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T03:18:13.389 INFO:teuthology.orchestra.run.smithi195.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T03:18:13.389 INFO:teuthology.orchestra.run.smithi195.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T03:18:13.390 INFO:teuthology.orchestra.run.smithi195.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T03:18:13.390 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-12T03:18:13.390 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-09-12T03:18:13.793 DEBUG:teuthology.parallel:result is None 2024-09-12T03:18:13.921 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T03:18:13.948 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T03:18:13.972 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T03:18:13.972 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:18:13.972 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T03:18:13.972 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T03:18:13.972 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T03:18:13.972 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T03:18:14.016 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T03:18:14.038 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T03:18:14.039 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:18:14.039 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T03:18:14.039 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T03:18:14.091 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T03:18:14.114 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T03:18:14.114 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:18:14.114 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T03:18:14.114 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T03:18:14.114 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T03:18:14.114 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T03:18:14.417 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T03:18:14.440 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T03:18:14.440 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:18:14.440 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T03:18:14.440 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T03:18:14.440 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T03:18:14.440 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T03:18:15.541 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:18:15.541 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T03:18:15.541 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T03:18:15.541 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T03:18:15.542 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T03:18:15.542 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T03:18:15.542 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T03:18:15.542 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T03:18:15.542 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:18:15.542 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:18:15.542 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T03:18:15.542 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T03:18:15.542 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T03:18:15.542 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:18:15.542 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T03:18:15.542 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T03:18:15.542 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T03:18:15.543 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T03:18:15.543 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T03:18:15.543 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T03:18:15.543 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T03:18:15.543 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T03:18:15.543 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T03:18:15.543 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T03:18:15.543 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T03:18:15.543 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T03:18:15.543 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T03:18:15.543 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T03:18:15.543 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T03:18:15.543 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T03:18:15.544 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T03:18:15.544 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T03:18:15.544 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T03:18:15.544 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T03:18:15.544 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T03:18:15.544 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T03:18:15.544 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T03:18:15.544 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T03:18:15.544 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T03:18:15.544 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T03:18:15.544 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T03:18:15.544 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T03:18:15.544 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T03:18:15.545 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T03:18:15.545 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T03:18:15.545 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T03:18:15.545 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T03:18:15.545 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T03:18:15.545 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T03:18:15.545 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T03:18:15.545 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T03:18:15.545 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T03:18:15.545 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T03:18:15.545 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T03:18:15.545 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T03:18:15.545 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T03:18:15.546 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T03:18:15.546 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T03:18:15.546 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T03:18:15.546 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T03:18:15.546 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T03:18:15.546 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T03:18:15.546 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T03:18:15.546 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T03:18:15.546 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T03:18:15.546 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T03:18:15.546 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T03:18:15.546 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T03:18:15.546 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T03:18:15.547 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T03:18:15.547 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T03:18:15.547 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T03:18:15.547 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T03:18:15.547 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T03:18:15.547 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T03:18:15.547 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T03:18:15.547 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T03:18:15.547 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T03:18:15.547 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T03:18:15.547 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T03:18:15.547 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T03:18:15.547 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T03:18:15.548 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T03:18:15.548 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T03:18:15.548 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T03:18:15.548 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T03:18:15.548 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T03:18:15.548 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T03:18:15.548 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T03:18:15.548 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T03:18:15.548 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T03:18:15.548 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T03:18:15.548 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T03:18:15.548 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T03:18:15.548 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T03:18:15.548 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T03:18:15.549 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T03:18:15.549 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T03:18:15.549 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T03:18:15.549 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T03:18:15.549 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T03:18:15.549 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T03:18:15.549 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T03:18:15.549 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T03:18:15.549 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T03:18:15.549 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T03:18:15.549 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T03:18:15.549 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-12T03:18:15.549 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T03:18:15.550 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T03:18:15.550 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T03:18:15.550 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T03:18:15.550 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T03:18:15.550 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T03:18:15.550 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T03:18:15.550 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T03:18:15.550 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T03:18:15.550 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T03:18:15.550 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T03:18:15.550 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T03:18:15.550 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T03:18:15.550 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T03:18:15.550 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T03:18:15.551 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T03:18:15.551 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T03:18:15.551 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T03:18:15.551 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T03:18:15.551 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T03:18:15.551 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T03:18:15.551 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T03:18:15.551 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T03:18:15.551 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T03:18:15.551 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T03:18:15.551 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T03:18:15.551 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T03:18:15.551 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T03:18:15.551 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T03:18:15.552 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T03:18:15.552 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T03:18:15.552 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T03:18:15.552 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T03:18:15.552 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T03:18:15.552 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T03:18:15.552 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T03:18:15.552 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T03:18:15.552 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T03:18:15.552 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T03:18:15.552 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T03:18:15.552 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T03:18:15.552 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T03:18:15.553 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T03:18:15.553 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T03:18:15.553 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T03:18:15.553 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T03:18:15.553 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T03:18:15.553 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T03:18:15.553 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T03:18:15.553 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T03:18:15.553 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T03:18:15.553 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T03:18:15.553 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T03:18:15.553 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T03:18:15.553 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T03:18:15.553 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T03:18:18.256 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:18:18.256 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:18:18.256 INFO:teuthology.orchestra.run.smithi064.stdout:Upgraded: 2024-09-12T03:18:18.256 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T03:18:18.256 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T03:18:18.256 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T03:18:18.256 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T03:18:18.257 INFO:teuthology.orchestra.run.smithi064.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:18.257 INFO:teuthology.orchestra.run.smithi064.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:18.257 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T03:18:18.257 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T03:18:18.257 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-09-12T03:18:18.257 INFO:teuthology.orchestra.run.smithi064.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T03:18:18.257 INFO:teuthology.orchestra.run.smithi064.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T03:18:18.257 INFO:teuthology.orchestra.run.smithi064.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T03:18:18.257 INFO:teuthology.orchestra.run.smithi064.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T03:18:18.257 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:18.257 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:18.258 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:18.258 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:18.258 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:18:18.258 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:18.258 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:18.258 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:18.258 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:18:18.258 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:18:18.258 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:18:18.258 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:18:18.258 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:18:18.258 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:18.258 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:18.259 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:18:18.259 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:18.259 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:18.259 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:18.259 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:18:18.259 INFO:teuthology.orchestra.run.smithi064.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:18:18.259 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:18.259 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:18.259 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:18:18.259 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T03:18:18.259 INFO:teuthology.orchestra.run.smithi064.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T03:18:18.259 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T03:18:18.259 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T03:18:18.260 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T03:18:18.260 INFO:teuthology.orchestra.run.smithi064.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T03:18:18.260 INFO:teuthology.orchestra.run.smithi064.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-12T03:18:18.260 INFO:teuthology.orchestra.run.smithi064.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T03:18:18.260 INFO:teuthology.orchestra.run.smithi064.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T03:18:18.260 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T03:18:18.260 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T03:18:18.260 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:18.260 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:18.260 INFO:teuthology.orchestra.run.smithi064.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:18.261 INFO:teuthology.orchestra.run.smithi064.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T03:18:18.261 INFO:teuthology.orchestra.run.smithi064.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T03:18:18.261 INFO:teuthology.orchestra.run.smithi064.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T03:18:18.261 INFO:teuthology.orchestra.run.smithi064.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T03:18:18.261 INFO:teuthology.orchestra.run.smithi064.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T03:18:18.261 INFO:teuthology.orchestra.run.smithi064.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:18.261 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:18.261 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T03:18:18.261 INFO:teuthology.orchestra.run.smithi064.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:18.262 INFO:teuthology.orchestra.run.smithi064.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T03:18:18.262 INFO:teuthology.orchestra.run.smithi064.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T03:18:18.262 INFO:teuthology.orchestra.run.smithi064.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T03:18:18.262 INFO:teuthology.orchestra.run.smithi064.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T03:18:18.262 INFO:teuthology.orchestra.run.smithi064.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T03:18:18.262 INFO:teuthology.orchestra.run.smithi064.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T03:18:18.262 INFO:teuthology.orchestra.run.smithi064.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T03:18:18.262 INFO:teuthology.orchestra.run.smithi064.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T03:18:18.262 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T03:18:18.262 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T03:18:18.262 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T03:18:18.263 INFO:teuthology.orchestra.run.smithi064.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T03:18:18.263 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T03:18:18.263 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T03:18:18.263 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T03:18:18.263 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T03:18:18.263 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T03:18:18.263 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T03:18:18.263 INFO:teuthology.orchestra.run.smithi064.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T03:18:18.263 INFO:teuthology.orchestra.run.smithi064.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T03:18:18.263 INFO:teuthology.orchestra.run.smithi064.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T03:18:18.263 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T03:18:18.263 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T03:18:18.263 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:18.263 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:18.264 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:18.264 INFO:teuthology.orchestra.run.smithi064.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T03:18:18.264 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T03:18:18.264 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T03:18:18.264 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T03:18:18.264 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T03:18:18.264 INFO:teuthology.orchestra.run.smithi064.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T03:18:18.264 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T03:18:18.264 INFO:teuthology.orchestra.run.smithi064.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T03:18:18.264 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T03:18:18.265 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T03:18:18.265 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T03:18:18.265 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T03:18:18.265 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T03:18:18.265 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T03:18:18.265 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T03:18:18.266 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T03:18:18.266 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T03:18:18.266 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T03:18:18.266 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T03:18:18.266 INFO:teuthology.orchestra.run.smithi064.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T03:18:18.266 INFO:teuthology.orchestra.run.smithi064.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T03:18:18.267 INFO:teuthology.orchestra.run.smithi064.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T03:18:18.267 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T03:18:18.267 INFO:teuthology.orchestra.run.smithi064.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T03:18:18.267 INFO:teuthology.orchestra.run.smithi064.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T03:18:18.267 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T03:18:18.267 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T03:18:18.267 INFO:teuthology.orchestra.run.smithi064.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T03:18:18.268 INFO:teuthology.orchestra.run.smithi064.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T03:18:18.268 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T03:18:18.268 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T03:18:18.268 INFO:teuthology.orchestra.run.smithi064.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T03:18:18.268 INFO:teuthology.orchestra.run.smithi064.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T03:18:18.268 INFO:teuthology.orchestra.run.smithi064.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T03:18:18.269 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T03:18:18.269 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T03:18:18.269 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T03:18:18.269 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T03:18:18.269 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T03:18:18.269 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T03:18:18.270 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:18.270 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:18.270 INFO:teuthology.orchestra.run.smithi064.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T03:18:18.271 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T03:18:18.271 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T03:18:18.271 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:18.271 INFO:teuthology.orchestra.run.smithi064.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T03:18:18.271 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T03:18:18.272 INFO:teuthology.orchestra.run.smithi064.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T03:18:18.272 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T03:18:18.272 INFO:teuthology.orchestra.run.smithi064.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T03:18:18.272 INFO:teuthology.orchestra.run.smithi064.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T03:18:18.272 INFO:teuthology.orchestra.run.smithi064.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T03:18:18.272 INFO:teuthology.orchestra.run.smithi064.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T03:18:18.273 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T03:18:18.273 INFO:teuthology.orchestra.run.smithi064.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T03:18:18.273 INFO:teuthology.orchestra.run.smithi064.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T03:18:18.273 INFO:teuthology.orchestra.run.smithi064.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T03:18:18.273 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T03:18:18.273 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T03:18:18.274 INFO:teuthology.orchestra.run.smithi064.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T03:18:18.274 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:18.274 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:18.274 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:18.274 INFO:teuthology.orchestra.run.smithi064.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T03:18:18.274 INFO:teuthology.orchestra.run.smithi064.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T03:18:18.275 INFO:teuthology.orchestra.run.smithi064.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T03:18:18.275 INFO:teuthology.orchestra.run.smithi064.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T03:18:18.275 INFO:teuthology.orchestra.run.smithi064.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T03:18:18.275 INFO:teuthology.orchestra.run.smithi064.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T03:18:18.276 INFO:teuthology.orchestra.run.smithi064.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T03:18:18.277 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:18:18.277 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-09-12T03:18:18.655 DEBUG:teuthology.parallel:result is None 2024-09-12T03:18:18.995 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T03:18:19.036 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T03:18:19.077 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T03:18:19.109 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T03:18:19.137 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T03:18:19.328 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T03:18:19.375 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T03:18:19.405 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T03:18:19.435 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T03:18:19.473 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T03:18:20.046 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:18:20.087 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:18:20.108 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:18:20.921 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T03:18:21.001 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T03:18:21.050 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T03:18:21.147 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T03:18:21.179 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T03:18:21.204 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T03:18:21.230 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T03:18:21.230 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T03:18:21.306 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T03:18:21.306 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T03:18:21.390 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T03:18:21.415 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T03:18:21.421 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:18:21.421 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:18:21.601 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:18:21.601 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T03:18:21.684 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T03:18:21.749 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:18:27.455 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:18:27.455 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T03:18:27.455 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T03:18:27.455 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T03:18:27.456 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T03:18:27.456 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T03:18:27.456 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T03:18:27.456 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T03:18:27.456 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:18:27.456 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:18:27.457 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T03:18:27.457 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T03:18:27.457 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T03:18:27.457 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:18:27.457 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T03:18:27.458 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T03:18:27.458 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T03:18:27.458 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T03:18:27.458 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T03:18:27.458 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T03:18:27.459 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T03:18:27.459 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T03:18:27.459 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T03:18:27.459 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T03:18:27.459 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T03:18:27.459 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T03:18:27.460 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T03:18:27.460 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T03:18:27.460 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T03:18:27.460 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T03:18:27.460 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T03:18:27.460 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T03:18:27.461 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T03:18:27.461 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T03:18:27.461 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T03:18:27.461 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T03:18:27.461 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T03:18:27.461 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T03:18:27.462 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T03:18:27.462 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T03:18:27.462 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T03:18:27.463 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T03:18:27.463 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T03:18:27.463 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T03:18:27.463 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T03:18:27.463 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T03:18:27.463 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T03:18:27.464 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T03:18:27.464 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T03:18:27.464 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T03:18:27.464 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T03:18:27.464 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T03:18:27.465 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T03:18:27.465 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T03:18:27.465 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T03:18:27.465 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T03:18:27.465 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T03:18:27.465 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T03:18:27.466 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T03:18:27.466 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T03:18:27.466 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T03:18:27.466 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T03:18:27.466 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T03:18:27.467 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T03:18:27.467 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T03:18:27.467 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T03:18:27.467 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T03:18:27.467 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T03:18:27.467 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T03:18:27.468 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T03:18:27.468 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T03:18:27.468 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T03:18:27.468 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T03:18:27.468 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T03:18:27.468 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T03:18:27.468 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T03:18:27.468 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T03:18:27.468 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T03:18:27.468 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T03:18:27.468 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T03:18:27.469 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T03:18:27.469 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T03:18:27.469 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T03:18:27.469 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T03:18:27.469 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T03:18:27.469 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T03:18:27.469 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T03:18:27.469 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T03:18:27.469 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T03:18:27.469 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T03:18:27.469 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T03:18:27.469 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T03:18:27.469 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T03:18:27.470 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T03:18:27.470 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T03:18:27.470 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T03:18:27.470 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T03:18:27.470 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T03:18:27.470 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T03:18:27.470 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T03:18:27.470 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T03:18:27.470 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T03:18:27.470 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T03:18:27.470 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T03:18:27.470 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T03:18:27.470 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T03:18:27.471 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T03:18:27.471 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-12T03:18:27.471 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T03:18:27.471 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T03:18:27.471 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T03:18:27.471 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T03:18:27.471 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T03:18:27.471 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T03:18:27.471 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T03:18:27.471 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T03:18:27.471 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T03:18:27.471 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T03:18:27.471 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T03:18:27.471 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T03:18:27.472 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T03:18:27.472 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T03:18:27.472 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T03:18:27.472 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T03:18:27.472 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T03:18:27.472 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T03:18:27.472 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T03:18:27.472 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T03:18:27.472 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T03:18:27.472 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T03:18:27.472 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T03:18:27.472 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T03:18:27.472 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T03:18:27.473 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T03:18:27.473 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T03:18:27.473 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T03:18:27.473 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T03:18:27.473 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T03:18:27.473 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T03:18:27.473 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T03:18:27.473 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T03:18:27.473 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T03:18:27.473 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T03:18:27.473 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T03:18:27.473 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T03:18:27.473 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T03:18:27.474 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T03:18:27.474 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T03:18:27.474 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T03:18:27.474 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T03:18:27.474 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T03:18:27.474 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T03:18:27.474 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T03:18:27.474 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T03:18:27.474 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T03:18:27.474 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T03:18:27.474 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T03:18:27.474 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T03:18:27.475 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T03:18:27.475 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T03:18:27.475 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T03:18:27.475 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T03:18:27.475 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T03:18:27.475 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T03:18:29.944 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:18:29.944 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T03:18:29.945 INFO:teuthology.orchestra.run.smithi115.stdout:Upgraded: 2024-09-12T03:18:29.945 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T03:18:29.945 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T03:18:29.945 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T03:18:29.945 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T03:18:29.945 INFO:teuthology.orchestra.run.smithi115.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:29.945 INFO:teuthology.orchestra.run.smithi115.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:29.945 INFO:teuthology.orchestra.run.smithi115.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T03:18:29.945 INFO:teuthology.orchestra.run.smithi115.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T03:18:29.945 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-12T03:18:29.945 INFO:teuthology.orchestra.run.smithi115.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T03:18:29.945 INFO:teuthology.orchestra.run.smithi115.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T03:18:29.946 INFO:teuthology.orchestra.run.smithi115.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T03:18:29.946 INFO:teuthology.orchestra.run.smithi115.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T03:18:29.946 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:29.946 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:29.946 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:29.946 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:29.946 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:18:29.946 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:29.946 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:29.946 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:29.946 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:18:29.946 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:18:29.947 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:18:29.947 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:18:29.947 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:18:29.947 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:29.947 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:29.947 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:18:29.947 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:29.947 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:29.947 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:29.947 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:18:29.947 INFO:teuthology.orchestra.run.smithi115.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:18:29.947 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:29.947 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:29.948 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:18:29.948 INFO:teuthology.orchestra.run.smithi115.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T03:18:29.948 INFO:teuthology.orchestra.run.smithi115.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T03:18:29.948 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T03:18:29.948 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T03:18:29.948 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T03:18:29.948 INFO:teuthology.orchestra.run.smithi115.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T03:18:29.948 INFO:teuthology.orchestra.run.smithi115.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-12T03:18:29.948 INFO:teuthology.orchestra.run.smithi115.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T03:18:29.948 INFO:teuthology.orchestra.run.smithi115.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T03:18:29.948 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T03:18:29.949 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T03:18:29.949 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:29.949 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:29.949 INFO:teuthology.orchestra.run.smithi115.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:29.949 INFO:teuthology.orchestra.run.smithi115.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T03:18:29.949 INFO:teuthology.orchestra.run.smithi115.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T03:18:29.949 INFO:teuthology.orchestra.run.smithi115.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T03:18:29.949 INFO:teuthology.orchestra.run.smithi115.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T03:18:29.949 INFO:teuthology.orchestra.run.smithi115.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T03:18:29.949 INFO:teuthology.orchestra.run.smithi115.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:29.949 INFO:teuthology.orchestra.run.smithi115.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:29.949 INFO:teuthology.orchestra.run.smithi115.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T03:18:29.949 INFO:teuthology.orchestra.run.smithi115.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:29.950 INFO:teuthology.orchestra.run.smithi115.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T03:18:29.950 INFO:teuthology.orchestra.run.smithi115.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T03:18:29.950 INFO:teuthology.orchestra.run.smithi115.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T03:18:29.950 INFO:teuthology.orchestra.run.smithi115.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T03:18:29.950 INFO:teuthology.orchestra.run.smithi115.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T03:18:29.950 INFO:teuthology.orchestra.run.smithi115.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T03:18:29.950 INFO:teuthology.orchestra.run.smithi115.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T03:18:29.950 INFO:teuthology.orchestra.run.smithi115.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T03:18:29.950 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T03:18:29.950 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T03:18:29.950 INFO:teuthology.orchestra.run.smithi115.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T03:18:29.950 INFO:teuthology.orchestra.run.smithi115.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T03:18:29.951 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T03:18:29.951 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T03:18:29.951 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T03:18:29.951 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T03:18:29.951 INFO:teuthology.orchestra.run.smithi115.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T03:18:29.951 INFO:teuthology.orchestra.run.smithi115.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T03:18:29.951 INFO:teuthology.orchestra.run.smithi115.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T03:18:29.951 INFO:teuthology.orchestra.run.smithi115.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T03:18:29.951 INFO:teuthology.orchestra.run.smithi115.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T03:18:29.951 INFO:teuthology.orchestra.run.smithi115.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T03:18:29.951 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T03:18:29.951 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:29.951 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:29.951 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:29.952 INFO:teuthology.orchestra.run.smithi115.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T03:18:29.952 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T03:18:29.952 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T03:18:29.952 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T03:18:29.952 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T03:18:29.952 INFO:teuthology.orchestra.run.smithi115.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T03:18:29.952 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T03:18:29.952 INFO:teuthology.orchestra.run.smithi115.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T03:18:29.952 INFO:teuthology.orchestra.run.smithi115.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T03:18:29.952 INFO:teuthology.orchestra.run.smithi115.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T03:18:29.952 INFO:teuthology.orchestra.run.smithi115.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T03:18:29.952 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T03:18:29.952 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T03:18:29.952 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T03:18:29.953 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T03:18:29.953 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T03:18:29.953 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T03:18:29.953 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T03:18:29.953 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T03:18:29.953 INFO:teuthology.orchestra.run.smithi115.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T03:18:29.953 INFO:teuthology.orchestra.run.smithi115.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T03:18:29.953 INFO:teuthology.orchestra.run.smithi115.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T03:18:29.953 INFO:teuthology.orchestra.run.smithi115.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T03:18:29.953 INFO:teuthology.orchestra.run.smithi115.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T03:18:29.953 INFO:teuthology.orchestra.run.smithi115.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T03:18:29.953 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T03:18:29.954 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T03:18:29.954 INFO:teuthology.orchestra.run.smithi115.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T03:18:29.954 INFO:teuthology.orchestra.run.smithi115.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T03:18:29.954 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T03:18:29.954 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T03:18:29.954 INFO:teuthology.orchestra.run.smithi115.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T03:18:29.954 INFO:teuthology.orchestra.run.smithi115.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T03:18:29.954 INFO:teuthology.orchestra.run.smithi115.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T03:18:29.954 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T03:18:29.954 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T03:18:29.954 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T03:18:29.954 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T03:18:29.954 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T03:18:29.955 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T03:18:29.955 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:29.955 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:29.955 INFO:teuthology.orchestra.run.smithi115.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T03:18:29.955 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T03:18:29.955 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T03:18:29.955 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:29.955 INFO:teuthology.orchestra.run.smithi115.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T03:18:29.955 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T03:18:29.955 INFO:teuthology.orchestra.run.smithi115.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T03:18:29.955 INFO:teuthology.orchestra.run.smithi115.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T03:18:29.955 INFO:teuthology.orchestra.run.smithi115.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T03:18:29.955 INFO:teuthology.orchestra.run.smithi115.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T03:18:29.955 INFO:teuthology.orchestra.run.smithi115.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T03:18:29.956 INFO:teuthology.orchestra.run.smithi115.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T03:18:29.956 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T03:18:29.956 INFO:teuthology.orchestra.run.smithi115.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T03:18:29.956 INFO:teuthology.orchestra.run.smithi115.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T03:18:29.956 INFO:teuthology.orchestra.run.smithi115.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T03:18:29.956 INFO:teuthology.orchestra.run.smithi115.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T03:18:29.956 INFO:teuthology.orchestra.run.smithi115.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T03:18:29.956 INFO:teuthology.orchestra.run.smithi115.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T03:18:29.956 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:29.956 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:29.956 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:18:29.956 INFO:teuthology.orchestra.run.smithi115.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T03:18:29.957 INFO:teuthology.orchestra.run.smithi115.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T03:18:29.957 INFO:teuthology.orchestra.run.smithi115.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T03:18:29.957 INFO:teuthology.orchestra.run.smithi115.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T03:18:29.957 INFO:teuthology.orchestra.run.smithi115.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T03:18:29.957 INFO:teuthology.orchestra.run.smithi115.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T03:18:29.957 INFO:teuthology.orchestra.run.smithi115.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T03:18:29.957 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T03:18:29.957 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-12T03:18:30.363 DEBUG:teuthology.parallel:result is None 2024-09-12T03:18:30.364 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:18:30.509 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T03:18:30.538 INFO:teuthology.orchestra.run.smithi064.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T03:18:30.539 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T03:18:30.539 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T03:18:30.542 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:18:30.681 DEBUG:teuthology.orchestra.run.smithi115:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T03:18:30.703 INFO:teuthology.orchestra.run.smithi115.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T03:18:30.704 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T03:18:30.704 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T03:18:30.706 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:18:30.838 DEBUG:teuthology.orchestra.run.smithi195:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T03:18:30.864 INFO:teuthology.orchestra.run.smithi195.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T03:18:30.864 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T03:18:30.865 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T03:18:30.867 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-12T03:18:30.867 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-12T03:18:30.867 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T03:18:30.900 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-12T03:18:30.901 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T03:18:30.928 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-12T03:18:30.928 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T03:18:30.960 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-12T03:18:30.960 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-12T03:18:30.960 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T03:18:30.990 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T03:18:31.060 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-12T03:18:31.060 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T03:18:31.085 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T03:18:31.151 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-12T03:18:31.151 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T03:18:31.181 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T03:18:31.250 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-12T03:18:31.251 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-12T03:18:31.251 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T03:18:31.282 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T03:18:31.354 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-12T03:18:31.354 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T03:18:31.379 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T03:18:31.443 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-12T03:18:31.443 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T03:18:31.469 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T03:18:31.534 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-12T03:18:31.534 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-12T03:18:31.535 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T03:18:31.565 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T03:18:31.636 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-12T03:18:31.636 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T03:18:31.661 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T03:18:31.726 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-12T03:18:31.726 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T03:18:31.751 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T03:18:31.818 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-12T03:18:32.441 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'subvols': {'create': 2}} 2024-09-12T03:18:32.441 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:18:32.442 INFO:tasks.cephadm:Cluster fsid is b0812460-70b5-11ef-bcea-c7b262605968 2024-09-12T03:18:32.442 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-12T03:18:32.442 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.64', 'mon.b': '172.21.15.115', 'mon.c': '172.21.15.195'} 2024-09-12T03:18:32.442 INFO:tasks.cephadm:First mon is mon.a on smithi064 2024-09-12T03:18:32.442 INFO:tasks.cephadm:First mgr is x 2024-09-12T03:18:32.442 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-12T03:18:32.442 DEBUG:teuthology.orchestra.run.smithi064:> sudo hostname $(hostname -s) 2024-09-12T03:18:32.474 DEBUG:teuthology.orchestra.run.smithi115:> sudo hostname $(hostname -s) 2024-09-12T03:18:32.500 DEBUG:teuthology.orchestra.run.smithi195:> sudo hostname $(hostname -s) 2024-09-12T03:18:32.526 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-12T03:18:32.526 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:18:32.671 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56038/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-298-gb53b6d3e', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-298.gb53b6d3e'}, 'url': 'https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 17:24:58.599585', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-12T03:18:32.818 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref squid, sha1 b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:18:32.819 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-12T03:18:32.819 INFO:tasks.cephadm:Downloading cephadm from url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-12T03:18:32.819 DEBUG:teuthology.orchestra.run.smithi064:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T03:18:33.211 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 03:18 /home/ubuntu/cephtest/cephadm 2024-09-12T03:18:33.212 DEBUG:teuthology.orchestra.run.smithi115:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T03:18:33.598 INFO:teuthology.orchestra.run.smithi115.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 03:18 /home/ubuntu/cephtest/cephadm 2024-09-12T03:18:33.598 DEBUG:teuthology.orchestra.run.smithi195:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T03:18:34.003 INFO:teuthology.orchestra.run.smithi195.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 03:18 /home/ubuntu/cephtest/cephadm 2024-09-12T03:18:34.003 DEBUG:teuthology.orchestra.run.smithi064:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T03:18:34.023 DEBUG:teuthology.orchestra.run.smithi115:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T03:18:34.043 DEBUG:teuthology.orchestra.run.smithi195:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T03:18:34.077 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc on all hosts... 2024-09-12T03:18:34.077 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T03:18:34.080 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T03:18:34.085 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T03:18:35.665 INFO:teuthology.orchestra.run.smithi064.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T03:18:35.741 INFO:teuthology.orchestra.run.smithi195.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T03:18:35.742 INFO:teuthology.orchestra.run.smithi115.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T03:19:04.894 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T03:19:04.894 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T03:19:04.894 INFO:teuthology.orchestra.run.smithi115.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T03:19:04.894 INFO:teuthology.orchestra.run.smithi115.stdout: "repo_digests": [ 2024-09-12T03:19:04.894 INFO:teuthology.orchestra.run.smithi115.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T03:19:04.894 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T03:19:04.894 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T03:19:05.792 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-12T03:19:05.792 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T03:19:05.792 INFO:teuthology.orchestra.run.smithi195.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T03:19:05.793 INFO:teuthology.orchestra.run.smithi195.stdout: "repo_digests": [ 2024-09-12T03:19:05.793 INFO:teuthology.orchestra.run.smithi195.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T03:19:05.793 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-12T03:19:05.793 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-12T03:19:06.838 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T03:19:06.838 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T03:19:06.838 INFO:teuthology.orchestra.run.smithi064.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T03:19:06.838 INFO:teuthology.orchestra.run.smithi064.stdout: "repo_digests": [ 2024-09-12T03:19:06.838 INFO:teuthology.orchestra.run.smithi064.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T03:19:06.839 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T03:19:06.839 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T03:19:06.872 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /etc/ceph 2024-09-12T03:19:07.008 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /etc/ceph 2024-09-12T03:19:07.043 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /etc/ceph 2024-09-12T03:19:07.077 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 777 /etc/ceph 2024-09-12T03:19:07.104 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 777 /etc/ceph 2024-09-12T03:19:07.133 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 777 /etc/ceph 2024-09-12T03:19:07.161 INFO:tasks.cephadm:Writing seed config... 2024-09-12T03:19:07.163 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-12T03:19:07.163 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-12T03:19:07.163 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-12T03:19:07.163 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-12T03:19:07.163 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-12T03:19:07.163 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-12T03:19:07.163 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-12T03:19:07.163 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-12T03:19:07.163 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-12T03:19:07.163 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-12T03:19:07.163 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-12T03:19:07.164 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-12T03:19:07.164 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-12T03:19:07.164 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-12T03:19:07.164 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-12T03:19:07.164 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-12T03:19:07.164 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-12T03:19:07.164 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-12T03:19:07.164 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-12T03:19:07.164 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-12T03:19:07.164 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-12T03:19:07.164 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-09-12T03:19:07.164 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-12T03:19:07.164 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-12T03:19:07.164 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-12T03:19:07.165 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-12T03:19:07.165 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-12T03:19:07.165 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-12T03:19:07.165 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-12T03:19:07.165 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-12T03:19:07.165 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-12T03:19:07.165 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-12T03:19:07.165 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-12T03:19:07.165 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-12T03:19:07.165 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-12T03:19:07.165 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-12T03:19:07.165 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-12T03:19:07.165 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-12T03:19:07.165 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-12T03:19:07.166 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-12T03:19:07.166 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-12T03:19:07.166 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-12T03:19:07.166 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-12T03:19:07.166 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-12T03:19:07.166 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-12T03:19:07.166 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-12T03:19:07.166 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-12T03:19:07.167 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-12T03:19:07.167 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-12T03:19:07.183 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = b0812460-70b5-11ef-bcea-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-12T03:19:07.183 DEBUG:teuthology.orchestra.run.smithi064:mon.a> sudo journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@mon.a.service 2024-09-12T03:19:07.225 DEBUG:teuthology.orchestra.run.smithi064:mgr.x> sudo journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@mgr.x.service 2024-09-12T03:19:07.267 INFO:tasks.cephadm:Bootstrapping... 2024-09-12T03:19:07.268 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc -v bootstrap --fsid b0812460-70b5-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.64 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:19:07.477 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-09-12T03:19:07.477 INFO:teuthology.orchestra.run.smithi064.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', '-v', 'bootstrap', '--fsid', 'b0812460-70b5-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.64', '--skip-admin-label'] 2024-09-12T03:19:07.477 INFO:teuthology.orchestra.run.smithi064.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-12T03:19:07.477 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying podman|docker is present... 2024-09-12T03:19:07.502 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stdout 5.2.2 2024-09-12T03:19:07.502 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying lvm2 is present... 2024-09-12T03:19:07.502 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying time synchronization is in place... 2024-09-12T03:19:07.510 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-12T03:19:07.510 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-12T03:19:07.516 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-12T03:19:07.517 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2024-09-12T03:19:07.523 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout enabled 2024-09-12T03:19:07.530 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout active 2024-09-12T03:19:07.530 INFO:teuthology.orchestra.run.smithi064.stdout:Unit chronyd.service is enabled and running 2024-09-12T03:19:07.530 INFO:teuthology.orchestra.run.smithi064.stdout:Repeating the final host check... 2024-09-12T03:19:07.554 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stdout 5.2.2 2024-09-12T03:19:07.555 INFO:teuthology.orchestra.run.smithi064.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-12T03:19:07.555 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl is present 2024-09-12T03:19:07.555 INFO:teuthology.orchestra.run.smithi064.stdout:lvcreate is present 2024-09-12T03:19:07.561 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-12T03:19:07.562 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-12T03:19:07.568 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-12T03:19:07.568 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2024-09-12T03:19:07.575 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout enabled 2024-09-12T03:19:07.582 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout active 2024-09-12T03:19:07.582 INFO:teuthology.orchestra.run.smithi064.stdout:Unit chronyd.service is enabled and running 2024-09-12T03:19:07.582 INFO:teuthology.orchestra.run.smithi064.stdout:Host looks OK 2024-09-12T03:19:07.582 INFO:teuthology.orchestra.run.smithi064.stdout:Cluster fsid: b0812460-70b5-11ef-bcea-c7b262605968 2024-09-12T03:19:07.582 INFO:teuthology.orchestra.run.smithi064.stdout:Acquiring lock 140478681355264 on /run/cephadm/b0812460-70b5-11ef-bcea-c7b262605968.lock 2024-09-12T03:19:07.582 INFO:teuthology.orchestra.run.smithi064.stdout:Lock 140478681355264 acquired on /run/cephadm/b0812460-70b5-11ef-bcea-c7b262605968.lock 2024-09-12T03:19:07.582 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying IP 172.21.15.64 port 3300 ... 2024-09-12T03:19:07.583 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying IP 172.21.15.64 port 6789 ... 2024-09-12T03:19:07.583 INFO:teuthology.orchestra.run.smithi064.stdout:Base mon IP(s) is [172.21.15.64:3300, 172.21.15.64:6789], mon addrv is [v2:172.21.15.64:3300,v1:172.21.15.64:6789] 2024-09-12T03:19:07.586 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.64 metric 100 2024-09-12T03:19:07.586 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.64 metric 100 2024-09-12T03:19:07.589 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-12T03:19:07.589 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-12T03:19:07.589 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-12T03:19:07.591 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-12T03:19:07.591 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-12T03:19:07.591 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-12T03:19:07.592 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-12T03:19:07.592 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:feb6:f8df/64 scope link noprefixroute 2024-09-12T03:19:07.592 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-12T03:19:07.592 INFO:teuthology.orchestra.run.smithi064.stdout:Mon IP `172.21.15.64` is in CIDR network `172.21.0.0/20` 2024-09-12T03:19:07.592 INFO:teuthology.orchestra.run.smithi064.stdout:Mon IP `172.21.15.64` is in CIDR network `172.21.0.0/20` 2024-09-12T03:19:07.592 INFO:teuthology.orchestra.run.smithi064.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-12T03:19:07.593 INFO:teuthology.orchestra.run.smithi064.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-12T03:19:07.594 INFO:teuthology.orchestra.run.smithi064.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T03:19:08.024 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stdout 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-12T03:19:08.024 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T03:19:08.024 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Getting image source signatures 2024-09-12T03:19:08.024 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-12T03:19:08.024 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Copying blob sha256:60180723200353b4e13b39fd162aecaf1bc097c859c7c6537324229e97c78567 2024-09-12T03:19:08.024 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Copying config sha256:54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-12T03:19:08.024 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-12T03:19:08.979 INFO:teuthology.orchestra.run.smithi064.stdout:ceph: stdout ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-12T03:19:08.979 INFO:teuthology.orchestra.run.smithi064.stdout:Ceph version: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-12T03:19:08.979 INFO:teuthology.orchestra.run.smithi064.stdout:Extracting ceph user uid/gid from container image... 2024-09-12T03:19:09.893 INFO:teuthology.orchestra.run.smithi064.stdout:stat: stdout 167 167 2024-09-12T03:19:09.893 INFO:teuthology.orchestra.run.smithi064.stdout:Creating initial keys... 2024-09-12T03:19:10.841 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-authtool: stdout AQCuXeJmrh+bDhAAgqCmnRRZ/2oq/l6m5/pdSw== 2024-09-12T03:19:11.754 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-authtool: stdout AQCvXeJmo/L9CBAAt82cjvyn5BqunpK18MwhYw== 2024-09-12T03:19:12.660 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-authtool: stdout AQCwXeJmYmI8AhAAFDQCDYLPeZKiK8ozvTTKXw== 2024-09-12T03:19:12.661 INFO:teuthology.orchestra.run.smithi064.stdout:Creating initial monmap... 2024-09-12T03:19:13.576 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-12T03:19:13.671 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-12T03:19:13.671 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to b0812460-70b5-11ef-bcea-c7b262605968 2024-09-12T03:19:13.671 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-12T03:19:13.672 INFO:teuthology.orchestra.run.smithi064.stdout:monmaptool for a [v2:172.21.15.64:3300,v1:172.21.15.64:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-12T03:19:13.672 INFO:teuthology.orchestra.run.smithi064.stdout:setting min_mon_release = quincy 2024-09-12T03:19:13.672 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: set fsid to b0812460-70b5-11ef-bcea-c7b262605968 2024-09-12T03:19:13.672 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-12T03:19:13.672 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:19:13.672 INFO:teuthology.orchestra.run.smithi064.stdout:Creating mon... 2024-09-12T03:19:14.455 INFO:teuthology.orchestra.run.smithi064.stdout:create mon.a on 2024-09-12T03:19:14.680 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-12T03:19:14.826 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-12T03:19:15.002 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-b0812460-70b5-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-b0812460-70b5-11ef-bcea-c7b262605968.target. 2024-09-12T03:19:15.003 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-b0812460-70b5-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-b0812460-70b5-11ef-bcea-c7b262605968.target. 2024-09-12T03:19:15.259 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b0812460-70b5-11ef-bcea-c7b262605968@mon.a 2024-09-12T03:19:15.259 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b0812460-70b5-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-b0812460-70b5-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-12T03:19:15.425 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b0812460-70b5-11ef-bcea-c7b262605968.target.wants/ceph-b0812460-70b5-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-b0812460-70b5-11ef-bcea-c7b262605968@.service. 2024-09-12T03:19:15.967 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:15 smithi064 ceph-mon[30585]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-12T03:19:15.967 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:15 smithi064 bash[30558]: a26fb510f734ad809a229e7a7ab8462e244961736ada9b4e15d0f5dd55f04c38 2024-09-12T03:19:15.985 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T03:19:15.986 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout disabled 2024-09-12T03:19:15.991 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T03:19:15.991 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2024-09-12T03:19:15.991 INFO:teuthology.orchestra.run.smithi064.stdout:firewalld.service is not enabled 2024-09-12T03:19:15.991 INFO:teuthology.orchestra.run.smithi064.stdout:Not possible to enable service . firewalld.service is not available 2024-09-12T03:19:15.991 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mon to start... 2024-09-12T03:19:15.991 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mon... 2024-09-12T03:19:16.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:15 smithi064 systemd[1]: Started Ceph mon.a for b0812460-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:19:17.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:16 smithi064 ceph-mon[30585]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-12T03:19:17.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:16 smithi064 ceph-mon[30585]: monmap epoch 1 2024-09-12T03:19:17.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:16 smithi064 ceph-mon[30585]: fsid b0812460-70b5-11ef-bcea-c7b262605968 2024-09-12T03:19:17.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:16 smithi064 ceph-mon[30585]: last_changed 2024-09-12T03:19:13.042174+0000 2024-09-12T03:19:17.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:16 smithi064 ceph-mon[30585]: created 2024-09-12T03:19:13.042174+0000 2024-09-12T03:19:17.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:16 smithi064 ceph-mon[30585]: min_mon_release 19 (squid) 2024-09-12T03:19:17.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:16 smithi064 ceph-mon[30585]: election_strategy: 1 2024-09-12T03:19:17.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:16 smithi064 ceph-mon[30585]: 0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2024-09-12T03:19:17.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:16 smithi064 ceph-mon[30585]: fsmap 2024-09-12T03:19:17.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:16 smithi064 ceph-mon[30585]: osdmap e1: 0 total, 0 up, 0 in 2024-09-12T03:19:17.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:16 smithi064 ceph-mon[30585]: mgrmap e1: no daemons active 2024-09-12T03:19:17.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:16 smithi064 ceph-mon[30585]: from='client.? 172.21.15.64:0/2976602201' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-12T03:19:17.073 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout cluster: 2024-09-12T03:19:17.073 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout id: b0812460-70b5-11ef-bcea-c7b262605968 2024-09-12T03:19:17.073 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-12T03:19:17.073 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-09-12T03:19:17.074 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout services: 2024-09-12T03:19:17.074 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.581808s) 2024-09-12T03:19:17.074 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-12T03:19:17.074 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-12T03:19:17.074 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-09-12T03:19:17.074 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout data: 2024-09-12T03:19:17.074 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-12T03:19:17.074 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-12T03:19:17.074 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-12T03:19:17.074 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout pgs: 2024-09-12T03:19:17.074 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-09-12T03:19:17.074 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.407+0000 7fd50d59e640 1 Processor -- start 2024-09-12T03:19:17.075 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.408+0000 7fd50d59e640 1 -- start start 2024-09-12T03:19:17.075 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.408+0000 7fd50d59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd508105c50 0x7fd508106050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:17.075 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.408+0000 7fd50d59e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd50807bde0 con 0x7fd508105c50 2024-09-12T03:19:17.075 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.408+0000 7fd507fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd508105c50 0x7fd508106050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:17.075 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.408+0000 7fd507fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd508105c50 0x7fd508106050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57178/0 (socket says 172.21.15.64:57178) 2024-09-12T03:19:17.075 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.408+0000 7fd507fff640 1 -- 172.21.15.64:0/2580197166 learned_addr learned my addr 172.21.15.64:0/2580197166 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:19:17.075 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.408+0000 7fd507fff640 1 -- 172.21.15.64:0/2580197166 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd508106590 con 0x7fd508105c50 2024-09-12T03:19:17.075 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.411+0000 7fd507fff640 1 --2- 172.21.15.64:0/2580197166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd508105c50 0x7fd508106050 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8009820 tx=0x7fd4f802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a69953e89e67916 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:17.075 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.412+0000 7fd506ffd640 1 -- 172.21.15.64:0/2580197166 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4f8037070 con 0x7fd508105c50 2024-09-12T03:19:17.075 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.412+0000 7fd506ffd640 1 -- 172.21.15.64:0/2580197166 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4f8032440 con 0x7fd508105c50 2024-09-12T03:19:17.075 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.412+0000 7fd506ffd640 1 -- 172.21.15.64:0/2580197166 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4f8032740 con 0x7fd508105c50 2024-09-12T03:19:17.075 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.412+0000 7fd50d59e640 1 -- 172.21.15.64:0/2580197166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd508105c50 msgr2=0x7fd508106050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:17.076 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.412+0000 7fd50d59e640 1 --2- 172.21.15.64:0/2580197166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd508105c50 0x7fd508106050 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8009820 tx=0x7fd4f802a270 comp rx=0 tx=0).stop 2024-09-12T03:19:17.076 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.412+0000 7fd50d59e640 1 -- 172.21.15.64:0/2580197166 shutdown_connections 2024-09-12T03:19:17.076 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.412+0000 7fd50d59e640 1 --2- 172.21.15.64:0/2580197166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd508105c50 0x7fd508106050 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.076 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.412+0000 7fd50d59e640 1 -- 172.21.15.64:0/2580197166 >> 172.21.15.64:0/2580197166 conn(0x7fd508101a00 msgr2=0x7fd508103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:17.076 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.412+0000 7fd50d59e640 1 -- 172.21.15.64:0/2580197166 shutdown_connections 2024-09-12T03:19:17.076 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.412+0000 7fd50d59e640 1 -- 172.21.15.64:0/2580197166 wait complete. 2024-09-12T03:19:17.076 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.413+0000 7fd50d59e640 1 Processor -- start 2024-09-12T03:19:17.076 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.413+0000 7fd50d59e640 1 -- start start 2024-09-12T03:19:17.076 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.414+0000 7fd50d59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd508105c50 0x7fd50819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:17.076 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.414+0000 7fd50d59e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd508106b30 con 0x7fd508105c50 2024-09-12T03:19:17.076 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.414+0000 7fd507fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd508105c50 0x7fd50819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:17.076 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.414+0000 7fd507fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd508105c50 0x7fd50819cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57190/0 (socket says 172.21.15.64:57190) 2024-09-12T03:19:17.077 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.414+0000 7fd507fff640 1 -- 172.21.15.64:0/2976602201 learned_addr learned my addr 172.21.15.64:0/2976602201 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:19:17.077 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.414+0000 7fd507fff640 1 -- 172.21.15.64:0/2976602201 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd50819d250 con 0x7fd508105c50 2024-09-12T03:19:17.077 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.414+0000 7fd507fff640 1 --2- 172.21.15.64:0/2976602201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd508105c50 0x7fd50819cd10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd4f80090b0 tx=0x7fd4f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:17.077 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.414+0000 7fd5057fa640 1 -- 172.21.15.64:0/2976602201 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4f8041070 con 0x7fd508105c50 2024-09-12T03:19:17.077 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.414+0000 7fd5057fa640 1 -- 172.21.15.64:0/2976602201 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4f803c7a0 con 0x7fd508105c50 2024-09-12T03:19:17.077 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.414+0000 7fd5057fa640 1 -- 172.21.15.64:0/2976602201 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4f8037040 con 0x7fd508105c50 2024-09-12T03:19:17.077 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.414+0000 7fd50d59e640 1 -- 172.21.15.64:0/2976602201 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd50819d4e0 con 0x7fd508105c50 2024-09-12T03:19:17.077 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.415+0000 7fd50d59e640 1 -- 172.21.15.64:0/2976602201 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5081a01c0 con 0x7fd508105c50 2024-09-12T03:19:17.077 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.415+0000 7fd5057fa640 1 -- 172.21.15.64:0/2976602201 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd4f8044070 con 0x7fd508105c50 2024-09-12T03:19:17.077 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.415+0000 7fd5057fa640 1 -- 172.21.15.64:0/2976602201 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd4f803d730 con 0x7fd508105c50 2024-09-12T03:19:17.077 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.416+0000 7fd50d59e640 1 -- 172.21.15.64:0/2976602201 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd508106050 con 0x7fd508105c50 2024-09-12T03:19:17.077 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.418+0000 7fd5057fa640 1 -- 172.21.15.64:0/2976602201 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fd4f803cbe0 con 0x7fd508105c50 2024-09-12T03:19:17.077 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.480+0000 7fd50d59e640 1 -- 172.21.15.64:0/2976602201 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fd5081a07a0 con 0x7fd508105c50 2024-09-12T03:19:17.078 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.480+0000 7fd5057fa640 1 -- 172.21.15.64:0/2976602201 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fd4f803cdc0 con 0x7fd508105c50 2024-09-12T03:19:17.078 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.481+0000 7fd50d59e640 1 -- 172.21.15.64:0/2976602201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd508105c50 msgr2=0x7fd50819cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:17.078 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.481+0000 7fd50d59e640 1 --2- 172.21.15.64:0/2976602201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd508105c50 0x7fd50819cd10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd4f80090b0 tx=0x7fd4f80046c0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.078 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.481+0000 7fd50d59e640 1 -- 172.21.15.64:0/2976602201 shutdown_connections 2024-09-12T03:19:17.078 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.481+0000 7fd50d59e640 1 --2- 172.21.15.64:0/2976602201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd508105c50 0x7fd50819cd10 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.078 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.481+0000 7fd50d59e640 1 -- 172.21.15.64:0/2976602201 >> 172.21.15.64:0/2976602201 conn(0x7fd508101a00 msgr2=0x7fd5081932d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:17.078 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.481+0000 7fd50d59e640 1 -- 172.21.15.64:0/2976602201 shutdown_connections 2024-09-12T03:19:17.078 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:16.481+0000 7fd50d59e640 1 -- 172.21.15.64:0/2976602201 wait complete. 2024-09-12T03:19:17.078 INFO:teuthology.orchestra.run.smithi064.stdout:mon is available 2024-09-12T03:19:17.078 INFO:teuthology.orchestra.run.smithi064.stdout:Assimilating anything we can from ceph.conf... 2024-09-12T03:19:18.046 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-09-12T03:19:18.046 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [global] 2024-09-12T03:19:18.046 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout fsid = b0812460-70b5-11ef-bcea-c7b262605968 2024-09-12T03:19:18.047 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.64:3300,v1:172.21.15.64:6789] 2024-09-12T03:19:18.047 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-12T03:19:18.047 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-12T03:19:18.047 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-12T03:19:18.047 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-12T03:19:18.047 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-09-12T03:19:18.047 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-12T03:19:18.047 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-12T03:19:18.047 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-09-12T03:19:18.047 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [osd] 2024-09-12T03:19:18.047 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-12T03:19:18.047 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-12T03:19:18.047 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-12T03:19:18.048 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-12T03:19:18.048 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-12T03:19:18.048 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.454+0000 7fe8a2642640 1 Processor -- start 2024-09-12T03:19:18.048 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.454+0000 7fe8a2642640 1 -- start start 2024-09-12T03:19:18.048 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.455+0000 7fe8a2642640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe89c107510 0x7fe89c107910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:18.048 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.455+0000 7fe8a2642640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c107ee0 con 0x7fe89c107510 2024-09-12T03:19:18.048 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.455+0000 7fe89bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe89c107510 0x7fe89c107910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:18.048 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.455+0000 7fe89bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe89c107510 0x7fe89c107910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57196/0 (socket says 172.21.15.64:57196) 2024-09-12T03:19:18.048 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.455+0000 7fe89bfff640 1 -- 172.21.15.64:0/1027955354 learned_addr learned my addr 172.21.15.64:0/1027955354 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:19:18.048 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.455+0000 7fe89bfff640 1 -- 172.21.15.64:0/1027955354 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe89c108710 con 0x7fe89c107510 2024-09-12T03:19:18.048 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.456+0000 7fe89bfff640 1 --2- 172.21.15.64:0/1027955354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe89c107510 0x7fe89c107910 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe88c009820 tx=0x7fe88c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ac7538b7d98b9b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:18.048 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.456+0000 7fe89affd640 1 -- 172.21.15.64:0/1027955354 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe88c037070 con 0x7fe89c107510 2024-09-12T03:19:18.049 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.456+0000 7fe89affd640 1 -- 172.21.15.64:0/1027955354 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe88c032440 con 0x7fe89c107510 2024-09-12T03:19:18.049 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.456+0000 7fe89affd640 1 -- 172.21.15.64:0/1027955354 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe88c032740 con 0x7fe89c107510 2024-09-12T03:19:18.049 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.457+0000 7fe8a2642640 1 -- 172.21.15.64:0/1027955354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe89c107510 msgr2=0x7fe89c107910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:18.049 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.457+0000 7fe8a2642640 1 --2- 172.21.15.64:0/1027955354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe89c107510 0x7fe89c107910 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe88c009820 tx=0x7fe88c02a270 comp rx=0 tx=0).stop 2024-09-12T03:19:18.049 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.457+0000 7fe8a2642640 1 -- 172.21.15.64:0/1027955354 shutdown_connections 2024-09-12T03:19:18.049 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.457+0000 7fe8a2642640 1 --2- 172.21.15.64:0/1027955354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe89c107510 0x7fe89c107910 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:18.049 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.457+0000 7fe8a2642640 1 -- 172.21.15.64:0/1027955354 >> 172.21.15.64:0/1027955354 conn(0x7fe89c102ca0 msgr2=0x7fe89c1050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:18.049 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.457+0000 7fe8a2642640 1 -- 172.21.15.64:0/1027955354 shutdown_connections 2024-09-12T03:19:18.049 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.457+0000 7fe8a2642640 1 -- 172.21.15.64:0/1027955354 wait complete. 2024-09-12T03:19:18.049 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.457+0000 7fe8a2642640 1 Processor -- start 2024-09-12T03:19:18.049 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.458+0000 7fe8a2642640 1 -- start start 2024-09-12T03:19:18.049 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.458+0000 7fe8a2642640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe89c107510 0x7fe89c07db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:18.049 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.458+0000 7fe8a2642640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c108cb0 con 0x7fe89c107510 2024-09-12T03:19:18.050 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.458+0000 7fe89bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe89c107510 0x7fe89c07db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:18.050 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.458+0000 7fe89bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe89c107510 0x7fe89c07db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57202/0 (socket says 172.21.15.64:57202) 2024-09-12T03:19:18.050 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.458+0000 7fe89bfff640 1 -- 172.21.15.64:0/1008871228 learned_addr learned my addr 172.21.15.64:0/1008871228 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:19:18.050 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.458+0000 7fe89bfff640 1 -- 172.21.15.64:0/1008871228 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe89c07e080 con 0x7fe89c107510 2024-09-12T03:19:18.050 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.458+0000 7fe89bfff640 1 --2- 172.21.15.64:0/1008871228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe89c107510 0x7fe89c07db40 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe88c0090b0 tx=0x7fe88c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:18.050 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.459+0000 7fe8997fa640 1 -- 172.21.15.64:0/1008871228 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe88c041070 con 0x7fe89c107510 2024-09-12T03:19:18.050 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.459+0000 7fe8997fa640 1 -- 172.21.15.64:0/1008871228 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe88c03c7a0 con 0x7fe89c107510 2024-09-12T03:19:18.050 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.459+0000 7fe8a2642640 1 -- 172.21.15.64:0/1008871228 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe89c07c270 con 0x7fe89c107510 2024-09-12T03:19:18.050 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.459+0000 7fe8997fa640 1 -- 172.21.15.64:0/1008871228 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe88c037040 con 0x7fe89c107510 2024-09-12T03:19:18.050 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.459+0000 7fe8a2642640 1 -- 172.21.15.64:0/1008871228 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe89c07c7b0 con 0x7fe89c107510 2024-09-12T03:19:18.050 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.459+0000 7fe8997fa640 1 -- 172.21.15.64:0/1008871228 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe88c044070 con 0x7fe89c107510 2024-09-12T03:19:18.050 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.459+0000 7fe8997fa640 1 -- 172.21.15.64:0/1008871228 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe88c03d3f0 con 0x7fe89c107510 2024-09-12T03:19:18.050 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.460+0000 7fe8a2642640 1 -- 172.21.15.64:0/1008871228 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe89c109300 con 0x7fe89c107510 2024-09-12T03:19:18.051 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.462+0000 7fe8997fa640 1 -- 172.21.15.64:0/1008871228 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fe88c046450 con 0x7fe89c107510 2024-09-12T03:19:18.051 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.521+0000 7fe8a2642640 1 -- 172.21.15.64:0/1008871228 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fe89c07cab0 con 0x7fe89c107510 2024-09-12T03:19:18.051 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.524+0000 7fe8997fa640 1 -- 172.21.15.64:0/1008871228 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7fe88c046630 con 0x7fe89c107510 2024-09-12T03:19:18.051 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.525+0000 7fe8997fa640 1 -- 172.21.15.64:0/1008871228 <== mon.0 v2:172.21.15.64:3300/0 8 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fe88c0584b0 con 0x7fe89c107510 2024-09-12T03:19:18.051 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.526+0000 7fe8a2642640 1 -- 172.21.15.64:0/1008871228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe89c107510 msgr2=0x7fe89c07db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:18.051 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.526+0000 7fe8a2642640 1 --2- 172.21.15.64:0/1008871228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe89c107510 0x7fe89c07db40 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe88c0090b0 tx=0x7fe88c0046c0 comp rx=0 tx=0).stop 2024-09-12T03:19:18.051 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.526+0000 7fe8a2642640 1 -- 172.21.15.64:0/1008871228 shutdown_connections 2024-09-12T03:19:18.051 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.526+0000 7fe8a2642640 1 --2- 172.21.15.64:0/1008871228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe89c107510 0x7fe89c07db40 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:18.051 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.526+0000 7fe8a2642640 1 -- 172.21.15.64:0/1008871228 >> 172.21.15.64:0/1008871228 conn(0x7fe89c102ca0 msgr2=0x7fe89c18efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:18.051 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.526+0000 7fe8a2642640 1 -- 172.21.15.64:0/1008871228 shutdown_connections 2024-09-12T03:19:18.051 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:17.526+0000 7fe8a2642640 1 -- 172.21.15.64:0/1008871228 wait complete. 2024-09-12T03:19:18.051 INFO:teuthology.orchestra.run.smithi064.stdout:Generating new minimal ceph.conf... 2024-09-12T03:19:18.164 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:17 smithi064 ceph-mon[30585]: from='client.? 172.21.15.64:0/1008871228' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-12T03:19:18.164 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:17 smithi064 ceph-mon[30585]: from='client.? 172.21.15.64:0/1008871228' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-12T03:19:19.003 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:18 smithi064 ceph-mon[30585]: from='client.? 172.21.15.64:0/3105769117' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:19:19.009 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.446+0000 7fc3b5afd640 1 Processor -- start 2024-09-12T03:19:19.009 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.446+0000 7fc3b5afd640 1 -- start start 2024-09-12T03:19:19.009 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.446+0000 7fc3b5afd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3b0107510 0x7fc3b0107910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:19.009 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.446+0000 7fc3b5afd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b0107ee0 con 0x7fc3b0107510 2024-09-12T03:19:19.009 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.446+0000 7fc3af7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3b0107510 0x7fc3b0107910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:19.009 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.446+0000 7fc3af7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3b0107510 0x7fc3b0107910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44820/0 (socket says 172.21.15.64:44820) 2024-09-12T03:19:19.009 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.446+0000 7fc3af7fe640 1 -- 172.21.15.64:0/1504389785 learned_addr learned my addr 172.21.15.64:0/1504389785 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:19:19.009 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.447+0000 7fc3af7fe640 1 -- 172.21.15.64:0/1504389785 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b0108710 con 0x7fc3b0107510 2024-09-12T03:19:19.009 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.447+0000 7fc3af7fe640 1 --2- 172.21.15.64:0/1504389785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3b0107510 0x7fc3b0107910 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0009820 tx=0x7fc3a002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=256fc08e01d6fcbf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:19.009 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.447+0000 7fc3ae7fc640 1 -- 172.21.15.64:0/1504389785 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3a0037070 con 0x7fc3b0107510 2024-09-12T03:19:19.015 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.448+0000 7fc3ae7fc640 1 -- 172.21.15.64:0/1504389785 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fc3a00328d0 con 0x7fc3b0107510 2024-09-12T03:19:19.015 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.448+0000 7fc3ae7fc640 1 -- 172.21.15.64:0/1504389785 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3a00307d0 con 0x7fc3b0107510 2024-09-12T03:19:19.015 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.448+0000 7fc3b5afd640 1 -- 172.21.15.64:0/1504389785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3b0107510 msgr2=0x7fc3b0107910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:19.015 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.448+0000 7fc3b5afd640 1 --2- 172.21.15.64:0/1504389785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3b0107510 0x7fc3b0107910 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0009820 tx=0x7fc3a002a270 comp rx=0 tx=0).stop 2024-09-12T03:19:19.015 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.448+0000 7fc3b5afd640 1 -- 172.21.15.64:0/1504389785 shutdown_connections 2024-09-12T03:19:19.015 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.448+0000 7fc3b5afd640 1 --2- 172.21.15.64:0/1504389785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3b0107510 0x7fc3b0107910 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:19.016 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.448+0000 7fc3b5afd640 1 -- 172.21.15.64:0/1504389785 >> 172.21.15.64:0/1504389785 conn(0x7fc3b0102ce0 msgr2=0x7fc3b0105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:19.016 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.448+0000 7fc3b5afd640 1 -- 172.21.15.64:0/1504389785 shutdown_connections 2024-09-12T03:19:19.016 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.448+0000 7fc3b5afd640 1 -- 172.21.15.64:0/1504389785 wait complete. 2024-09-12T03:19:19.016 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.449+0000 7fc3b5afd640 1 Processor -- start 2024-09-12T03:19:19.016 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.449+0000 7fc3b5afd640 1 -- start start 2024-09-12T03:19:19.016 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.449+0000 7fc3b5afd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3b0107510 0x7fc3b019d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:19.016 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.449+0000 7fc3b5afd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b010bac0 con 0x7fc3b0107510 2024-09-12T03:19:19.016 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.449+0000 7fc3af7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3b0107510 0x7fc3b019d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:19.017 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.449+0000 7fc3af7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3b0107510 0x7fc3b019d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44826/0 (socket says 172.21.15.64:44826) 2024-09-12T03:19:19.017 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.449+0000 7fc3af7fe640 1 -- 172.21.15.64:0/3105769117 learned_addr learned my addr 172.21.15.64:0/3105769117 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:19:19.017 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.449+0000 7fc3af7fe640 1 -- 172.21.15.64:0/3105769117 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b019dc10 con 0x7fc3b0107510 2024-09-12T03:19:19.017 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.450+0000 7fc3af7fe640 1 --2- 172.21.15.64:0/3105769117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3b0107510 0x7fc3b019d6d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0009940 tx=0x7fc3a0032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:19.017 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.450+0000 7fc3acff9640 1 -- 172.21.15.64:0/3105769117 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3a0042070 con 0x7fc3b0107510 2024-09-12T03:19:19.017 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.450+0000 7fc3acff9640 1 -- 172.21.15.64:0/3105769117 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fc3a0030c80 con 0x7fc3b0107510 2024-09-12T03:19:19.017 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.450+0000 7fc3b5afd640 1 -- 172.21.15.64:0/3105769117 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3b019dea0 con 0x7fc3b0107510 2024-09-12T03:19:19.017 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.450+0000 7fc3b5afd640 1 -- 172.21.15.64:0/3105769117 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3b01a0b90 con 0x7fc3b0107510 2024-09-12T03:19:19.017 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.450+0000 7fc3acff9640 1 -- 172.21.15.64:0/3105769117 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3a0037040 con 0x7fc3b0107510 2024-09-12T03:19:19.017 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.450+0000 7fc3acff9640 1 -- 172.21.15.64:0/3105769117 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc3a0045070 con 0x7fc3b0107510 2024-09-12T03:19:19.017 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.450+0000 7fc3acff9640 1 -- 172.21.15.64:0/3105769117 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc3a0047a40 con 0x7fc3b0107510 2024-09-12T03:19:19.018 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.451+0000 7fc3b5afd640 1 -- 172.21.15.64:0/3105769117 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3b010bc40 con 0x7fc3b0107510 2024-09-12T03:19:19.018 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.453+0000 7fc3acff9640 1 -- 172.21.15.64:0/3105769117 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fc3a003c700 con 0x7fc3b0107510 2024-09-12T03:19:19.018 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.513+0000 7fc3b5afd640 1 -- 172.21.15.64:0/3105769117 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fc3b01a0d60 con 0x7fc3b0107510 2024-09-12T03:19:19.018 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.513+0000 7fc3acff9640 1 -- 172.21.15.64:0/3105769117 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fc3a003c8e0 con 0x7fc3b0107510 2024-09-12T03:19:19.018 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.514+0000 7fc3b5afd640 1 -- 172.21.15.64:0/3105769117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3b0107510 msgr2=0x7fc3b019d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:19.018 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.514+0000 7fc3b5afd640 1 --2- 172.21.15.64:0/3105769117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3b0107510 0x7fc3b019d6d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0009940 tx=0x7fc3a0032e10 comp rx=0 tx=0).stop 2024-09-12T03:19:19.018 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.514+0000 7fc3b5afd640 1 -- 172.21.15.64:0/3105769117 shutdown_connections 2024-09-12T03:19:19.018 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.514+0000 7fc3b5afd640 1 --2- 172.21.15.64:0/3105769117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3b0107510 0x7fc3b019d6d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:19.018 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.514+0000 7fc3b5afd640 1 -- 172.21.15.64:0/3105769117 >> 172.21.15.64:0/3105769117 conn(0x7fc3b0102ce0 msgr2=0x7fc3b0103780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:19.018 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.514+0000 7fc3b5afd640 1 -- 172.21.15.64:0/3105769117 shutdown_connections 2024-09-12T03:19:19.018 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:18.514+0000 7fc3b5afd640 1 -- 172.21.15.64:0/3105769117 wait complete. 2024-09-12T03:19:19.019 INFO:teuthology.orchestra.run.smithi064.stdout:Restarting the monitor... 2024-09-12T03:19:19.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:19 smithi064 systemd[1]: Stopping Ceph mon.a for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:19:19.581 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:19 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mon-a[30581]: 2024-09-12T03:19:19.281+0000 7efe37a71640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:19:19.581 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:19 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mon-a[30581]: 2024-09-12T03:19:19.281+0000 7efe37a71640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-12T03:19:19.581 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:19 smithi064 podman[30830]: 2024-09-12 03:19:19.333271814 +0000 UTC m=+0.145389888 container died a26fb510f734ad809a229e7a7ab8462e244961736ada9b4e15d0f5dd55f04c38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True) 2024-09-12T03:19:19.907 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:19 smithi064 podman[30830]: 2024-09-12 03:19:19.681245436 +0000 UTC m=+0.493363509 container cleanup a26fb510f734ad809a229e7a7ab8462e244961736ada9b4e15d0f5dd55f04c38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mon-a, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-12T03:19:19.907 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:19 smithi064 bash[30830]: ceph-b0812460-70b5-11ef-bcea-c7b262605968-mon-a 2024-09-12T03:19:20.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:19 smithi064 podman[30842]: 2024-09-12 03:19:19.907201806 +0000 UTC m=+0.569178514 container remove a26fb510f734ad809a229e7a7ab8462e244961736ada9b4e15d0f5dd55f04c38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-12T03:19:20.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 systemd[1]: ceph-b0812460-70b5-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-12T03:19:20.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 systemd[1]: Stopped Ceph mon.a for b0812460-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:19:20.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 systemd[1]: Starting Ceph mon.a for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:19:20.601 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 podman[30933]: 2024-09-12 03:19:20.251762882 +0000 UTC m=+0.016363136 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:19:20.602 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 podman[30933]: 2024-09-12 03:19:20.366714957 +0000 UTC m=+0.131315193 container create 9aa76b73f7af2ea7831e04fc80191f235927cc5c7db6073b4e687510da1b6a80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-12T03:19:20.602 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 podman[30933]: 2024-09-12 03:19:20.577268908 +0000 UTC m=+0.341869154 container init 9aa76b73f7af2ea7831e04fc80191f235927cc5c7db6073b4e687510da1b6a80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mon-a, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:19:20.602 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 podman[30933]: 2024-09-12 03:19:20.580522302 +0000 UTC m=+0.345122541 container start 9aa76b73f7af2ea7831e04fc80191f235927cc5c7db6073b4e687510da1b6a80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, ceph=True) 2024-09-12T03:19:20.656 INFO:teuthology.orchestra.run.smithi064.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-12T03:19:20.853 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: set uid:gid to 167:167 (ceph:ceph) 2024-09-12T03:19:20.853 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable), process ceph-mon, pid 2 2024-09-12T03:19:20.853 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: pidfile_write: ignore empty --pid-file 2024-09-12T03:19:20.853 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: load: jerasure load: lrc 2024-09-12T03:19:20.853 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: RocksDB version: 7.9.2 2024-09-12T03:19:20.853 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Git sha 0 2024-09-12T03:19:20.853 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Compile date 2024-09-09 16:23:28 2024-09-12T03:19:20.854 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: DB SUMMARY 2024-09-12T03:19:20.854 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: DB Session ID: HCLCXM3OF4TFU39P8GK9 2024-09-12T03:19:20.854 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: CURRENT file: CURRENT 2024-09-12T03:19:20.854 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: IDENTITY file: IDENTITY 2024-09-12T03:19:20.854 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-12T03:19:20.854 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-12T03:19:20.854 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105262 ; 2024-09-12T03:19:20.854 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.error_if_exists: 0 2024-09-12T03:19:20.854 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.create_if_missing: 0 2024-09-12T03:19:20.854 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.paranoid_checks: 1 2024-09-12T03:19:20.854 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-12T03:19:20.854 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-12T03:19:20.854 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-12T03:19:20.855 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.env: 0x559e7899f940 2024-09-12T03:19:20.855 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.fs: PosixFileSystem 2024-09-12T03:19:20.855 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.info_log: 0x559e7a11f6e0 2024-09-12T03:19:20.855 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.max_file_opening_threads: 16 2024-09-12T03:19:20.855 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.statistics: (nil) 2024-09-12T03:19:20.855 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.use_fsync: 0 2024-09-12T03:19:20.855 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.max_log_file_size: 0 2024-09-12T03:19:20.855 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-12T03:19:20.855 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-12T03:19:20.855 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.keep_log_file_num: 1000 2024-09-12T03:19:20.855 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.recycle_log_file_num: 0 2024-09-12T03:19:20.855 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.allow_fallocate: 1 2024-09-12T03:19:20.855 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.allow_mmap_reads: 0 2024-09-12T03:19:20.855 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.allow_mmap_writes: 0 2024-09-12T03:19:20.856 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.use_direct_reads: 0 2024-09-12T03:19:20.856 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-12T03:19:20.856 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.create_missing_column_families: 0 2024-09-12T03:19:20.856 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.db_log_dir: 2024-09-12T03:19:20.856 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.wal_dir: 2024-09-12T03:19:20.856 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-12T03:19:20.857 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-12T03:19:20.857 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-12T03:19:20.857 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-12T03:19:20.857 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-12T03:19:20.857 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-12T03:19:20.858 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.advise_random_on_open: 1 2024-09-12T03:19:20.858 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.db_write_buffer_size: 0 2024-09-12T03:19:20.858 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.write_buffer_manager: 0x559e7a12edc0 2024-09-12T03:19:20.858 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-12T03:19:20.858 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-12T03:19:20.858 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-12T03:19:20.858 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.rate_limiter: (nil) 2024-09-12T03:19:20.858 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-12T03:19:20.858 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.wal_recovery_mode: 2 2024-09-12T03:19:20.858 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.enable_thread_tracking: 0 2024-09-12T03:19:20.858 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.enable_pipelined_write: 0 2024-09-12T03:19:20.858 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.unordered_write: 0 2024-09-12T03:19:20.858 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-12T03:19:20.859 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-12T03:19:20.859 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-12T03:19:20.859 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-12T03:19:20.859 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.row_cache: None 2024-09-12T03:19:20.859 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.wal_filter: None 2024-09-12T03:19:20.859 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-12T03:19:20.859 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.allow_ingest_behind: 0 2024-09-12T03:19:20.859 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.two_write_queues: 0 2024-09-12T03:19:20.859 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.manual_wal_flush: 0 2024-09-12T03:19:20.859 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.wal_compression: 0 2024-09-12T03:19:20.859 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.atomic_flush: 0 2024-09-12T03:19:20.859 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-12T03:19:20.859 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-12T03:19:20.860 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-12T03:19:20.860 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.log_readahead_size: 0 2024-09-12T03:19:20.860 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-12T03:19:20.860 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.best_efforts_recovery: 0 2024-09-12T03:19:20.860 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-12T03:19:20.860 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-12T03:19:20.860 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.allow_data_in_errors: 0 2024-09-12T03:19:20.860 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.db_host_id: __hostname__ 2024-09-12T03:19:20.860 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-12T03:19:20.860 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.max_background_jobs: 2 2024-09-12T03:19:20.860 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.max_background_compactions: -1 2024-09-12T03:19:20.860 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.max_subcompactions: 1 2024-09-12T03:19:20.860 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-12T03:19:20.860 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-12T03:19:20.861 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-12T03:19:20.861 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.max_total_wal_size: 0 2024-09-12T03:19:20.861 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-12T03:19:20.861 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-12T03:19:20.861 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-12T03:19:20.861 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-12T03:19:20.861 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.max_open_files: -1 2024-09-12T03:19:20.861 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.bytes_per_sync: 0 2024-09-12T03:19:20.861 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-12T03:19:20.861 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-12T03:19:20.861 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.compaction_readahead_size: 0 2024-09-12T03:19:20.861 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.max_background_flushes: -1 2024-09-12T03:19:20.861 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Compression algorithms supported: 2024-09-12T03:19:20.861 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: kZSTD supported: 0 2024-09-12T03:19:20.862 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: kXpressCompression supported: 0 2024-09-12T03:19:20.862 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: kBZip2Compression supported: 0 2024-09-12T03:19:20.862 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-12T03:19:20.862 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: kLZ4Compression supported: 1 2024-09-12T03:19:20.862 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: kZlibCompression supported: 1 2024-09-12T03:19:20.862 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: kLZ4HCCompression supported: 1 2024-09-12T03:19:20.862 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: kSnappyCompression supported: 1 2024-09-12T03:19:20.862 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-12T03:19:20.862 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-12T03:19:20.862 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-12T03:19:20.862 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-12T03:19:20.862 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-12T03:19:20.862 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.merge_operator: 2024-09-12T03:19:20.862 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.compaction_filter: None 2024-09-12T03:19:20.863 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.compaction_filter_factory: None 2024-09-12T03:19:20.863 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.sst_partitioner_factory: None 2024-09-12T03:19:20.863 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-12T03:19:20.863 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-12T03:19:20.863 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559e7a11f320) 2024-09-12T03:19:20.863 INFO:journalctl@ceph.mon.a.smithi064.stdout: cache_index_and_filter_blocks: 1 2024-09-12T03:19:20.863 INFO:journalctl@ceph.mon.a.smithi064.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-12T03:19:20.863 INFO:journalctl@ceph.mon.a.smithi064.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-12T03:19:20.863 INFO:journalctl@ceph.mon.a.smithi064.stdout: pin_top_level_index_and_filter: 1 2024-09-12T03:19:20.863 INFO:journalctl@ceph.mon.a.smithi064.stdout: index_type: 0 2024-09-12T03:19:20.863 INFO:journalctl@ceph.mon.a.smithi064.stdout: data_block_index_type: 0 2024-09-12T03:19:20.863 INFO:journalctl@ceph.mon.a.smithi064.stdout: index_shortening: 1 2024-09-12T03:19:20.863 INFO:journalctl@ceph.mon.a.smithi064.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-12T03:19:20.863 INFO:journalctl@ceph.mon.a.smithi064.stdout: checksum: 4 2024-09-12T03:19:20.864 INFO:journalctl@ceph.mon.a.smithi064.stdout: no_block_cache: 0 2024-09-12T03:19:20.864 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_cache: 0x559e7a1131f0 2024-09-12T03:19:20.864 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_cache_name: BinnedLRUCache 2024-09-12T03:19:20.864 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_cache_options: 2024-09-12T03:19:20.864 INFO:journalctl@ceph.mon.a.smithi064.stdout: capacity : 536870912 2024-09-12T03:19:20.864 INFO:journalctl@ceph.mon.a.smithi064.stdout: num_shard_bits : 4 2024-09-12T03:19:20.864 INFO:journalctl@ceph.mon.a.smithi064.stdout: strict_capacity_limit : 0 2024-09-12T03:19:20.864 INFO:journalctl@ceph.mon.a.smithi064.stdout: high_pri_pool_ratio: 0.000 2024-09-12T03:19:20.864 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_cache_compressed: (nil) 2024-09-12T03:19:20.864 INFO:journalctl@ceph.mon.a.smithi064.stdout: persistent_cache: (nil) 2024-09-12T03:19:20.864 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_size: 4096 2024-09-12T03:19:20.864 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_size_deviation: 10 2024-09-12T03:19:20.864 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_restart_interval: 16 2024-09-12T03:19:20.865 INFO:journalctl@ceph.mon.a.smithi064.stdout: index_block_restart_interval: 1 2024-09-12T03:19:20.865 INFO:journalctl@ceph.mon.a.smithi064.stdout: metadata_block_size: 4096 2024-09-12T03:19:20.865 INFO:journalctl@ceph.mon.a.smithi064.stdout: partition_filters: 0 2024-09-12T03:19:20.865 INFO:journalctl@ceph.mon.a.smithi064.stdout: use_delta_encoding: 1 2024-09-12T03:19:20.865 INFO:journalctl@ceph.mon.a.smithi064.stdout: filter_policy: bloomfilter 2024-09-12T03:19:20.865 INFO:journalctl@ceph.mon.a.smithi064.stdout: whole_key_filtering: 1 2024-09-12T03:19:20.865 INFO:journalctl@ceph.mon.a.smithi064.stdout: verify_compression: 0 2024-09-12T03:19:20.865 INFO:journalctl@ceph.mon.a.smithi064.stdout: read_amp_bytes_per_bit: 0 2024-09-12T03:19:20.865 INFO:journalctl@ceph.mon.a.smithi064.stdout: format_version: 5 2024-09-12T03:19:20.865 INFO:journalctl@ceph.mon.a.smithi064.stdout: enable_index_compression: 1 2024-09-12T03:19:20.865 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_align: 0 2024-09-12T03:19:20.865 INFO:journalctl@ceph.mon.a.smithi064.stdout: max_auto_readahead_size: 262144 2024-09-12T03:19:20.866 INFO:journalctl@ceph.mon.a.smithi064.stdout: prepopulate_block_cache: 0 2024-09-12T03:19:20.866 INFO:journalctl@ceph.mon.a.smithi064.stdout: initial_auto_readahead_size: 8192 2024-09-12T03:19:20.866 INFO:journalctl@ceph.mon.a.smithi064.stdout: num_file_reads_for_auto_readahead: 2 2024-09-12T03:19:20.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.write_buffer_size: 33554432 2024-09-12T03:19:20.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.max_write_buffer_number: 2 2024-09-12T03:19:20.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.compression: NoCompression 2024-09-12T03:19:20.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.bottommost_compression: Disabled 2024-09-12T03:19:20.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.prefix_extractor: nullptr 2024-09-12T03:19:20.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-12T03:19:20.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.num_levels: 7 2024-09-12T03:19:20.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-12T03:19:20.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-12T03:19:20.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-12T03:19:20.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-12T03:19:20.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-12T03:19:20.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-12T03:19:20.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-12T03:19:20.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-12T03:19:20.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-12T03:19:20.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-12T03:19:20.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-12T03:19:20.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-12T03:19:20.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-12T03:19:20.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.compression_opts.level: 32767 2024-09-12T03:19:20.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.compression_opts.strategy: 0 2024-09-12T03:19:20.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-12T03:19:20.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-12T03:19:20.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-12T03:19:20.867 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-12T03:19:20.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.compression_opts.enabled: false 2024-09-12T03:19:20.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-12T03:19:20.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-12T03:19:20.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-12T03:19:20.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-12T03:19:20.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.target_file_size_base: 67108864 2024-09-12T03:19:20.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-12T03:19:20.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-12T03:19:20.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-12T03:19:20.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-12T03:19:20.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-12T03:19:20.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-12T03:19:20.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-12T03:19:20.868 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-12T03:19:20.869 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-12T03:19:20.869 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-12T03:19:20.869 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-12T03:19:20.869 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-12T03:19:20.869 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-12T03:19:20.869 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-12T03:19:20.869 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.arena_block_size: 1048576 2024-09-12T03:19:20.869 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-12T03:19:20.869 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-12T03:19:20.869 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.disable_auto_compactions: 0 2024-09-12T03:19:20.869 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-12T03:19:20.869 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-12T03:19:20.869 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-12T03:19:20.869 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-12T03:19:20.870 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-12T03:19:20.870 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-12T03:19:20.870 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-12T03:19:20.870 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-12T03:19:20.870 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-12T03:19:20.870 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-12T03:19:20.870 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-12T03:19:20.870 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.inplace_update_support: 0 2024-09-12T03:19:20.870 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-12T03:19:20.870 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-12T03:19:20.870 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-12T03:19:20.870 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-12T03:19:20.870 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.bloom_locality: 0 2024-09-12T03:19:20.870 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.max_successive_merges: 0 2024-09-12T03:19:20.871 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-12T03:19:20.871 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.paranoid_file_checks: 0 2024-09-12T03:19:20.871 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.force_consistency_checks: 1 2024-09-12T03:19:20.871 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.report_bg_io_stats: 0 2024-09-12T03:19:20.871 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.ttl: 2592000 2024-09-12T03:19:20.871 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-12T03:19:20.871 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-12T03:19:20.871 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-12T03:19:20.871 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.enable_blob_files: false 2024-09-12T03:19:20.871 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.min_blob_size: 0 2024-09-12T03:19:20.871 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.blob_file_size: 268435456 2024-09-12T03:19:20.871 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-12T03:19:20.871 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-12T03:19:20.871 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-12T03:19:20.872 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-12T03:19:20.872 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-12T03:19:20.872 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.blob_file_starting_level: 0 2024-09-12T03:19:20.872 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-12T03:19:20.872 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-12T03:19:20.872 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-12T03:19:20.872 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f1943d25-febd-46aa-bd0c-0c945f629d3f 2024-09-12T03:19:20.872 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726111160607780, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-12T03:19:20.872 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-12T03:19:20.872 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726111160608882, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100337, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98221, "index_size": 308, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14288, "raw_average_key_size": 49, "raw_value_size": 90489, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726111160, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f1943d25-febd-46aa-bd0c-0c945f629d3f", "db_session_id": "HCLCXM3OF4TFU39P8GK9", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-12T03:19:20.872 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726111160608981, "job": 1, "event": "recovery_finished"} 2024-09-12T03:19:20.872 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-12T03:19:20.872 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-12T03:19:20.872 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x559e7a140e00 2024-09-12T03:19:20.873 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: DB pointer 0x559e7a23c000 2024-09-12T03:19:20.873 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-12T03:19:20.873 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-12T03:19:20.873 INFO:journalctl@ceph.mon.a.smithi064.stdout: ** DB Stats ** 2024-09-12T03:19:20.873 INFO:journalctl@ceph.mon.a.smithi064.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T03:19:20.873 INFO:journalctl@ceph.mon.a.smithi064.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-12T03:19:20.873 INFO:journalctl@ceph.mon.a.smithi064.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T03:19:20.873 INFO:journalctl@ceph.mon.a.smithi064.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T03:19:20.873 INFO:journalctl@ceph.mon.a.smithi064.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-12T03:19:20.873 INFO:journalctl@ceph.mon.a.smithi064.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T03:19:20.873 INFO:journalctl@ceph.mon.a.smithi064.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T03:19:20.873 INFO:journalctl@ceph.mon.a.smithi064.stdout: 2024-09-12T03:19:20.873 INFO:journalctl@ceph.mon.a.smithi064.stdout: ** Compaction Stats [default] ** 2024-09-12T03:19:20.873 INFO:journalctl@ceph.mon.a.smithi064.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T03:19:20.874 INFO:journalctl@ceph.mon.a.smithi064.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-12T03:19:20.874 INFO:journalctl@ceph.mon.a.smithi064.stdout: L0 2/0 99.85 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 138.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T03:19:20.874 INFO:journalctl@ceph.mon.a.smithi064.stdout: Sum 2/0 99.85 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 138.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T03:19:20.874 INFO:journalctl@ceph.mon.a.smithi064.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 138.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T03:19:20.874 INFO:journalctl@ceph.mon.a.smithi064.stdout: 2024-09-12T03:19:20.874 INFO:journalctl@ceph.mon.a.smithi064.stdout: ** Compaction Stats [default] ** 2024-09-12T03:19:20.874 INFO:journalctl@ceph.mon.a.smithi064.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T03:19:20.874 INFO:journalctl@ceph.mon.a.smithi064.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-12T03:19:20.874 INFO:journalctl@ceph.mon.a.smithi064.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 138.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T03:19:20.874 INFO:journalctl@ceph.mon.a.smithi064.stdout: 2024-09-12T03:19:20.874 INFO:journalctl@ceph.mon.a.smithi064.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-12T03:19:20.874 INFO:journalctl@ceph.mon.a.smithi064.stdout: 2024-09-12T03:19:20.874 INFO:journalctl@ceph.mon.a.smithi064.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T03:19:20.874 INFO:journalctl@ceph.mon.a.smithi064.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-12T03:19:20.875 INFO:journalctl@ceph.mon.a.smithi064.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-12T03:19:20.875 INFO:journalctl@ceph.mon.a.smithi064.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-12T03:19:20.875 INFO:journalctl@ceph.mon.a.smithi064.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-12T03:19:20.875 INFO:journalctl@ceph.mon.a.smithi064.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-12T03:19:20.875 INFO:journalctl@ceph.mon.a.smithi064.stdout: Cumulative compaction: 0.00 GB write, 28.97 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T03:19:20.875 INFO:journalctl@ceph.mon.a.smithi064.stdout: Interval compaction: 0.00 GB write, 28.97 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T03:19:20.875 INFO:journalctl@ceph.mon.a.smithi064.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-12T03:19:20.875 INFO:journalctl@ceph.mon.a.smithi064.stdout: Block cache BinnedLRUCache@0x559e7a1131f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-12T03:19:20.875 INFO:journalctl@ceph.mon.a.smithi064.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-12T03:19:20.875 INFO:journalctl@ceph.mon.a.smithi064.stdout: 2024-09-12T03:19:20.875 INFO:journalctl@ceph.mon.a.smithi064.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-12T03:19:20.875 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: starting mon.a rank 0 at public addrs [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] at bind addrs [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid b0812460-70b5-11ef-bcea-c7b262605968 2024-09-12T03:19:20.875 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: mon.a@-1(???) e1 preinit fsid b0812460-70b5-11ef-bcea-c7b262605968 2024-09-12T03:19:20.875 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: mon.a@-1(???).mds e1 new map 2024-09-12T03:19:20.876 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: mon.a@-1(???).mds e1 print_map 2024-09-12T03:19:20.876 INFO:journalctl@ceph.mon.a.smithi064.stdout: e1 2024-09-12T03:19:20.876 INFO:journalctl@ceph.mon.a.smithi064.stdout: btime 2024-09-12T03:19:15:900094+0000 2024-09-12T03:19:20.876 INFO:journalctl@ceph.mon.a.smithi064.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T03:19:20.876 INFO:journalctl@ceph.mon.a.smithi064.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T03:19:20.876 INFO:journalctl@ceph.mon.a.smithi064.stdout: legacy client fscid: -1 2024-09-12T03:19:20.876 INFO:journalctl@ceph.mon.a.smithi064.stdout: 2024-09-12T03:19:20.876 INFO:journalctl@ceph.mon.a.smithi064.stdout: No filesystems configured 2024-09-12T03:19:20.877 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-12T03:19:20.877 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T03:19:20.877 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T03:19:20.877 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T03:19:20.877 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-12T03:19:20.877 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 bash[30933]: 9aa76b73f7af2ea7831e04fc80191f235927cc5c7db6073b4e687510da1b6a80 2024-09-12T03:19:20.877 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 systemd[1]: Started Ceph mon.a for b0812460-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:19:20.877 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-12T03:19:20.877 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: monmap epoch 1 2024-09-12T03:19:20.877 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: fsid b0812460-70b5-11ef-bcea-c7b262605968 2024-09-12T03:19:20.877 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: last_changed 2024-09-12T03:19:13.042174+0000 2024-09-12T03:19:20.878 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: created 2024-09-12T03:19:13.042174+0000 2024-09-12T03:19:20.878 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: min_mon_release 19 (squid) 2024-09-12T03:19:20.878 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: election_strategy: 1 2024-09-12T03:19:20.878 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: 0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2024-09-12T03:19:20.878 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: fsmap 2024-09-12T03:19:20.878 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: osdmap e1: 0 total, 0 up, 0 in 2024-09-12T03:19:20.878 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:20 smithi064 ceph-mon[30952]: mgrmap e1: no daemons active 2024-09-12T03:19:21.671 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.039+0000 7f2884921640 1 Processor -- start 2024-09-12T03:19:21.678 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.039+0000 7f2884921640 1 -- start start 2024-09-12T03:19:21.679 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.040+0000 7f2884921640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2880106400 0x7f2880106800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:21.679 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.040+0000 7f2884921640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2880106dd0 con 0x7f2880106400 2024-09-12T03:19:21.679 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.040+0000 7f287e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2880106400 0x7f2880106800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:21.679 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.040+0000 7f287e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2880106400 0x7f2880106800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44840/0 (socket says 172.21.15.64:44840) 2024-09-12T03:19:21.679 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.040+0000 7f287e59c640 1 -- 172.21.15.64:0/307788978 learned_addr learned my addr 172.21.15.64:0/307788978 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:19:21.679 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.040+0000 7f287e59c640 1 -- 172.21.15.64:0/307788978 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2880107600 con 0x7f2880106400 2024-09-12T03:19:21.679 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.042+0000 7f287e59c640 1 --2- 172.21.15.64:0/307788978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2880106400 0x7f2880106800 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2870009820 tx=0x7f287002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb4581253344534 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:21.679 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.042+0000 7f287d59a640 1 -- 172.21.15.64:0/307788978 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2870037070 con 0x7f2880106400 2024-09-12T03:19:21.679 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.042+0000 7f287d59a640 1 -- 172.21.15.64:0/307788978 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f28700328d0 con 0x7f2880106400 2024-09-12T03:19:21.680 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.042+0000 7f287d59a640 1 -- 172.21.15.64:0/307788978 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f28700307d0 con 0x7f2880106400 2024-09-12T03:19:21.680 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.043+0000 7f2884921640 1 -- 172.21.15.64:0/307788978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2880106400 msgr2=0x7f2880106800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:21.680 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.043+0000 7f2884921640 1 --2- 172.21.15.64:0/307788978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2880106400 0x7f2880106800 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2870009820 tx=0x7f287002a270 comp rx=0 tx=0).stop 2024-09-12T03:19:21.680 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.043+0000 7f2884921640 1 -- 172.21.15.64:0/307788978 shutdown_connections 2024-09-12T03:19:21.681 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.043+0000 7f2884921640 1 --2- 172.21.15.64:0/307788978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2880106400 0x7f2880106800 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:21.681 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.043+0000 7f2884921640 1 -- 172.21.15.64:0/307788978 >> 172.21.15.64:0/307788978 conn(0x7f2880101b70 msgr2=0x7f2880103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:21.681 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.043+0000 7f2884921640 1 -- 172.21.15.64:0/307788978 shutdown_connections 2024-09-12T03:19:21.681 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.043+0000 7f2884921640 1 -- 172.21.15.64:0/307788978 wait complete. 2024-09-12T03:19:21.681 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.044+0000 7f2884921640 1 Processor -- start 2024-09-12T03:19:21.681 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.044+0000 7f2884921640 1 -- start start 2024-09-12T03:19:21.681 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.044+0000 7f2884921640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2880106400 0x7f288019b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:21.681 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.044+0000 7f2884921640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f288010a9b0 con 0x7f2880106400 2024-09-12T03:19:21.681 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.044+0000 7f287e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2880106400 0x7f288019b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:21.681 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.044+0000 7f287e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2880106400 0x7f288019b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44854/0 (socket says 172.21.15.64:44854) 2024-09-12T03:19:21.681 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.044+0000 7f287e59c640 1 -- 172.21.15.64:0/239967097 learned_addr learned my addr 172.21.15.64:0/239967097 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:19:21.681 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.044+0000 7f287e59c640 1 -- 172.21.15.64:0/239967097 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f288019ba80 con 0x7f2880106400 2024-09-12T03:19:21.682 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.045+0000 7f287e59c640 1 --2- 172.21.15.64:0/239967097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2880106400 0x7f288019b540 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2870030520 tx=0x7f2870032dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:21.682 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.045+0000 7f286f7fe640 1 -- 172.21.15.64:0/239967097 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2870042070 con 0x7f2880106400 2024-09-12T03:19:21.682 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.045+0000 7f286f7fe640 1 -- 172.21.15.64:0/239967097 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f2870030cc0 con 0x7f2880106400 2024-09-12T03:19:21.682 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.045+0000 7f2884921640 1 -- 172.21.15.64:0/239967097 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f288019bd10 con 0x7f2880106400 2024-09-12T03:19:21.682 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.045+0000 7f286f7fe640 1 -- 172.21.15.64:0/239967097 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2870037040 con 0x7f2880106400 2024-09-12T03:19:21.682 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.045+0000 7f2884921640 1 -- 172.21.15.64:0/239967097 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f288019ea00 con 0x7f2880106400 2024-09-12T03:19:21.682 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.045+0000 7f286f7fe640 1 -- 172.21.15.64:0/239967097 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2870045070 con 0x7f2880106400 2024-09-12T03:19:21.682 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.045+0000 7f286f7fe640 1 -- 172.21.15.64:0/239967097 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f28700479a0 con 0x7f2880106400 2024-09-12T03:19:21.682 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.046+0000 7f2884921640 1 -- 172.21.15.64:0/239967097 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f288010ab30 con 0x7f2880106400 2024-09-12T03:19:21.682 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.048+0000 7f286f7fe640 1 -- 172.21.15.64:0/239967097 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f287003d9c0 con 0x7f2880106400 2024-09-12T03:19:21.682 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.107+0000 7f2884921640 1 -- 172.21.15.64:0/239967097 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f2880106880 con 0x7f2880106400 2024-09-12T03:19:21.682 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.112+0000 7f286f7fe640 1 -- 172.21.15.64:0/239967097 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f2870047ba0 con 0x7f2880106400 2024-09-12T03:19:21.683 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.113+0000 7f2884921640 1 -- 172.21.15.64:0/239967097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2880106400 msgr2=0x7f288019b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:21.683 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.113+0000 7f2884921640 1 --2- 172.21.15.64:0/239967097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2880106400 0x7f288019b540 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2870030520 tx=0x7f2870032dd0 comp rx=0 tx=0).stop 2024-09-12T03:19:21.683 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.113+0000 7f2884921640 1 -- 172.21.15.64:0/239967097 shutdown_connections 2024-09-12T03:19:21.683 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.113+0000 7f2884921640 1 --2- 172.21.15.64:0/239967097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2880106400 0x7f288019b540 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:21.683 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.113+0000 7f2884921640 1 -- 172.21.15.64:0/239967097 >> 172.21.15.64:0/239967097 conn(0x7f2880101b70 msgr2=0x7f2880102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:21.683 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.113+0000 7f2884921640 1 -- 172.21.15.64:0/239967097 shutdown_connections 2024-09-12T03:19:21.683 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:21.113+0000 7f2884921640 1 -- 172.21.15.64:0/239967097 wait complete. 2024-09-12T03:19:21.683 INFO:teuthology.orchestra.run.smithi064.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-12T03:19:21.701 INFO:teuthology.orchestra.run.smithi064.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:19:21.701 INFO:teuthology.orchestra.run.smithi064.stdout:Creating mgr... 2024-09-12T03:19:21.701 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-12T03:19:21.702 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-12T03:19:21.965 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b0812460-70b5-11ef-bcea-c7b262605968@mgr.x 2024-09-12T03:19:21.965 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b0812460-70b5-11ef-bcea-c7b262605968@mgr.x.service: Unit ceph-b0812460-70b5-11ef-bcea-c7b262605968@mgr.x.service not loaded. 2024-09-12T03:19:22.125 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b0812460-70b5-11ef-bcea-c7b262605968.target.wants/ceph-b0812460-70b5-11ef-bcea-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-b0812460-70b5-11ef-bcea-c7b262605968@.service. 2024-09-12T03:19:22.408 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:22 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/239967097' entity='client.admin' 2024-09-12T03:19:22.611 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T03:19:22.611 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout disabled 2024-09-12T03:19:22.618 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T03:19:22.619 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2024-09-12T03:19:22.619 INFO:teuthology.orchestra.run.smithi064.stdout:firewalld.service is not enabled 2024-09-12T03:19:22.619 INFO:teuthology.orchestra.run.smithi064.stdout:Not possible to enable service . firewalld.service is not available 2024-09-12T03:19:22.630 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T03:19:22.630 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout disabled 2024-09-12T03:19:22.639 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T03:19:22.639 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2024-09-12T03:19:22.639 INFO:teuthology.orchestra.run.smithi064.stdout:firewalld.service is not enabled 2024-09-12T03:19:22.639 INFO:teuthology.orchestra.run.smithi064.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-12T03:19:22.639 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mgr to start... 2024-09-12T03:19:22.640 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mgr... 2024-09-12T03:19:22.707 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:22 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:22.645+0000 7f6ab550c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T03:19:22.708 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:22 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:22.706+0000 7f6ab550c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T03:19:22.991 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:22 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:22.866+0000 7f6ab550c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T03:19:22.992 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:22 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:22.936+0000 7f6ab550c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T03:19:23.246 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:23 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/2453148499' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T03:19:23.246 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:23 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:23.097+0000 7f6ab550c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:19:23.246 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:23 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:23.245+0000 7f6ab550c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T03:19:23.720 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-09-12T03:19:23.721 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-09-12T03:19:23.721 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsid": "b0812460-70b5-11ef-bcea-c7b262605968", 2024-09-12T03:19:23.721 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T03:19:23.721 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T03:19:23.721 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T03:19:23.722 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T03:19:23.722 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:19:23.722 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T03:19:23.722 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T03:19:23.722 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 0 2024-09-12T03:19:23.723 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-09-12T03:19:23.723 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T03:19:23.723 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T03:19:23.723 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-09-12T03:19:23.723 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-12T03:19:23.723 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T03:19:23.724 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:19:23.724 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T03:19:23.724 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T03:19:23.724 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:19:23.724 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T03:19:23.724 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:19:23.725 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T03:19:23.725 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T03:19:23.725 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T03:19:23.725 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T03:19:23.725 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T03:19:23.725 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T03:19:23.726 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:19:23.726 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T03:19:23.726 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T03:19:23.727 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T03:19:23.727 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T03:19:23.727 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T03:19:23.727 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T03:19:23.727 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T03:19:23.727 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T03:19:23.728 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T03:19:23.728 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:19:23.728 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T03:19:23.728 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:19:23.728 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T03:19:15:900094+0000", 2024-09-12T03:19:23.728 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T03:19:23.729 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T03:19:23.729 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:19:23.729 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T03:19:23.729 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-12T03:19:23.729 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T03:19:23.729 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T03:19:23.730 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T03:19:23.730 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T03:19:23.730 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T03:19:23.730 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-09-12T03:19:23.730 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T03:19:23.731 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:19:23.731 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T03:19:23.731 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:19:23.731 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T03:19:15.900949+0000", 2024-09-12T03:19:23.731 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T03:19:23.731 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:19:23.732 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T03:19:23.732 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-09-12T03:19:23.732 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.014+0000 7ffb6a14d640 1 Processor -- start 2024-09-12T03:19:23.732 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.014+0000 7ffb6a14d640 1 -- start start 2024-09-12T03:19:23.732 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.014+0000 7ffb6a14d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb64107f70 0x7ffb6410a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:23.733 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.014+0000 7ffb6a14d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6407be80 con 0x7ffb64107f70 2024-09-12T03:19:23.733 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.015+0000 7ffb637fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb64107f70 0x7ffb6410a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:23.733 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.015+0000 7ffb637fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb64107f70 0x7ffb6410a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44862/0 (socket says 172.21.15.64:44862) 2024-09-12T03:19:23.733 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.015+0000 7ffb637fe640 1 -- 172.21.15.64:0/366867144 learned_addr learned my addr 172.21.15.64:0/366867144 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:19:23.733 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.015+0000 7ffb637fe640 1 -- 172.21.15.64:0/366867144 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb6410a8a0 con 0x7ffb64107f70 2024-09-12T03:19:23.733 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.016+0000 7ffb637fe640 1 --2- 172.21.15.64:0/366867144 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb64107f70 0x7ffb6410a360 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ffb54009820 tx=0x7ffb5402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75dda51ffd92dc65 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:23.734 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.016+0000 7ffb627fc640 1 -- 172.21.15.64:0/366867144 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb54037070 con 0x7ffb64107f70 2024-09-12T03:19:23.734 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.016+0000 7ffb627fc640 1 -- 172.21.15.64:0/366867144 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ffb540328d0 con 0x7ffb64107f70 2024-09-12T03:19:23.734 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.016+0000 7ffb627fc640 1 -- 172.21.15.64:0/366867144 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb540307d0 con 0x7ffb64107f70 2024-09-12T03:19:23.734 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.017+0000 7ffb6a14d640 1 -- 172.21.15.64:0/366867144 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb64107f70 msgr2=0x7ffb6410a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:23.735 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.017+0000 7ffb6a14d640 1 --2- 172.21.15.64:0/366867144 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb64107f70 0x7ffb6410a360 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ffb54009820 tx=0x7ffb5402a270 comp rx=0 tx=0).stop 2024-09-12T03:19:23.735 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.017+0000 7ffb6a14d640 1 -- 172.21.15.64:0/366867144 shutdown_connections 2024-09-12T03:19:23.735 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.017+0000 7ffb6a14d640 1 --2- 172.21.15.64:0/366867144 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb64107f70 0x7ffb6410a360 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:23.736 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.017+0000 7ffb6a14d640 1 -- 172.21.15.64:0/366867144 >> 172.21.15.64:0/366867144 conn(0x7ffb64101b70 msgr2=0x7ffb64103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:23.736 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.017+0000 7ffb6a14d640 1 -- 172.21.15.64:0/366867144 shutdown_connections 2024-09-12T03:19:23.736 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.017+0000 7ffb6a14d640 1 -- 172.21.15.64:0/366867144 wait complete. 2024-09-12T03:19:23.736 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.018+0000 7ffb6a14d640 1 Processor -- start 2024-09-12T03:19:23.736 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.018+0000 7ffb6a14d640 1 -- start start 2024-09-12T03:19:23.736 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.018+0000 7ffb6a14d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb64107f70 0x7ffb641992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:23.737 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.018+0000 7ffb6a14d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffb64106910 con 0x7ffb64107f70 2024-09-12T03:19:23.737 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.018+0000 7ffb637fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb64107f70 0x7ffb641992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:23.737 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.018+0000 7ffb637fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb64107f70 0x7ffb641992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44874/0 (socket says 172.21.15.64:44874) 2024-09-12T03:19:23.737 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.018+0000 7ffb637fe640 1 -- 172.21.15.64:0/2453148499 learned_addr learned my addr 172.21.15.64:0/2453148499 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:19:23.737 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.018+0000 7ffb637fe640 1 -- 172.21.15.64:0/2453148499 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb641997f0 con 0x7ffb64107f70 2024-09-12T03:19:23.737 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.019+0000 7ffb637fe640 1 --2- 172.21.15.64:0/2453148499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb64107f70 0x7ffb641992b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ffb540097f0 tx=0x7ffb54030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:23.738 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.019+0000 7ffb60ff9640 1 -- 172.21.15.64:0/2453148499 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb54037040 con 0x7ffb64107f70 2024-09-12T03:19:23.738 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.019+0000 7ffb60ff9640 1 -- 172.21.15.64:0/2453148499 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ffb5403b3f0 con 0x7ffb64107f70 2024-09-12T03:19:23.738 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.019+0000 7ffb6a14d640 1 -- 172.21.15.64:0/2453148499 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb64199a80 con 0x7ffb64107f70 2024-09-12T03:19:23.738 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.019+0000 7ffb60ff9640 1 -- 172.21.15.64:0/2453148499 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb5403a520 con 0x7ffb64107f70 2024-09-12T03:19:23.738 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.019+0000 7ffb6a14d640 1 -- 172.21.15.64:0/2453148499 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb6419c770 con 0x7ffb64107f70 2024-09-12T03:19:23.738 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.019+0000 7ffb60ff9640 1 -- 172.21.15.64:0/2453148499 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ffb5403a710 con 0x7ffb64107f70 2024-09-12T03:19:23.739 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.019+0000 7ffb60ff9640 1 -- 172.21.15.64:0/2453148499 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ffb54044b50 con 0x7ffb64107f70 2024-09-12T03:19:23.739 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.020+0000 7ffb6a14d640 1 -- 172.21.15.64:0/2453148499 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb64106fb0 con 0x7ffb64107f70 2024-09-12T03:19:23.739 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.022+0000 7ffb60ff9640 1 -- 172.21.15.64:0/2453148499 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7ffb5403b980 con 0x7ffb64107f70 2024-09-12T03:19:23.739 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.085+0000 7ffb6a14d640 1 -- 172.21.15.64:0/2453148499 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ffb6402d940 con 0x7ffb64107f70 2024-09-12T03:19:23.739 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.085+0000 7ffb60ff9640 1 -- 172.21.15.64:0/2453148499 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7ffb5403b5a0 con 0x7ffb64107f70 2024-09-12T03:19:23.739 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.086+0000 7ffb6a14d640 1 -- 172.21.15.64:0/2453148499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb64107f70 msgr2=0x7ffb641992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:23.740 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.086+0000 7ffb6a14d640 1 --2- 172.21.15.64:0/2453148499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb64107f70 0x7ffb641992b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ffb540097f0 tx=0x7ffb54030d00 comp rx=0 tx=0).stop 2024-09-12T03:19:23.740 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.086+0000 7ffb6a14d640 1 -- 172.21.15.64:0/2453148499 shutdown_connections 2024-09-12T03:19:23.740 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.086+0000 7ffb6a14d640 1 --2- 172.21.15.64:0/2453148499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb64107f70 0x7ffb641992b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:23.740 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.086+0000 7ffb6a14d640 1 -- 172.21.15.64:0/2453148499 >> 172.21.15.64:0/2453148499 conn(0x7ffb64101b70 msgr2=0x7ffb64102480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:23.740 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.086+0000 7ffb6a14d640 1 -- 172.21.15.64:0/2453148499 shutdown_connections 2024-09-12T03:19:23.741 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:23.086+0000 7ffb6a14d640 1 -- 172.21.15.64:0/2453148499 wait complete. 2024-09-12T03:19:23.741 INFO:teuthology.orchestra.run.smithi064.stdout:mgr not available, waiting (1/15)... 2024-09-12T03:19:23.772 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:23 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:23.579+0000 7f6ab550c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T03:19:23.772 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:23 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:23.643+0000 7f6ab550c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T03:19:24.272 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:23 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:23.837+0000 7f6ab550c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T03:19:25.022 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:24 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:24.760+0000 7f6ab550c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T03:19:25.022 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:24 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:24.830+0000 7f6ab550c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T03:19:25.837 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:25 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:25.585+0000 7f6ab550c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T03:19:25.837 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:25 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:25.705+0000 7f6ab550c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T03:19:26.095 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:25 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:25.866+0000 7f6ab550c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:19:26.095 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:25 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:25.932+0000 7f6ab550c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T03:19:26.095 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:26 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:26.094+0000 7f6ab550c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T03:19:26.409 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:26 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/3645171504' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T03:19:26.410 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:26 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:26.169+0000 7f6ab550c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T03:19:26.410 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:26 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:26.240+0000 7f6ab550c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T03:19:26.410 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:26 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:26.406+0000 7f6ab550c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T03:19:26.670 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:26 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:26.478+0000 7f6ab550c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T03:19:26.671 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:26 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:26.549+0000 7f6ab550c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T03:19:26.723 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-09-12T03:19:26.723 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-09-12T03:19:26.724 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsid": "b0812460-70b5-11ef-bcea-c7b262605968", 2024-09-12T03:19:26.724 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T03:19:26.724 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T03:19:26.724 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T03:19:26.724 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T03:19:26.724 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:19:26.724 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T03:19:26.724 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T03:19:26.724 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 0 2024-09-12T03:19:26.724 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-09-12T03:19:26.724 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T03:19:26.724 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T03:19:26.724 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-09-12T03:19:26.725 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-12T03:19:26.725 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T03:19:26.725 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:19:26.725 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T03:19:26.725 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T03:19:26.725 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:19:26.725 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T03:19:26.725 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:19:26.725 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T03:19:26.725 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T03:19:26.725 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T03:19:26.725 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T03:19:26.725 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T03:19:26.726 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T03:19:26.726 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:19:26.726 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T03:19:26.726 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T03:19:26.726 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T03:19:26.726 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T03:19:26.726 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T03:19:26.726 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T03:19:26.726 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T03:19:26.726 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T03:19:26.726 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T03:19:26.726 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:19:26.727 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T03:19:26.727 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:19:26.727 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T03:19:15:900094+0000", 2024-09-12T03:19:26.727 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T03:19:26.727 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T03:19:26.727 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:19:26.727 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T03:19:26.727 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-12T03:19:26.727 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T03:19:26.727 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T03:19:26.727 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T03:19:26.727 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T03:19:26.727 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T03:19:26.728 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-09-12T03:19:26.728 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T03:19:26.728 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:19:26.728 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T03:19:26.728 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:19:26.728 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T03:19:15.900949+0000", 2024-09-12T03:19:26.728 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T03:19:26.728 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:19:26.728 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T03:19:26.728 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-09-12T03:19:26.728 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.119+0000 7f5962f0f640 1 Processor -- start 2024-09-12T03:19:26.728 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.119+0000 7f5962f0f640 1 -- start start 2024-09-12T03:19:26.728 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.119+0000 7f5962f0f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f595c107f70 0x7f595c10a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:26.729 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.119+0000 7f5962f0f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f595c07be80 con 0x7f595c107f70 2024-09-12T03:19:26.729 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.119+0000 7f5960cab640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f595c107f70 0x7f595c10a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:26.729 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.119+0000 7f5960cab640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f595c107f70 0x7f595c10a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44878/0 (socket says 172.21.15.64:44878) 2024-09-12T03:19:26.729 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.119+0000 7f5960cab640 1 -- 172.21.15.64:0/2447394245 learned_addr learned my addr 172.21.15.64:0/2447394245 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:19:26.729 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.119+0000 7f5960cab640 1 -- 172.21.15.64:0/2447394245 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f595c10a8a0 con 0x7f595c107f70 2024-09-12T03:19:26.729 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.120+0000 7f5960cab640 1 --2- 172.21.15.64:0/2447394245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f595c107f70 0x7f595c10a360 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5950009820 tx=0x7f595002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74666c4c0f07f19c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:26.729 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.121+0000 7f595b7fe640 1 -- 172.21.15.64:0/2447394245 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5950037070 con 0x7f595c107f70 2024-09-12T03:19:26.729 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.121+0000 7f595b7fe640 1 -- 172.21.15.64:0/2447394245 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f59500328d0 con 0x7f595c107f70 2024-09-12T03:19:26.729 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.121+0000 7f595b7fe640 1 -- 172.21.15.64:0/2447394245 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59500307d0 con 0x7f595c107f70 2024-09-12T03:19:26.729 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.121+0000 7f5962f0f640 1 -- 172.21.15.64:0/2447394245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f595c107f70 msgr2=0x7f595c10a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:26.729 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.121+0000 7f5962f0f640 1 --2- 172.21.15.64:0/2447394245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f595c107f70 0x7f595c10a360 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5950009820 tx=0x7f595002a270 comp rx=0 tx=0).stop 2024-09-12T03:19:26.729 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.121+0000 7f5962f0f640 1 -- 172.21.15.64:0/2447394245 shutdown_connections 2024-09-12T03:19:26.729 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.121+0000 7f5962f0f640 1 --2- 172.21.15.64:0/2447394245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f595c107f70 0x7f595c10a360 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.729 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.121+0000 7f5962f0f640 1 -- 172.21.15.64:0/2447394245 >> 172.21.15.64:0/2447394245 conn(0x7f595c101b70 msgr2=0x7f595c103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:26.730 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.122+0000 7f5962f0f640 1 -- 172.21.15.64:0/2447394245 shutdown_connections 2024-09-12T03:19:26.730 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.122+0000 7f5962f0f640 1 -- 172.21.15.64:0/2447394245 wait complete. 2024-09-12T03:19:26.730 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.122+0000 7f5962f0f640 1 Processor -- start 2024-09-12T03:19:26.730 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.122+0000 7f5962f0f640 1 -- start start 2024-09-12T03:19:26.730 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.122+0000 7f5962f0f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f595c107f70 0x7f595c19b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:26.730 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.122+0000 7f5962f0f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f595c10b8e0 con 0x7f595c107f70 2024-09-12T03:19:26.730 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.123+0000 7f5960cab640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f595c107f70 0x7f595c19b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:26.730 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.123+0000 7f5960cab640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f595c107f70 0x7f595c19b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44886/0 (socket says 172.21.15.64:44886) 2024-09-12T03:19:26.730 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.123+0000 7f5960cab640 1 -- 172.21.15.64:0/3645171504 learned_addr learned my addr 172.21.15.64:0/3645171504 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:19:26.730 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.123+0000 7f5960cab640 1 -- 172.21.15.64:0/3645171504 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f595c19ba00 con 0x7f595c107f70 2024-09-12T03:19:26.730 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.123+0000 7f5960cab640 1 --2- 172.21.15.64:0/3645171504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f595c107f70 0x7f595c19b4c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5950030520 tx=0x7f5950032dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:26.730 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.123+0000 7f5959ffb640 1 -- 172.21.15.64:0/3645171504 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5950042070 con 0x7f595c107f70 2024-09-12T03:19:26.730 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.123+0000 7f5959ffb640 1 -- 172.21.15.64:0/3645171504 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f5950030cc0 con 0x7f595c107f70 2024-09-12T03:19:26.731 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.123+0000 7f5962f0f640 1 -- 172.21.15.64:0/3645171504 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f595c19bc90 con 0x7f595c107f70 2024-09-12T03:19:26.731 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.123+0000 7f5959ffb640 1 -- 172.21.15.64:0/3645171504 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5950037040 con 0x7f595c107f70 2024-09-12T03:19:26.731 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.124+0000 7f5962f0f640 1 -- 172.21.15.64:0/3645171504 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f595c19e980 con 0x7f595c107f70 2024-09-12T03:19:26.731 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.124+0000 7f5959ffb640 1 -- 172.21.15.64:0/3645171504 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5950045070 con 0x7f595c107f70 2024-09-12T03:19:26.731 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.124+0000 7f5959ffb640 1 -- 172.21.15.64:0/3645171504 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5950047df0 con 0x7f595c107f70 2024-09-12T03:19:26.731 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.125+0000 7f5962f0f640 1 -- 172.21.15.64:0/3645171504 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f595c1083d0 con 0x7f595c107f70 2024-09-12T03:19:26.731 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.127+0000 7f5959ffb640 1 -- 172.21.15.64:0/3645171504 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f5950047440 con 0x7f595c107f70 2024-09-12T03:19:26.731 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.190+0000 7f5962f0f640 1 -- 172.21.15.64:0/3645171504 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f595c19eb50 con 0x7f595c107f70 2024-09-12T03:19:26.731 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.190+0000 7f5959ffb640 1 -- 172.21.15.64:0/3645171504 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f595003c790 con 0x7f595c107f70 2024-09-12T03:19:26.732 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.191+0000 7f5962f0f640 1 -- 172.21.15.64:0/3645171504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f595c107f70 msgr2=0x7f595c19b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:26.732 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.191+0000 7f5962f0f640 1 --2- 172.21.15.64:0/3645171504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f595c107f70 0x7f595c19b4c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5950030520 tx=0x7f5950032dd0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.732 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.191+0000 7f5962f0f640 1 -- 172.21.15.64:0/3645171504 shutdown_connections 2024-09-12T03:19:26.732 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.191+0000 7f5962f0f640 1 --2- 172.21.15.64:0/3645171504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f595c107f70 0x7f595c19b4c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.732 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.191+0000 7f5962f0f640 1 -- 172.21.15.64:0/3645171504 >> 172.21.15.64:0/3645171504 conn(0x7f595c101b70 msgr2=0x7f595c102770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:26.732 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.191+0000 7f5962f0f640 1 -- 172.21.15.64:0/3645171504 shutdown_connections 2024-09-12T03:19:26.732 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:26.191+0000 7f5962f0f640 1 -- 172.21.15.64:0/3645171504 wait complete. 2024-09-12T03:19:26.732 INFO:teuthology.orchestra.run.smithi064.stdout:mgr not available, waiting (2/15)... 2024-09-12T03:19:27.022 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:26 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:26.670+0000 7f6ab550c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T03:19:27.022 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:26 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:26.735+0000 7f6ab550c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T03:19:27.022 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:26 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:26.796+0000 7f6ab550c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T03:19:28.022 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:27 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:27.625+0000 7f6ab550c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T03:19:28.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:27 smithi064 ceph-mon[30952]: Activating manager daemon x 2024-09-12T03:19:28.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:27 smithi064 ceph-mon[30952]: mgrmap e2: x(active, starting, since 0.00309557s) 2024-09-12T03:19:28.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:27 smithi064 ceph-mon[30952]: from='mgr.14100 172.21.15.64:0/1316290174' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T03:19:28.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:27 smithi064 ceph-mon[30952]: from='mgr.14100 172.21.15.64:0/1316290174' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T03:19:28.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:27 smithi064 ceph-mon[30952]: from='mgr.14100 172.21.15.64:0/1316290174' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T03:19:28.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:27 smithi064 ceph-mon[30952]: from='mgr.14100 172.21.15.64:0/1316290174' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:19:28.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:27 smithi064 ceph-mon[30952]: from='mgr.14100 172.21.15.64:0/1316290174' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T03:19:28.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:27 smithi064 ceph-mon[30952]: Manager daemon x is now available 2024-09-12T03:19:28.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:27 smithi064 ceph-mon[30952]: from='mgr.14100 172.21.15.64:0/1316290174' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T03:19:28.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:27 smithi064 ceph-mon[30952]: from='mgr.14100 172.21.15.64:0/1316290174' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T03:19:28.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:27 smithi064 ceph-mon[30952]: from='mgr.14100 172.21.15.64:0/1316290174' entity='mgr.x' 2024-09-12T03:19:28.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:27 smithi064 ceph-mon[30952]: from='mgr.14100 172.21.15.64:0/1316290174' entity='mgr.x' 2024-09-12T03:19:28.024 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:27 smithi064 ceph-mon[30952]: from='mgr.14100 172.21.15.64:0/1316290174' entity='mgr.x' 2024-09-12T03:19:29.878 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-09-12T03:19:29.879 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-09-12T03:19:29.879 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsid": "b0812460-70b5-11ef-bcea-c7b262605968", 2024-09-12T03:19:29.879 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T03:19:29.879 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T03:19:29.879 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T03:19:29.879 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T03:19:29.879 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:19:29.879 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T03:19:29.879 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T03:19:29.879 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 0 2024-09-12T03:19:29.879 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-09-12T03:19:29.880 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T03:19:29.880 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T03:19:29.880 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-09-12T03:19:29.880 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-09-12T03:19:29.880 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T03:19:29.880 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:19:29.880 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T03:19:29.880 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T03:19:29.880 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:19:29.880 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T03:19:29.881 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:19:29.881 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T03:19:29.885 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T03:19:29.886 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T03:19:29.895 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T03:19:29.895 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T03:19:29.895 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T03:19:29.895 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:19:29.895 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T03:19:29.895 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T03:19:29.895 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T03:19:29.895 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T03:19:29.895 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T03:19:29.895 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T03:19:29.896 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T03:19:29.896 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T03:19:29.896 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T03:19:29.896 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:19:29.896 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T03:19:29.896 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:19:29.896 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T03:19:15:900094+0000", 2024-09-12T03:19:29.896 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T03:19:29.896 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T03:19:29.896 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:19:29.896 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T03:19:29.896 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T03:19:29.897 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T03:19:29.897 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T03:19:29.897 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T03:19:29.897 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T03:19:29.897 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T03:19:29.897 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-09-12T03:19:29.897 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T03:19:29.897 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:19:29.897 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T03:19:29.897 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:19:29.897 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T03:19:15.900949+0000", 2024-09-12T03:19:29.897 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T03:19:29.897 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:19:29.898 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T03:19:29.898 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-09-12T03:19:29.898 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.093+0000 7fb49ff7b640 1 Processor -- start 2024-09-12T03:19:29.898 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.093+0000 7fb49ff7b640 1 -- start start 2024-09-12T03:19:29.898 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.093+0000 7fb49ff7b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb49807de70 0x7fb49807c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:29.898 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.093+0000 7fb49ff7b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb49807c8a0 con 0x7fb49807de70 2024-09-12T03:19:29.898 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.094+0000 7fb49dd17640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb49807de70 0x7fb49807c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:29.898 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.094+0000 7fb49dd17640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb49807de70 0x7fb49807c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55400/0 (socket says 172.21.15.64:55400) 2024-09-12T03:19:29.898 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.094+0000 7fb49dd17640 1 -- 172.21.15.64:0/2466717474 learned_addr learned my addr 172.21.15.64:0/2466717474 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:19:29.898 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.094+0000 7fb49dd17640 1 -- 172.21.15.64:0/2466717474 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb49807ca20 con 0x7fb49807de70 2024-09-12T03:19:29.898 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.095+0000 7fb49dd17640 1 --2- 172.21.15.64:0/2466717474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb49807de70 0x7fb49807c2d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb494009820 tx=0x7fb49402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b5383d0a29a7849 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:29.898 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.095+0000 7fb49cd15640 1 -- 172.21.15.64:0/2466717474 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb494037070 con 0x7fb49807de70 2024-09-12T03:19:29.898 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.095+0000 7fb49cd15640 1 -- 172.21.15.64:0/2466717474 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb4940328d0 con 0x7fb49807de70 2024-09-12T03:19:29.899 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.095+0000 7fb49cd15640 1 -- 172.21.15.64:0/2466717474 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4940307d0 con 0x7fb49807de70 2024-09-12T03:19:29.899 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.096+0000 7fb49ff7b640 1 -- 172.21.15.64:0/2466717474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb49807de70 msgr2=0x7fb49807c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:29.899 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.096+0000 7fb49ff7b640 1 --2- 172.21.15.64:0/2466717474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb49807de70 0x7fb49807c2d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb494009820 tx=0x7fb49402a270 comp rx=0 tx=0).stop 2024-09-12T03:19:29.899 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.096+0000 7fb49ff7b640 1 -- 172.21.15.64:0/2466717474 shutdown_connections 2024-09-12T03:19:29.899 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.096+0000 7fb49ff7b640 1 --2- 172.21.15.64:0/2466717474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb49807de70 0x7fb49807c2d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.899 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.096+0000 7fb49ff7b640 1 -- 172.21.15.64:0/2466717474 >> 172.21.15.64:0/2466717474 conn(0x7fb498102d30 msgr2=0x7fb498105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:29.899 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.096+0000 7fb49ff7b640 1 -- 172.21.15.64:0/2466717474 shutdown_connections 2024-09-12T03:19:29.899 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.096+0000 7fb49ff7b640 1 -- 172.21.15.64:0/2466717474 wait complete. 2024-09-12T03:19:29.899 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.096+0000 7fb49ff7b640 1 Processor -- start 2024-09-12T03:19:29.899 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.096+0000 7fb49ff7b640 1 -- start start 2024-09-12T03:19:29.899 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.097+0000 7fb49ff7b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb49807de70 0x7fb49819d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:29.899 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.097+0000 7fb49ff7b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb49810db50 con 0x7fb49807de70 2024-09-12T03:19:29.899 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.097+0000 7fb49dd17640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb49807de70 0x7fb49819d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:29.900 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.097+0000 7fb49dd17640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb49807de70 0x7fb49819d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55416/0 (socket says 172.21.15.64:55416) 2024-09-12T03:19:29.900 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.097+0000 7fb49dd17640 1 -- 172.21.15.64:0/3523082289 learned_addr learned my addr 172.21.15.64:0/3523082289 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:19:29.900 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.097+0000 7fb49dd17640 1 -- 172.21.15.64:0/3523082289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb49819dc00 con 0x7fb49807de70 2024-09-12T03:19:29.900 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.097+0000 7fb49dd17640 1 --2- 172.21.15.64:0/3523082289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb49807de70 0x7fb49819d6c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb4940097f0 tx=0x7fb494030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:29.900 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.098+0000 7fb48effd640 1 -- 172.21.15.64:0/3523082289 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb494037040 con 0x7fb49807de70 2024-09-12T03:19:29.900 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.098+0000 7fb49ff7b640 1 -- 172.21.15.64:0/3523082289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb49819de90 con 0x7fb49807de70 2024-09-12T03:19:29.900 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.098+0000 7fb48effd640 1 -- 172.21.15.64:0/3523082289 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb494032bb0 con 0x7fb49807de70 2024-09-12T03:19:29.900 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.098+0000 7fb48effd640 1 -- 172.21.15.64:0/3523082289 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb49403a390 con 0x7fb49807de70 2024-09-12T03:19:29.900 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.098+0000 7fb49ff7b640 1 -- 172.21.15.64:0/3523082289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4981a0b80 con 0x7fb49807de70 2024-09-12T03:19:29.900 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.099+0000 7fb48effd640 1 -- 172.21.15.64:0/3523082289 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 3) ==== 46170+0+0 (secure 0 0 0) 0x7fb49403b3f0 con 0x7fb49807de70 2024-09-12T03:19:29.900 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.099+0000 7fb48effd640 1 --2- 172.21.15.64:0/3523082289 >> [v2:172.21.15.64:6800/3186640984,v1:172.21.15.64:6801/3186640984] conn(0x7fb484038e10 0x7fb48403b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:29.900 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.099+0000 7fb49ff7b640 1 -- 172.21.15.64:0/3523082289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb49807c350 con 0x7fb49807de70 2024-09-12T03:19:29.900 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.099+0000 7fb48effd640 1 -- 172.21.15.64:0/3523082289 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb49406bfc0 con 0x7fb49807de70 2024-09-12T03:19:29.901 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.099+0000 7fb49d516640 1 --2- 172.21.15.64:0/3523082289 >> [v2:172.21.15.64:6800/3186640984,v1:172.21.15.64:6801/3186640984] conn(0x7fb484038e10 0x7fb48403b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:29.901 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.100+0000 7fb49d516640 1 --2- 172.21.15.64:0/3523082289 >> [v2:172.21.15.64:6800/3186640984,v1:172.21.15.64:6801/3186640984] conn(0x7fb484038e10 0x7fb48403b2d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb4880098c0 tx=0x7fb488006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:29.901 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.104+0000 7fb48effd640 1 -- 172.21.15.64:0/3523082289 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb494040210 con 0x7fb49807de70 2024-09-12T03:19:29.901 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.348+0000 7fb49ff7b640 1 -- 172.21.15.64:0/3523082289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb4981a0d50 con 0x7fb49807de70 2024-09-12T03:19:29.901 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.348+0000 7fb48effd640 1 -- 172.21.15.64:0/3523082289 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7fb4981a0d50 con 0x7fb49807de70 2024-09-12T03:19:29.901 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.350+0000 7fb49ff7b640 1 -- 172.21.15.64:0/3523082289 >> [v2:172.21.15.64:6800/3186640984,v1:172.21.15.64:6801/3186640984] conn(0x7fb484038e10 msgr2=0x7fb48403b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:29.901 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.350+0000 7fb49ff7b640 1 --2- 172.21.15.64:0/3523082289 >> [v2:172.21.15.64:6800/3186640984,v1:172.21.15.64:6801/3186640984] conn(0x7fb484038e10 0x7fb48403b2d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb4880098c0 tx=0x7fb488006db0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.901 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.350+0000 7fb49ff7b640 1 -- 172.21.15.64:0/3523082289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb49807de70 msgr2=0x7fb49819d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:29.901 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.350+0000 7fb49ff7b640 1 --2- 172.21.15.64:0/3523082289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb49807de70 0x7fb49819d6c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb4940097f0 tx=0x7fb494030d00 comp rx=0 tx=0).stop 2024-09-12T03:19:29.901 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.350+0000 7fb49ff7b640 1 -- 172.21.15.64:0/3523082289 shutdown_connections 2024-09-12T03:19:29.901 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.350+0000 7fb49ff7b640 1 --2- 172.21.15.64:0/3523082289 >> [v2:172.21.15.64:6800/3186640984,v1:172.21.15.64:6801/3186640984] conn(0x7fb484038e10 0x7fb48403b2d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.901 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.350+0000 7fb49ff7b640 1 --2- 172.21.15.64:0/3523082289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb49807de70 0x7fb49819d6c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.901 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.350+0000 7fb49ff7b640 1 -- 172.21.15.64:0/3523082289 >> 172.21.15.64:0/3523082289 conn(0x7fb498102d30 msgr2=0x7fb4981038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:29.902 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.351+0000 7fb49ff7b640 1 -- 172.21.15.64:0/3523082289 shutdown_connections 2024-09-12T03:19:29.902 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:29.351+0000 7fb49ff7b640 1 -- 172.21.15.64:0/3523082289 wait complete. 2024-09-12T03:19:29.902 INFO:teuthology.orchestra.run.smithi064.stdout:mgr is available 2024-09-12T03:19:29.983 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:29 smithi064 ceph-mon[30952]: mgrmap e3: x(active, since 1.00749s) 2024-09-12T03:19:29.983 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:29 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/3523082289' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T03:19:31.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:30 smithi064 ceph-mon[30952]: mgrmap e4: x(active, since 2s) 2024-09-12T03:19:31.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:30 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/1316804066' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-12T03:19:31.073 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-09-12T03:19:31.073 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [global] 2024-09-12T03:19:31.073 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout fsid = b0812460-70b5-11ef-bcea-c7b262605968 2024-09-12T03:19:31.073 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.64:3300,v1:172.21.15.64:6789] 2024-09-12T03:19:31.073 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-12T03:19:31.073 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-12T03:19:31.073 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-12T03:19:31.073 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-12T03:19:31.073 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-09-12T03:19:31.073 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-12T03:19:31.073 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-12T03:19:31.074 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-09-12T03:19:31.074 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [osd] 2024-09-12T03:19:31.074 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-12T03:19:31.074 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-12T03:19:31.074 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-12T03:19:31.074 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-12T03:19:31.074 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-12T03:19:31.074 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.240+0000 7f0bbd8b5640 1 Processor -- start 2024-09-12T03:19:31.074 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.240+0000 7f0bbd8b5640 1 -- start start 2024-09-12T03:19:31.074 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.240+0000 7f0bbd8b5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bb807de70 0x7f0bb807c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:31.075 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.240+0000 7f0bbd8b5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb807c8a0 con 0x7f0bb807de70 2024-09-12T03:19:31.075 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.240+0000 7f0bb6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bb807de70 0x7f0bb807c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:31.075 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.240+0000 7f0bb6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bb807de70 0x7f0bb807c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55424/0 (socket says 172.21.15.64:55424) 2024-09-12T03:19:31.075 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.240+0000 7f0bb6ffd640 1 -- 172.21.15.64:0/675126779 learned_addr learned my addr 172.21.15.64:0/675126779 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:19:31.075 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.240+0000 7f0bb6ffd640 1 -- 172.21.15.64:0/675126779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb807ca20 con 0x7f0bb807de70 2024-09-12T03:19:31.075 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.241+0000 7f0bb6ffd640 1 --2- 172.21.15.64:0/675126779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bb807de70 0x7f0bb807c2d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8009820 tx=0x7f0ba802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb644b5fbdcc494 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:31.075 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.241+0000 7f0bb67fc640 1 -- 172.21.15.64:0/675126779 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ba8037070 con 0x7f0bb807de70 2024-09-12T03:19:31.075 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.241+0000 7f0bb67fc640 1 -- 172.21.15.64:0/675126779 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0ba80328d0 con 0x7f0bb807de70 2024-09-12T03:19:31.075 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.241+0000 7f0bb67fc640 1 -- 172.21.15.64:0/675126779 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ba80307d0 con 0x7f0bb807de70 2024-09-12T03:19:31.075 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.242+0000 7f0bbd8b5640 1 -- 172.21.15.64:0/675126779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bb807de70 msgr2=0x7f0bb807c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:31.075 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.242+0000 7f0bbd8b5640 1 --2- 172.21.15.64:0/675126779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bb807de70 0x7f0bb807c2d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8009820 tx=0x7f0ba802a270 comp rx=0 tx=0).stop 2024-09-12T03:19:31.075 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.242+0000 7f0bbd8b5640 1 -- 172.21.15.64:0/675126779 shutdown_connections 2024-09-12T03:19:31.075 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.242+0000 7f0bbd8b5640 1 --2- 172.21.15.64:0/675126779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bb807de70 0x7f0bb807c2d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:31.076 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.242+0000 7f0bbd8b5640 1 -- 172.21.15.64:0/675126779 >> 172.21.15.64:0/675126779 conn(0x7f0bb8102d30 msgr2=0x7f0bb8105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:31.076 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.242+0000 7f0bbd8b5640 1 -- 172.21.15.64:0/675126779 shutdown_connections 2024-09-12T03:19:31.076 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.242+0000 7f0bbd8b5640 1 -- 172.21.15.64:0/675126779 wait complete. 2024-09-12T03:19:31.076 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.243+0000 7f0bbd8b5640 1 Processor -- start 2024-09-12T03:19:31.076 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.243+0000 7f0bbd8b5640 1 -- start start 2024-09-12T03:19:31.076 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.243+0000 7f0bbd8b5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bb807de70 0x7f0bb819d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:31.076 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.243+0000 7f0bbd8b5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb810db50 con 0x7f0bb807de70 2024-09-12T03:19:31.076 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.243+0000 7f0bb6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bb807de70 0x7f0bb819d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:31.076 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.243+0000 7f0bb6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bb807de70 0x7f0bb819d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55430/0 (socket says 172.21.15.64:55430) 2024-09-12T03:19:31.076 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.243+0000 7f0bb6ffd640 1 -- 172.21.15.64:0/1316804066 learned_addr learned my addr 172.21.15.64:0/1316804066 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:19:31.076 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.243+0000 7f0bb6ffd640 1 -- 172.21.15.64:0/1316804066 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb819dc00 con 0x7f0bb807de70 2024-09-12T03:19:31.076 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.243+0000 7f0bb6ffd640 1 --2- 172.21.15.64:0/1316804066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bb807de70 0x7f0bb819d6c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0ba80097f0 tx=0x7f0ba8030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:31.076 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.243+0000 7f0bbc8b3640 1 -- 172.21.15.64:0/1316804066 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ba8037040 con 0x7f0bb807de70 2024-09-12T03:19:31.077 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.244+0000 7f0bbd8b5640 1 -- 172.21.15.64:0/1316804066 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bb819de90 con 0x7f0bb807de70 2024-09-12T03:19:31.077 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.244+0000 7f0bbc8b3640 1 -- 172.21.15.64:0/1316804066 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0ba8032bb0 con 0x7f0bb807de70 2024-09-12T03:19:31.077 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.244+0000 7f0bbc8b3640 1 -- 172.21.15.64:0/1316804066 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ba803a390 con 0x7f0bb807de70 2024-09-12T03:19:31.077 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.244+0000 7f0bbd8b5640 1 -- 172.21.15.64:0/1316804066 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bb81a0b80 con 0x7f0bb807de70 2024-09-12T03:19:31.077 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.244+0000 7f0bbc8b3640 1 -- 172.21.15.64:0/1316804066 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f0ba803a5a0 con 0x7f0bb807de70 2024-09-12T03:19:31.077 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.245+0000 7f0bbc8b3640 1 --2- 172.21.15.64:0/1316804066 >> [v2:172.21.15.64:6800/3186640984,v1:172.21.15.64:6801/3186640984] conn(0x7f0ba4038f70 0x7f0ba403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:31.077 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.245+0000 7f0bbd8b5640 1 -- 172.21.15.64:0/1316804066 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bb807c350 con 0x7f0bb807de70 2024-09-12T03:19:31.077 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.245+0000 7f0bbc8b3640 1 -- 172.21.15.64:0/1316804066 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0ba806c0c0 con 0x7f0bb807de70 2024-09-12T03:19:31.077 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.245+0000 7f0baffff640 1 --2- 172.21.15.64:0/1316804066 >> [v2:172.21.15.64:6800/3186640984,v1:172.21.15.64:6801/3186640984] conn(0x7f0ba4038f70 0x7f0ba403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:31.077 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.245+0000 7f0baffff640 1 --2- 172.21.15.64:0/1316804066 >> [v2:172.21.15.64:6800/3186640984,v1:172.21.15.64:6801/3186640984] conn(0x7f0ba4038f70 0x7f0ba403b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0ba00098c0 tx=0x7f0ba0006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:31.077 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.250+0000 7f0bbc8b3640 1 -- 172.21.15.64:0/1316804066 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0ba8032d60 con 0x7f0bb807de70 2024-09-12T03:19:31.077 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.427+0000 7f0bbd8b5640 1 -- 172.21.15.64:0/1316804066 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f0bb81a2380 con 0x7f0bb807de70 2024-09-12T03:19:31.077 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.428+0000 7f0bbc8b3640 1 -- 172.21.15.64:0/1316804066 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+429 (secure 0 0 0) 0x7f0ba80317a0 con 0x7f0bb807de70 2024-09-12T03:19:31.078 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.430+0000 7f0bbd8b5640 1 -- 172.21.15.64:0/1316804066 >> [v2:172.21.15.64:6800/3186640984,v1:172.21.15.64:6801/3186640984] conn(0x7f0ba4038f70 msgr2=0x7f0ba403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:31.078 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.430+0000 7f0bbd8b5640 1 --2- 172.21.15.64:0/1316804066 >> [v2:172.21.15.64:6800/3186640984,v1:172.21.15.64:6801/3186640984] conn(0x7f0ba4038f70 0x7f0ba403b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0ba00098c0 tx=0x7f0ba0006db0 comp rx=0 tx=0).stop 2024-09-12T03:19:31.078 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.430+0000 7f0bbd8b5640 1 -- 172.21.15.64:0/1316804066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bb807de70 msgr2=0x7f0bb819d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:31.078 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.430+0000 7f0bbd8b5640 1 --2- 172.21.15.64:0/1316804066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bb807de70 0x7f0bb819d6c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0ba80097f0 tx=0x7f0ba8030d00 comp rx=0 tx=0).stop 2024-09-12T03:19:31.078 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.430+0000 7f0bbd8b5640 1 -- 172.21.15.64:0/1316804066 shutdown_connections 2024-09-12T03:19:31.078 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.430+0000 7f0bbd8b5640 1 --2- 172.21.15.64:0/1316804066 >> [v2:172.21.15.64:6800/3186640984,v1:172.21.15.64:6801/3186640984] conn(0x7f0ba4038f70 0x7f0ba403b430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:31.078 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.430+0000 7f0bbd8b5640 1 --2- 172.21.15.64:0/1316804066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bb807de70 0x7f0bb819d6c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:31.078 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.430+0000 7f0bbd8b5640 1 -- 172.21.15.64:0/1316804066 >> 172.21.15.64:0/1316804066 conn(0x7f0bb8102d30 msgr2=0x7f0bb81038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:31.078 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.430+0000 7f0bbd8b5640 1 -- 172.21.15.64:0/1316804066 shutdown_connections 2024-09-12T03:19:31.078 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:30.430+0000 7f0bbd8b5640 1 -- 172.21.15.64:0/1316804066 wait complete. 2024-09-12T03:19:31.078 INFO:teuthology.orchestra.run.smithi064.stdout:Enabling cephadm module... 2024-09-12T03:19:31.930 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:31 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/257350234' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-12T03:19:31.931 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:31 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: ignoring --setuser ceph since I am not root 2024-09-12T03:19:31.931 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:31 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: ignoring --setgroup ceph since I am not root 2024-09-12T03:19:31.931 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:31 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:31.870+0000 7f5e3c2c4040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T03:19:31.931 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:31 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:31.930+0000 7f5e3c2c4040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T03:19:32.272 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:32 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:32.085+0000 7f5e3c2c4040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T03:19:32.272 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:32 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:32.154+0000 7f5e3c2c4040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T03:19:32.728 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:32 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:32.309+0000 7f5e3c2c4040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:19:32.729 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:32 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:32.457+0000 7f5e3c2c4040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T03:19:33.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:32 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/257350234' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-12T03:19:33.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:32 smithi064 ceph-mon[30952]: mgrmap e5: x(active, since 4s) 2024-09-12T03:19:33.022 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:32 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:32.787+0000 7f5e3c2c4040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T03:19:33.022 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:32 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:32.850+0000 7f5e3c2c4040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T03:19:33.522 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:33 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:33.045+0000 7f5e3c2c4040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T03:19:34.272 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:33 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:33.972+0000 7f5e3c2c4040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T03:19:34.272 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:34 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:34.044+0000 7f5e3c2c4040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T03:19:35.086 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:34 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:34.811+0000 7f5e3c2c4040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T03:19:35.087 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:34 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:34.933+0000 7f5e3c2c4040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T03:19:35.367 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:35 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:35.086+0000 7f5e3c2c4040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:19:35.368 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:35 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:35.156+0000 7f5e3c2c4040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T03:19:35.368 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:35 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:35.319+0000 7f5e3c2c4040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T03:19:35.644 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:35 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:35.396+0000 7f5e3c2c4040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T03:19:35.645 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:35 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:35.464+0000 7f5e3c2c4040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T03:19:35.645 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:35 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:35.644+0000 7f5e3c2c4040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T03:19:35.938 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:35 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:35.714+0000 7f5e3c2c4040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T03:19:35.938 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:35 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:35.785+0000 7f5e3c2c4040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T03:19:35.938 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:35 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:35.875+0000 7f5e3c2c4040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T03:19:35.938 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:35 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:35.938+0000 7f5e3c2c4040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T03:19:36.272 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:36 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:36.000+0000 7f5e3c2c4040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T03:19:37.031 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.440+0000 7f63fe8de640 1 Processor -- start 2024-09-12T03:19:37.038 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.440+0000 7f63fe8de640 1 -- start start 2024-09-12T03:19:37.038 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.440+0000 7f63fe8de640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63f8107510 0x7f63f8107910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:37.038 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.440+0000 7f63fe8de640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f63f8107ee0 con 0x7f63f8107510 2024-09-12T03:19:37.038 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.440+0000 7f63f7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63f8107510 0x7f63f8107910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:37.038 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.440+0000 7f63f7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63f8107510 0x7f63f8107910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55446/0 (socket says 172.21.15.64:55446) 2024-09-12T03:19:37.039 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.440+0000 7f63f7fff640 1 -- 172.21.15.64:0/2572393192 learned_addr learned my addr 172.21.15.64:0/2572393192 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:19:37.039 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.440+0000 7f63f7fff640 1 -- 172.21.15.64:0/2572393192 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63f8108710 con 0x7f63f8107510 2024-09-12T03:19:37.039 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.441+0000 7f63f7fff640 1 --2- 172.21.15.64:0/2572393192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63f8107510 0x7f63f8107910 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f63e8009820 tx=0x7f63e802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae74e6e4402b8e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:37.039 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.441+0000 7f63f6ffd640 1 -- 172.21.15.64:0/2572393192 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63e8037070 con 0x7f63f8107510 2024-09-12T03:19:37.039 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.441+0000 7f63f6ffd640 1 -- 172.21.15.64:0/2572393192 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f63e80328d0 con 0x7f63f8107510 2024-09-12T03:19:37.039 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.441+0000 7f63f6ffd640 1 -- 172.21.15.64:0/2572393192 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63e80307d0 con 0x7f63f8107510 2024-09-12T03:19:37.039 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.442+0000 7f63fe8de640 1 -- 172.21.15.64:0/2572393192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63f8107510 msgr2=0x7f63f8107910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:37.039 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.442+0000 7f63fe8de640 1 --2- 172.21.15.64:0/2572393192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63f8107510 0x7f63f8107910 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f63e8009820 tx=0x7f63e802a270 comp rx=0 tx=0).stop 2024-09-12T03:19:37.039 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.442+0000 7f63fe8de640 1 -- 172.21.15.64:0/2572393192 shutdown_connections 2024-09-12T03:19:37.039 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.442+0000 7f63fe8de640 1 --2- 172.21.15.64:0/2572393192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63f8107510 0x7f63f8107910 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.039 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.442+0000 7f63fe8de640 1 -- 172.21.15.64:0/2572393192 >> 172.21.15.64:0/2572393192 conn(0x7f63f8102ce0 msgr2=0x7f63f8105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:37.040 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.442+0000 7f63fe8de640 1 -- 172.21.15.64:0/2572393192 shutdown_connections 2024-09-12T03:19:37.040 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.442+0000 7f63fe8de640 1 -- 172.21.15.64:0/2572393192 wait complete. 2024-09-12T03:19:37.040 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.442+0000 7f63fe8de640 1 Processor -- start 2024-09-12T03:19:37.040 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.443+0000 7f63fe8de640 1 -- start start 2024-09-12T03:19:37.040 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.443+0000 7f63fe8de640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63f8107510 0x7f63f819b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:37.040 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.443+0000 7f63fe8de640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f63f810bac0 con 0x7f63f8107510 2024-09-12T03:19:37.040 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.443+0000 7f63f7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63f8107510 0x7f63f819b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:37.040 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.443+0000 7f63f7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63f8107510 0x7f63f819b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55462/0 (socket says 172.21.15.64:55462) 2024-09-12T03:19:37.040 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.443+0000 7f63f7fff640 1 -- 172.21.15.64:0/257350234 learned_addr learned my addr 172.21.15.64:0/257350234 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:19:37.040 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.443+0000 7f63f7fff640 1 -- 172.21.15.64:0/257350234 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63f819b9f0 con 0x7f63f8107510 2024-09-12T03:19:37.041 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.443+0000 7f63f7fff640 1 --2- 172.21.15.64:0/257350234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63f8107510 0x7f63f819b4b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f63e8009940 tx=0x7f63e8032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:37.041 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.444+0000 7f63f57fa640 1 -- 172.21.15.64:0/257350234 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63e8042070 con 0x7f63f8107510 2024-09-12T03:19:37.041 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.444+0000 7f63f57fa640 1 -- 172.21.15.64:0/257350234 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f63e8030c80 con 0x7f63f8107510 2024-09-12T03:19:37.041 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.444+0000 7f63fe8de640 1 -- 172.21.15.64:0/257350234 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63f819bc80 con 0x7f63f8107510 2024-09-12T03:19:37.041 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.444+0000 7f63f57fa640 1 -- 172.21.15.64:0/257350234 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63e8037040 con 0x7f63f8107510 2024-09-12T03:19:37.041 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.444+0000 7f63fe8de640 1 -- 172.21.15.64:0/257350234 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63f819e970 con 0x7f63f8107510 2024-09-12T03:19:37.041 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.444+0000 7f63f57fa640 1 -- 172.21.15.64:0/257350234 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f63e8045070 con 0x7f63f8107510 2024-09-12T03:19:37.041 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.445+0000 7f63f57fa640 1 --2- 172.21.15.64:0/257350234 >> [v2:172.21.15.64:6800/3186640984,v1:172.21.15.64:6801/3186640984] conn(0x7f63e0038f70 0x7f63e003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:37.041 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.445+0000 7f63fe8de640 1 -- 172.21.15.64:0/257350234 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63f810bc40 con 0x7f63f8107510 2024-09-12T03:19:37.042 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.445+0000 7f63f57fa640 1 -- 172.21.15.64:0/257350234 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f63e806bf70 con 0x7f63f8107510 2024-09-12T03:19:37.042 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.445+0000 7f63f77fe640 1 --2- 172.21.15.64:0/257350234 >> [v2:172.21.15.64:6800/3186640984,v1:172.21.15.64:6801/3186640984] conn(0x7f63e0038f70 0x7f63e003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:37.042 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.445+0000 7f63f77fe640 1 --2- 172.21.15.64:0/257350234 >> [v2:172.21.15.64:6800/3186640984,v1:172.21.15.64:6801/3186640984] conn(0x7f63e0038f70 0x7f63e003b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f63ec0098c0 tx=0x7f63ec006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:37.042 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.450+0000 7f63f57fa640 1 -- 172.21.15.64:0/257350234 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f63e803ac60 con 0x7f63f8107510 2024-09-12T03:19:37.042 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.673+0000 7f63fe8de640 1 -- 172.21.15.64:0/257350234 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f63f819eb40 con 0x7f63f8107510 2024-09-12T03:19:37.042 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.725+0000 7f63f57fa640 1 -- 172.21.15.64:0/257350234 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f63e806b990 con 0x7f63f8107510 2024-09-12T03:19:37.042 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.726+0000 7f63f57fa640 1 -- 172.21.15.64:0/257350234 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f63e803c640 con 0x7f63f8107510 2024-09-12T03:19:37.042 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.727+0000 7f63fe8de640 1 -- 172.21.15.64:0/257350234 >> [v2:172.21.15.64:6800/3186640984,v1:172.21.15.64:6801/3186640984] conn(0x7f63e0038f70 msgr2=0x7f63e003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:37.042 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.727+0000 7f63fe8de640 1 --2- 172.21.15.64:0/257350234 >> [v2:172.21.15.64:6800/3186640984,v1:172.21.15.64:6801/3186640984] conn(0x7f63e0038f70 0x7f63e003b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f63ec0098c0 tx=0x7f63ec006db0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.043 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.727+0000 7f63fe8de640 1 -- 172.21.15.64:0/257350234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63f8107510 msgr2=0x7f63f819b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:37.043 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.727+0000 7f63fe8de640 1 --2- 172.21.15.64:0/257350234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63f8107510 0x7f63f819b4b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f63e8009940 tx=0x7f63e8032e10 comp rx=0 tx=0).stop 2024-09-12T03:19:37.043 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.727+0000 7f63fe8de640 1 -- 172.21.15.64:0/257350234 shutdown_connections 2024-09-12T03:19:37.043 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.727+0000 7f63fe8de640 1 --2- 172.21.15.64:0/257350234 >> [v2:172.21.15.64:6800/3186640984,v1:172.21.15.64:6801/3186640984] conn(0x7f63e0038f70 0x7f63e003b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.043 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.727+0000 7f63fe8de640 1 --2- 172.21.15.64:0/257350234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63f8107510 0x7f63f819b4b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.043 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.727+0000 7f63fe8de640 1 -- 172.21.15.64:0/257350234 >> 172.21.15.64:0/257350234 conn(0x7f63f8102ce0 msgr2=0x7f63f81037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:37.043 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.728+0000 7f63fe8de640 1 -- 172.21.15.64:0/257350234 shutdown_connections 2024-09-12T03:19:37.043 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:31.728+0000 7f63fe8de640 1 -- 172.21.15.64:0/257350234 wait complete. 2024-09-12T03:19:37.160 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:36 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:36.837+0000 7f5e3c2c4040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T03:19:37.160 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:36 smithi064 ceph-mon[30952]: Active manager daemon x restarted 2024-09-12T03:19:37.161 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:36 smithi064 ceph-mon[30952]: Activating manager daemon x 2024-09-12T03:19:37.161 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:36 smithi064 ceph-mon[30952]: osdmap e2: 0 total, 0 up, 0 in 2024-09-12T03:19:37.161 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:36 smithi064 ceph-mon[30952]: mgrmap e6: x(active, starting, since 0.00340438s) 2024-09-12T03:19:37.161 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:36 smithi064 ceph-mon[30952]: from='mgr.14118 172.21.15.64:0/358528520' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:19:37.161 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:36 smithi064 ceph-mon[30952]: from='mgr.14118 172.21.15.64:0/358528520' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T03:19:37.161 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:36 smithi064 ceph-mon[30952]: from='mgr.14118 172.21.15.64:0/358528520' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T03:19:37.161 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:36 smithi064 ceph-mon[30952]: from='mgr.14118 172.21.15.64:0/358528520' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T03:19:37.161 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:36 smithi064 ceph-mon[30952]: from='mgr.14118 172.21.15.64:0/358528520' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T03:19:37.161 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:36 smithi064 ceph-mon[30952]: Manager daemon x is now available 2024-09-12T03:19:37.161 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:36 smithi064 ceph-mon[30952]: from='mgr.14118 172.21.15.64:0/358528520' entity='mgr.x' 2024-09-12T03:19:37.161 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:36 smithi064 ceph-mon[30952]: from='mgr.14118 172.21.15.64:0/358528520' entity='mgr.x' 2024-09-12T03:19:37.162 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:36 smithi064 ceph-mon[30952]: from='mgr.14118 172.21.15.64:0/358528520' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:19:37.162 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:36 smithi064 ceph-mon[30952]: from='mgr.14118 172.21.15.64:0/358528520' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:19:37.162 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:36 smithi064 ceph-mon[30952]: from='mgr.14118 172.21.15.64:0/358528520' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T03:19:37.162 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:36 smithi064 ceph-mon[30952]: from='mgr.14118 172.21.15.64:0/358528520' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T03:19:38.271 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:37 smithi064 ceph-mon[30952]: Found migration_current of "None". Setting to last migration. 2024-09-12T03:19:38.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:37 smithi064 ceph-mon[30952]: mgrmap e7: x(active, since 1.00519s) 2024-09-12T03:19:39.245 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:38 smithi064 ceph-mon[30952]: from='mgr.14118 172.21.15.64:0/358528520' entity='mgr.x' 2024-09-12T03:19:39.245 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:38 smithi064 ceph-mon[30952]: from='mgr.14118 172.21.15.64:0/358528520' entity='mgr.x' 2024-09-12T03:19:39.522 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:39 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:39.243+0000 7f5da8e29640 -1 log_channel(cephadm) log [ERR] : [12/Sep/2024:03:19:39] ENGINE Error in HTTPServer.serve 2024-09-12T03:19:39.671 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:39 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: Traceback (most recent call last): 2024-09-12T03:19:39.671 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:39 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T03:19:39.671 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:39 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: self._connections.run(self.expiration_interval) 2024-09-12T03:19:39.671 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:39 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T03:19:39.671 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:39 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: self._run(expiration_interval) 2024-09-12T03:19:39.672 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:39 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T03:19:39.672 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:39 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T03:19:39.672 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:39 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T03:19:39.672 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:39 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T03:19:39.672 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:39 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T03:19:39.672 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:39 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: s = self.context.wrap_socket( 2024-09-12T03:19:39.672 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:39 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T03:19:39.672 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:39 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: return self.sslsocket_class._create( 2024-09-12T03:19:39.672 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:39 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T03:19:39.672 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:39 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: self.do_handshake() 2024-09-12T03:19:39.672 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:39 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T03:19:39.673 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:39 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: self._sslobj.do_handshake() 2024-09-12T03:19:39.673 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:39 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T03:19:39.673 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:39 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:19:40.200 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:39 smithi064 ceph-mon[30952]: mgrmap e8: x(active, since 2s) 2024-09-12T03:19:40.200 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:39 smithi064 ceph-mon[30952]: [12/Sep/2024:03:19:39] ENGINE Bus STARTING 2024-09-12T03:19:40.200 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:39 smithi064 ceph-mon[30952]: [12/Sep/2024:03:19:39] ENGINE Serving on https://172.21.15.64:7150 2024-09-12T03:19:40.200 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:39 smithi064 ceph-mon[30952]: [12/Sep/2024:03:19:39] ENGINE Error in HTTPServer.serve 2024-09-12T03:19:40.201 INFO:journalctl@ceph.mon.a.smithi064.stdout: Traceback (most recent call last): 2024-09-12T03:19:40.201 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T03:19:40.201 INFO:journalctl@ceph.mon.a.smithi064.stdout: self._connections.run(self.expiration_interval) 2024-09-12T03:19:40.201 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T03:19:40.201 INFO:journalctl@ceph.mon.a.smithi064.stdout: self._run(expiration_interval) 2024-09-12T03:19:40.201 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T03:19:40.201 INFO:journalctl@ceph.mon.a.smithi064.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T03:19:40.201 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T03:19:40.201 INFO:journalctl@ceph.mon.a.smithi064.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T03:19:40.201 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T03:19:40.201 INFO:journalctl@ceph.mon.a.smithi064.stdout: s = self.context.wrap_socket( 2024-09-12T03:19:40.201 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T03:19:40.201 INFO:journalctl@ceph.mon.a.smithi064.stdout: return self.sslsocket_class._create( 2024-09-12T03:19:40.201 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T03:19:40.202 INFO:journalctl@ceph.mon.a.smithi064.stdout: self.do_handshake() 2024-09-12T03:19:40.202 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T03:19:40.202 INFO:journalctl@ceph.mon.a.smithi064.stdout: self._sslobj.do_handshake() 2024-09-12T03:19:40.202 INFO:journalctl@ceph.mon.a.smithi064.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T03:19:40.202 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:39 smithi064 ceph-mon[30952]: [12/Sep/2024:03:19:39] ENGINE Serving on http://172.21.15.64:8765 2024-09-12T03:19:40.202 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:39 smithi064 ceph-mon[30952]: [12/Sep/2024:03:19:39] ENGINE Bus STARTED 2024-09-12T03:19:40.202 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:39 smithi064 ceph-mon[30952]: from='mgr.14118 172.21.15.64:0/358528520' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:19:41.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:40 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/3838382742' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-12T03:19:44.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-09-12T03:19:44.062 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 8, 2024-09-12T03:19:44.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T03:19:44.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-12T03:19:44.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-12T03:19:44.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-09-12T03:19:44.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:39.962+0000 7fdf5b566640 1 Processor -- start 2024-09-12T03:19:44.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:39.963+0000 7fdf5b566640 1 -- start start 2024-09-12T03:19:44.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:39.963+0000 7fdf5b566640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf54105820 0x7fdf54105c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:44.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:39.963+0000 7fdf5b566640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdf541061f0 con 0x7fdf54105820 2024-09-12T03:19:44.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:39.963+0000 7fdf59302640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf54105820 0x7fdf54105c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:44.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:39.963+0000 7fdf59302640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf54105820 0x7fdf54105c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43548/0 (socket says 172.21.15.64:43548) 2024-09-12T03:19:44.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:39.963+0000 7fdf59302640 1 -- 172.21.15.64:0/568754695 learned_addr learned my addr 172.21.15.64:0/568754695 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:19:44.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:39.963+0000 7fdf59302640 1 -- 172.21.15.64:0/568754695 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf54106370 con 0x7fdf54105820 2024-09-12T03:19:44.063 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:39.964+0000 7fdf59302640 1 --2- 172.21.15.64:0/568754695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf54105820 0x7fdf54105c20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdf44009820 tx=0x7fdf4402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16fef9281f33d6cd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:44.064 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:39.964+0000 7fdf53fff640 1 -- 172.21.15.64:0/568754695 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf44037070 con 0x7fdf54105820 2024-09-12T03:19:44.064 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:39.964+0000 7fdf53fff640 1 -- 172.21.15.64:0/568754695 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fdf440328d0 con 0x7fdf54105820 2024-09-12T03:19:44.064 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:39.964+0000 7fdf53fff640 1 -- 172.21.15.64:0/568754695 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf440307d0 con 0x7fdf54105820 2024-09-12T03:19:44.064 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:39.965+0000 7fdf5b566640 1 -- 172.21.15.64:0/568754695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf54105820 msgr2=0x7fdf54105c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:44.064 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:39.965+0000 7fdf5b566640 1 --2- 172.21.15.64:0/568754695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf54105820 0x7fdf54105c20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdf44009820 tx=0x7fdf4402a270 comp rx=0 tx=0).stop 2024-09-12T03:19:44.064 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:39.965+0000 7fdf5b566640 1 -- 172.21.15.64:0/568754695 shutdown_connections 2024-09-12T03:19:44.064 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:39.965+0000 7fdf5b566640 1 --2- 172.21.15.64:0/568754695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf54105820 0x7fdf54105c20 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:44.064 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:39.965+0000 7fdf5b566640 1 -- 172.21.15.64:0/568754695 >> 172.21.15.64:0/568754695 conn(0x7fdf54101390 msgr2=0x7fdf541037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:44.064 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:39.965+0000 7fdf5b566640 1 -- 172.21.15.64:0/568754695 shutdown_connections 2024-09-12T03:19:44.065 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:39.965+0000 7fdf5b566640 1 -- 172.21.15.64:0/568754695 wait complete. 2024-09-12T03:19:44.065 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:39.966+0000 7fdf5b566640 1 Processor -- start 2024-09-12T03:19:44.065 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:39.966+0000 7fdf5b566640 1 -- start start 2024-09-12T03:19:44.065 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:39.966+0000 7fdf5b566640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf54105820 0x7fdf5419b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:44.065 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:39.966+0000 7fdf5b566640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5410bad0 con 0x7fdf54105820 2024-09-12T03:19:44.065 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:39.966+0000 7fdf59302640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf54105820 0x7fdf5419b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:44.065 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:39.966+0000 7fdf59302640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf54105820 0x7fdf5419b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43558/0 (socket says 172.21.15.64:43558) 2024-09-12T03:19:44.065 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:39.966+0000 7fdf59302640 1 -- 172.21.15.64:0/3838382742 learned_addr learned my addr 172.21.15.64:0/3838382742 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:19:44.065 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:39.966+0000 7fdf59302640 1 -- 172.21.15.64:0/3838382742 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf5419ba10 con 0x7fdf54105820 2024-09-12T03:19:44.065 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:39.966+0000 7fdf59302640 1 --2- 172.21.15.64:0/3838382742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf54105820 0x7fdf5419b4d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdf44009940 tx=0x7fdf44032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:44.065 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:39.967+0000 7fdf527fc640 1 -- 172.21.15.64:0/3838382742 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf44042070 con 0x7fdf54105820 2024-09-12T03:19:44.065 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:39.967+0000 7fdf527fc640 1 -- 172.21.15.64:0/3838382742 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fdf44030c80 con 0x7fdf54105820 2024-09-12T03:19:44.065 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:39.967+0000 7fdf5b566640 1 -- 172.21.15.64:0/3838382742 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf5419bca0 con 0x7fdf54105820 2024-09-12T03:19:44.066 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:39.967+0000 7fdf527fc640 1 -- 172.21.15.64:0/3838382742 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf44037040 con 0x7fdf54105820 2024-09-12T03:19:44.066 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:39.967+0000 7fdf5b566640 1 -- 172.21.15.64:0/3838382742 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf5419e990 con 0x7fdf54105820 2024-09-12T03:19:44.066 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:39.968+0000 7fdf527fc640 1 -- 172.21.15.64:0/3838382742 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fdf44045070 con 0x7fdf54105820 2024-09-12T03:19:44.066 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:39.968+0000 7fdf5b566640 1 -- 172.21.15.64:0/3838382742 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf54105c20 con 0x7fdf54105820 2024-09-12T03:19:44.066 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:39.968+0000 7fdf527fc640 1 --2- 172.21.15.64:0/3838382742 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fdf3c038fc0 0x7fdf3c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:44.066 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:39.968+0000 7fdf527fc640 1 -- 172.21.15.64:0/3838382742 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdf4406be70 con 0x7fdf54105820 2024-09-12T03:19:44.066 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:39.968+0000 7fdf58b01640 1 --2- 172.21.15.64:0/3838382742 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fdf3c038fc0 0x7fdf3c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:44.066 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:39.968+0000 7fdf58b01640 1 --2- 172.21.15.64:0/3838382742 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fdf3c038fc0 0x7fdf3c03b480 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdf480098c0 tx=0x7fdf48006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:44.066 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:39.973+0000 7fdf527fc640 1 -- 172.21.15.64:0/3838382742 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdf4403d8b0 con 0x7fdf54105820 2024-09-12T03:19:44.066 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:40.199+0000 7fdf5b566640 1 -- 172.21.15.64:0/3838382742 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fdf5419f010 con 0x7fdf54105820 2024-09-12T03:19:44.066 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:40.199+0000 7fdf527fc640 1 -- 172.21.15.64:0/3838382742 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v8) ==== 56+0+88 (secure 0 0 0) 0x7fdf44066070 con 0x7fdf54105820 2024-09-12T03:19:44.066 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:40.201+0000 7fdf5b566640 1 -- 172.21.15.64:0/3838382742 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fdf3c038fc0 msgr2=0x7fdf3c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:44.066 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:40.201+0000 7fdf5b566640 1 --2- 172.21.15.64:0/3838382742 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fdf3c038fc0 0x7fdf3c03b480 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdf480098c0 tx=0x7fdf48006db0 comp rx=0 tx=0).stop 2024-09-12T03:19:44.066 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:40.202+0000 7fdf5b566640 1 -- 172.21.15.64:0/3838382742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf54105820 msgr2=0x7fdf5419b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:44.067 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:40.202+0000 7fdf5b566640 1 --2- 172.21.15.64:0/3838382742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf54105820 0x7fdf5419b4d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdf44009940 tx=0x7fdf44032e10 comp rx=0 tx=0).stop 2024-09-12T03:19:44.067 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:40.202+0000 7fdf5b566640 1 -- 172.21.15.64:0/3838382742 shutdown_connections 2024-09-12T03:19:44.067 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:40.202+0000 7fdf5b566640 1 --2- 172.21.15.64:0/3838382742 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fdf3c038fc0 0x7fdf3c03b480 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:44.067 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:40.202+0000 7fdf5b566640 1 --2- 172.21.15.64:0/3838382742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf54105820 0x7fdf5419b4d0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:44.067 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:40.202+0000 7fdf5b566640 1 -- 172.21.15.64:0/3838382742 >> 172.21.15.64:0/3838382742 conn(0x7fdf54101390 msgr2=0x7fdf54101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:44.067 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:40.202+0000 7fdf5b566640 1 -- 172.21.15.64:0/3838382742 shutdown_connections 2024-09-12T03:19:44.067 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:40.202+0000 7fdf5b566640 1 -- 172.21.15.64:0/3838382742 wait complete. 2024-09-12T03:19:44.067 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for the mgr to restart... 2024-09-12T03:19:44.067 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mgr epoch 8... 2024-09-12T03:19:45.013 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:44 smithi064 ceph-mon[30952]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-12T03:19:45.013 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:44 smithi064 ceph-mon[30952]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-12T03:19:45.014 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-09-12T03:19:45.014 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 8, 2024-09-12T03:19:45.014 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-12T03:19:45.015 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-09-12T03:19:45.015 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.445+0000 7fe9306d7640 1 Processor -- start 2024-09-12T03:19:45.015 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.445+0000 7fe9306d7640 1 -- start start 2024-09-12T03:19:45.015 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.445+0000 7fe9306d7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe928105e10 0x7fe928106210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:45.015 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.445+0000 7fe9306d7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe92807bdf0 con 0x7fe928105e10 2024-09-12T03:19:45.019 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.445+0000 7fe92e473640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe928105e10 0x7fe928106210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:45.020 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.445+0000 7fe92e473640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe928105e10 0x7fe928106210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43564/0 (socket says 172.21.15.64:43564) 2024-09-12T03:19:45.020 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.445+0000 7fe92e473640 1 -- 172.21.15.64:0/2654447435 learned_addr learned my addr 172.21.15.64:0/2654447435 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:19:45.020 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.446+0000 7fe92e473640 1 -- 172.21.15.64:0/2654447435 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe928106750 con 0x7fe928105e10 2024-09-12T03:19:45.020 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.447+0000 7fe92e473640 1 --2- 172.21.15.64:0/2654447435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe928105e10 0x7fe928106210 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe924009820 tx=0x7fe92402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f301644ffbb9e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:45.020 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.447+0000 7fe92d471640 1 -- 172.21.15.64:0/2654447435 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe924037070 con 0x7fe928105e10 2024-09-12T03:19:45.020 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.447+0000 7fe92d471640 1 -- 172.21.15.64:0/2654447435 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fe9240328d0 con 0x7fe928105e10 2024-09-12T03:19:45.020 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.447+0000 7fe92d471640 1 -- 172.21.15.64:0/2654447435 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9240307d0 con 0x7fe928105e10 2024-09-12T03:19:45.020 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.447+0000 7fe9306d7640 1 -- 172.21.15.64:0/2654447435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe928105e10 msgr2=0x7fe928106210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:45.020 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.447+0000 7fe9306d7640 1 --2- 172.21.15.64:0/2654447435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe928105e10 0x7fe928106210 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe924009820 tx=0x7fe92402a270 comp rx=0 tx=0).stop 2024-09-12T03:19:45.020 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.448+0000 7fe9306d7640 1 -- 172.21.15.64:0/2654447435 shutdown_connections 2024-09-12T03:19:45.020 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.448+0000 7fe9306d7640 1 --2- 172.21.15.64:0/2654447435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe928105e10 0x7fe928106210 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:45.020 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.448+0000 7fe9306d7640 1 -- 172.21.15.64:0/2654447435 >> 172.21.15.64:0/2654447435 conn(0x7fe928101c00 msgr2=0x7fe928104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:45.021 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.448+0000 7fe9306d7640 1 -- 172.21.15.64:0/2654447435 shutdown_connections 2024-09-12T03:19:45.021 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.448+0000 7fe9306d7640 1 -- 172.21.15.64:0/2654447435 wait complete. 2024-09-12T03:19:45.021 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.448+0000 7fe9306d7640 1 Processor -- start 2024-09-12T03:19:45.021 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.448+0000 7fe9306d7640 1 -- start start 2024-09-12T03:19:45.021 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.448+0000 7fe9306d7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe928105e10 0x7fe92819d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:45.021 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.448+0000 7fe9306d7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe9281077f0 con 0x7fe928105e10 2024-09-12T03:19:45.021 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.449+0000 7fe92e473640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe928105e10 0x7fe92819d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:45.021 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.449+0000 7fe92e473640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe928105e10 0x7fe92819d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43578/0 (socket says 172.21.15.64:43578) 2024-09-12T03:19:45.021 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.449+0000 7fe92e473640 1 -- 172.21.15.64:0/131632116 learned_addr learned my addr 172.21.15.64:0/131632116 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:19:45.021 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.449+0000 7fe92e473640 1 -- 172.21.15.64:0/131632116 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe92819dc20 con 0x7fe928105e10 2024-09-12T03:19:45.021 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.449+0000 7fe92e473640 1 --2- 172.21.15.64:0/131632116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe928105e10 0x7fe92819d6e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe9240097f0 tx=0x7fe924030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:45.021 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.449+0000 7fe91f7fe640 1 -- 172.21.15.64:0/131632116 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe924037040 con 0x7fe928105e10 2024-09-12T03:19:45.021 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.449+0000 7fe91f7fe640 1 -- 172.21.15.64:0/131632116 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fe92403b3f0 con 0x7fe928105e10 2024-09-12T03:19:45.021 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.449+0000 7fe9306d7640 1 -- 172.21.15.64:0/131632116 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe92819deb0 con 0x7fe928105e10 2024-09-12T03:19:45.022 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.449+0000 7fe91f7fe640 1 -- 172.21.15.64:0/131632116 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe92403a520 con 0x7fe928105e10 2024-09-12T03:19:45.022 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.449+0000 7fe9306d7640 1 -- 172.21.15.64:0/131632116 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9281a0ba0 con 0x7fe928105e10 2024-09-12T03:19:45.022 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.450+0000 7fe91f7fe640 1 -- 172.21.15.64:0/131632116 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fe924032820 con 0x7fe928105e10 2024-09-12T03:19:45.022 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.451+0000 7fe91f7fe640 1 --2- 172.21.15.64:0/131632116 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fe914038f70 0x7fe91403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:45.022 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.451+0000 7fe91f7fe640 1 -- 172.21.15.64:0/131632116 --> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe91403bb40 con 0x7fe914038f70 2024-09-12T03:19:45.022 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.451+0000 7fe91f7fe640 1 -- 172.21.15.64:0/131632116 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe92406c180 con 0x7fe928105e10 2024-09-12T03:19:45.022 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.451+0000 7fe92dc72640 1 --2- 172.21.15.64:0/131632116 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fe914038f70 0x7fe91403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:45.022 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.451+0000 7fe92dc72640 1 --2- 172.21.15.64:0/131632116 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fe914038f70 0x7fe91403b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe9180098c0 tx=0x7fe918006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:45.022 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.452+0000 7fe91f7fe640 1 -- 172.21.15.64:0/131632116 <== mgr.14118 v2:172.21.15.64:6800/68603379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fe91403bb40 con 0x7fe914038f70 2024-09-12T03:19:45.022 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.458+0000 7fe9306d7640 1 -- 172.21.15.64:0/131632116 --> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fe928106210 con 0x7fe914038f70 2024-09-12T03:19:45.022 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.459+0000 7fe91f7fe640 1 -- 172.21.15.64:0/131632116 <== mgr.14118 v2:172.21.15.64:6800/68603379 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fe928106210 con 0x7fe914038f70 2024-09-12T03:19:45.022 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.459+0000 7fe9306d7640 1 -- 172.21.15.64:0/131632116 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fe914038f70 msgr2=0x7fe91403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:45.022 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.459+0000 7fe9306d7640 1 --2- 172.21.15.64:0/131632116 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fe914038f70 0x7fe91403b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe9180098c0 tx=0x7fe918006db0 comp rx=0 tx=0).stop 2024-09-12T03:19:45.023 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.459+0000 7fe9306d7640 1 -- 172.21.15.64:0/131632116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe928105e10 msgr2=0x7fe92819d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:45.023 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.459+0000 7fe9306d7640 1 --2- 172.21.15.64:0/131632116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe928105e10 0x7fe92819d6e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe9240097f0 tx=0x7fe924030d00 comp rx=0 tx=0).stop 2024-09-12T03:19:45.023 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.459+0000 7fe9306d7640 1 -- 172.21.15.64:0/131632116 shutdown_connections 2024-09-12T03:19:45.023 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.459+0000 7fe9306d7640 1 --2- 172.21.15.64:0/131632116 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fe914038f70 0x7fe91403b430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:45.023 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.459+0000 7fe9306d7640 1 --2- 172.21.15.64:0/131632116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe928105e10 0x7fe92819d6e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:45.023 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.459+0000 7fe9306d7640 1 -- 172.21.15.64:0/131632116 >> 172.21.15.64:0/131632116 conn(0x7fe928101c00 msgr2=0x7fe9281919e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:45.023 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.459+0000 7fe9306d7640 1 -- 172.21.15.64:0/131632116 shutdown_connections 2024-09-12T03:19:45.023 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:44.459+0000 7fe9306d7640 1 -- 172.21.15.64:0/131632116 wait complete. 2024-09-12T03:19:45.023 INFO:teuthology.orchestra.run.smithi064.stdout:mgr epoch 8 is available 2024-09-12T03:19:45.023 INFO:teuthology.orchestra.run.smithi064.stdout:Setting orchestrator backend to cephadm... 2024-09-12T03:19:46.151 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.395+0000 7f85149b5640 1 Processor -- start 2024-09-12T03:19:46.151 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.395+0000 7f85149b5640 1 -- start start 2024-09-12T03:19:46.151 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.395+0000 7f85149b5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f851007de50 0x7f851007c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:46.152 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.395+0000 7f85149b5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f851007c880 con 0x7f851007de50 2024-09-12T03:19:46.152 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.395+0000 7f850e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f851007de50 0x7f851007c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:46.152 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.396+0000 7f850e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f851007de50 0x7f851007c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43586/0 (socket says 172.21.15.64:43586) 2024-09-12T03:19:46.152 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.396+0000 7f850e59c640 1 -- 172.21.15.64:0/2661727086 learned_addr learned my addr 172.21.15.64:0/2661727086 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:19:46.152 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.396+0000 7f850e59c640 1 -- 172.21.15.64:0/2661727086 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f851007ca00 con 0x7f851007de50 2024-09-12T03:19:46.152 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.396+0000 7f850e59c640 1 --2- 172.21.15.64:0/2661727086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f851007de50 0x7f851007c2b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8500009820 tx=0x7f850002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e460dea215f0fd39 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:46.152 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.397+0000 7f850d59a640 1 -- 172.21.15.64:0/2661727086 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8500037070 con 0x7f851007de50 2024-09-12T03:19:46.153 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.397+0000 7f850d59a640 1 -- 172.21.15.64:0/2661727086 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f85000328d0 con 0x7f851007de50 2024-09-12T03:19:46.153 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.397+0000 7f850d59a640 1 -- 172.21.15.64:0/2661727086 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85000307d0 con 0x7f851007de50 2024-09-12T03:19:46.153 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.397+0000 7f85149b5640 1 -- 172.21.15.64:0/2661727086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f851007de50 msgr2=0x7f851007c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:46.153 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.397+0000 7f85149b5640 1 --2- 172.21.15.64:0/2661727086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f851007de50 0x7f851007c2b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8500009820 tx=0x7f850002a270 comp rx=0 tx=0).stop 2024-09-12T03:19:46.153 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.397+0000 7f85149b5640 1 -- 172.21.15.64:0/2661727086 shutdown_connections 2024-09-12T03:19:46.153 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.397+0000 7f85149b5640 1 --2- 172.21.15.64:0/2661727086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f851007de50 0x7f851007c2b0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.153 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.397+0000 7f85149b5640 1 -- 172.21.15.64:0/2661727086 >> 172.21.15.64:0/2661727086 conn(0x7f8510102ce0 msgr2=0x7f8510105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:46.154 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.397+0000 7f85149b5640 1 -- 172.21.15.64:0/2661727086 shutdown_connections 2024-09-12T03:19:46.154 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.397+0000 7f85149b5640 1 -- 172.21.15.64:0/2661727086 wait complete. 2024-09-12T03:19:46.154 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.398+0000 7f85149b5640 1 Processor -- start 2024-09-12T03:19:46.154 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.398+0000 7f85149b5640 1 -- start start 2024-09-12T03:19:46.154 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.398+0000 7f85149b5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f851007de50 0x7f851019d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:46.154 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.398+0000 7f85149b5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f851010dae0 con 0x7f851007de50 2024-09-12T03:19:46.154 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.398+0000 7f850e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f851007de50 0x7f851019d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:46.154 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.398+0000 7f850e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f851007de50 0x7f851019d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43594/0 (socket says 172.21.15.64:43594) 2024-09-12T03:19:46.154 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.399+0000 7f850e59c640 1 -- 172.21.15.64:0/216076308 learned_addr learned my addr 172.21.15.64:0/216076308 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:19:46.154 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.399+0000 7f850e59c640 1 -- 172.21.15.64:0/216076308 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f851019dbe0 con 0x7f851007de50 2024-09-12T03:19:46.155 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.399+0000 7f850e59c640 1 --2- 172.21.15.64:0/216076308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f851007de50 0x7f851019d6a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f85000097f0 tx=0x7f8500030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:46.155 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.399+0000 7f84ff7fe640 1 -- 172.21.15.64:0/216076308 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8500037040 con 0x7f851007de50 2024-09-12T03:19:46.155 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.399+0000 7f84ff7fe640 1 -- 172.21.15.64:0/216076308 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f8500032bb0 con 0x7f851007de50 2024-09-12T03:19:46.155 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.399+0000 7f85149b5640 1 -- 172.21.15.64:0/216076308 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f851019de70 con 0x7f851007de50 2024-09-12T03:19:46.155 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.399+0000 7f84ff7fe640 1 -- 172.21.15.64:0/216076308 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f850003a390 con 0x7f851007de50 2024-09-12T03:19:46.155 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.399+0000 7f85149b5640 1 -- 172.21.15.64:0/216076308 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85101a0b60 con 0x7f851007de50 2024-09-12T03:19:46.155 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.400+0000 7f84ff7fe640 1 -- 172.21.15.64:0/216076308 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f850003a5d0 con 0x7f851007de50 2024-09-12T03:19:46.155 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.400+0000 7f84ff7fe640 1 --2- 172.21.15.64:0/216076308 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f84f8038f70 0x7f84f803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:46.155 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.400+0000 7f85149b5640 1 -- 172.21.15.64:0/216076308 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f851007c330 con 0x7f851007de50 2024-09-12T03:19:46.155 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.400+0000 7f84ff7fe640 1 -- 172.21.15.64:0/216076308 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f850006c370 con 0x7f851007de50 2024-09-12T03:19:46.155 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.400+0000 7f850dd9b640 1 --2- 172.21.15.64:0/216076308 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f84f8038f70 0x7f84f803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:46.155 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.401+0000 7f850dd9b640 1 --2- 172.21.15.64:0/216076308 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f84f8038f70 0x7f84f803b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f85040098c0 tx=0x7f8504006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:46.156 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.405+0000 7f84ff7fe640 1 -- 172.21.15.64:0/216076308 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f85000313a0 con 0x7f851007de50 2024-09-12T03:19:46.156 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.597+0000 7f85149b5640 1 -- 172.21.15.64:0/216076308 --> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f8510103f60 con 0x7f84f8038f70 2024-09-12T03:19:46.156 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.602+0000 7f84ff7fe640 1 -- 172.21.15.64:0/216076308 <== mgr.14118 v2:172.21.15.64:6800/68603379 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8510103f60 con 0x7f84f8038f70 2024-09-12T03:19:46.156 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.604+0000 7f85149b5640 1 -- 172.21.15.64:0/216076308 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f84f8038f70 msgr2=0x7f84f803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:46.156 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.604+0000 7f85149b5640 1 --2- 172.21.15.64:0/216076308 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f84f8038f70 0x7f84f803b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f85040098c0 tx=0x7f8504006db0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.156 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.604+0000 7f85149b5640 1 -- 172.21.15.64:0/216076308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f851007de50 msgr2=0x7f851019d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:46.156 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.604+0000 7f85149b5640 1 --2- 172.21.15.64:0/216076308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f851007de50 0x7f851019d6a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f85000097f0 tx=0x7f8500030d00 comp rx=0 tx=0).stop 2024-09-12T03:19:46.156 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.604+0000 7f85149b5640 1 -- 172.21.15.64:0/216076308 shutdown_connections 2024-09-12T03:19:46.157 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.604+0000 7f85149b5640 1 --2- 172.21.15.64:0/216076308 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f84f8038f70 0x7f84f803b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.157 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.605+0000 7f85149b5640 1 --2- 172.21.15.64:0/216076308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f851007de50 0x7f851019d6a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.157 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.605+0000 7f85149b5640 1 -- 172.21.15.64:0/216076308 >> 172.21.15.64:0/216076308 conn(0x7f8510102ce0 msgr2=0x7f8510103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:46.157 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.605+0000 7f85149b5640 1 -- 172.21.15.64:0/216076308 shutdown_connections 2024-09-12T03:19:46.157 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:45.605+0000 7f85149b5640 1 -- 172.21.15.64:0/216076308 wait complete. 2024-09-12T03:19:46.703 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:46 smithi064 ceph-mon[30952]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:19:46.703 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:46 smithi064 ceph-mon[30952]: from='mgr.14118 172.21.15.64:0/358528520' entity='mgr.x' 2024-09-12T03:19:46.703 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:46 smithi064 ceph-mon[30952]: from='mgr.14118 172.21.15.64:0/358528520' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:19:47.222 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-12T03:19:47.222 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.514+0000 7f1c9f5c2640 1 Processor -- start 2024-09-12T03:19:47.222 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.514+0000 7f1c9f5c2640 1 -- start start 2024-09-12T03:19:47.222 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.515+0000 7f1c9f5c2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c9807de40 0x7f1c9807c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:47.222 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.515+0000 7f1c9f5c2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9807c8c0 con 0x7f1c9807de40 2024-09-12T03:19:47.222 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.515+0000 7f1c9d35e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c9807de40 0x7f1c9807c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:47.222 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.515+0000 7f1c9d35e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c9807de40 0x7f1c9807c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43602/0 (socket says 172.21.15.64:43602) 2024-09-12T03:19:47.222 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.515+0000 7f1c9d35e640 1 -- 172.21.15.64:0/4203999794 learned_addr learned my addr 172.21.15.64:0/4203999794 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:19:47.222 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.515+0000 7f1c9d35e640 1 -- 172.21.15.64:0/4203999794 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c9807ca40 con 0x7f1c9807de40 2024-09-12T03:19:47.222 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.516+0000 7f1c9d35e640 1 --2- 172.21.15.64:0/4203999794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c9807de40 0x7f1c9807c2f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1c90009820 tx=0x7f1c9002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91ae3ab6c46afee0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:47.222 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.516+0000 7f1c97fff640 1 -- 172.21.15.64:0/4203999794 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c90037070 con 0x7f1c9807de40 2024-09-12T03:19:47.222 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.516+0000 7f1c97fff640 1 -- 172.21.15.64:0/4203999794 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f1c900328d0 con 0x7f1c9807de40 2024-09-12T03:19:47.223 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.516+0000 7f1c97fff640 1 -- 172.21.15.64:0/4203999794 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c900307d0 con 0x7f1c9807de40 2024-09-12T03:19:47.223 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.516+0000 7f1c9f5c2640 1 -- 172.21.15.64:0/4203999794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c9807de40 msgr2=0x7f1c9807c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:47.223 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.516+0000 7f1c9f5c2640 1 --2- 172.21.15.64:0/4203999794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c9807de40 0x7f1c9807c2f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1c90009820 tx=0x7f1c9002a270 comp rx=0 tx=0).stop 2024-09-12T03:19:47.223 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.517+0000 7f1c9f5c2640 1 -- 172.21.15.64:0/4203999794 shutdown_connections 2024-09-12T03:19:47.223 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.517+0000 7f1c9f5c2640 1 --2- 172.21.15.64:0/4203999794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c9807de40 0x7f1c9807c2f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.223 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.517+0000 7f1c9f5c2640 1 -- 172.21.15.64:0/4203999794 >> 172.21.15.64:0/4203999794 conn(0x7f1c98102d00 msgr2=0x7f1c98105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:47.223 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.517+0000 7f1c9f5c2640 1 -- 172.21.15.64:0/4203999794 shutdown_connections 2024-09-12T03:19:47.223 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.517+0000 7f1c9f5c2640 1 -- 172.21.15.64:0/4203999794 wait complete. 2024-09-12T03:19:47.223 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.517+0000 7f1c9f5c2640 1 Processor -- start 2024-09-12T03:19:47.223 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.517+0000 7f1c9f5c2640 1 -- start start 2024-09-12T03:19:47.224 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.517+0000 7f1c9f5c2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c9807de40 0x7f1c9819d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:47.224 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.517+0000 7f1c9f5c2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9810db20 con 0x7f1c9807de40 2024-09-12T03:19:47.224 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.518+0000 7f1c9d35e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c9807de40 0x7f1c9819d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:47.224 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.518+0000 7f1c9d35e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c9807de40 0x7f1c9819d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43612/0 (socket says 172.21.15.64:43612) 2024-09-12T03:19:47.224 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.518+0000 7f1c9d35e640 1 -- 172.21.15.64:0/1163001492 learned_addr learned my addr 172.21.15.64:0/1163001492 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:19:47.224 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.518+0000 7f1c9d35e640 1 -- 172.21.15.64:0/1163001492 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c9819dca0 con 0x7f1c9807de40 2024-09-12T03:19:47.224 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.518+0000 7f1c9d35e640 1 --2- 172.21.15.64:0/1163001492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c9807de40 0x7f1c9819d760 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1c90030520 tx=0x7f1c90032dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:47.224 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.518+0000 7f1c967fc640 1 -- 172.21.15.64:0/1163001492 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c90042070 con 0x7f1c9807de40 2024-09-12T03:19:47.224 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.518+0000 7f1c967fc640 1 -- 172.21.15.64:0/1163001492 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f1c90030cc0 con 0x7f1c9807de40 2024-09-12T03:19:47.224 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.518+0000 7f1c9f5c2640 1 -- 172.21.15.64:0/1163001492 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c9819df30 con 0x7f1c9807de40 2024-09-12T03:19:47.224 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.518+0000 7f1c967fc640 1 -- 172.21.15.64:0/1163001492 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c90037040 con 0x7f1c9807de40 2024-09-12T03:19:47.224 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.518+0000 7f1c9f5c2640 1 -- 172.21.15.64:0/1163001492 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c981a0c20 con 0x7f1c9807de40 2024-09-12T03:19:47.225 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.519+0000 7f1c967fc640 1 -- 172.21.15.64:0/1163001492 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f1c90045070 con 0x7f1c9807de40 2024-09-12T03:19:47.225 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.519+0000 7f1c967fc640 1 --2- 172.21.15.64:0/1163001492 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f1c80038fc0 0x7f1c8003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:47.225 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.519+0000 7f1c967fc640 1 -- 172.21.15.64:0/1163001492 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1c9006bf90 con 0x7f1c9807de40 2024-09-12T03:19:47.225 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.519+0000 7f1c9f5c2640 1 -- 172.21.15.64:0/1163001492 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c9807c370 con 0x7f1c9807de40 2024-09-12T03:19:47.225 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.519+0000 7f1c9cb5d640 1 --2- 172.21.15.64:0/1163001492 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f1c80038fc0 0x7f1c8003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:47.225 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.520+0000 7f1c9cb5d640 1 --2- 172.21.15.64:0/1163001492 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f1c80038fc0 0x7f1c8003b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c00ad30 tx=0x7f1c8c0093f0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:47.225 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.524+0000 7f1c967fc640 1 -- 172.21.15.64:0/1163001492 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1c90046e00 con 0x7f1c9807de40 2024-09-12T03:19:47.225 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.702+0000 7f1c9f5c2640 1 -- 172.21.15.64:0/1163001492 --> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f1c98104000 con 0x7f1c80038fc0 2024-09-12T03:19:47.225 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.702+0000 7f1c967fc640 1 -- 172.21.15.64:0/1163001492 <== mgr.14118 v2:172.21.15.64:6800/68603379 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f1c98104000 con 0x7f1c80038fc0 2024-09-12T03:19:47.225 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.705+0000 7f1c9f5c2640 1 -- 172.21.15.64:0/1163001492 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f1c80038fc0 msgr2=0x7f1c8003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:47.226 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.705+0000 7f1c9f5c2640 1 --2- 172.21.15.64:0/1163001492 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f1c80038fc0 0x7f1c8003b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c00ad30 tx=0x7f1c8c0093f0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.226 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.705+0000 7f1c9f5c2640 1 -- 172.21.15.64:0/1163001492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c9807de40 msgr2=0x7f1c9819d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:47.226 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.705+0000 7f1c9f5c2640 1 --2- 172.21.15.64:0/1163001492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c9807de40 0x7f1c9819d760 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1c90030520 tx=0x7f1c90032dd0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.226 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.705+0000 7f1c9f5c2640 1 -- 172.21.15.64:0/1163001492 shutdown_connections 2024-09-12T03:19:47.226 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.705+0000 7f1c9f5c2640 1 --2- 172.21.15.64:0/1163001492 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f1c80038fc0 0x7f1c8003b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.226 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.705+0000 7f1c9f5c2640 1 --2- 172.21.15.64:0/1163001492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c9807de40 0x7f1c9819d760 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.226 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.705+0000 7f1c9f5c2640 1 -- 172.21.15.64:0/1163001492 >> 172.21.15.64:0/1163001492 conn(0x7f1c98102d00 msgr2=0x7f1c981038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:47.226 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.705+0000 7f1c9f5c2640 1 -- 172.21.15.64:0/1163001492 shutdown_connections 2024-09-12T03:19:47.226 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:46.705+0000 7f1c9f5c2640 1 -- 172.21.15.64:0/1163001492 wait complete. 2024-09-12T03:19:47.226 INFO:teuthology.orchestra.run.smithi064.stdout:Generating ssh key... 2024-09-12T03:19:48.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:47 smithi064 ceph-mon[30952]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:19:48.453 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:48 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: Generating public/private rsa key pair. 2024-09-12T03:19:48.453 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:48 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: Your identification has been saved in /tmp/tmpnujw1718/key 2024-09-12T03:19:48.453 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:48 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: Your public key has been saved in /tmp/tmpnujw1718/key.pub 2024-09-12T03:19:48.453 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:48 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: The key fingerprint is: 2024-09-12T03:19:48.453 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:48 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: SHA256:BE0/E4ca+Tknl+iL4QorxHMElWiL27vFDMUtLVkgAUM ceph-b0812460-70b5-11ef-bcea-c7b262605968 2024-09-12T03:19:48.453 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:48 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: The key's randomart image is: 2024-09-12T03:19:48.453 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:48 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: +---[RSA 3072]----+ 2024-09-12T03:19:48.453 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:48 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: |+Eo+oooo..... | 2024-09-12T03:19:48.454 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:48 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: | .+o.= .+..o | 2024-09-12T03:19:48.454 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:48 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: | o o* o .++o . | 2024-09-12T03:19:48.454 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:48 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: |. ...o .. *o+ | 2024-09-12T03:19:48.454 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:48 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: | +.. S. = | 2024-09-12T03:19:48.454 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:48 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: |. =+. . . | 2024-09-12T03:19:48.454 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:48 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: | . += . o . | 2024-09-12T03:19:48.454 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:48 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: | o. o o . | 2024-09-12T03:19:48.454 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:48 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: | .o. .. | 2024-09-12T03:19:48.454 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:19:48 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: +----[SHA256]-----+ 2024-09-12T03:19:48.719 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:47.586+0000 7fa10f7a7640 1 Processor -- start 2024-09-12T03:19:48.720 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:47.586+0000 7fa10f7a7640 1 -- start start 2024-09-12T03:19:48.720 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:47.586+0000 7fa10f7a7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa10807de50 0x7fa10807c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:48.720 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:47.586+0000 7fa10f7a7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa10807c880 con 0x7fa10807de50 2024-09-12T03:19:48.720 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:47.586+0000 7fa10d543640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa10807de50 0x7fa10807c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:48.720 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:47.586+0000 7fa10d543640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa10807de50 0x7fa10807c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42276/0 (socket says 172.21.15.64:42276) 2024-09-12T03:19:48.720 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:47.586+0000 7fa10d543640 1 -- 172.21.15.64:0/2314386729 learned_addr learned my addr 172.21.15.64:0/2314386729 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:19:48.720 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:47.587+0000 7fa10d543640 1 -- 172.21.15.64:0/2314386729 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa10807ca00 con 0x7fa10807de50 2024-09-12T03:19:48.720 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:47.587+0000 7fa10d543640 1 --2- 172.21.15.64:0/2314386729 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa10807de50 0x7fa10807c2b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa104009820 tx=0x7fa10402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba639a7358d68c44 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:48.720 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:47.587+0000 7fa0fffff640 1 -- 172.21.15.64:0/2314386729 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa104037070 con 0x7fa10807de50 2024-09-12T03:19:48.720 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:47.588+0000 7fa0fffff640 1 -- 172.21.15.64:0/2314386729 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa1040328d0 con 0x7fa10807de50 2024-09-12T03:19:48.720 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:47.588+0000 7fa0fffff640 1 -- 172.21.15.64:0/2314386729 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1040307d0 con 0x7fa10807de50 2024-09-12T03:19:48.721 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:47.588+0000 7fa10f7a7640 1 -- 172.21.15.64:0/2314386729 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa10807de50 msgr2=0x7fa10807c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:48.721 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:47.588+0000 7fa10f7a7640 1 --2- 172.21.15.64:0/2314386729 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa10807de50 0x7fa10807c2b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa104009820 tx=0x7fa10402a270 comp rx=0 tx=0).stop 2024-09-12T03:19:48.721 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:47.588+0000 7fa10f7a7640 1 -- 172.21.15.64:0/2314386729 shutdown_connections 2024-09-12T03:19:48.721 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:47.588+0000 7fa10f7a7640 1 --2- 172.21.15.64:0/2314386729 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa10807de50 0x7fa10807c2b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:48.721 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:47.588+0000 7fa10f7a7640 1 -- 172.21.15.64:0/2314386729 >> 172.21.15.64:0/2314386729 conn(0x7fa108102d30 msgr2=0x7fa108105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:48.721 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:47.588+0000 7fa10f7a7640 1 -- 172.21.15.64:0/2314386729 shutdown_connections 2024-09-12T03:19:48.721 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:47.588+0000 7fa10f7a7640 1 -- 172.21.15.64:0/2314386729 wait complete. 2024-09-12T03:19:48.721 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:47.589+0000 7fa10f7a7640 1 Processor -- start 2024-09-12T03:19:48.721 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:47.589+0000 7fa10f7a7640 1 -- start start 2024-09-12T03:19:48.722 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:47.589+0000 7fa10f7a7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa10807de50 0x7fa10819d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:48.722 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:47.589+0000 7fa10f7a7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa10810db10 con 0x7fa10807de50 2024-09-12T03:19:48.722 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:47.589+0000 7fa10d543640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa10807de50 0x7fa10819d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:48.722 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:47.589+0000 7fa10d543640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa10807de50 0x7fa10819d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42284/0 (socket says 172.21.15.64:42284) 2024-09-12T03:19:48.722 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:47.589+0000 7fa10d543640 1 -- 172.21.15.64:0/2005597711 learned_addr learned my addr 172.21.15.64:0/2005597711 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:19:48.722 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:47.589+0000 7fa10d543640 1 -- 172.21.15.64:0/2005597711 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa10819dbf0 con 0x7fa10807de50 2024-09-12T03:19:48.722 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:47.589+0000 7fa10d543640 1 --2- 172.21.15.64:0/2005597711 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa10807de50 0x7fa10819d6b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa104032e10 tx=0x7fa104032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:48.722 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:47.590+0000 7fa0fe7fc640 1 -- 172.21.15.64:0/2005597711 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa104042070 con 0x7fa10807de50 2024-09-12T03:19:48.722 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:47.590+0000 7fa0fe7fc640 1 -- 172.21.15.64:0/2005597711 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa104030c80 con 0x7fa10807de50 2024-09-12T03:19:48.722 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:47.590+0000 7fa10f7a7640 1 -- 172.21.15.64:0/2005597711 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa10819de80 con 0x7fa10807de50 2024-09-12T03:19:48.722 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:47.590+0000 7fa0fe7fc640 1 -- 172.21.15.64:0/2005597711 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa104037040 con 0x7fa10807de50 2024-09-12T03:19:48.722 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:47.590+0000 7fa10f7a7640 1 -- 172.21.15.64:0/2005597711 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1081a0b70 con 0x7fa10807de50 2024-09-12T03:19:48.723 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:47.591+0000 7fa10f7a7640 1 -- 172.21.15.64:0/2005597711 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa10807c330 con 0x7fa10807de50 2024-09-12T03:19:48.723 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:47.591+0000 7fa0fe7fc640 1 -- 172.21.15.64:0/2005597711 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fa104045070 con 0x7fa10807de50 2024-09-12T03:19:48.723 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:47.591+0000 7fa0fe7fc640 1 --2- 172.21.15.64:0/2005597711 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fa0f4038fc0 0x7fa0f403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:48.723 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:47.591+0000 7fa0fe7fc640 1 -- 172.21.15.64:0/2005597711 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa10406be60 con 0x7fa10807de50 2024-09-12T03:19:48.723 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:47.591+0000 7fa10cd42640 1 --2- 172.21.15.64:0/2005597711 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fa0f4038fc0 0x7fa0f403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:48.723 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:47.592+0000 7fa10cd42640 1 --2- 172.21.15.64:0/2005597711 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fa0f4038fc0 0x7fa0f403b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa0f80098c0 tx=0x7fa0f8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:48.724 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:47.597+0000 7fa0fe7fc640 1 -- 172.21.15.64:0/2005597711 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa10404b170 con 0x7fa10807de50 2024-09-12T03:19:48.724 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:47.775+0000 7fa10f7a7640 1 -- 172.21.15.64:0/2005597711 --> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fa108103fb0 con 0x7fa0f4038fc0 2024-09-12T03:19:48.724 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:48.158+0000 7fa0fe7fc640 1 -- 172.21.15.64:0/2005597711 <== mgr.14118 v2:172.21.15.64:6800/68603379 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa108103fb0 con 0x7fa0f4038fc0 2024-09-12T03:19:48.724 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:48.160+0000 7fa10f7a7640 1 -- 172.21.15.64:0/2005597711 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fa0f4038fc0 msgr2=0x7fa0f403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:48.724 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:48.160+0000 7fa10f7a7640 1 --2- 172.21.15.64:0/2005597711 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fa0f4038fc0 0x7fa0f403b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa0f80098c0 tx=0x7fa0f8006db0 comp rx=0 tx=0).stop 2024-09-12T03:19:48.724 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:48.160+0000 7fa10f7a7640 1 -- 172.21.15.64:0/2005597711 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa10807de50 msgr2=0x7fa10819d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:48.724 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:48.160+0000 7fa10f7a7640 1 --2- 172.21.15.64:0/2005597711 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa10807de50 0x7fa10819d6b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa104032e10 tx=0x7fa104032e40 comp rx=0 tx=0).stop 2024-09-12T03:19:48.725 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:48.160+0000 7fa10f7a7640 1 -- 172.21.15.64:0/2005597711 shutdown_connections 2024-09-12T03:19:48.725 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:48.160+0000 7fa10f7a7640 1 --2- 172.21.15.64:0/2005597711 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fa0f4038fc0 0x7fa0f403b480 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:48.725 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:48.160+0000 7fa10f7a7640 1 --2- 172.21.15.64:0/2005597711 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa10807de50 0x7fa10819d6b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:48.725 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:48.160+0000 7fa10f7a7640 1 -- 172.21.15.64:0/2005597711 >> 172.21.15.64:0/2005597711 conn(0x7fa108102d30 msgr2=0x7fa1081038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:48.725 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:48.160+0000 7fa10f7a7640 1 -- 172.21.15.64:0/2005597711 shutdown_connections 2024-09-12T03:19:48.725 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:48.160+0000 7fa10f7a7640 1 -- 172.21.15.64:0/2005597711 wait complete. 2024-09-12T03:19:49.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:49 smithi064 ceph-mon[30952]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:19:49.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:49 smithi064 ceph-mon[30952]: Generating ssh key... 2024-09-12T03:19:49.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:49 smithi064 ceph-mon[30952]: from='mgr.14118 172.21.15.64:0/358528520' entity='mgr.x' 2024-09-12T03:19:49.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:49 smithi064 ceph-mon[30952]: from='mgr.14118 172.21.15.64:0/358528520' entity='mgr.x' 2024-09-12T03:19:49.899 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjKZXV/vMRD/iDwNqYR5tmOeS4R7ZF2rCDqp+J9IMU1lop70Rz+kPbLst++O0MCBxCIy743TcXpd8ulruGb+NeTKvkozXFXGCJZsdkYKcyqhcMh0NCJ2iGbRIEwIM3e0hNTi1dRHFNiAU+rmwj3PBS8K3g8QcdPHrFhTwozow1yy/P2/nfJ2aox2H4KPLavxZv/4UicUxJp4Np8DzIdAckBGWghbXv3Js0BHEgW7pd2QSD8tNl9xJse/G4RZp+B1KpB1WEanEgFoHGeketQqqyXt+x9kL1U9wdbGz3+O7fU0sGGRg6VGF+9zg7Jah6BPwqDFgv817NzRKvNHQkfdgUwWw2pOLiFOUDf7ZtKO07mhzcAu6PGOZxZJb1Edpx8QisUdgCACemNoY3/quLapsMjsbqKjqLe5vR1ckBKivnEnhfM2MIRue6/AY1xnZq5NwGMReqOSFjiyaRXSLCPaBKM1Ui33UE0oTXw3j7C0t47LZVx8elKY4TWOOpycFwNb8= ceph-b0812460-70b5-11ef-bcea-c7b262605968 2024-09-12T03:19:49.900 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.128+0000 7fc303b75640 1 Processor -- start 2024-09-12T03:19:49.900 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.128+0000 7fc303b75640 1 -- start start 2024-09-12T03:19:49.900 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.128+0000 7fc303b75640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2fc109890 0x7fc2fc109c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:49.900 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.128+0000 7fc303b75640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc2fc10a260 con 0x7fc2fc109890 2024-09-12T03:19:49.900 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.128+0000 7fc301911640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2fc109890 0x7fc2fc109c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:49.900 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.128+0000 7fc301911640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2fc109890 0x7fc2fc109c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42288/0 (socket says 172.21.15.64:42288) 2024-09-12T03:19:49.900 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.128+0000 7fc301911640 1 -- 172.21.15.64:0/3455756276 learned_addr learned my addr 172.21.15.64:0/3455756276 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:19:49.900 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.129+0000 7fc301911640 1 -- 172.21.15.64:0/3455756276 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2fc10a900 con 0x7fc2fc109890 2024-09-12T03:19:49.901 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.129+0000 7fc301911640 1 --2- 172.21.15.64:0/3455756276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2fc109890 0x7fc2fc109c90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009820 tx=0x7fc2f802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b0f3c467e6164e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:49.901 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.129+0000 7fc30090f640 1 -- 172.21.15.64:0/3455756276 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2f8037070 con 0x7fc2fc109890 2024-09-12T03:19:49.901 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.130+0000 7fc30090f640 1 -- 172.21.15.64:0/3455756276 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fc2f80328d0 con 0x7fc2fc109890 2024-09-12T03:19:49.901 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.130+0000 7fc30090f640 1 -- 172.21.15.64:0/3455756276 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2f80307d0 con 0x7fc2fc109890 2024-09-12T03:19:49.901 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.130+0000 7fc303b75640 1 -- 172.21.15.64:0/3455756276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2fc109890 msgr2=0x7fc2fc109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:49.901 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.130+0000 7fc303b75640 1 --2- 172.21.15.64:0/3455756276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2fc109890 0x7fc2fc109c90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009820 tx=0x7fc2f802a270 comp rx=0 tx=0).stop 2024-09-12T03:19:49.901 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.130+0000 7fc303b75640 1 -- 172.21.15.64:0/3455756276 shutdown_connections 2024-09-12T03:19:49.901 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.130+0000 7fc303b75640 1 --2- 172.21.15.64:0/3455756276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2fc109890 0x7fc2fc109c90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:49.901 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.130+0000 7fc303b75640 1 -- 172.21.15.64:0/3455756276 >> 172.21.15.64:0/3455756276 conn(0x7fc2fc07cd00 msgr2=0x7fc2fc07d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:49.902 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.130+0000 7fc303b75640 1 -- 172.21.15.64:0/3455756276 shutdown_connections 2024-09-12T03:19:49.902 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.130+0000 7fc303b75640 1 -- 172.21.15.64:0/3455756276 wait complete. 2024-09-12T03:19:49.902 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.131+0000 7fc303b75640 1 Processor -- start 2024-09-12T03:19:49.902 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.131+0000 7fc303b75640 1 -- start start 2024-09-12T03:19:49.902 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.131+0000 7fc303b75640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2fc109890 0x7fc2fc19d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:49.902 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.131+0000 7fc303b75640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc2fc10dcb0 con 0x7fc2fc109890 2024-09-12T03:19:49.902 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.131+0000 7fc301911640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2fc109890 0x7fc2fc19d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:49.902 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.131+0000 7fc301911640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2fc109890 0x7fc2fc19d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42294/0 (socket says 172.21.15.64:42294) 2024-09-12T03:19:49.902 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.131+0000 7fc301911640 1 -- 172.21.15.64:0/1762775362 learned_addr learned my addr 172.21.15.64:0/1762775362 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:19:49.903 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.131+0000 7fc301911640 1 -- 172.21.15.64:0/1762775362 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2fc19dc00 con 0x7fc2fc109890 2024-09-12T03:19:49.903 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.132+0000 7fc301911640 1 --2- 172.21.15.64:0/1762775362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2fc109890 0x7fc2fc19d6c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc2f80097f0 tx=0x7fc2f8030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:49.903 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.132+0000 7fc2eeffd640 1 -- 172.21.15.64:0/1762775362 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2f8037040 con 0x7fc2fc109890 2024-09-12T03:19:49.903 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.132+0000 7fc2eeffd640 1 -- 172.21.15.64:0/1762775362 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fc2f8032bb0 con 0x7fc2fc109890 2024-09-12T03:19:49.903 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.132+0000 7fc303b75640 1 -- 172.21.15.64:0/1762775362 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2fc19de90 con 0x7fc2fc109890 2024-09-12T03:19:49.903 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.132+0000 7fc2eeffd640 1 -- 172.21.15.64:0/1762775362 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2f803a390 con 0x7fc2fc109890 2024-09-12T03:19:49.903 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.132+0000 7fc303b75640 1 -- 172.21.15.64:0/1762775362 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2fc1a0b80 con 0x7fc2fc109890 2024-09-12T03:19:49.903 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.133+0000 7fc2eeffd640 1 -- 172.21.15.64:0/1762775362 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fc2f803a5d0 con 0x7fc2fc109890 2024-09-12T03:19:49.903 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.133+0000 7fc303b75640 1 -- 172.21.15.64:0/1762775362 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2fc10de30 con 0x7fc2fc109890 2024-09-12T03:19:49.904 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.133+0000 7fc2eeffd640 1 --2- 172.21.15.64:0/1762775362 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fc2e8038f70 0x7fc2e803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:49.904 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.133+0000 7fc2eeffd640 1 -- 172.21.15.64:0/1762775362 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc2f806c310 con 0x7fc2fc109890 2024-09-12T03:19:49.904 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.133+0000 7fc301110640 1 --2- 172.21.15.64:0/1762775362 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fc2e8038f70 0x7fc2e803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:49.904 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.133+0000 7fc301110640 1 --2- 172.21.15.64:0/1762775362 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fc2e8038f70 0x7fc2e803b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc2f00098c0 tx=0x7fc2f0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:49.904 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.138+0000 7fc2eeffd640 1 -- 172.21.15.64:0/1762775362 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc2f803a8c0 con 0x7fc2fc109890 2024-09-12T03:19:49.904 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.315+0000 7fc303b75640 1 -- 172.21.15.64:0/1762775362 --> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fc2fc107bd0 con 0x7fc2e8038f70 2024-09-12T03:19:49.905 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.316+0000 7fc2eeffd640 1 -- 172.21.15.64:0/1762775362 <== mgr.14118 v2:172.21.15.64:6800/68603379 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fc2fc107bd0 con 0x7fc2e8038f70 2024-09-12T03:19:49.905 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.318+0000 7fc303b75640 1 -- 172.21.15.64:0/1762775362 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fc2e8038f70 msgr2=0x7fc2e803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:49.905 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.318+0000 7fc303b75640 1 --2- 172.21.15.64:0/1762775362 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fc2e8038f70 0x7fc2e803b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc2f00098c0 tx=0x7fc2f0006db0 comp rx=0 tx=0).stop 2024-09-12T03:19:49.905 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.318+0000 7fc303b75640 1 -- 172.21.15.64:0/1762775362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2fc109890 msgr2=0x7fc2fc19d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:49.905 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.318+0000 7fc303b75640 1 --2- 172.21.15.64:0/1762775362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2fc109890 0x7fc2fc19d6c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc2f80097f0 tx=0x7fc2f8030d00 comp rx=0 tx=0).stop 2024-09-12T03:19:49.905 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.318+0000 7fc303b75640 1 -- 172.21.15.64:0/1762775362 shutdown_connections 2024-09-12T03:19:49.905 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.318+0000 7fc303b75640 1 --2- 172.21.15.64:0/1762775362 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fc2e8038f70 0x7fc2e803b430 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:49.905 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.318+0000 7fc303b75640 1 --2- 172.21.15.64:0/1762775362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2fc109890 0x7fc2fc19d6c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:49.906 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.318+0000 7fc303b75640 1 -- 172.21.15.64:0/1762775362 >> 172.21.15.64:0/1762775362 conn(0x7fc2fc07cd00 msgr2=0x7fc2fc1074c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:49.906 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.318+0000 7fc303b75640 1 -- 172.21.15.64:0/1762775362 shutdown_connections 2024-09-12T03:19:49.906 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:49.318+0000 7fc303b75640 1 -- 172.21.15.64:0/1762775362 wait complete. 2024-09-12T03:19:49.906 INFO:teuthology.orchestra.run.smithi064.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-12T03:19:49.906 INFO:teuthology.orchestra.run.smithi064.stdout:Adding key to root@localhost authorized_keys... 2024-09-12T03:19:49.906 INFO:teuthology.orchestra.run.smithi064.stdout:Adding host smithi064... 2024-09-12T03:19:50.509 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:50 smithi064 ceph-mon[30952]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:19:51.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:51 smithi064 ceph-mon[30952]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi064", "addr": "172.21.15.64", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:19:52.475 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:52 smithi064 ceph-mon[30952]: Deploying cephadm binary to smithi064 2024-09-12T03:19:54.958 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout Added host 'smithi064' with addr '172.21.15.64' 2024-09-12T03:19:54.959 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:50.320+0000 7f9c0e1e5640 1 Processor -- start 2024-09-12T03:19:54.959 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:50.320+0000 7f9c0e1e5640 1 -- start start 2024-09-12T03:19:54.959 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:50.320+0000 7f9c0e1e5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c08107510 0x7f9c08107910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:54.959 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:50.320+0000 7f9c0e1e5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9c08107ee0 con 0x7f9c08107510 2024-09-12T03:19:54.959 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:50.320+0000 7f9c077fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c08107510 0x7f9c08107910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:54.959 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:50.320+0000 7f9c077fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c08107510 0x7f9c08107910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42298/0 (socket says 172.21.15.64:42298) 2024-09-12T03:19:54.959 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:50.320+0000 7f9c077fe640 1 -- 172.21.15.64:0/2875328544 learned_addr learned my addr 172.21.15.64:0/2875328544 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:19:54.959 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:50.320+0000 7f9c077fe640 1 -- 172.21.15.64:0/2875328544 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c08108710 con 0x7f9c08107510 2024-09-12T03:19:54.959 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:50.321+0000 7f9c077fe640 1 --2- 172.21.15.64:0/2875328544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c08107510 0x7f9c08107910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8009820 tx=0x7f9bf802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e31ea672562d42c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:54.959 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:50.321+0000 7f9c067fc640 1 -- 172.21.15.64:0/2875328544 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9bf8037070 con 0x7f9c08107510 2024-09-12T03:19:54.959 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:50.321+0000 7f9c067fc640 1 -- 172.21.15.64:0/2875328544 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f9bf80328d0 con 0x7f9c08107510 2024-09-12T03:19:54.960 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:50.321+0000 7f9c067fc640 1 -- 172.21.15.64:0/2875328544 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9bf80307d0 con 0x7f9c08107510 2024-09-12T03:19:54.960 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:50.322+0000 7f9c0e1e5640 1 -- 172.21.15.64:0/2875328544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c08107510 msgr2=0x7f9c08107910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:54.960 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:50.322+0000 7f9c0e1e5640 1 --2- 172.21.15.64:0/2875328544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c08107510 0x7f9c08107910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8009820 tx=0x7f9bf802a270 comp rx=0 tx=0).stop 2024-09-12T03:19:54.960 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:50.322+0000 7f9c0e1e5640 1 -- 172.21.15.64:0/2875328544 shutdown_connections 2024-09-12T03:19:54.960 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:50.322+0000 7f9c0e1e5640 1 --2- 172.21.15.64:0/2875328544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c08107510 0x7f9c08107910 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:54.960 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:50.322+0000 7f9c0e1e5640 1 -- 172.21.15.64:0/2875328544 >> 172.21.15.64:0/2875328544 conn(0x7f9c08102ce0 msgr2=0x7f9c08105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:54.960 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:50.322+0000 7f9c0e1e5640 1 -- 172.21.15.64:0/2875328544 shutdown_connections 2024-09-12T03:19:54.960 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:50.322+0000 7f9c0e1e5640 1 -- 172.21.15.64:0/2875328544 wait complete. 2024-09-12T03:19:54.960 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:50.323+0000 7f9c0e1e5640 1 Processor -- start 2024-09-12T03:19:54.960 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:50.323+0000 7f9c0e1e5640 1 -- start start 2024-09-12T03:19:54.960 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:50.323+0000 7f9c0e1e5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c08107510 0x7f9c0819b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:54.960 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:50.323+0000 7f9c0e1e5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0810bac0 con 0x7f9c08107510 2024-09-12T03:19:54.960 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:50.323+0000 7f9c077fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c08107510 0x7f9c0819b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:54.961 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:50.323+0000 7f9c077fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c08107510 0x7f9c0819b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42304/0 (socket says 172.21.15.64:42304) 2024-09-12T03:19:54.961 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:50.323+0000 7f9c077fe640 1 -- 172.21.15.64:0/2850185964 learned_addr learned my addr 172.21.15.64:0/2850185964 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:19:54.961 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:50.324+0000 7f9c077fe640 1 -- 172.21.15.64:0/2850185964 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c0819b9f0 con 0x7f9c08107510 2024-09-12T03:19:54.961 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:50.324+0000 7f9c077fe640 1 --2- 172.21.15.64:0/2850185964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c08107510 0x7f9c0819b4b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8009940 tx=0x7f9bf8032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:54.961 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:50.324+0000 7f9c04ff9640 1 -- 172.21.15.64:0/2850185964 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9bf8042070 con 0x7f9c08107510 2024-09-12T03:19:54.961 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:50.324+0000 7f9c04ff9640 1 -- 172.21.15.64:0/2850185964 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f9bf8030c80 con 0x7f9c08107510 2024-09-12T03:19:54.961 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:50.324+0000 7f9c0e1e5640 1 -- 172.21.15.64:0/2850185964 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c0819bc80 con 0x7f9c08107510 2024-09-12T03:19:54.961 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:50.324+0000 7f9c04ff9640 1 -- 172.21.15.64:0/2850185964 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9bf8037040 con 0x7f9c08107510 2024-09-12T03:19:54.961 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:50.324+0000 7f9c0e1e5640 1 -- 172.21.15.64:0/2850185964 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c0819e970 con 0x7f9c08107510 2024-09-12T03:19:54.961 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:50.325+0000 7f9c04ff9640 1 -- 172.21.15.64:0/2850185964 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f9bf8045070 con 0x7f9c08107510 2024-09-12T03:19:54.961 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:50.325+0000 7f9c0e1e5640 1 -- 172.21.15.64:0/2850185964 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c0810bc40 con 0x7f9c08107510 2024-09-12T03:19:54.961 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:50.325+0000 7f9c04ff9640 1 --2- 172.21.15.64:0/2850185964 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f9bf0038f70 0x7f9bf003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:54.961 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:50.325+0000 7f9c04ff9640 1 -- 172.21.15.64:0/2850185964 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9bf806c060 con 0x7f9c08107510 2024-09-12T03:19:54.961 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:50.325+0000 7f9c06ffd640 1 --2- 172.21.15.64:0/2850185964 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f9bf0038f70 0x7f9bf003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:54.962 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:50.326+0000 7f9c06ffd640 1 --2- 172.21.15.64:0/2850185964 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f9bf0038f70 0x7f9bf003b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0098c0 tx=0x7f9bfc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:54.962 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:50.330+0000 7f9c04ff9640 1 -- 172.21.15.64:0/2850185964 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9bf8045360 con 0x7f9c08107510 2024-09-12T03:19:54.962 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:50.508+0000 7f9c0e1e5640 1 -- 172.21.15.64:0/2850185964 --> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi064", "addr": "172.21.15.64", "target": ["mon-mgr", ""]}) -- 0x7f9c08103f40 con 0x7f9bf0038f70 2024-09-12T03:19:54.962 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:54.396+0000 7f9c04ff9640 1 -- 172.21.15.64:0/2850185964 <== mgr.14118 v2:172.21.15.64:6800/68603379 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f9c08103f40 con 0x7f9bf0038f70 2024-09-12T03:19:54.962 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:54.400+0000 7f9c0e1e5640 1 -- 172.21.15.64:0/2850185964 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f9bf0038f70 msgr2=0x7f9bf003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:54.962 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:54.400+0000 7f9c0e1e5640 1 --2- 172.21.15.64:0/2850185964 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f9bf0038f70 0x7f9bf003b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0098c0 tx=0x7f9bfc006db0 comp rx=0 tx=0).stop 2024-09-12T03:19:54.962 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:54.400+0000 7f9c0e1e5640 1 -- 172.21.15.64:0/2850185964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c08107510 msgr2=0x7f9c0819b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:54.962 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:54.400+0000 7f9c0e1e5640 1 --2- 172.21.15.64:0/2850185964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c08107510 0x7f9c0819b4b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8009940 tx=0x7f9bf8032e10 comp rx=0 tx=0).stop 2024-09-12T03:19:54.962 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:54.400+0000 7f9c0e1e5640 1 -- 172.21.15.64:0/2850185964 shutdown_connections 2024-09-12T03:19:54.962 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:54.400+0000 7f9c0e1e5640 1 --2- 172.21.15.64:0/2850185964 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f9bf0038f70 0x7f9bf003b430 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:54.962 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:54.400+0000 7f9c0e1e5640 1 --2- 172.21.15.64:0/2850185964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c08107510 0x7f9c0819b4b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:54.962 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:54.400+0000 7f9c0e1e5640 1 -- 172.21.15.64:0/2850185964 >> 172.21.15.64:0/2850185964 conn(0x7f9c08102ce0 msgr2=0x7f9c08103830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:54.962 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:54.400+0000 7f9c0e1e5640 1 -- 172.21.15.64:0/2850185964 shutdown_connections 2024-09-12T03:19:54.962 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:54.400+0000 7f9c0e1e5640 1 -- 172.21.15.64:0/2850185964 wait complete. 2024-09-12T03:19:54.963 INFO:teuthology.orchestra.run.smithi064.stdout:Deploying unmanaged mon service... 2024-09-12T03:19:55.649 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:55 smithi064 ceph-mon[30952]: from='mgr.14118 172.21.15.64:0/358528520' entity='mgr.x' 2024-09-12T03:19:55.649 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:55 smithi064 ceph-mon[30952]: Added host smithi064 2024-09-12T03:19:55.649 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:55 smithi064 ceph-mon[30952]: from='mgr.14118 172.21.15.64:0/358528520' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:19:56.340 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-12T03:19:56.341 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.549+0000 7f76f7e06640 1 Processor -- start 2024-09-12T03:19:56.341 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.550+0000 7f76f7e06640 1 -- start start 2024-09-12T03:19:56.341 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.550+0000 7f76f7e06640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f0105820 0x7f76f0105c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:56.341 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.550+0000 7f76f7e06640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f76f01061f0 con 0x7f76f0105820 2024-09-12T03:19:56.341 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.550+0000 7f76f5ba2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f0105820 0x7f76f0105c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:56.341 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.550+0000 7f76f5ba2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f0105820 0x7f76f0105c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42312/0 (socket says 172.21.15.64:42312) 2024-09-12T03:19:56.342 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.550+0000 7f76f5ba2640 1 -- 172.21.15.64:0/2440347606 learned_addr learned my addr 172.21.15.64:0/2440347606 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:19:56.342 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.550+0000 7f76f5ba2640 1 -- 172.21.15.64:0/2440347606 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f0108800 con 0x7f76f0105820 2024-09-12T03:19:56.342 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.551+0000 7f76f5ba2640 1 --2- 172.21.15.64:0/2440347606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f0105820 0x7f76f0105c20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f76ec009e80 tx=0x7f76ec02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de3570a5941d86f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:56.342 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.551+0000 7f76f4ba0640 1 -- 172.21.15.64:0/2440347606 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76ec036070 con 0x7f76f0105820 2024-09-12T03:19:56.343 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.551+0000 7f76f4ba0640 1 -- 172.21.15.64:0/2440347606 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f76ec031890 con 0x7f76f0105820 2024-09-12T03:19:56.343 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.551+0000 7f76f4ba0640 1 -- 172.21.15.64:0/2440347606 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76ec007980 con 0x7f76f0105820 2024-09-12T03:19:56.343 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.552+0000 7f76f7e06640 1 -- 172.21.15.64:0/2440347606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f0105820 msgr2=0x7f76f0105c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:56.343 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.552+0000 7f76f7e06640 1 --2- 172.21.15.64:0/2440347606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f0105820 0x7f76f0105c20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f76ec009e80 tx=0x7f76ec02a7a0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.343 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.552+0000 7f76f7e06640 1 -- 172.21.15.64:0/2440347606 shutdown_connections 2024-09-12T03:19:56.343 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.552+0000 7f76f7e06640 1 --2- 172.21.15.64:0/2440347606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f0105820 0x7f76f0105c20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.344 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.552+0000 7f76f7e06640 1 -- 172.21.15.64:0/2440347606 >> 172.21.15.64:0/2440347606 conn(0x7f76f0101390 msgr2=0x7f76f01037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:56.344 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.552+0000 7f76f7e06640 1 -- 172.21.15.64:0/2440347606 shutdown_connections 2024-09-12T03:19:56.344 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.552+0000 7f76f7e06640 1 -- 172.21.15.64:0/2440347606 wait complete. 2024-09-12T03:19:56.344 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.552+0000 7f76f7e06640 1 Processor -- start 2024-09-12T03:19:56.344 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.552+0000 7f76f7e06640 1 -- start start 2024-09-12T03:19:56.344 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.553+0000 7f76f7e06640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f0105820 0x7f76f019b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:56.345 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.553+0000 7f76f7e06640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f76f010ba30 con 0x7f76f0105820 2024-09-12T03:19:56.345 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.553+0000 7f76f5ba2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f0105820 0x7f76f019b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:56.345 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.553+0000 7f76f5ba2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f0105820 0x7f76f019b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42316/0 (socket says 172.21.15.64:42316) 2024-09-12T03:19:56.345 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.553+0000 7f76f5ba2640 1 -- 172.21.15.64:0/1384375616 learned_addr learned my addr 172.21.15.64:0/1384375616 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:19:56.345 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.553+0000 7f76f5ba2640 1 -- 172.21.15.64:0/1384375616 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f019ba40 con 0x7f76f0105820 2024-09-12T03:19:56.346 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.553+0000 7f76f5ba2640 1 --2- 172.21.15.64:0/1384375616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f0105820 0x7f76f019b500 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f76ec02acc0 tx=0x7f76ec031c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:56.346 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.553+0000 7f76e2ffd640 1 -- 172.21.15.64:0/1384375616 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76ec041030 con 0x7f76f0105820 2024-09-12T03:19:56.346 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.553+0000 7f76e2ffd640 1 -- 172.21.15.64:0/1384375616 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f76ec03c900 con 0x7f76f0105820 2024-09-12T03:19:56.346 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.553+0000 7f76f7e06640 1 -- 172.21.15.64:0/1384375616 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76f019bcd0 con 0x7f76f0105820 2024-09-12T03:19:56.346 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.553+0000 7f76e2ffd640 1 -- 172.21.15.64:0/1384375616 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76ec036040 con 0x7f76f0105820 2024-09-12T03:19:56.346 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.553+0000 7f76f7e06640 1 -- 172.21.15.64:0/1384375616 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76f019e9c0 con 0x7f76f0105820 2024-09-12T03:19:56.347 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.554+0000 7f76e2ffd640 1 -- 172.21.15.64:0/1384375616 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f76ec044070 con 0x7f76f0105820 2024-09-12T03:19:56.347 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.554+0000 7f76f7e06640 1 -- 172.21.15.64:0/1384375616 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76f0105ca0 con 0x7f76f0105820 2024-09-12T03:19:56.347 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.554+0000 7f76e2ffd640 1 --2- 172.21.15.64:0/1384375616 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f76dc038fc0 0x7f76dc03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:56.347 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.554+0000 7f76e2ffd640 1 -- 172.21.15.64:0/1384375616 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f76ec042070 con 0x7f76f0105820 2024-09-12T03:19:56.347 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.554+0000 7f76f53a1640 1 --2- 172.21.15.64:0/1384375616 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f76dc038fc0 0x7f76dc03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:56.347 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.555+0000 7f76f53a1640 1 --2- 172.21.15.64:0/1384375616 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f76dc038fc0 0x7f76dc03b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f76e40098c0 tx=0x7f76e4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:56.348 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.559+0000 7f76e2ffd640 1 -- 172.21.15.64:0/1384375616 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f76ec04b070 con 0x7f76f0105820 2024-09-12T03:19:56.348 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.752+0000 7f76f7e06640 1 -- 172.21.15.64:0/1384375616 --> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f76f0102600 con 0x7f76dc038fc0 2024-09-12T03:19:56.348 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.757+0000 7f76e2ffd640 1 -- 172.21.15.64:0/1384375616 <== mgr.14118 v2:172.21.15.64:6800/68603379 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f76f0102600 con 0x7f76dc038fc0 2024-09-12T03:19:56.348 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.759+0000 7f76f7e06640 1 -- 172.21.15.64:0/1384375616 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f76dc038fc0 msgr2=0x7f76dc03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:56.348 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.759+0000 7f76f7e06640 1 --2- 172.21.15.64:0/1384375616 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f76dc038fc0 0x7f76dc03b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f76e40098c0 tx=0x7f76e4006db0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.348 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.759+0000 7f76f7e06640 1 -- 172.21.15.64:0/1384375616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f0105820 msgr2=0x7f76f019b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:56.348 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.759+0000 7f76f7e06640 1 --2- 172.21.15.64:0/1384375616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f0105820 0x7f76f019b500 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f76ec02acc0 tx=0x7f76ec031c70 comp rx=0 tx=0).stop 2024-09-12T03:19:56.348 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.759+0000 7f76f7e06640 1 -- 172.21.15.64:0/1384375616 shutdown_connections 2024-09-12T03:19:56.348 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.759+0000 7f76f7e06640 1 --2- 172.21.15.64:0/1384375616 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f76dc038fc0 0x7f76dc03b480 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.348 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.759+0000 7f76f7e06640 1 --2- 172.21.15.64:0/1384375616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f0105820 0x7f76f019b500 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.349 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.759+0000 7f76f7e06640 1 -- 172.21.15.64:0/1384375616 >> 172.21.15.64:0/1384375616 conn(0x7f76f0101390 msgr2=0x7f76f0101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:56.349 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.759+0000 7f76f7e06640 1 -- 172.21.15.64:0/1384375616 shutdown_connections 2024-09-12T03:19:56.349 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:55.759+0000 7f76f7e06640 1 -- 172.21.15.64:0/1384375616 wait complete. 2024-09-12T03:19:56.349 INFO:teuthology.orchestra.run.smithi064.stdout:Deploying unmanaged mgr service... 2024-09-12T03:19:56.854 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:56 smithi064 ceph-mon[30952]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:19:56.854 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:56 smithi064 ceph-mon[30952]: Saving service mon spec with placement count:5 2024-09-12T03:19:56.855 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:56 smithi064 ceph-mon[30952]: from='mgr.14118 172.21.15.64:0/358528520' entity='mgr.x' 2024-09-12T03:19:57.939 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:57 smithi064 ceph-mon[30952]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:19:57.939 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:57 smithi064 ceph-mon[30952]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:19:57.939 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:57 smithi064 ceph-mon[30952]: Saving service mgr spec with placement count:2 2024-09-12T03:19:57.940 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:57 smithi064 ceph-mon[30952]: from='mgr.14118 172.21.15.64:0/358528520' entity='mgr.x' 2024-09-12T03:19:57.940 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:57 smithi064 ceph-mon[30952]: from='mgr.14118 172.21.15.64:0/358528520' entity='mgr.x' 2024-09-12T03:19:57.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-12T03:19:57.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.688+0000 7f938a3b5640 1 Processor -- start 2024-09-12T03:19:57.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.688+0000 7f938a3b5640 1 -- start start 2024-09-12T03:19:57.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.688+0000 7f938a3b5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9384107510 0x7f9384107910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:57.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.688+0000 7f938a3b5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9384107ee0 con 0x7f9384107510 2024-09-12T03:19:57.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.688+0000 7f9383fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9384107510 0x7f9384107910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:57.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.688+0000 7f9383fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9384107510 0x7f9384107910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42330/0 (socket says 172.21.15.64:42330) 2024-09-12T03:19:57.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.688+0000 7f9383fff640 1 -- 172.21.15.64:0/643074185 learned_addr learned my addr 172.21.15.64:0/643074185 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:19:57.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.688+0000 7f9383fff640 1 -- 172.21.15.64:0/643074185 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9384108710 con 0x7f9384107510 2024-09-12T03:19:57.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.689+0000 7f9383fff640 1 --2- 172.21.15.64:0/643074185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9384107510 0x7f9384107910 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9374009820 tx=0x7f937402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7515a3b5eb3f50d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:57.949 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.689+0000 7f93837fe640 1 -- 172.21.15.64:0/643074185 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9374037070 con 0x7f9384107510 2024-09-12T03:19:57.949 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.689+0000 7f93837fe640 1 -- 172.21.15.64:0/643074185 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f93740328d0 con 0x7f9384107510 2024-09-12T03:19:57.949 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.689+0000 7f93837fe640 1 -- 172.21.15.64:0/643074185 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93740307d0 con 0x7f9384107510 2024-09-12T03:19:57.949 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.690+0000 7f938a3b5640 1 -- 172.21.15.64:0/643074185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9384107510 msgr2=0x7f9384107910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:57.949 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.690+0000 7f938a3b5640 1 --2- 172.21.15.64:0/643074185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9384107510 0x7f9384107910 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9374009820 tx=0x7f937402a270 comp rx=0 tx=0).stop 2024-09-12T03:19:57.949 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.690+0000 7f938a3b5640 1 -- 172.21.15.64:0/643074185 shutdown_connections 2024-09-12T03:19:57.949 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.690+0000 7f938a3b5640 1 --2- 172.21.15.64:0/643074185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9384107510 0x7f9384107910 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:57.949 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.690+0000 7f938a3b5640 1 -- 172.21.15.64:0/643074185 >> 172.21.15.64:0/643074185 conn(0x7f9384102ce0 msgr2=0x7f9384105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:57.949 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.690+0000 7f938a3b5640 1 -- 172.21.15.64:0/643074185 shutdown_connections 2024-09-12T03:19:57.949 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.690+0000 7f938a3b5640 1 -- 172.21.15.64:0/643074185 wait complete. 2024-09-12T03:19:57.949 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.691+0000 7f938a3b5640 1 Processor -- start 2024-09-12T03:19:57.949 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.691+0000 7f938a3b5640 1 -- start start 2024-09-12T03:19:57.950 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.691+0000 7f938a3b5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9384107510 0x7f938407db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:57.950 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.691+0000 7f938a3b5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f938410bac0 con 0x7f9384107510 2024-09-12T03:19:57.950 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.691+0000 7f9383fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9384107510 0x7f938407db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:57.950 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.691+0000 7f9383fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9384107510 0x7f938407db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42332/0 (socket says 172.21.15.64:42332) 2024-09-12T03:19:57.950 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.691+0000 7f9383fff640 1 -- 172.21.15.64:0/2520628038 learned_addr learned my addr 172.21.15.64:0/2520628038 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:19:57.950 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.691+0000 7f9383fff640 1 -- 172.21.15.64:0/2520628038 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f938407e080 con 0x7f9384107510 2024-09-12T03:19:57.950 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.691+0000 7f9383fff640 1 --2- 172.21.15.64:0/2520628038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9384107510 0x7f938407db40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f93740097f0 tx=0x7f9374030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:57.950 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.692+0000 7f9381ffb640 1 -- 172.21.15.64:0/2520628038 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9374037040 con 0x7f9384107510 2024-09-12T03:19:57.950 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.692+0000 7f9381ffb640 1 -- 172.21.15.64:0/2520628038 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f9374032bb0 con 0x7f9384107510 2024-09-12T03:19:57.950 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.692+0000 7f938a3b5640 1 -- 172.21.15.64:0/2520628038 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f938407c270 con 0x7f9384107510 2024-09-12T03:19:57.950 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.692+0000 7f9381ffb640 1 -- 172.21.15.64:0/2520628038 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f937403a390 con 0x7f9384107510 2024-09-12T03:19:57.950 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.692+0000 7f938a3b5640 1 -- 172.21.15.64:0/2520628038 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f938407c7b0 con 0x7f9384107510 2024-09-12T03:19:57.950 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.693+0000 7f9381ffb640 1 -- 172.21.15.64:0/2520628038 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f937403b3f0 con 0x7f9384107510 2024-09-12T03:19:57.951 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.693+0000 7f938a3b5640 1 -- 172.21.15.64:0/2520628038 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f938410bc40 con 0x7f9384107510 2024-09-12T03:19:57.951 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.693+0000 7f9381ffb640 1 --2- 172.21.15.64:0/2520628038 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f9370038f70 0x7f937003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:57.951 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.693+0000 7f9381ffb640 1 -- 172.21.15.64:0/2520628038 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f937406c1e0 con 0x7f9384107510 2024-09-12T03:19:57.951 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.693+0000 7f937bfff640 1 --2- 172.21.15.64:0/2520628038 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f9370038f70 0x7f937003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:57.951 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.693+0000 7f937bfff640 1 --2- 172.21.15.64:0/2520628038 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f9370038f70 0x7f937003b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f936c0098c0 tx=0x7f936c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:57.951 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.698+0000 7f9381ffb640 1 -- 172.21.15.64:0/2520628038 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f937403a5d0 con 0x7f9384107510 2024-09-12T03:19:57.951 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.891+0000 7f938a3b5640 1 -- 172.21.15.64:0/2520628038 --> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f938407d040 con 0x7f9370038f70 2024-09-12T03:19:57.951 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.898+0000 7f9381ffb640 1 -- 172.21.15.64:0/2520628038 <== mgr.14118 v2:172.21.15.64:6800/68603379 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f938407d040 con 0x7f9370038f70 2024-09-12T03:19:57.951 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.900+0000 7f938a3b5640 1 -- 172.21.15.64:0/2520628038 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f9370038f70 msgr2=0x7f937003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:57.951 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.900+0000 7f938a3b5640 1 --2- 172.21.15.64:0/2520628038 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f9370038f70 0x7f937003b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f936c0098c0 tx=0x7f936c006db0 comp rx=0 tx=0).stop 2024-09-12T03:19:57.951 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.900+0000 7f938a3b5640 1 -- 172.21.15.64:0/2520628038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9384107510 msgr2=0x7f938407db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:57.951 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.900+0000 7f938a3b5640 1 --2- 172.21.15.64:0/2520628038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9384107510 0x7f938407db40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f93740097f0 tx=0x7f9374030d00 comp rx=0 tx=0).stop 2024-09-12T03:19:57.951 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.900+0000 7f938a3b5640 1 -- 172.21.15.64:0/2520628038 shutdown_connections 2024-09-12T03:19:57.952 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.900+0000 7f938a3b5640 1 --2- 172.21.15.64:0/2520628038 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f9370038f70 0x7f937003b430 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:57.952 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.900+0000 7f938a3b5640 1 --2- 172.21.15.64:0/2520628038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9384107510 0x7f938407db40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:57.952 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.900+0000 7f938a3b5640 1 -- 172.21.15.64:0/2520628038 >> 172.21.15.64:0/2520628038 conn(0x7f9384102ce0 msgr2=0x7f93841037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:57.952 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.900+0000 7f938a3b5640 1 -- 172.21.15.64:0/2520628038 shutdown_connections 2024-09-12T03:19:57.952 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:56.900+0000 7f938a3b5640 1 -- 172.21.15.64:0/2520628038 wait complete. 2024-09-12T03:19:59.280 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:59 smithi064 ceph-mon[30952]: from='mgr.14118 172.21.15.64:0/358528520' entity='mgr.x' 2024-09-12T03:19:59.280 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:19:59 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/4285908462' entity='client.admin' 2024-09-12T03:19:59.287 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.369+0000 7fa33fad5640 1 Processor -- start 2024-09-12T03:19:59.287 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.369+0000 7fa33fad5640 1 -- start start 2024-09-12T03:19:59.287 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.369+0000 7fa33fad5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa33807de70 0x7fa33807c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:59.287 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.369+0000 7fa33fad5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa33807c8a0 con 0x7fa33807de70 2024-09-12T03:19:59.287 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.369+0000 7fa33d871640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa33807de70 0x7fa33807c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:59.288 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.369+0000 7fa33d871640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa33807de70 0x7fa33807c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50514/0 (socket says 172.21.15.64:50514) 2024-09-12T03:19:59.288 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.369+0000 7fa33d871640 1 -- 172.21.15.64:0/922659510 learned_addr learned my addr 172.21.15.64:0/922659510 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:19:59.288 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.369+0000 7fa33d871640 1 -- 172.21.15.64:0/922659510 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa33807ca20 con 0x7fa33807de70 2024-09-12T03:19:59.288 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.370+0000 7fa33d871640 1 --2- 172.21.15.64:0/922659510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa33807de70 0x7fa33807c2d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa334009820 tx=0x7fa33402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14a45d9c136a39eb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:59.288 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.370+0000 7fa33c86f640 1 -- 172.21.15.64:0/922659510 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa334037070 con 0x7fa33807de70 2024-09-12T03:19:59.288 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.370+0000 7fa33c86f640 1 -- 172.21.15.64:0/922659510 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3340328f0 con 0x7fa33807de70 2024-09-12T03:19:59.288 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.370+0000 7fa33c86f640 1 -- 172.21.15.64:0/922659510 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa334030740 con 0x7fa33807de70 2024-09-12T03:19:59.288 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.371+0000 7fa33fad5640 1 -- 172.21.15.64:0/922659510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa33807de70 msgr2=0x7fa33807c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:59.288 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.371+0000 7fa33fad5640 1 --2- 172.21.15.64:0/922659510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa33807de70 0x7fa33807c2d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa334009820 tx=0x7fa33402a270 comp rx=0 tx=0).stop 2024-09-12T03:19:59.288 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.371+0000 7fa33fad5640 1 -- 172.21.15.64:0/922659510 shutdown_connections 2024-09-12T03:19:59.288 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.371+0000 7fa33fad5640 1 --2- 172.21.15.64:0/922659510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa33807de70 0x7fa33807c2d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:59.288 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.371+0000 7fa33fad5640 1 -- 172.21.15.64:0/922659510 >> 172.21.15.64:0/922659510 conn(0x7fa338102ce0 msgr2=0x7fa338105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:59.288 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.371+0000 7fa33fad5640 1 -- 172.21.15.64:0/922659510 shutdown_connections 2024-09-12T03:19:59.289 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.371+0000 7fa33fad5640 1 -- 172.21.15.64:0/922659510 wait complete. 2024-09-12T03:19:59.289 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.372+0000 7fa33fad5640 1 Processor -- start 2024-09-12T03:19:59.289 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.372+0000 7fa33fad5640 1 -- start start 2024-09-12T03:19:59.289 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.372+0000 7fa33fad5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa33807de70 0x7fa33819d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:59.289 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.372+0000 7fa33fad5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa33810db80 con 0x7fa33807de70 2024-09-12T03:19:59.289 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.372+0000 7fa33d871640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa33807de70 0x7fa33819d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:59.289 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.372+0000 7fa33d871640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa33807de70 0x7fa33819d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50522/0 (socket says 172.21.15.64:50522) 2024-09-12T03:19:59.289 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.372+0000 7fa33d871640 1 -- 172.21.15.64:0/4285908462 learned_addr learned my addr 172.21.15.64:0/4285908462 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:19:59.289 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.372+0000 7fa33d871640 1 -- 172.21.15.64:0/4285908462 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa33819dbe0 con 0x7fa33807de70 2024-09-12T03:19:59.289 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.372+0000 7fa33d871640 1 --2- 172.21.15.64:0/4285908462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa33807de70 0x7fa33819d6a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa3340097f0 tx=0x7fa334030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:59.289 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.373+0000 7fa32effd640 1 -- 172.21.15.64:0/4285908462 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa334042070 con 0x7fa33807de70 2024-09-12T03:19:59.289 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.373+0000 7fa32effd640 1 -- 172.21.15.64:0/4285908462 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa334030d80 con 0x7fa33807de70 2024-09-12T03:19:59.289 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.373+0000 7fa33fad5640 1 -- 172.21.15.64:0/4285908462 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa33819de70 con 0x7fa33807de70 2024-09-12T03:19:59.289 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.373+0000 7fa32effd640 1 -- 172.21.15.64:0/4285908462 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa334037040 con 0x7fa33807de70 2024-09-12T03:19:59.290 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.373+0000 7fa33fad5640 1 -- 172.21.15.64:0/4285908462 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3381a0b60 con 0x7fa33807de70 2024-09-12T03:19:59.290 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.373+0000 7fa32effd640 1 -- 172.21.15.64:0/4285908462 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fa334045070 con 0x7fa33807de70 2024-09-12T03:19:59.290 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.374+0000 7fa32effd640 1 --2- 172.21.15.64:0/4285908462 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fa324038fc0 0x7fa32403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:59.290 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.374+0000 7fa32effd640 1 -- 172.21.15.64:0/4285908462 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa33406bfe0 con 0x7fa33807de70 2024-09-12T03:19:59.290 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.374+0000 7fa33fad5640 1 -- 172.21.15.64:0/4285908462 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa33807c350 con 0x7fa33807de70 2024-09-12T03:19:59.290 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.374+0000 7fa33d070640 1 --2- 172.21.15.64:0/4285908462 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fa324038fc0 0x7fa32403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:59.290 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.374+0000 7fa33d070640 1 --2- 172.21.15.64:0/4285908462 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fa324038fc0 0x7fa32403b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa3280098c0 tx=0x7fa328006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:59.290 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.378+0000 7fa32effd640 1 -- 172.21.15.64:0/4285908462 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa33402ed90 con 0x7fa33807de70 2024-09-12T03:19:59.290 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.583+0000 7fa33fad5640 1 -- 172.21.15.64:0/4285908462 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fa3381a10b0 con 0x7fa33807de70 2024-09-12T03:19:59.290 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.586+0000 7fa32effd640 1 -- 172.21.15.64:0/4285908462 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fa3340313a0 con 0x7fa33807de70 2024-09-12T03:19:59.291 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.588+0000 7fa33fad5640 1 -- 172.21.15.64:0/4285908462 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fa324038fc0 msgr2=0x7fa32403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:59.291 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.588+0000 7fa33fad5640 1 --2- 172.21.15.64:0/4285908462 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fa324038fc0 0x7fa32403b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa3280098c0 tx=0x7fa328006db0 comp rx=0 tx=0).stop 2024-09-12T03:19:59.291 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.588+0000 7fa33fad5640 1 -- 172.21.15.64:0/4285908462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa33807de70 msgr2=0x7fa33819d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:59.291 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.588+0000 7fa33fad5640 1 --2- 172.21.15.64:0/4285908462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa33807de70 0x7fa33819d6a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa3340097f0 tx=0x7fa334030c70 comp rx=0 tx=0).stop 2024-09-12T03:19:59.291 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.588+0000 7fa33fad5640 1 -- 172.21.15.64:0/4285908462 shutdown_connections 2024-09-12T03:19:59.291 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.588+0000 7fa33fad5640 1 --2- 172.21.15.64:0/4285908462 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fa324038fc0 0x7fa32403b480 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:59.291 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.588+0000 7fa33fad5640 1 --2- 172.21.15.64:0/4285908462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa33807de70 0x7fa33819d6a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:59.291 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.588+0000 7fa33fad5640 1 -- 172.21.15.64:0/4285908462 >> 172.21.15.64:0/4285908462 conn(0x7fa338102ce0 msgr2=0x7fa338103890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:59.291 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.589+0000 7fa33fad5640 1 -- 172.21.15.64:0/4285908462 shutdown_connections 2024-09-12T03:19:59.291 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:58.589+0000 7fa33fad5640 1 -- 172.21.15.64:0/4285908462 wait complete. 2024-09-12T03:20:00.691 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:00 smithi064 ceph-mon[30952]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:20:00.691 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:00 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/2765527084' entity='client.admin' 2024-09-12T03:20:00.691 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:00 smithi064 ceph-mon[30952]: from='mgr.14118 172.21.15.64:0/358528520' entity='mgr.x' 2024-09-12T03:20:00.699 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.754+0000 7fcf93a69640 1 Processor -- start 2024-09-12T03:20:00.699 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.754+0000 7fcf93a69640 1 -- start start 2024-09-12T03:20:00.699 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.754+0000 7fcf93a69640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf8c108020 0x7fcf8c10a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:00.699 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.754+0000 7fcf93a69640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8c07be00 con 0x7fcf8c108020 2024-09-12T03:20:00.699 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.755+0000 7fcf91805640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf8c108020 0x7fcf8c10a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:00.699 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.755+0000 7fcf91805640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf8c108020 0x7fcf8c10a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50532/0 (socket says 172.21.15.64:50532) 2024-09-12T03:20:00.699 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.755+0000 7fcf91805640 1 -- 172.21.15.64:0/1649694564 learned_addr learned my addr 172.21.15.64:0/1649694564 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:20:00.699 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.755+0000 7fcf91805640 1 -- 172.21.15.64:0/1649694564 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf8c10a950 con 0x7fcf8c108020 2024-09-12T03:20:00.699 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.756+0000 7fcf91805640 1 --2- 172.21.15.64:0/1649694564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf8c108020 0x7fcf8c10a410 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcf88009820 tx=0x7fcf8802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c3b0f6ac209bfd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:00.699 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.756+0000 7fcf90803640 1 -- 172.21.15.64:0/1649694564 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf88037070 con 0x7fcf8c108020 2024-09-12T03:20:00.700 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.756+0000 7fcf90803640 1 -- 172.21.15.64:0/1649694564 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf880328f0 con 0x7fcf8c108020 2024-09-12T03:20:00.700 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.756+0000 7fcf90803640 1 -- 172.21.15.64:0/1649694564 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf88030740 con 0x7fcf8c108020 2024-09-12T03:20:00.700 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.756+0000 7fcf93a69640 1 -- 172.21.15.64:0/1649694564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf8c108020 msgr2=0x7fcf8c10a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:00.700 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.756+0000 7fcf93a69640 1 --2- 172.21.15.64:0/1649694564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf8c108020 0x7fcf8c10a410 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcf88009820 tx=0x7fcf8802a270 comp rx=0 tx=0).stop 2024-09-12T03:20:00.700 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.756+0000 7fcf93a69640 1 -- 172.21.15.64:0/1649694564 shutdown_connections 2024-09-12T03:20:00.700 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.756+0000 7fcf93a69640 1 --2- 172.21.15.64:0/1649694564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf8c108020 0x7fcf8c10a410 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.700 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.756+0000 7fcf93a69640 1 -- 172.21.15.64:0/1649694564 >> 172.21.15.64:0/1649694564 conn(0x7fcf8c101ca0 msgr2=0x7fcf8c1040c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:00.700 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.757+0000 7fcf93a69640 1 -- 172.21.15.64:0/1649694564 shutdown_connections 2024-09-12T03:20:00.700 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.757+0000 7fcf93a69640 1 -- 172.21.15.64:0/1649694564 wait complete. 2024-09-12T03:20:00.700 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.757+0000 7fcf93a69640 1 Processor -- start 2024-09-12T03:20:00.700 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.757+0000 7fcf93a69640 1 -- start start 2024-09-12T03:20:00.700 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.757+0000 7fcf93a69640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf8c108020 0x7fcf8c19b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:00.700 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.757+0000 7fcf93a69640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8c10b910 con 0x7fcf8c108020 2024-09-12T03:20:00.701 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.757+0000 7fcf91805640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf8c108020 0x7fcf8c19b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:00.701 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.757+0000 7fcf91805640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf8c108020 0x7fcf8c19b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50546/0 (socket says 172.21.15.64:50546) 2024-09-12T03:20:00.701 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.757+0000 7fcf91805640 1 -- 172.21.15.64:0/2765527084 learned_addr learned my addr 172.21.15.64:0/2765527084 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:20:00.701 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.758+0000 7fcf91805640 1 -- 172.21.15.64:0/2765527084 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf8c19b9e0 con 0x7fcf8c108020 2024-09-12T03:20:00.701 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.758+0000 7fcf91805640 1 --2- 172.21.15.64:0/2765527084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf8c108020 0x7fcf8c19b4a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcf88030cc0 tx=0x7fcf88030cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:00.701 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.758+0000 7fcf82ffd640 1 -- 172.21.15.64:0/2765527084 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf88042070 con 0x7fcf8c108020 2024-09-12T03:20:00.701 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.758+0000 7fcf82ffd640 1 -- 172.21.15.64:0/2765527084 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf88030e60 con 0x7fcf8c108020 2024-09-12T03:20:00.701 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.758+0000 7fcf93a69640 1 -- 172.21.15.64:0/2765527084 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf8c19bc70 con 0x7fcf8c108020 2024-09-12T03:20:00.701 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.758+0000 7fcf82ffd640 1 -- 172.21.15.64:0/2765527084 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf88037040 con 0x7fcf8c108020 2024-09-12T03:20:00.701 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.758+0000 7fcf93a69640 1 -- 172.21.15.64:0/2765527084 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf8c19e960 con 0x7fcf8c108020 2024-09-12T03:20:00.702 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.759+0000 7fcf93a69640 1 -- 172.21.15.64:0/2765527084 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf8c108480 con 0x7fcf8c108020 2024-09-12T03:20:00.702 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.759+0000 7fcf82ffd640 1 -- 172.21.15.64:0/2765527084 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fcf88045070 con 0x7fcf8c108020 2024-09-12T03:20:00.702 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.760+0000 7fcf82ffd640 1 --2- 172.21.15.64:0/2765527084 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fcf78038fc0 0x7fcf7803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:00.702 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.760+0000 7fcf82ffd640 1 -- 172.21.15.64:0/2765527084 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcf8806c020 con 0x7fcf8c108020 2024-09-12T03:20:00.702 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.760+0000 7fcf91004640 1 --2- 172.21.15.64:0/2765527084 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fcf78038fc0 0x7fcf7803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:00.702 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.760+0000 7fcf91004640 1 --2- 172.21.15.64:0/2765527084 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fcf78038fc0 0x7fcf7803b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c0098c0 tx=0x7fcf7c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:00.702 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.764+0000 7fcf82ffd640 1 -- 172.21.15.64:0/2765527084 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcf8804b170 con 0x7fcf8c108020 2024-09-12T03:20:00.702 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.940+0000 7fcf93a69640 1 -- 172.21.15.64:0/2765527084 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fcf8c19efc0 con 0x7fcf8c108020 2024-09-12T03:20:00.702 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.943+0000 7fcf82ffd640 1 -- 172.21.15.64:0/2765527084 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fcf8802fd90 con 0x7fcf8c108020 2024-09-12T03:20:00.702 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.945+0000 7fcf93a69640 1 -- 172.21.15.64:0/2765527084 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fcf78038fc0 msgr2=0x7fcf7803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:00.702 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.945+0000 7fcf93a69640 1 --2- 172.21.15.64:0/2765527084 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fcf78038fc0 0x7fcf7803b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c0098c0 tx=0x7fcf7c006db0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.702 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.945+0000 7fcf93a69640 1 -- 172.21.15.64:0/2765527084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf8c108020 msgr2=0x7fcf8c19b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:00.702 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.945+0000 7fcf93a69640 1 --2- 172.21.15.64:0/2765527084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf8c108020 0x7fcf8c19b4a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcf88030cc0 tx=0x7fcf88030cf0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.703 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.945+0000 7fcf93a69640 1 -- 172.21.15.64:0/2765527084 shutdown_connections 2024-09-12T03:20:00.703 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.945+0000 7fcf93a69640 1 --2- 172.21.15.64:0/2765527084 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fcf78038fc0 0x7fcf7803b480 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.703 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.945+0000 7fcf93a69640 1 --2- 172.21.15.64:0/2765527084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf8c108020 0x7fcf8c19b4a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.703 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.945+0000 7fcf93a69640 1 -- 172.21.15.64:0/2765527084 >> 172.21.15.64:0/2765527084 conn(0x7fcf8c101ca0 msgr2=0x7fcf8c1027e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:00.703 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.945+0000 7fcf93a69640 1 -- 172.21.15.64:0/2765527084 shutdown_connections 2024-09-12T03:20:00.703 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:19:59.945+0000 7fcf93a69640 1 -- 172.21.15.64:0/2765527084 wait complete. 2024-09-12T03:20:00.703 INFO:teuthology.orchestra.run.smithi064.stdout:Enabling the dashboard module... 2024-09-12T03:20:02.601 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:02 smithi064 ceph-mon[30952]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:20:02.601 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:02 smithi064 ceph-mon[30952]: from='mgr.14118 172.21.15.64:0/358528520' entity='mgr.x' 2024-09-12T03:20:02.601 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:02 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/3710956235' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-12T03:20:02.601 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:02 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: ignoring --setuser ceph since I am not root 2024-09-12T03:20:02.601 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:02 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: ignoring --setgroup ceph since I am not root 2024-09-12T03:20:02.601 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:02 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:20:02.461+0000 7f44aa354040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T03:20:02.601 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:02 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:20:02.521+0000 7f44aa354040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T03:20:02.903 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:02 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:20:02.678+0000 7f44aa354040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T03:20:02.903 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:02 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:20:02.748+0000 7f44aa354040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T03:20:02.904 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:02 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:20:02.903+0000 7f44aa354040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:20:03.237 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:03 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:20:03.050+0000 7f44aa354040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T03:20:03.245 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:01.063+0000 7f0893adc640 1 Processor -- start 2024-09-12T03:20:03.245 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:01.063+0000 7f0893adc640 1 -- start start 2024-09-12T03:20:03.245 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:01.063+0000 7f0893adc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f088c105de0 0x7f088c1061e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:03.245 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:01.063+0000 7f0893adc640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f088c07be00 con 0x7f088c105de0 2024-09-12T03:20:03.245 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:01.064+0000 7f0891878640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f088c105de0 0x7f088c1061e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:03.246 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:01.064+0000 7f0891878640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f088c105de0 0x7f088c1061e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50560/0 (socket says 172.21.15.64:50560) 2024-09-12T03:20:03.246 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:01.064+0000 7f0891878640 1 -- 172.21.15.64:0/1198446968 learned_addr learned my addr 172.21.15.64:0/1198446968 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:20:03.246 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:01.064+0000 7f0891878640 1 -- 172.21.15.64:0/1198446968 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f088c106720 con 0x7f088c105de0 2024-09-12T03:20:03.246 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:01.065+0000 7f0891878640 1 --2- 172.21.15.64:0/1198446968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f088c105de0 0x7f088c1061e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0888009820 tx=0x7f088802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89ac5a18fdef2efd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:03.246 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:01.065+0000 7f0890876640 1 -- 172.21.15.64:0/1198446968 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0888037070 con 0x7f088c105de0 2024-09-12T03:20:03.246 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:01.065+0000 7f0890876640 1 -- 172.21.15.64:0/1198446968 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08880328f0 con 0x7f088c105de0 2024-09-12T03:20:03.246 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:01.065+0000 7f0890876640 1 -- 172.21.15.64:0/1198446968 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0888030740 con 0x7f088c105de0 2024-09-12T03:20:03.246 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:01.065+0000 7f0893adc640 1 -- 172.21.15.64:0/1198446968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f088c105de0 msgr2=0x7f088c1061e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:03.246 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:01.065+0000 7f0893adc640 1 --2- 172.21.15.64:0/1198446968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f088c105de0 0x7f088c1061e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0888009820 tx=0x7f088802a270 comp rx=0 tx=0).stop 2024-09-12T03:20:03.246 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:01.066+0000 7f0893adc640 1 -- 172.21.15.64:0/1198446968 shutdown_connections 2024-09-12T03:20:03.246 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:01.066+0000 7f0893adc640 1 --2- 172.21.15.64:0/1198446968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f088c105de0 0x7f088c1061e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:03.246 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:01.066+0000 7f0893adc640 1 -- 172.21.15.64:0/1198446968 >> 172.21.15.64:0/1198446968 conn(0x7f088c101b70 msgr2=0x7f088c103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:03.247 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:01.066+0000 7f0893adc640 1 -- 172.21.15.64:0/1198446968 shutdown_connections 2024-09-12T03:20:03.247 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:01.066+0000 7f0893adc640 1 -- 172.21.15.64:0/1198446968 wait complete. 2024-09-12T03:20:03.247 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:01.066+0000 7f0893adc640 1 Processor -- start 2024-09-12T03:20:03.247 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:01.067+0000 7f0893adc640 1 -- start start 2024-09-12T03:20:03.247 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:01.067+0000 7f0893adc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f088c105de0 0x7f088c19d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:03.247 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:01.067+0000 7f0893adc640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f088c107800 con 0x7f088c105de0 2024-09-12T03:20:03.247 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:01.067+0000 7f0891878640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f088c105de0 0x7f088c19d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:03.247 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:01.067+0000 7f0891878640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f088c105de0 0x7f088c19d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50574/0 (socket says 172.21.15.64:50574) 2024-09-12T03:20:03.248 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:01.067+0000 7f0891878640 1 -- 172.21.15.64:0/3710956235 learned_addr learned my addr 172.21.15.64:0/3710956235 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:20:03.248 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:01.067+0000 7f0891878640 1 -- 172.21.15.64:0/3710956235 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f088c19dbe0 con 0x7f088c105de0 2024-09-12T03:20:03.248 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:01.067+0000 7f0891878640 1 --2- 172.21.15.64:0/3710956235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f088c105de0 0x7f088c19d6a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f08880097f0 tx=0x7f0888030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:03.248 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:01.068+0000 7f0882ffd640 1 -- 172.21.15.64:0/3710956235 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0888042070 con 0x7f088c105de0 2024-09-12T03:20:03.248 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:01.068+0000 7f0882ffd640 1 -- 172.21.15.64:0/3710956235 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0888030e80 con 0x7f088c105de0 2024-09-12T03:20:03.248 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:01.068+0000 7f0893adc640 1 -- 172.21.15.64:0/3710956235 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f088c19de70 con 0x7f088c105de0 2024-09-12T03:20:03.248 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:01.068+0000 7f0882ffd640 1 -- 172.21.15.64:0/3710956235 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0888037040 con 0x7f088c105de0 2024-09-12T03:20:03.248 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:01.068+0000 7f0893adc640 1 -- 172.21.15.64:0/3710956235 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f088c1a0b60 con 0x7f088c105de0 2024-09-12T03:20:03.248 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:01.069+0000 7f0882ffd640 1 -- 172.21.15.64:0/3710956235 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f0888045070 con 0x7f088c105de0 2024-09-12T03:20:03.248 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:01.069+0000 7f0882ffd640 1 --2- 172.21.15.64:0/3710956235 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f0878038f70 0x7f087803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:03.248 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:01.069+0000 7f0882ffd640 1 -- 172.21.15.64:0/3710956235 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f088806cf80 con 0x7f088c105de0 2024-09-12T03:20:03.248 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:01.069+0000 7f0893adc640 1 -- 172.21.15.64:0/3710956235 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f088c1061e0 con 0x7f088c105de0 2024-09-12T03:20:03.249 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:01.069+0000 7f0891077640 1 --2- 172.21.15.64:0/3710956235 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f0878038f70 0x7f087803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:03.249 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:01.070+0000 7f0891077640 1 --2- 172.21.15.64:0/3710956235 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f0878038f70 0x7f087803b430 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f087c0098c0 tx=0x7f087c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:03.249 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:01.074+0000 7f0882ffd640 1 -- 172.21.15.64:0/3710956235 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0888045360 con 0x7f088c105de0 2024-09-12T03:20:03.249 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:01.306+0000 7f0893adc640 1 -- 172.21.15.64:0/3710956235 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f088c02d950 con 0x7f088c105de0 2024-09-12T03:20:03.249 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:02.304+0000 7f0882ffd640 1 -- 172.21.15.64:0/3710956235 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f088802ed90 con 0x7f088c105de0 2024-09-12T03:20:03.249 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:02.307+0000 7f0882ffd640 1 -- 172.21.15.64:0/3710956235 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f0888067070 con 0x7f088c105de0 2024-09-12T03:20:03.249 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:02.308+0000 7f0893adc640 1 -- 172.21.15.64:0/3710956235 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f0878038f70 msgr2=0x7f087803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:03.249 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:02.308+0000 7f0893adc640 1 --2- 172.21.15.64:0/3710956235 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f0878038f70 0x7f087803b430 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f087c0098c0 tx=0x7f087c006db0 comp rx=0 tx=0).stop 2024-09-12T03:20:03.249 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:02.308+0000 7f0893adc640 1 -- 172.21.15.64:0/3710956235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f088c105de0 msgr2=0x7f088c19d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:03.249 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:02.308+0000 7f0893adc640 1 --2- 172.21.15.64:0/3710956235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f088c105de0 0x7f088c19d6a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f08880097f0 tx=0x7f0888030c70 comp rx=0 tx=0).stop 2024-09-12T03:20:03.249 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:02.308+0000 7f0893adc640 1 -- 172.21.15.64:0/3710956235 shutdown_connections 2024-09-12T03:20:03.249 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:02.308+0000 7f0893adc640 1 --2- 172.21.15.64:0/3710956235 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f0878038f70 0x7f087803b430 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:03.249 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:02.308+0000 7f0893adc640 1 --2- 172.21.15.64:0/3710956235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f088c105de0 0x7f088c19d6a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:03.250 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:02.308+0000 7f0893adc640 1 -- 172.21.15.64:0/3710956235 >> 172.21.15.64:0/3710956235 conn(0x7f088c101b70 msgr2=0x7f088c194340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:03.250 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:02.308+0000 7f0893adc640 1 -- 172.21.15.64:0/3710956235 shutdown_connections 2024-09-12T03:20:03.250 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:02.308+0000 7f0893adc640 1 -- 172.21.15.64:0/3710956235 wait complete. 2024-09-12T03:20:03.492 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:03 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/3710956235' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-12T03:20:03.493 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:03 smithi064 ceph-mon[30952]: mgrmap e9: x(active, since 25s) 2024-09-12T03:20:03.493 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:03 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:20:03.383+0000 7f44aa354040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T03:20:03.493 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:03 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:20:03.446+0000 7f44aa354040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T03:20:03.772 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:03 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:20:03.644+0000 7f44aa354040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T03:20:04.477 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:04 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/602574245' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-12T03:20:04.485 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-09-12T03:20:04.485 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-12T03:20:04.486 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T03:20:04.486 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-12T03:20:04.486 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-12T03:20:04.486 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-09-12T03:20:04.486 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.679+0000 7f984b607640 1 Processor -- start 2024-09-12T03:20:04.486 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.679+0000 7f984b607640 1 -- start start 2024-09-12T03:20:04.486 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.679+0000 7f984b607640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9844107520 0x7f9844107920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:04.486 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.679+0000 7f984b607640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9844107ef0 con 0x7f9844107520 2024-09-12T03:20:04.486 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.680+0000 7f98493a3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9844107520 0x7f9844107920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:04.486 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.680+0000 7f98493a3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9844107520 0x7f9844107920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50598/0 (socket says 172.21.15.64:50598) 2024-09-12T03:20:04.486 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.680+0000 7f98493a3640 1 -- 172.21.15.64:0/2165120018 learned_addr learned my addr 172.21.15.64:0/2165120018 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:20:04.486 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.680+0000 7f98493a3640 1 -- 172.21.15.64:0/2165120018 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9844108720 con 0x7f9844107520 2024-09-12T03:20:04.486 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.681+0000 7f98493a3640 1 --2- 172.21.15.64:0/2165120018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9844107520 0x7f9844107920 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9840009820 tx=0x7f984002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d99f7bb4046a4dbf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:04.487 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.681+0000 7f983bfff640 1 -- 172.21.15.64:0/2165120018 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9840037070 con 0x7f9844107520 2024-09-12T03:20:04.487 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.681+0000 7f983bfff640 1 -- 172.21.15.64:0/2165120018 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f98400328f0 con 0x7f9844107520 2024-09-12T03:20:04.487 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.681+0000 7f983bfff640 1 -- 172.21.15.64:0/2165120018 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9840030740 con 0x7f9844107520 2024-09-12T03:20:04.487 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.681+0000 7f984b607640 1 -- 172.21.15.64:0/2165120018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9844107520 msgr2=0x7f9844107920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:04.487 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.681+0000 7f984b607640 1 --2- 172.21.15.64:0/2165120018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9844107520 0x7f9844107920 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9840009820 tx=0x7f984002a270 comp rx=0 tx=0).stop 2024-09-12T03:20:04.487 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.682+0000 7f984b607640 1 -- 172.21.15.64:0/2165120018 shutdown_connections 2024-09-12T03:20:04.487 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.682+0000 7f984b607640 1 --2- 172.21.15.64:0/2165120018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9844107520 0x7f9844107920 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.487 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.682+0000 7f984b607640 1 -- 172.21.15.64:0/2165120018 >> 172.21.15.64:0/2165120018 conn(0x7f9844102c70 msgr2=0x7f98441050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:04.487 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.682+0000 7f984b607640 1 -- 172.21.15.64:0/2165120018 shutdown_connections 2024-09-12T03:20:04.487 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.682+0000 7f984b607640 1 -- 172.21.15.64:0/2165120018 wait complete. 2024-09-12T03:20:04.487 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.682+0000 7f984b607640 1 Processor -- start 2024-09-12T03:20:04.487 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.682+0000 7f984b607640 1 -- start start 2024-09-12T03:20:04.487 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.683+0000 7f984b607640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9844107520 0x7f984419b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:04.488 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.683+0000 7f984b607640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f984410bad0 con 0x7f9844107520 2024-09-12T03:20:04.488 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.683+0000 7f98493a3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9844107520 0x7f984419b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:04.488 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.683+0000 7f98493a3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9844107520 0x7f984419b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50608/0 (socket says 172.21.15.64:50608) 2024-09-12T03:20:04.488 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.683+0000 7f98493a3640 1 -- 172.21.15.64:0/602574245 learned_addr learned my addr 172.21.15.64:0/602574245 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:20:04.488 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.683+0000 7f98493a3640 1 -- 172.21.15.64:0/602574245 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f984419ba50 con 0x7f9844107520 2024-09-12T03:20:04.488 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.683+0000 7f98493a3640 1 --2- 172.21.15.64:0/602574245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9844107520 0x7f984419b510 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9840009940 tx=0x7f9840030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:04.488 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.683+0000 7f983a7fc640 1 -- 172.21.15.64:0/602574245 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9840042070 con 0x7f9844107520 2024-09-12T03:20:04.488 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.683+0000 7f984b607640 1 -- 172.21.15.64:0/602574245 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f984419bce0 con 0x7f9844107520 2024-09-12T03:20:04.488 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.684+0000 7f984b607640 1 -- 172.21.15.64:0/602574245 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f984419e9d0 con 0x7f9844107520 2024-09-12T03:20:04.488 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.684+0000 7f983a7fc640 1 -- 172.21.15.64:0/602574245 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9840030e30 con 0x7f9844107520 2024-09-12T03:20:04.488 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.684+0000 7f983a7fc640 1 -- 172.21.15.64:0/602574245 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9840037040 con 0x7f9844107520 2024-09-12T03:20:04.489 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.684+0000 7f983a7fc640 1 -- 172.21.15.64:0/602574245 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f9840045070 con 0x7f9844107520 2024-09-12T03:20:04.489 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.685+0000 7f983a7fc640 1 --2- 172.21.15.64:0/602574245 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f9830039020 0x7f983003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:04.489 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.685+0000 7f983a7fc640 1 -- 172.21.15.64:0/602574245 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f984006c000 con 0x7f9844107520 2024-09-12T03:20:04.489 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.685+0000 7f9848ba2640 1 -- 172.21.15.64:0/602574245 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f9830039020 msgr2=0x7f983003b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/68603379 2024-09-12T03:20:04.489 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.685+0000 7f9848ba2640 1 --2- 172.21.15.64:0/602574245 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f9830039020 0x7f983003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T03:20:04.489 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.685+0000 7f984b607640 1 -- 172.21.15.64:0/602574245 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98280052c0 con 0x7f9844107520 2024-09-12T03:20:04.489 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.690+0000 7f983a7fc640 1 -- 172.21.15.64:0/602574245 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9840045520 con 0x7f9844107520 2024-09-12T03:20:04.489 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.886+0000 7f9848ba2640 1 -- 172.21.15.64:0/602574245 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f9830039020 msgr2=0x7f983003b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/68603379 2024-09-12T03:20:04.489 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.886+0000 7f9848ba2640 1 --2- 172.21.15.64:0/602574245 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f9830039020 0x7f983003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T03:20:04.489 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.913+0000 7f984b607640 1 -- 172.21.15.64:0/602574245 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f9828005dc0 con 0x7f9844107520 2024-09-12T03:20:04.489 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.913+0000 7f983a7fc640 1 -- 172.21.15.64:0/602574245 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f984003add0 con 0x7f9844107520 2024-09-12T03:20:04.489 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.915+0000 7f984b607640 1 -- 172.21.15.64:0/602574245 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f9830039020 msgr2=0x7f983003b4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:20:04.490 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.915+0000 7f984b607640 1 --2- 172.21.15.64:0/602574245 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f9830039020 0x7f983003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.490 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.915+0000 7f984b607640 1 -- 172.21.15.64:0/602574245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9844107520 msgr2=0x7f984419b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:04.490 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.915+0000 7f984b607640 1 --2- 172.21.15.64:0/602574245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9844107520 0x7f984419b510 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9840009940 tx=0x7f9840030cc0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.490 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.915+0000 7f984b607640 1 -- 172.21.15.64:0/602574245 shutdown_connections 2024-09-12T03:20:04.490 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.915+0000 7f984b607640 1 --2- 172.21.15.64:0/602574245 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7f9830039020 0x7f983003b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.490 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.915+0000 7f984b607640 1 --2- 172.21.15.64:0/602574245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9844107520 0x7f984419b510 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.490 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.915+0000 7f984b607640 1 -- 172.21.15.64:0/602574245 >> 172.21.15.64:0/602574245 conn(0x7f9844102c70 msgr2=0x7f9844103810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:04.490 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.915+0000 7f984b607640 1 -- 172.21.15.64:0/602574245 shutdown_connections 2024-09-12T03:20:04.490 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:03.915+0000 7f984b607640 1 -- 172.21.15.64:0/602574245 wait complete. 2024-09-12T03:20:04.490 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for the mgr to restart... 2024-09-12T03:20:04.490 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mgr epoch 9... 2024-09-12T03:20:04.772 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:04 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:20:04.598+0000 7f44aa354040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T03:20:04.772 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:04 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:20:04.669+0000 7f44aa354040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T03:20:05.706 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:05 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:20:05.427+0000 7f44aa354040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T03:20:05.706 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:05 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:20:05.552+0000 7f44aa354040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T03:20:06.000 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:05 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:20:05.705+0000 7f44aa354040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:20:06.000 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:05 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:20:05.770+0000 7f44aa354040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T03:20:06.000 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:05 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:20:05.928+0000 7f44aa354040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T03:20:06.000 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:06 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:20:06.000+0000 7f44aa354040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T03:20:06.271 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:06 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:20:06.063+0000 7f44aa354040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T03:20:06.272 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:06 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:20:06.230+0000 7f44aa354040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T03:20:06.581 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:06 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:20:06.299+0000 7f44aa354040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T03:20:06.581 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:06 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:20:06.370+0000 7f44aa354040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T03:20:06.582 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:06 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:20:06.457+0000 7f44aa354040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T03:20:06.582 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:06 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:20:06.519+0000 7f44aa354040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T03:20:06.582 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:06 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:20:06.581+0000 7f44aa354040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T03:20:07.661 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:07 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:20:07.409+0000 7f44aa354040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T03:20:07.662 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:07 smithi064 ceph-mon[30952]: Active manager daemon x restarted 2024-09-12T03:20:07.662 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:07 smithi064 ceph-mon[30952]: Activating manager daemon x 2024-09-12T03:20:07.662 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:07 smithi064 ceph-mon[30952]: osdmap e3: 0 total, 0 up, 0 in 2024-09-12T03:20:07.662 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:07 smithi064 ceph-mon[30952]: mgrmap e10: x(active, starting, since 0.00311272s) 2024-09-12T03:20:07.662 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:07 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:20:07.662 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:07 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T03:20:07.662 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:07 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T03:20:07.662 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:07 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T03:20:07.662 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:07 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T03:20:07.662 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:07 smithi064 ceph-mon[30952]: Manager daemon x is now available 2024-09-12T03:20:07.663 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:07 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:20:07.663 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:07 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T03:20:07.663 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:07 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T03:20:09.205 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-09-12T03:20:09.205 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-12T03:20:09.205 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-12T03:20:09.205 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-09-12T03:20:09.205 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:04.885+0000 7fb9b1e68640 1 Processor -- start 2024-09-12T03:20:09.205 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:04.885+0000 7fb9b1e68640 1 -- start start 2024-09-12T03:20:09.206 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:04.885+0000 7fb9b1e68640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9ac108000 0x7fb9ac10a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:09.206 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:04.885+0000 7fb9b1e68640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb9ac07be00 con 0x7fb9ac108000 2024-09-12T03:20:09.206 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:04.886+0000 7fb9ab7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9ac108000 0x7fb9ac10a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:09.206 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:04.886+0000 7fb9ab7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9ac108000 0x7fb9ac10a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50612/0 (socket says 172.21.15.64:50612) 2024-09-12T03:20:09.206 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:04.886+0000 7fb9ab7fe640 1 -- 172.21.15.64:0/1498175479 learned_addr learned my addr 172.21.15.64:0/1498175479 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:20:09.206 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:04.886+0000 7fb9ab7fe640 1 -- 172.21.15.64:0/1498175479 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9ac10a930 con 0x7fb9ac108000 2024-09-12T03:20:09.206 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:04.887+0000 7fb9ab7fe640 1 --2- 172.21.15.64:0/1498175479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9ac108000 0x7fb9ac10a3f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb99c009820 tx=0x7fb99c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2306a76e87f105 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:09.206 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:04.887+0000 7fb9aa7fc640 1 -- 172.21.15.64:0/1498175479 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb99c037070 con 0x7fb9ac108000 2024-09-12T03:20:09.206 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:04.887+0000 7fb9aa7fc640 1 -- 172.21.15.64:0/1498175479 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb99c0328f0 con 0x7fb9ac108000 2024-09-12T03:20:09.206 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:04.887+0000 7fb9aa7fc640 1 -- 172.21.15.64:0/1498175479 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb99c030740 con 0x7fb9ac108000 2024-09-12T03:20:09.207 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:04.887+0000 7fb9b1e68640 1 -- 172.21.15.64:0/1498175479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9ac108000 msgr2=0x7fb9ac10a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:09.207 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:04.887+0000 7fb9b1e68640 1 --2- 172.21.15.64:0/1498175479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9ac108000 0x7fb9ac10a3f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb99c009820 tx=0x7fb99c02a270 comp rx=0 tx=0).stop 2024-09-12T03:20:09.207 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:04.888+0000 7fb9b1e68640 1 -- 172.21.15.64:0/1498175479 shutdown_connections 2024-09-12T03:20:09.207 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:04.888+0000 7fb9b1e68640 1 --2- 172.21.15.64:0/1498175479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9ac108000 0x7fb9ac10a3f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.207 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:04.888+0000 7fb9b1e68640 1 -- 172.21.15.64:0/1498175479 >> 172.21.15.64:0/1498175479 conn(0x7fb9ac101b80 msgr2=0x7fb9ac103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:09.207 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:04.888+0000 7fb9b1e68640 1 -- 172.21.15.64:0/1498175479 shutdown_connections 2024-09-12T03:20:09.207 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:04.888+0000 7fb9b1e68640 1 -- 172.21.15.64:0/1498175479 wait complete. 2024-09-12T03:20:09.207 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:04.888+0000 7fb9b1e68640 1 Processor -- start 2024-09-12T03:20:09.207 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:04.888+0000 7fb9b1e68640 1 -- start start 2024-09-12T03:20:09.207 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:04.889+0000 7fb9b1e68640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9ac108000 0x7fb9ac107880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:09.207 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:04.889+0000 7fb9b1e68640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb9ac10b8f0 con 0x7fb9ac108000 2024-09-12T03:20:09.208 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:04.889+0000 7fb9ab7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9ac108000 0x7fb9ac107880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:09.208 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:04.889+0000 7fb9ab7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9ac108000 0x7fb9ac107880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50614/0 (socket says 172.21.15.64:50614) 2024-09-12T03:20:09.208 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:04.889+0000 7fb9ab7fe640 1 -- 172.21.15.64:0/483733546 learned_addr learned my addr 172.21.15.64:0/483733546 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:20:09.208 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:04.889+0000 7fb9ab7fe640 1 -- 172.21.15.64:0/483733546 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9ac107dc0 con 0x7fb9ac108000 2024-09-12T03:20:09.208 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:04.889+0000 7fb9ab7fe640 1 --2- 172.21.15.64:0/483733546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9ac108000 0x7fb9ac107880 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb99c0327b0 tx=0x7fb99c0327e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:09.208 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:04.889+0000 7fb9a8ff9640 1 -- 172.21.15.64:0/483733546 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb99c042070 con 0x7fb9ac108000 2024-09-12T03:20:09.208 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:04.889+0000 7fb9a8ff9640 1 -- 172.21.15.64:0/483733546 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb99c030d50 con 0x7fb9ac108000 2024-09-12T03:20:09.208 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:04.889+0000 7fb9a8ff9640 1 -- 172.21.15.64:0/483733546 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb99c037040 con 0x7fb9ac108000 2024-09-12T03:20:09.208 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:04.890+0000 7fb9b1e68640 1 -- 172.21.15.64:0/483733546 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9ac105fb0 con 0x7fb9ac108000 2024-09-12T03:20:09.208 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:04.890+0000 7fb9b1e68640 1 -- 172.21.15.64:0/483733546 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9ac1064f0 con 0x7fb9ac108000 2024-09-12T03:20:09.208 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:04.891+0000 7fb9a8ff9640 1 -- 172.21.15.64:0/483733546 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fb99c045070 con 0x7fb9ac108000 2024-09-12T03:20:09.208 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:04.891+0000 7fb9a8ff9640 1 --2- 172.21.15.64:0/483733546 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fb994038f70 0x7fb99403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:09.209 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:04.891+0000 7fb9a8ff9640 1 -- 172.21.15.64:0/483733546 --> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb99403bb40 con 0x7fb994038f70 2024-09-12T03:20:09.209 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:04.891+0000 7fb9a8ff9640 1 -- 172.21.15.64:0/483733546 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb99c06c060 con 0x7fb9ac108000 2024-09-12T03:20:09.209 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:04.891+0000 7fb9aaffd640 1 -- 172.21.15.64:0/483733546 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fb994038f70 msgr2=0x7fb99403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/68603379 2024-09-12T03:20:09.209 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:04.891+0000 7fb9aaffd640 1 --2- 172.21.15.64:0/483733546 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fb994038f70 0x7fb99403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T03:20:09.209 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:05.092+0000 7fb9aaffd640 1 -- 172.21.15.64:0/483733546 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fb994038f70 msgr2=0x7fb99403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/68603379 2024-09-12T03:20:09.209 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:05.092+0000 7fb9aaffd640 1 --2- 172.21.15.64:0/483733546 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fb994038f70 0x7fb99403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T03:20:09.209 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:05.493+0000 7fb9aaffd640 1 -- 172.21.15.64:0/483733546 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fb994038f70 msgr2=0x7fb99403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/68603379 2024-09-12T03:20:09.209 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:05.493+0000 7fb9aaffd640 1 --2- 172.21.15.64:0/483733546 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fb994038f70 0x7fb99403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-12T03:20:09.209 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:06.294+0000 7fb9aaffd640 1 -- 172.21.15.64:0/483733546 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fb994038f70 msgr2=0x7fb99403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/68603379 2024-09-12T03:20:09.209 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:06.294+0000 7fb9aaffd640 1 --2- 172.21.15.64:0/483733546 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fb994038f70 0x7fb99403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-12T03:20:09.209 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:07.412+0000 7fb9a8ff9640 1 -- 172.21.15.64:0/483733546 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7fb99c06ab10 con 0x7fb9ac108000 2024-09-12T03:20:09.209 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:07.412+0000 7fb9a8ff9640 1 -- 172.21.15.64:0/483733546 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fb994038f70 msgr2=0x7fb99403b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:20:09.210 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:07.412+0000 7fb9a8ff9640 1 --2- 172.21.15.64:0/483733546 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fb994038f70 0x7fb99403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.210 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:08.414+0000 7fb9a8ff9640 1 -- 172.21.15.64:0/483733546 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7fb99c03d430 con 0x7fb9ac108000 2024-09-12T03:20:09.210 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:08.414+0000 7fb9a8ff9640 1 --2- 172.21.15.64:0/483733546 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fb99403ca30 0x7fb99403ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:09.210 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:08.414+0000 7fb9a8ff9640 1 -- 172.21.15.64:0/483733546 --> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb99c03ee50 con 0x7fb99403ca30 2024-09-12T03:20:09.210 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:08.415+0000 7fb9aaffd640 1 --2- 172.21.15.64:0/483733546 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fb99403ca30 0x7fb99403ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:09.210 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:08.415+0000 7fb9aaffd640 1 --2- 172.21.15.64:0/483733546 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fb99403ca30 0x7fb99403ee20 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0003980 tx=0x7fb9a0006e10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:09.210 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:08.416+0000 7fb9a8ff9640 1 -- 172.21.15.64:0/483733546 <== mgr.14150 v2:172.21.15.64:6800/759931848 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fb99c03ee50 con 0x7fb99403ca30 2024-09-12T03:20:09.210 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:08.423+0000 7fb9b1e68640 1 -- 172.21.15.64:0/483733546 --> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fb9ac108460 con 0x7fb99403ca30 2024-09-12T03:20:09.210 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:08.423+0000 7fb9a8ff9640 1 -- 172.21.15.64:0/483733546 <== mgr.14150 v2:172.21.15.64:6800/759931848 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fb9ac108460 con 0x7fb99403ca30 2024-09-12T03:20:09.210 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:08.423+0000 7fb9b1e68640 1 -- 172.21.15.64:0/483733546 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fb99403ca30 msgr2=0x7fb99403ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:09.210 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:08.423+0000 7fb9b1e68640 1 --2- 172.21.15.64:0/483733546 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fb99403ca30 0x7fb99403ee20 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0003980 tx=0x7fb9a0006e10 comp rx=0 tx=0).stop 2024-09-12T03:20:09.210 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:08.423+0000 7fb9b1e68640 1 -- 172.21.15.64:0/483733546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9ac108000 msgr2=0x7fb9ac107880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:09.210 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:08.423+0000 7fb9b1e68640 1 --2- 172.21.15.64:0/483733546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9ac108000 0x7fb9ac107880 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb99c0327b0 tx=0x7fb99c0327e0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.211 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:08.424+0000 7fb9b1e68640 1 -- 172.21.15.64:0/483733546 shutdown_connections 2024-09-12T03:20:09.211 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:08.424+0000 7fb9b1e68640 1 --2- 172.21.15.64:0/483733546 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fb99403ca30 0x7fb99403ee20 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.211 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:08.424+0000 7fb9b1e68640 1 --2- 172.21.15.64:0/483733546 >> [v2:172.21.15.64:6800/68603379,v1:172.21.15.64:6801/68603379] conn(0x7fb994038f70 0x7fb99403b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.211 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:08.424+0000 7fb9b1e68640 1 --2- 172.21.15.64:0/483733546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9ac108000 0x7fb9ac107880 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.211 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:08.424+0000 7fb9b1e68640 1 -- 172.21.15.64:0/483733546 >> 172.21.15.64:0/483733546 conn(0x7fb9ac101b80 msgr2=0x7fb9ac1027d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:09.211 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:08.424+0000 7fb9b1e68640 1 -- 172.21.15.64:0/483733546 shutdown_connections 2024-09-12T03:20:09.211 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:08.424+0000 7fb9b1e68640 1 -- 172.21.15.64:0/483733546 wait complete. 2024-09-12T03:20:09.211 INFO:teuthology.orchestra.run.smithi064.stdout:mgr epoch 9 is available 2024-09-12T03:20:09.211 INFO:teuthology.orchestra.run.smithi064.stdout:Generating a dashboard self-signed certificate... 2024-09-12T03:20:09.644 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:09 smithi064 ceph-mon[30952]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-12T03:20:09.644 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:09 smithi064 ceph-mon[30952]: mgrmap e11: x(active, since 1.00607s) 2024-09-12T03:20:09.644 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:09 smithi064 ceph-mon[30952]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-12T03:20:09.644 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:09 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:10.459 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-12T03:20:10.459 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.643+0000 7f77cf4ae640 1 Processor -- start 2024-09-12T03:20:10.459 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.644+0000 7f77cf4ae640 1 -- start start 2024-09-12T03:20:10.459 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.644+0000 7f77cf4ae640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77c8107f70 0x7f77c810a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:10.460 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.644+0000 7f77cf4ae640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f77c807be80 con 0x7f77c8107f70 2024-09-12T03:20:10.460 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.644+0000 7f77cd24a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77c8107f70 0x7f77c810a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:10.460 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.644+0000 7f77cd24a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77c8107f70 0x7f77c810a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43250/0 (socket says 172.21.15.64:43250) 2024-09-12T03:20:10.460 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.644+0000 7f77cd24a640 1 -- 172.21.15.64:0/1352587131 learned_addr learned my addr 172.21.15.64:0/1352587131 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:20:10.460 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.644+0000 7f77cd24a640 1 -- 172.21.15.64:0/1352587131 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c810a8a0 con 0x7f77c8107f70 2024-09-12T03:20:10.460 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.645+0000 7f77cd24a640 1 --2- 172.21.15.64:0/1352587131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77c8107f70 0x7f77c810a360 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f77c4009820 tx=0x7f77c402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=512d4e3fa62476b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:10.460 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.645+0000 7f77bffff640 1 -- 172.21.15.64:0/1352587131 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77c4037070 con 0x7f77c8107f70 2024-09-12T03:20:10.460 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.645+0000 7f77bffff640 1 -- 172.21.15.64:0/1352587131 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77c40328f0 con 0x7f77c8107f70 2024-09-12T03:20:10.460 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.645+0000 7f77bffff640 1 -- 172.21.15.64:0/1352587131 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77c4030740 con 0x7f77c8107f70 2024-09-12T03:20:10.460 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.646+0000 7f77cf4ae640 1 -- 172.21.15.64:0/1352587131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77c8107f70 msgr2=0x7f77c810a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:10.460 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.646+0000 7f77cf4ae640 1 --2- 172.21.15.64:0/1352587131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77c8107f70 0x7f77c810a360 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f77c4009820 tx=0x7f77c402a270 comp rx=0 tx=0).stop 2024-09-12T03:20:10.460 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.646+0000 7f77cf4ae640 1 -- 172.21.15.64:0/1352587131 shutdown_connections 2024-09-12T03:20:10.461 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.646+0000 7f77cf4ae640 1 --2- 172.21.15.64:0/1352587131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77c8107f70 0x7f77c810a360 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.461 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.646+0000 7f77cf4ae640 1 -- 172.21.15.64:0/1352587131 >> 172.21.15.64:0/1352587131 conn(0x7f77c8101b70 msgr2=0x7f77c8103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:10.461 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.646+0000 7f77cf4ae640 1 -- 172.21.15.64:0/1352587131 shutdown_connections 2024-09-12T03:20:10.461 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.646+0000 7f77cf4ae640 1 -- 172.21.15.64:0/1352587131 wait complete. 2024-09-12T03:20:10.461 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.647+0000 7f77cf4ae640 1 Processor -- start 2024-09-12T03:20:10.461 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.647+0000 7f77cf4ae640 1 -- start start 2024-09-12T03:20:10.461 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.647+0000 7f77cf4ae640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77c8107f70 0x7f77c81077f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:10.461 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.647+0000 7f77cf4ae640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f77c810b920 con 0x7f77c8107f70 2024-09-12T03:20:10.461 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.647+0000 7f77cd24a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77c8107f70 0x7f77c81077f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:10.461 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.647+0000 7f77cd24a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77c8107f70 0x7f77c81077f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43252/0 (socket says 172.21.15.64:43252) 2024-09-12T03:20:10.461 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.647+0000 7f77cd24a640 1 -- 172.21.15.64:0/3016999724 learned_addr learned my addr 172.21.15.64:0/3016999724 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:20:10.461 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.647+0000 7f77cd24a640 1 -- 172.21.15.64:0/3016999724 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c8107d30 con 0x7f77c8107f70 2024-09-12T03:20:10.461 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.648+0000 7f77cd24a640 1 --2- 172.21.15.64:0/3016999724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77c8107f70 0x7f77c81077f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f77c40097f0 tx=0x7f77c4030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:10.462 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.648+0000 7f77be7fc640 1 -- 172.21.15.64:0/3016999724 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77c4042070 con 0x7f77c8107f70 2024-09-12T03:20:10.462 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.648+0000 7f77be7fc640 1 -- 172.21.15.64:0/3016999724 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77c4030e80 con 0x7f77c8107f70 2024-09-12T03:20:10.462 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.648+0000 7f77be7fc640 1 -- 172.21.15.64:0/3016999724 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77c4037040 con 0x7f77c8107f70 2024-09-12T03:20:10.462 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.648+0000 7f77cf4ae640 1 -- 172.21.15.64:0/3016999724 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77c8105f20 con 0x7f77c8107f70 2024-09-12T03:20:10.462 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.648+0000 7f77cf4ae640 1 -- 172.21.15.64:0/3016999724 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77c8106460 con 0x7f77c8107f70 2024-09-12T03:20:10.462 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.649+0000 7f77be7fc640 1 -- 172.21.15.64:0/3016999724 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f77c4045070 con 0x7f77c8107f70 2024-09-12T03:20:10.462 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.649+0000 7f77be7fc640 1 --2- 172.21.15.64:0/3016999724 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f77b4038f00 0x7f77b403b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:10.462 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.649+0000 7f77cf4ae640 1 -- 172.21.15.64:0/3016999724 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77c81083d0 con 0x7f77c8107f70 2024-09-12T03:20:10.462 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.649+0000 7f77be7fc640 1 -- 172.21.15.64:0/3016999724 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f77c406bf50 con 0x7f77c8107f70 2024-09-12T03:20:10.462 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.649+0000 7f77cca49640 1 --2- 172.21.15.64:0/3016999724 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f77b4038f00 0x7f77b403b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:10.463 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.650+0000 7f77cca49640 1 --2- 172.21.15.64:0/3016999724 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f77b4038f00 0x7f77b403b3c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f77b80098c0 tx=0x7f77b8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:10.463 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.654+0000 7f77be7fc640 1 -- 172.21.15.64:0/3016999724 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f77c404b170 con 0x7f77c8107f70 2024-09-12T03:20:10.463 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.845+0000 7f77cf4ae640 1 -- 172.21.15.64:0/3016999724 --> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f77c8106d80 con 0x7f77b4038f00 2024-09-12T03:20:10.463 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.896+0000 7f77be7fc640 1 -- 172.21.15.64:0/3016999724 <== mgr.14150 v2:172.21.15.64:6800/759931848 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f77c8106d80 con 0x7f77b4038f00 2024-09-12T03:20:10.463 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.900+0000 7f77cf4ae640 1 -- 172.21.15.64:0/3016999724 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f77b4038f00 msgr2=0x7f77b403b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:10.463 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.900+0000 7f77cf4ae640 1 --2- 172.21.15.64:0/3016999724 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f77b4038f00 0x7f77b403b3c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f77b80098c0 tx=0x7f77b8006db0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.463 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.900+0000 7f77cf4ae640 1 -- 172.21.15.64:0/3016999724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77c8107f70 msgr2=0x7f77c81077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:10.463 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.900+0000 7f77cf4ae640 1 --2- 172.21.15.64:0/3016999724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77c8107f70 0x7f77c81077f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f77c40097f0 tx=0x7f77c4030c70 comp rx=0 tx=0).stop 2024-09-12T03:20:10.463 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.900+0000 7f77cf4ae640 1 -- 172.21.15.64:0/3016999724 shutdown_connections 2024-09-12T03:20:10.463 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.900+0000 7f77cf4ae640 1 --2- 172.21.15.64:0/3016999724 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f77b4038f00 0x7f77b403b3c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.463 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.900+0000 7f77cf4ae640 1 --2- 172.21.15.64:0/3016999724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77c8107f70 0x7f77c81077f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.463 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.900+0000 7f77cf4ae640 1 -- 172.21.15.64:0/3016999724 >> 172.21.15.64:0/3016999724 conn(0x7f77c8101b70 msgr2=0x7f77c8102770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:10.463 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.900+0000 7f77cf4ae640 1 -- 172.21.15.64:0/3016999724 shutdown_connections 2024-09-12T03:20:10.464 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:09.900+0000 7f77cf4ae640 1 -- 172.21.15.64:0/3016999724 wait complete. 2024-09-12T03:20:10.464 INFO:teuthology.orchestra.run.smithi064.stdout:Creating initial admin user... 2024-09-12T03:20:10.522 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:10 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:20:10.258+0000 7f44154a8640 -1 log_channel(cephadm) log [ERR] : [12/Sep/2024:03:20:10] ENGINE Error in HTTPServer.serve 2024-09-12T03:20:10.522 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:10 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: Traceback (most recent call last): 2024-09-12T03:20:10.523 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:10 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T03:20:10.523 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:10 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: self._connections.run(self.expiration_interval) 2024-09-12T03:20:10.523 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:10 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T03:20:10.523 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:10 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: self._run(expiration_interval) 2024-09-12T03:20:10.523 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:10 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T03:20:10.523 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:10 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T03:20:10.523 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:10 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T03:20:10.523 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:10 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T03:20:10.523 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:10 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T03:20:10.523 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:10 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: s = self.context.wrap_socket( 2024-09-12T03:20:10.523 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:10 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T03:20:10.524 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:10 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: return self.sslsocket_class._create( 2024-09-12T03:20:10.524 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:10 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T03:20:10.524 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:10 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: self.do_handshake() 2024-09-12T03:20:10.524 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:10 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T03:20:10.524 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:10 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: self._sslobj.do_handshake() 2024-09-12T03:20:10.524 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:10 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T03:20:10.524 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:20:10 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:20:11.103 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:10 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:11.103 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:10 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:11.103 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:10 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:11.103 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:10 smithi064 ceph-mon[30952]: [12/Sep/2024:03:20:10] ENGINE Bus STARTING 2024-09-12T03:20:11.103 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:10 smithi064 ceph-mon[30952]: mgrmap e12: x(active, since 2s) 2024-09-12T03:20:11.103 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:10 smithi064 ceph-mon[30952]: [12/Sep/2024:03:20:10] ENGINE Serving on https://172.21.15.64:7150 2024-09-12T03:20:11.103 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:10 smithi064 ceph-mon[30952]: [12/Sep/2024:03:20:10] ENGINE Error in HTTPServer.serve 2024-09-12T03:20:11.103 INFO:journalctl@ceph.mon.a.smithi064.stdout: Traceback (most recent call last): 2024-09-12T03:20:11.103 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T03:20:11.103 INFO:journalctl@ceph.mon.a.smithi064.stdout: self._connections.run(self.expiration_interval) 2024-09-12T03:20:11.104 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T03:20:11.104 INFO:journalctl@ceph.mon.a.smithi064.stdout: self._run(expiration_interval) 2024-09-12T03:20:11.104 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T03:20:11.104 INFO:journalctl@ceph.mon.a.smithi064.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T03:20:11.104 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T03:20:11.104 INFO:journalctl@ceph.mon.a.smithi064.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T03:20:11.104 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T03:20:11.104 INFO:journalctl@ceph.mon.a.smithi064.stdout: s = self.context.wrap_socket( 2024-09-12T03:20:11.104 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T03:20:11.104 INFO:journalctl@ceph.mon.a.smithi064.stdout: return self.sslsocket_class._create( 2024-09-12T03:20:11.104 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T03:20:11.105 INFO:journalctl@ceph.mon.a.smithi064.stdout: self.do_handshake() 2024-09-12T03:20:11.105 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T03:20:11.105 INFO:journalctl@ceph.mon.a.smithi064.stdout: self._sslobj.do_handshake() 2024-09-12T03:20:11.105 INFO:journalctl@ceph.mon.a.smithi064.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T03:20:11.105 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:10 smithi064 ceph-mon[30952]: [12/Sep/2024:03:20:10] ENGINE Serving on http://172.21.15.64:8765 2024-09-12T03:20:11.105 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:10 smithi064 ceph-mon[30952]: [12/Sep/2024:03:20:10] ENGINE Bus STARTED 2024-09-12T03:20:11.997 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$f3KPiu/8Tov02vpyIVlZc..NcmKp6B7vRE9hSs/17h6n.NxyL02F.", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726111211, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-12T03:20:11.997 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:10.840+0000 7faa2354e640 1 Processor -- start 2024-09-12T03:20:11.997 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:10.841+0000 7faa2354e640 1 -- start start 2024-09-12T03:20:11.997 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:10.841+0000 7faa2354e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa1c107f70 0x7faa1c10a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:11.997 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:10.841+0000 7faa2354e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c07be80 con 0x7faa1c107f70 2024-09-12T03:20:11.998 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:10.841+0000 7faa212ea640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa1c107f70 0x7faa1c10a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:11.998 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:10.841+0000 7faa212ea640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa1c107f70 0x7faa1c10a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43266/0 (socket says 172.21.15.64:43266) 2024-09-12T03:20:11.998 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:10.841+0000 7faa212ea640 1 -- 172.21.15.64:0/2537362898 learned_addr learned my addr 172.21.15.64:0/2537362898 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:20:11.998 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:10.841+0000 7faa212ea640 1 -- 172.21.15.64:0/2537362898 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa1c10a8a0 con 0x7faa1c107f70 2024-09-12T03:20:11.998 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:10.842+0000 7faa212ea640 1 --2- 172.21.15.64:0/2537362898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa1c107f70 0x7faa1c10a360 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faa18009820 tx=0x7faa1802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a81663340b3f02a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:11.998 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:10.842+0000 7faa13fff640 1 -- 172.21.15.64:0/2537362898 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa18037070 con 0x7faa1c107f70 2024-09-12T03:20:11.998 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:10.842+0000 7faa13fff640 1 -- 172.21.15.64:0/2537362898 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa180328f0 con 0x7faa1c107f70 2024-09-12T03:20:11.998 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:10.842+0000 7faa13fff640 1 -- 172.21.15.64:0/2537362898 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa18030740 con 0x7faa1c107f70 2024-09-12T03:20:11.999 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:10.843+0000 7faa2354e640 1 -- 172.21.15.64:0/2537362898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa1c107f70 msgr2=0x7faa1c10a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:11.999 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:10.843+0000 7faa2354e640 1 --2- 172.21.15.64:0/2537362898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa1c107f70 0x7faa1c10a360 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faa18009820 tx=0x7faa1802a270 comp rx=0 tx=0).stop 2024-09-12T03:20:11.999 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:10.844+0000 7faa2354e640 1 -- 172.21.15.64:0/2537362898 shutdown_connections 2024-09-12T03:20:11.999 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:10.844+0000 7faa2354e640 1 --2- 172.21.15.64:0/2537362898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa1c107f70 0x7faa1c10a360 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:12.000 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:10.844+0000 7faa2354e640 1 -- 172.21.15.64:0/2537362898 >> 172.21.15.64:0/2537362898 conn(0x7faa1c101b70 msgr2=0x7faa1c103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:12.000 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:10.844+0000 7faa2354e640 1 -- 172.21.15.64:0/2537362898 shutdown_connections 2024-09-12T03:20:12.000 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:10.844+0000 7faa2354e640 1 -- 172.21.15.64:0/2537362898 wait complete. 2024-09-12T03:20:12.000 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:10.844+0000 7faa2354e640 1 Processor -- start 2024-09-12T03:20:12.000 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:10.845+0000 7faa2354e640 1 -- start start 2024-09-12T03:20:12.000 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:10.845+0000 7faa2354e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa1c107f70 0x7faa1c19b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:12.000 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:10.845+0000 7faa2354e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c10b920 con 0x7faa1c107f70 2024-09-12T03:20:12.000 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:10.845+0000 7faa212ea640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa1c107f70 0x7faa1c19b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:12.000 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:10.845+0000 7faa212ea640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa1c107f70 0x7faa1c19b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43268/0 (socket says 172.21.15.64:43268) 2024-09-12T03:20:12.000 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:10.845+0000 7faa212ea640 1 -- 172.21.15.64:0/3822039818 learned_addr learned my addr 172.21.15.64:0/3822039818 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:20:12.000 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:10.845+0000 7faa212ea640 1 -- 172.21.15.64:0/3822039818 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa1c19ba20 con 0x7faa1c107f70 2024-09-12T03:20:12.001 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:10.845+0000 7faa212ea640 1 --2- 172.21.15.64:0/3822039818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa1c107f70 0x7faa1c19b4e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faa180097f0 tx=0x7faa18030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:12.001 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:10.846+0000 7faa127fc640 1 -- 172.21.15.64:0/3822039818 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa18042070 con 0x7faa1c107f70 2024-09-12T03:20:12.001 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:10.846+0000 7faa2354e640 1 -- 172.21.15.64:0/3822039818 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa1c19bcb0 con 0x7faa1c107f70 2024-09-12T03:20:12.001 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:10.846+0000 7faa127fc640 1 -- 172.21.15.64:0/3822039818 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa18030e80 con 0x7faa1c107f70 2024-09-12T03:20:12.001 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:10.846+0000 7faa127fc640 1 -- 172.21.15.64:0/3822039818 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa18037040 con 0x7faa1c107f70 2024-09-12T03:20:12.001 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:10.846+0000 7faa2354e640 1 -- 172.21.15.64:0/3822039818 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa1c19e9a0 con 0x7faa1c107f70 2024-09-12T03:20:12.001 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:10.847+0000 7faa127fc640 1 -- 172.21.15.64:0/3822039818 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7faa18045070 con 0x7faa1c107f70 2024-09-12T03:20:12.001 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:10.847+0000 7faa2354e640 1 -- 172.21.15.64:0/3822039818 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa1c1083d0 con 0x7faa1c107f70 2024-09-12T03:20:12.001 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:10.847+0000 7faa127fc640 1 --2- 172.21.15.64:0/3822039818 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7faa08039020 0x7faa0803b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:12.001 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:10.847+0000 7faa127fc640 1 -- 172.21.15.64:0/3822039818 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7faa1806c150 con 0x7faa1c107f70 2024-09-12T03:20:12.002 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:10.847+0000 7faa20ae9640 1 --2- 172.21.15.64:0/3822039818 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7faa08039020 0x7faa0803b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:12.002 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:10.848+0000 7faa20ae9640 1 --2- 172.21.15.64:0/3822039818 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7faa08039020 0x7faa0803b4e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faa0c0098c0 tx=0x7faa0c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:12.002 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:10.856+0000 7faa127fc640 1 -- 172.21.15.64:0/3822039818 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faa18045360 con 0x7faa1c107f70 2024-09-12T03:20:12.002 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:11.058+0000 7faa2354e640 1 -- 172.21.15.64:0/3822039818 --> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7faa1c19c190 con 0x7faa08039020 2024-09-12T03:20:12.002 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:11.307+0000 7faa127fc640 1 -- 172.21.15.64:0/3822039818 <== mgr.14150 v2:172.21.15.64:6800/759931848 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7faa1c19c190 con 0x7faa08039020 2024-09-12T03:20:12.002 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:11.309+0000 7faa2354e640 1 -- 172.21.15.64:0/3822039818 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7faa08039020 msgr2=0x7faa0803b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:12.002 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:11.309+0000 7faa2354e640 1 --2- 172.21.15.64:0/3822039818 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7faa08039020 0x7faa0803b4e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faa0c0098c0 tx=0x7faa0c006db0 comp rx=0 tx=0).stop 2024-09-12T03:20:12.002 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:11.309+0000 7faa2354e640 1 -- 172.21.15.64:0/3822039818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa1c107f70 msgr2=0x7faa1c19b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:12.002 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:11.309+0000 7faa2354e640 1 --2- 172.21.15.64:0/3822039818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa1c107f70 0x7faa1c19b4e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faa180097f0 tx=0x7faa18030c70 comp rx=0 tx=0).stop 2024-09-12T03:20:12.002 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:11.309+0000 7faa2354e640 1 -- 172.21.15.64:0/3822039818 shutdown_connections 2024-09-12T03:20:12.002 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:11.309+0000 7faa2354e640 1 --2- 172.21.15.64:0/3822039818 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7faa08039020 0x7faa0803b4e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:12.002 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:11.309+0000 7faa2354e640 1 --2- 172.21.15.64:0/3822039818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa1c107f70 0x7faa1c19b4e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:12.003 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:11.309+0000 7faa2354e640 1 -- 172.21.15.64:0/3822039818 >> 172.21.15.64:0/3822039818 conn(0x7faa1c101b70 msgr2=0x7faa1c102770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:12.003 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:11.310+0000 7faa2354e640 1 -- 172.21.15.64:0/3822039818 shutdown_connections 2024-09-12T03:20:12.003 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:11.310+0000 7faa2354e640 1 -- 172.21.15.64:0/3822039818 wait complete. 2024-09-12T03:20:12.003 INFO:teuthology.orchestra.run.smithi064.stdout:Fetching dashboard port number... 2024-09-12T03:20:12.381 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:12 smithi064 ceph-mon[30952]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:20:12.381 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:12 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:13.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:13 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/2081702116' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-12T03:20:13.430 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 8443 2024-09-12T03:20:13.430 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.399+0000 7ff8dcc3b640 1 Processor -- start 2024-09-12T03:20:13.430 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.399+0000 7ff8dcc3b640 1 -- start start 2024-09-12T03:20:13.430 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.400+0000 7ff8dcc3b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8d807de50 0x7ff8d807c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:13.430 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.400+0000 7ff8dcc3b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d807c880 con 0x7ff8d807de50 2024-09-12T03:20:13.430 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.400+0000 7ff8d659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8d807de50 0x7ff8d807c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:13.431 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.400+0000 7ff8d659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8d807de50 0x7ff8d807c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43282/0 (socket says 172.21.15.64:43282) 2024-09-12T03:20:13.431 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.400+0000 7ff8d659c640 1 -- 172.21.15.64:0/1028084342 learned_addr learned my addr 172.21.15.64:0/1028084342 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:20:13.431 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.400+0000 7ff8d659c640 1 -- 172.21.15.64:0/1028084342 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8d807ca00 con 0x7ff8d807de50 2024-09-12T03:20:13.431 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.401+0000 7ff8d659c640 1 --2- 172.21.15.64:0/1028084342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8d807de50 0x7ff8d807c2b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8009e80 tx=0x7ff8c802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5fc99023d52f807 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:13.431 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.401+0000 7ff8d559a640 1 -- 172.21.15.64:0/1028084342 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8c8036070 con 0x7ff8d807de50 2024-09-12T03:20:13.431 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.401+0000 7ff8d559a640 1 -- 172.21.15.64:0/1028084342 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8c80318b0 con 0x7ff8d807de50 2024-09-12T03:20:13.432 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.401+0000 7ff8d559a640 1 -- 172.21.15.64:0/1028084342 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8c8007930 con 0x7ff8d807de50 2024-09-12T03:20:13.432 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.402+0000 7ff8dcc3b640 1 -- 172.21.15.64:0/1028084342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8d807de50 msgr2=0x7ff8d807c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:13.432 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.402+0000 7ff8dcc3b640 1 --2- 172.21.15.64:0/1028084342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8d807de50 0x7ff8d807c2b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8009e80 tx=0x7ff8c802a7a0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.432 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.402+0000 7ff8dcc3b640 1 -- 172.21.15.64:0/1028084342 shutdown_connections 2024-09-12T03:20:13.432 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.402+0000 7ff8dcc3b640 1 --2- 172.21.15.64:0/1028084342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8d807de50 0x7ff8d807c2b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.432 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.402+0000 7ff8dcc3b640 1 -- 172.21.15.64:0/1028084342 >> 172.21.15.64:0/1028084342 conn(0x7ff8d8102d30 msgr2=0x7ff8d8105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:13.432 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.402+0000 7ff8dcc3b640 1 -- 172.21.15.64:0/1028084342 shutdown_connections 2024-09-12T03:20:13.432 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.402+0000 7ff8dcc3b640 1 -- 172.21.15.64:0/1028084342 wait complete. 2024-09-12T03:20:13.432 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.403+0000 7ff8dcc3b640 1 Processor -- start 2024-09-12T03:20:13.432 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.403+0000 7ff8dcc3b640 1 -- start start 2024-09-12T03:20:13.433 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.403+0000 7ff8dcc3b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8d807de50 0x7ff8d819d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:13.433 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.403+0000 7ff8dcc3b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d810db70 con 0x7ff8d807de50 2024-09-12T03:20:13.433 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.403+0000 7ff8d659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8d807de50 0x7ff8d819d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:13.433 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.403+0000 7ff8d659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8d807de50 0x7ff8d819d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43298/0 (socket says 172.21.15.64:43298) 2024-09-12T03:20:13.433 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.403+0000 7ff8d659c640 1 -- 172.21.15.64:0/2081702116 learned_addr learned my addr 172.21.15.64:0/2081702116 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:20:13.433 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.403+0000 7ff8d659c640 1 -- 172.21.15.64:0/2081702116 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8d819dc20 con 0x7ff8d807de50 2024-09-12T03:20:13.433 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.404+0000 7ff8d659c640 1 --2- 172.21.15.64:0/2081702116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8d807de50 0x7ff8d819d6e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8031c90 tx=0x7ff8c8031cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:13.433 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.404+0000 7ff8cf7fe640 1 -- 172.21.15.64:0/2081702116 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8c8036050 con 0x7ff8d807de50 2024-09-12T03:20:13.433 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.404+0000 7ff8cf7fe640 1 -- 172.21.15.64:0/2081702116 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8c8042070 con 0x7ff8d807de50 2024-09-12T03:20:13.433 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.404+0000 7ff8dcc3b640 1 -- 172.21.15.64:0/2081702116 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8d819deb0 con 0x7ff8d807de50 2024-09-12T03:20:13.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.404+0000 7ff8cf7fe640 1 -- 172.21.15.64:0/2081702116 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8c803d9c0 con 0x7ff8d807de50 2024-09-12T03:20:13.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.404+0000 7ff8dcc3b640 1 -- 172.21.15.64:0/2081702116 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8d81a0ba0 con 0x7ff8d807de50 2024-09-12T03:20:13.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.405+0000 7ff8cf7fe640 1 -- 172.21.15.64:0/2081702116 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7ff8c803dcc0 con 0x7ff8d807de50 2024-09-12T03:20:13.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.405+0000 7ff8cf7fe640 1 --2- 172.21.15.64:0/2081702116 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff8c4038fc0 0x7ff8c403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:13.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.405+0000 7ff8dcc3b640 1 -- 172.21.15.64:0/2081702116 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8d807c330 con 0x7ff8d807de50 2024-09-12T03:20:13.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.405+0000 7ff8cf7fe640 1 -- 172.21.15.64:0/2081702116 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff8c806d810 con 0x7ff8d807de50 2024-09-12T03:20:13.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.405+0000 7ff8d5d9b640 1 --2- 172.21.15.64:0/2081702116 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff8c4038fc0 0x7ff8c403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:13.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.406+0000 7ff8d5d9b640 1 --2- 172.21.15.64:0/2081702116 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff8c4038fc0 0x7ff8c403b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff8c00098c0 tx=0x7ff8c0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:13.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.410+0000 7ff8cf7fe640 1 -- 172.21.15.64:0/2081702116 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff8c8044310 con 0x7ff8d807de50 2024-09-12T03:20:13.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.614+0000 7ff8dcc3b640 1 -- 172.21.15.64:0/2081702116 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7ff8d81a1160 con 0x7ff8d807de50 2024-09-12T03:20:13.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.614+0000 7ff8cf7fe640 1 -- 172.21.15.64:0/2081702116 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7ff8c806a780 con 0x7ff8d807de50 2024-09-12T03:20:13.434 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.617+0000 7ff8dcc3b640 1 -- 172.21.15.64:0/2081702116 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff8c4038fc0 msgr2=0x7ff8c403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:13.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.617+0000 7ff8dcc3b640 1 --2- 172.21.15.64:0/2081702116 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff8c4038fc0 0x7ff8c403b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff8c00098c0 tx=0x7ff8c0006db0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.617+0000 7ff8dcc3b640 1 -- 172.21.15.64:0/2081702116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8d807de50 msgr2=0x7ff8d819d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:13.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.617+0000 7ff8dcc3b640 1 --2- 172.21.15.64:0/2081702116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8d807de50 0x7ff8d819d6e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8031c90 tx=0x7ff8c8031cc0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.617+0000 7ff8dcc3b640 1 -- 172.21.15.64:0/2081702116 shutdown_connections 2024-09-12T03:20:13.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.617+0000 7ff8dcc3b640 1 --2- 172.21.15.64:0/2081702116 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff8c4038fc0 0x7ff8c403b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.617+0000 7ff8dcc3b640 1 --2- 172.21.15.64:0/2081702116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8d807de50 0x7ff8d819d6e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.617+0000 7ff8dcc3b640 1 -- 172.21.15.64:0/2081702116 >> 172.21.15.64:0/2081702116 conn(0x7ff8d8102d30 msgr2=0x7ff8d81038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:13.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.617+0000 7ff8dcc3b640 1 -- 172.21.15.64:0/2081702116 shutdown_connections 2024-09-12T03:20:13.435 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:12.617+0000 7ff8dcc3b640 1 -- 172.21.15.64:0/2081702116 wait complete. 2024-09-12T03:20:13.437 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T03:20:13.437 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout disabled 2024-09-12T03:20:13.445 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T03:20:13.445 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2024-09-12T03:20:13.445 INFO:teuthology.orchestra.run.smithi064.stdout:firewalld.service is not enabled 2024-09-12T03:20:13.446 INFO:teuthology.orchestra.run.smithi064.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-12T03:20:13.447 INFO:teuthology.orchestra.run.smithi064.stdout:Ceph Dashboard is now available at: 2024-09-12T03:20:13.447 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:20:13.447 INFO:teuthology.orchestra.run.smithi064.stdout: URL: https://smithi064.front.sepia.ceph.com:8443/ 2024-09-12T03:20:13.447 INFO:teuthology.orchestra.run.smithi064.stdout: User: admin 2024-09-12T03:20:13.447 INFO:teuthology.orchestra.run.smithi064.stdout: Password: grai8jwqun 2024-09-12T03:20:13.447 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:20:13.448 INFO:teuthology.orchestra.run.smithi064.stdout:Saving cluster configuration to /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/config directory 2024-09-12T03:20:14.768 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:13.816+0000 7f857446a640 1 Processor -- start 2024-09-12T03:20:14.769 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:13.816+0000 7f857446a640 1 -- start start 2024-09-12T03:20:14.769 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:13.817+0000 7f857446a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f856c108030 0x7f856c10a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:14.770 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:13.817+0000 7f857446a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f856c07bde0 con 0x7f856c108030 2024-09-12T03:20:14.770 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:13.817+0000 7f8572206640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f856c108030 0x7f856c10a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:14.770 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:13.817+0000 7f8572206640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f856c108030 0x7f856c10a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43308/0 (socket says 172.21.15.64:43308) 2024-09-12T03:20:14.770 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:13.817+0000 7f8572206640 1 -- 172.21.15.64:0/4042841701 learned_addr learned my addr 172.21.15.64:0/4042841701 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:20:14.770 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:13.817+0000 7f8572206640 1 -- 172.21.15.64:0/4042841701 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f856c10a960 con 0x7f856c108030 2024-09-12T03:20:14.770 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:13.818+0000 7f8572206640 1 --2- 172.21.15.64:0/4042841701 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f856c108030 0x7f856c10a420 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8564009820 tx=0x7f856402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dff776ce5f7ba1e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:14.770 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:13.818+0000 7f8571204640 1 -- 172.21.15.64:0/4042841701 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8564037070 con 0x7f856c108030 2024-09-12T03:20:14.770 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:13.818+0000 7f8571204640 1 -- 172.21.15.64:0/4042841701 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85640328f0 con 0x7f856c108030 2024-09-12T03:20:14.770 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:13.818+0000 7f8571204640 1 -- 172.21.15.64:0/4042841701 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8564030740 con 0x7f856c108030 2024-09-12T03:20:14.770 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:13.819+0000 7f857446a640 1 -- 172.21.15.64:0/4042841701 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f856c108030 msgr2=0x7f856c10a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:14.771 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:13.819+0000 7f857446a640 1 --2- 172.21.15.64:0/4042841701 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f856c108030 0x7f856c10a420 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8564009820 tx=0x7f856402a270 comp rx=0 tx=0).stop 2024-09-12T03:20:14.771 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:13.819+0000 7f857446a640 1 -- 172.21.15.64:0/4042841701 shutdown_connections 2024-09-12T03:20:14.771 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:13.819+0000 7f857446a640 1 --2- 172.21.15.64:0/4042841701 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f856c108030 0x7f856c10a420 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:14.771 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:13.819+0000 7f857446a640 1 -- 172.21.15.64:0/4042841701 >> 172.21.15.64:0/4042841701 conn(0x7f856c101b70 msgr2=0x7f856c103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:14.771 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:13.819+0000 7f857446a640 1 -- 172.21.15.64:0/4042841701 shutdown_connections 2024-09-12T03:20:14.771 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:13.819+0000 7f857446a640 1 -- 172.21.15.64:0/4042841701 wait complete. 2024-09-12T03:20:14.771 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:13.819+0000 7f857446a640 1 Processor -- start 2024-09-12T03:20:14.771 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:13.820+0000 7f857446a640 1 -- start start 2024-09-12T03:20:14.771 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:13.820+0000 7f857446a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f856c108030 0x7f856c19b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:14.771 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:13.820+0000 7f857446a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f856c10b920 con 0x7f856c108030 2024-09-12T03:20:14.771 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:13.820+0000 7f8572206640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f856c108030 0x7f856c19b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:14.771 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:13.820+0000 7f8572206640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f856c108030 0x7f856c19b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43322/0 (socket says 172.21.15.64:43322) 2024-09-12T03:20:14.771 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:13.820+0000 7f8572206640 1 -- 172.21.15.64:0/1671566771 learned_addr learned my addr 172.21.15.64:0/1671566771 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:20:14.772 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:13.820+0000 7f8572206640 1 -- 172.21.15.64:0/1671566771 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f856c19b9e0 con 0x7f856c108030 2024-09-12T03:20:14.772 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:13.820+0000 7f8572206640 1 --2- 172.21.15.64:0/1671566771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f856c108030 0x7f856c19b4a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8564030cc0 tx=0x7f8564030cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:14.772 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:13.820+0000 7f856b7fe640 1 -- 172.21.15.64:0/1671566771 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8564042070 con 0x7f856c108030 2024-09-12T03:20:14.772 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:13.820+0000 7f856b7fe640 1 -- 172.21.15.64:0/1671566771 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8564030e60 con 0x7f856c108030 2024-09-12T03:20:14.772 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:13.820+0000 7f857446a640 1 -- 172.21.15.64:0/1671566771 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f856c19bc70 con 0x7f856c108030 2024-09-12T03:20:14.772 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:13.820+0000 7f856b7fe640 1 -- 172.21.15.64:0/1671566771 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8564037040 con 0x7f856c108030 2024-09-12T03:20:14.772 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:13.820+0000 7f857446a640 1 -- 172.21.15.64:0/1671566771 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f856c19e960 con 0x7f856c108030 2024-09-12T03:20:14.772 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:13.821+0000 7f856b7fe640 1 -- 172.21.15.64:0/1671566771 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f8564045070 con 0x7f856c108030 2024-09-12T03:20:14.772 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:13.821+0000 7f857446a640 1 -- 172.21.15.64:0/1671566771 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f856c108490 con 0x7f856c108030 2024-09-12T03:20:14.772 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:13.822+0000 7f856b7fe640 1 --2- 172.21.15.64:0/1671566771 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f8554039020 0x7f855403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:14.772 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:13.822+0000 7f856b7fe640 1 -- 172.21.15.64:0/1671566771 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f856406c310 con 0x7f856c108030 2024-09-12T03:20:14.772 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:13.822+0000 7f8571a05640 1 --2- 172.21.15.64:0/1671566771 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f8554039020 0x7f855403b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:14.772 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:13.822+0000 7f8571a05640 1 --2- 172.21.15.64:0/1671566771 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f8554039020 0x7f855403b4e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f85600098c0 tx=0x7f8560006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:14.773 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:13.826+0000 7f856b7fe640 1 -- 172.21.15.64:0/1671566771 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f856403add0 con 0x7f856c108030 2024-09-12T03:20:14.773 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:14.095+0000 7f857446a640 1 -- 172.21.15.64:0/1671566771 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f856c19eb30 con 0x7f856c108030 2024-09-12T03:20:14.773 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:14.097+0000 7f856b7fe640 1 -- 172.21.15.64:0/1671566771 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f8564045520 con 0x7f856c108030 2024-09-12T03:20:14.773 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-12T03:20:14.773 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:14.099+0000 7f857446a640 1 -- 172.21.15.64:0/1671566771 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f8554039020 msgr2=0x7f855403b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:14.773 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:14.099+0000 7f857446a640 1 --2- 172.21.15.64:0/1671566771 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f8554039020 0x7f855403b4e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f85600098c0 tx=0x7f8560006db0 comp rx=0 tx=0).stop 2024-09-12T03:20:14.773 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:14.099+0000 7f857446a640 1 -- 172.21.15.64:0/1671566771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f856c108030 msgr2=0x7f856c19b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:14.773 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:14.099+0000 7f857446a640 1 --2- 172.21.15.64:0/1671566771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f856c108030 0x7f856c19b4a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8564030cc0 tx=0x7f8564030cf0 comp rx=0 tx=0).stop 2024-09-12T03:20:14.773 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:14.099+0000 7f857446a640 1 -- 172.21.15.64:0/1671566771 shutdown_connections 2024-09-12T03:20:14.773 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:14.099+0000 7f857446a640 1 --2- 172.21.15.64:0/1671566771 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f8554039020 0x7f855403b4e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:14.773 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:14.099+0000 7f857446a640 1 --2- 172.21.15.64:0/1671566771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f856c108030 0x7f856c19b4a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:14.774 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:14.099+0000 7f857446a640 1 -- 172.21.15.64:0/1671566771 >> 172.21.15.64:0/1671566771 conn(0x7f856c101b70 msgr2=0x7f856c1027a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:14.774 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:14.099+0000 7f857446a640 1 -- 172.21.15.64:0/1671566771 shutdown_connections 2024-09-12T03:20:14.774 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr 2024-09-12T03:20:14.099+0000 7f857446a640 1 -- 172.21.15.64:0/1671566771 wait complete. 2024-09-12T03:20:14.774 INFO:teuthology.orchestra.run.smithi064.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-12T03:20:14.774 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:20:14.774 INFO:teuthology.orchestra.run.smithi064.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:20:14.774 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:20:14.774 INFO:teuthology.orchestra.run.smithi064.stdout:Or, if you are only running a single cluster on this host: 2024-09-12T03:20:14.774 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:20:14.775 INFO:teuthology.orchestra.run.smithi064.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-12T03:20:14.775 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:20:14.775 INFO:teuthology.orchestra.run.smithi064.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-12T03:20:14.775 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:20:14.775 INFO:teuthology.orchestra.run.smithi064.stdout: ceph telemetry on 2024-09-12T03:20:14.775 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:20:14.775 INFO:teuthology.orchestra.run.smithi064.stdout:For more information see: 2024-09-12T03:20:14.775 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:20:14.775 INFO:teuthology.orchestra.run.smithi064.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-12T03:20:14.775 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:20:14.776 INFO:teuthology.orchestra.run.smithi064.stdout:Bootstrap complete. 2024-09-12T03:20:14.816 INFO:tasks.cephadm:Fetching config... 2024-09-12T03:20:14.816 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-12T03:20:14.816 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-12T03:20:14.834 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-12T03:20:14.835 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-12T03:20:14.835 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-12T03:20:14.891 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-12T03:20:14.891 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-12T03:20:14.892 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-12T03:20:14.959 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-12T03:20:14.959 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-12T03:20:14.959 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-12T03:20:15.015 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-12T03:20:15.015 DEBUG:teuthology.orchestra.run.smithi064:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjKZXV/vMRD/iDwNqYR5tmOeS4R7ZF2rCDqp+J9IMU1lop70Rz+kPbLst++O0MCBxCIy743TcXpd8ulruGb+NeTKvkozXFXGCJZsdkYKcyqhcMh0NCJ2iGbRIEwIM3e0hNTi1dRHFNiAU+rmwj3PBS8K3g8QcdPHrFhTwozow1yy/P2/nfJ2aox2H4KPLavxZv/4UicUxJp4Np8DzIdAckBGWghbXv3Js0BHEgW7pd2QSD8tNl9xJse/G4RZp+B1KpB1WEanEgFoHGeketQqqyXt+x9kL1U9wdbGz3+O7fU0sGGRg6VGF+9zg7Jah6BPwqDFgv817NzRKvNHQkfdgUwWw2pOLiFOUDf7ZtKO07mhzcAu6PGOZxZJb1Edpx8QisUdgCACemNoY3/quLapsMjsbqKjqLe5vR1ckBKivnEnhfM2MIRue6/AY1xnZq5NwGMReqOSFjiyaRXSLCPaBKM1Ui33UE0oTXw3j7C0t47LZVx8elKY4TWOOpycFwNb8= ceph-b0812460-70b5-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T03:20:15.093 INFO:teuthology.orchestra.run.smithi064.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjKZXV/vMRD/iDwNqYR5tmOeS4R7ZF2rCDqp+J9IMU1lop70Rz+kPbLst++O0MCBxCIy743TcXpd8ulruGb+NeTKvkozXFXGCJZsdkYKcyqhcMh0NCJ2iGbRIEwIM3e0hNTi1dRHFNiAU+rmwj3PBS8K3g8QcdPHrFhTwozow1yy/P2/nfJ2aox2H4KPLavxZv/4UicUxJp4Np8DzIdAckBGWghbXv3Js0BHEgW7pd2QSD8tNl9xJse/G4RZp+B1KpB1WEanEgFoHGeketQqqyXt+x9kL1U9wdbGz3+O7fU0sGGRg6VGF+9zg7Jah6BPwqDFgv817NzRKvNHQkfdgUwWw2pOLiFOUDf7ZtKO07mhzcAu6PGOZxZJb1Edpx8QisUdgCACemNoY3/quLapsMjsbqKjqLe5vR1ckBKivnEnhfM2MIRue6/AY1xnZq5NwGMReqOSFjiyaRXSLCPaBKM1Ui33UE0oTXw3j7C0t47LZVx8elKY4TWOOpycFwNb8= ceph-b0812460-70b5-11ef-bcea-c7b262605968 2024-09-12T03:20:15.106 DEBUG:teuthology.orchestra.run.smithi115:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjKZXV/vMRD/iDwNqYR5tmOeS4R7ZF2rCDqp+J9IMU1lop70Rz+kPbLst++O0MCBxCIy743TcXpd8ulruGb+NeTKvkozXFXGCJZsdkYKcyqhcMh0NCJ2iGbRIEwIM3e0hNTi1dRHFNiAU+rmwj3PBS8K3g8QcdPHrFhTwozow1yy/P2/nfJ2aox2H4KPLavxZv/4UicUxJp4Np8DzIdAckBGWghbXv3Js0BHEgW7pd2QSD8tNl9xJse/G4RZp+B1KpB1WEanEgFoHGeketQqqyXt+x9kL1U9wdbGz3+O7fU0sGGRg6VGF+9zg7Jah6BPwqDFgv817NzRKvNHQkfdgUwWw2pOLiFOUDf7ZtKO07mhzcAu6PGOZxZJb1Edpx8QisUdgCACemNoY3/quLapsMjsbqKjqLe5vR1ckBKivnEnhfM2MIRue6/AY1xnZq5NwGMReqOSFjiyaRXSLCPaBKM1Ui33UE0oTXw3j7C0t47LZVx8elKY4TWOOpycFwNb8= ceph-b0812460-70b5-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T03:20:15.248 INFO:teuthology.orchestra.run.smithi115.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjKZXV/vMRD/iDwNqYR5tmOeS4R7ZF2rCDqp+J9IMU1lop70Rz+kPbLst++O0MCBxCIy743TcXpd8ulruGb+NeTKvkozXFXGCJZsdkYKcyqhcMh0NCJ2iGbRIEwIM3e0hNTi1dRHFNiAU+rmwj3PBS8K3g8QcdPHrFhTwozow1yy/P2/nfJ2aox2H4KPLavxZv/4UicUxJp4Np8DzIdAckBGWghbXv3Js0BHEgW7pd2QSD8tNl9xJse/G4RZp+B1KpB1WEanEgFoHGeketQqqyXt+x9kL1U9wdbGz3+O7fU0sGGRg6VGF+9zg7Jah6BPwqDFgv817NzRKvNHQkfdgUwWw2pOLiFOUDf7ZtKO07mhzcAu6PGOZxZJb1Edpx8QisUdgCACemNoY3/quLapsMjsbqKjqLe5vR1ckBKivnEnhfM2MIRue6/AY1xnZq5NwGMReqOSFjiyaRXSLCPaBKM1Ui33UE0oTXw3j7C0t47LZVx8elKY4TWOOpycFwNb8= ceph-b0812460-70b5-11ef-bcea-c7b262605968 2024-09-12T03:20:15.261 DEBUG:teuthology.orchestra.run.smithi195:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjKZXV/vMRD/iDwNqYR5tmOeS4R7ZF2rCDqp+J9IMU1lop70Rz+kPbLst++O0MCBxCIy743TcXpd8ulruGb+NeTKvkozXFXGCJZsdkYKcyqhcMh0NCJ2iGbRIEwIM3e0hNTi1dRHFNiAU+rmwj3PBS8K3g8QcdPHrFhTwozow1yy/P2/nfJ2aox2H4KPLavxZv/4UicUxJp4Np8DzIdAckBGWghbXv3Js0BHEgW7pd2QSD8tNl9xJse/G4RZp+B1KpB1WEanEgFoHGeketQqqyXt+x9kL1U9wdbGz3+O7fU0sGGRg6VGF+9zg7Jah6BPwqDFgv817NzRKvNHQkfdgUwWw2pOLiFOUDf7ZtKO07mhzcAu6PGOZxZJb1Edpx8QisUdgCACemNoY3/quLapsMjsbqKjqLe5vR1ckBKivnEnhfM2MIRue6/AY1xnZq5NwGMReqOSFjiyaRXSLCPaBKM1Ui33UE0oTXw3j7C0t47LZVx8elKY4TWOOpycFwNb8= ceph-b0812460-70b5-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T03:20:15.331 INFO:teuthology.orchestra.run.smithi195.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjKZXV/vMRD/iDwNqYR5tmOeS4R7ZF2rCDqp+J9IMU1lop70Rz+kPbLst++O0MCBxCIy743TcXpd8ulruGb+NeTKvkozXFXGCJZsdkYKcyqhcMh0NCJ2iGbRIEwIM3e0hNTi1dRHFNiAU+rmwj3PBS8K3g8QcdPHrFhTwozow1yy/P2/nfJ2aox2H4KPLavxZv/4UicUxJp4Np8DzIdAckBGWghbXv3Js0BHEgW7pd2QSD8tNl9xJse/G4RZp+B1KpB1WEanEgFoHGeketQqqyXt+x9kL1U9wdbGz3+O7fU0sGGRg6VGF+9zg7Jah6BPwqDFgv817NzRKvNHQkfdgUwWw2pOLiFOUDf7ZtKO07mhzcAu6PGOZxZJb1Edpx8QisUdgCACemNoY3/quLapsMjsbqKjqLe5vR1ckBKivnEnhfM2MIRue6/AY1xnZq5NwGMReqOSFjiyaRXSLCPaBKM1Ui33UE0oTXw3j7C0t47LZVx8elKY4TWOOpycFwNb8= ceph-b0812460-70b5-11ef-bcea-c7b262605968 2024-09-12T03:20:15.345 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-12T03:20:15.367 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:15 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/1671566771' entity='client.admin' 2024-09-12T03:20:15.367 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:15 smithi064 ceph-mon[30952]: mgrmap e13: x(active, since 6s) 2024-09-12T03:20:15.601 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:16.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:16.045+0000 7fb065880640 1 -- 172.21.15.64:0/2951492818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb060102bd0 msgr2=0x7fb060102fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:16.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:16.045+0000 7fb065880640 1 --2- 172.21.15.64:0/2951492818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb060102bd0 0x7fb060102fd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb050009c20 tx=0x7fb05002aa90 comp rx=0 tx=0).stop 2024-09-12T03:20:16.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:16.045+0000 7fb065880640 1 -- 172.21.15.64:0/2951492818 shutdown_connections 2024-09-12T03:20:16.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:16.045+0000 7fb065880640 1 --2- 172.21.15.64:0/2951492818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb060102bd0 0x7fb060102fd0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:16.045+0000 7fb065880640 1 -- 172.21.15.64:0/2951492818 >> 172.21.15.64:0/2951492818 conn(0x7fb0600fb400 msgr2=0x7fb0600fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:16.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:16.045+0000 7fb065880640 1 -- 172.21.15.64:0/2951492818 shutdown_connections 2024-09-12T03:20:16.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:16.045+0000 7fb065880640 1 -- 172.21.15.64:0/2951492818 wait complete. 2024-09-12T03:20:16.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:16.046+0000 7fb065880640 1 Processor -- start 2024-09-12T03:20:16.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:16.046+0000 7fb065880640 1 -- start start 2024-09-12T03:20:16.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:16.046+0000 7fb065880640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb060102bd0 0x7fb0601021f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:16.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:16.046+0000 7fb065880640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb0601075b0 con 0x7fb060102bd0 2024-09-12T03:20:16.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:16.046+0000 7fb05effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb060102bd0 0x7fb0601021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:16.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:16.046+0000 7fb05effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb060102bd0 0x7fb0601021f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43328/0 (socket says 172.21.15.64:43328) 2024-09-12T03:20:16.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:16.046+0000 7fb05effd640 1 -- 172.21.15.64:0/359885263 learned_addr learned my addr 172.21.15.64:0/359885263 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:20:16.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:16.046+0000 7fb05effd640 1 -- 172.21.15.64:0/359885263 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb060102730 con 0x7fb060102bd0 2024-09-12T03:20:16.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:16.047+0000 7fb05effd640 1 --2- 172.21.15.64:0/359885263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb060102bd0 0x7fb0601021f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb050005cc0 tx=0x7fb05002b020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:16.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:16.047+0000 7fb06487e640 1 -- 172.21.15.64:0/359885263 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb050047070 con 0x7fb060102bd0 2024-09-12T03:20:16.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:16.047+0000 7fb065880640 1 -- 172.21.15.64:0/359885263 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb060100920 con 0x7fb060102bd0 2024-09-12T03:20:16.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:16.047+0000 7fb06487e640 1 -- 172.21.15.64:0/359885263 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb050003e20 con 0x7fb060102bd0 2024-09-12T03:20:16.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:16.047+0000 7fb06487e640 1 -- 172.21.15.64:0/359885263 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb050033050 con 0x7fb060102bd0 2024-09-12T03:20:16.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:16.047+0000 7fb065880640 1 -- 172.21.15.64:0/359885263 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb060100e60 con 0x7fb060102bd0 2024-09-12T03:20:16.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:16.048+0000 7fb06487e640 1 -- 172.21.15.64:0/359885263 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fb05004a070 con 0x7fb060102bd0 2024-09-12T03:20:16.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:16.048+0000 7fb065880640 1 -- 172.21.15.64:0/359885263 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb060107d30 con 0x7fb060102bd0 2024-09-12T03:20:16.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:16.048+0000 7fb06487e640 1 --2- 172.21.15.64:0/359885263 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fb04c039060 0x7fb04c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:16.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:16.048+0000 7fb06487e640 1 -- 172.21.15.64:0/359885263 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb05003e030 con 0x7fb060102bd0 2024-09-12T03:20:16.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:16.048+0000 7fb05e7fc640 1 --2- 172.21.15.64:0/359885263 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fb04c039060 0x7fb04c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:16.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:16.049+0000 7fb05e7fc640 1 --2- 172.21.15.64:0/359885263 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fb04c039060 0x7fb04c03b520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb0480098c0 tx=0x7fb048006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:16.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:16.053+0000 7fb06487e640 1 -- 172.21.15.64:0/359885263 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb0500373b0 con 0x7fb060102bd0 2024-09-12T03:20:16.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:16.231+0000 7fb065880640 1 -- 172.21.15.64:0/359885263 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fb060190070 con 0x7fb060102bd0 2024-09-12T03:20:16.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:16.233+0000 7fb06487e640 1 -- 172.21.15.64:0/359885263 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fb0500492c0 con 0x7fb060102bd0 2024-09-12T03:20:16.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:16.235+0000 7fb065880640 1 -- 172.21.15.64:0/359885263 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fb04c039060 msgr2=0x7fb04c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:16.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:16.235+0000 7fb065880640 1 --2- 172.21.15.64:0/359885263 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fb04c039060 0x7fb04c03b520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb0480098c0 tx=0x7fb048006db0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:16.235+0000 7fb065880640 1 -- 172.21.15.64:0/359885263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb060102bd0 msgr2=0x7fb0601021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:16.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:16.235+0000 7fb065880640 1 --2- 172.21.15.64:0/359885263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb060102bd0 0x7fb0601021f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb050005cc0 tx=0x7fb05002b020 comp rx=0 tx=0).stop 2024-09-12T03:20:16.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:16.236+0000 7fb065880640 1 -- 172.21.15.64:0/359885263 shutdown_connections 2024-09-12T03:20:16.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:16.236+0000 7fb065880640 1 --2- 172.21.15.64:0/359885263 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fb04c039060 0x7fb04c03b520 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:16.236+0000 7fb065880640 1 --2- 172.21.15.64:0/359885263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb060102bd0 0x7fb0601021f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:16.236+0000 7fb065880640 1 -- 172.21.15.64:0/359885263 >> 172.21.15.64:0/359885263 conn(0x7fb0600fb400 msgr2=0x7fb0600fbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:16.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:16.236+0000 7fb065880640 1 -- 172.21.15.64:0/359885263 shutdown_connections 2024-09-12T03:20:16.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:16.236+0000 7fb065880640 1 -- 172.21.15.64:0/359885263 wait complete. 2024-09-12T03:20:17.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:17 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/359885263' entity='client.admin' 2024-09-12T03:20:17.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:17 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:17.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:17 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:17.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:17 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi064", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:20:17.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:17 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:17.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:17 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:20:17.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:17 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:17.715 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-12T03:20:17.715 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-12T03:20:18.425 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:18.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:18.831+0000 7fe05be12640 1 -- 172.21.15.64:0/4275631267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0541020d0 msgr2=0x7fe0541044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:18.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:18.831+0000 7fe05be12640 1 --2- 172.21.15.64:0/4275631267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0541020d0 0x7fe0541044c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe04c009880 tx=0x7fe04c02a5d0 comp rx=0 tx=0).stop 2024-09-12T03:20:18.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:18.831+0000 7fe05be12640 1 -- 172.21.15.64:0/4275631267 shutdown_connections 2024-09-12T03:20:18.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:18.831+0000 7fe05be12640 1 --2- 172.21.15.64:0/4275631267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0541020d0 0x7fe0541044c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:18.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:18.831+0000 7fe05be12640 1 -- 172.21.15.64:0/4275631267 >> 172.21.15.64:0/4275631267 conn(0x7fe0540fbcd0 msgr2=0x7fe0540fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:18.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:18.831+0000 7fe05be12640 1 -- 172.21.15.64:0/4275631267 shutdown_connections 2024-09-12T03:20:18.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:18.831+0000 7fe05be12640 1 -- 172.21.15.64:0/4275631267 wait complete. 2024-09-12T03:20:18.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:18.831+0000 7fe05be12640 1 Processor -- start 2024-09-12T03:20:18.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:18.831+0000 7fe05be12640 1 -- start start 2024-09-12T03:20:18.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:18.832+0000 7fe05be12640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0541020d0 0x7fe054196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:18.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:18.832+0000 7fe05be12640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe054107520 con 0x7fe0541020d0 2024-09-12T03:20:18.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:18.832+0000 7fe059bae640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0541020d0 0x7fe054196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:18.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:18.832+0000 7fe059bae640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0541020d0 0x7fe054196c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48980/0 (socket says 172.21.15.64:48980) 2024-09-12T03:20:18.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:18.832+0000 7fe059bae640 1 -- 172.21.15.64:0/3845252968 learned_addr learned my addr 172.21.15.64:0/3845252968 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:20:18.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:18.832+0000 7fe059bae640 1 -- 172.21.15.64:0/3845252968 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe054197150 con 0x7fe0541020d0 2024-09-12T03:20:18.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:18.832+0000 7fe059bae640 1 --2- 172.21.15.64:0/3845252968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0541020d0 0x7fe054196c10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe04c009230 tx=0x7fe04c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:18.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:18.832+0000 7fe052ffd640 1 -- 172.21.15.64:0/3845252968 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe04c03f070 con 0x7fe0541020d0 2024-09-12T03:20:18.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:18.832+0000 7fe05be12640 1 -- 172.21.15.64:0/3845252968 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe054197440 con 0x7fe0541020d0 2024-09-12T03:20:18.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:18.832+0000 7fe052ffd640 1 -- 172.21.15.64:0/3845252968 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe04c003aa0 con 0x7fe0541020d0 2024-09-12T03:20:18.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:18.832+0000 7fe052ffd640 1 -- 172.21.15.64:0/3845252968 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe04c035030 con 0x7fe0541020d0 2024-09-12T03:20:18.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:18.832+0000 7fe05be12640 1 -- 172.21.15.64:0/3845252968 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe05419a0d0 con 0x7fe0541020d0 2024-09-12T03:20:18.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:18.833+0000 7fe052ffd640 1 -- 172.21.15.64:0/3845252968 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fe04c042070 con 0x7fe0541020d0 2024-09-12T03:20:18.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:18.833+0000 7fe05be12640 1 -- 172.21.15.64:0/3845252968 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe054107ca0 con 0x7fe0541020d0 2024-09-12T03:20:18.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:18.833+0000 7fe052ffd640 1 --2- 172.21.15.64:0/3845252968 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe03c039060 0x7fe03c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:18.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:18.834+0000 7fe052ffd640 1 -- 172.21.15.64:0/3845252968 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe04c06ce80 con 0x7fe0541020d0 2024-09-12T03:20:18.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:18.834+0000 7fe0593ad640 1 --2- 172.21.15.64:0/3845252968 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe03c039060 0x7fe03c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:18.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:18.834+0000 7fe0593ad640 1 --2- 172.21.15.64:0/3845252968 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe03c039060 0x7fe03c03b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe0480098c0 tx=0x7fe048006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:18.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:18.838+0000 7fe052ffd640 1 -- 172.21.15.64:0/3845252968 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe04c04c350 con 0x7fe0541020d0 2024-09-12T03:20:19.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:19.016+0000 7fe05be12640 1 -- 172.21.15.64:0/3845252968 --> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fe05402d9a0 con 0x7fe03c039060 2024-09-12T03:20:19.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:19.020+0000 7fe052ffd640 1 -- 172.21.15.64:0/3845252968 <== mgr.14150 v2:172.21.15.64:6800/759931848 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe05402d9a0 con 0x7fe03c039060 2024-09-12T03:20:19.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:19.022+0000 7fe05be12640 1 -- 172.21.15.64:0/3845252968 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe03c039060 msgr2=0x7fe03c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:19.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:19.022+0000 7fe05be12640 1 --2- 172.21.15.64:0/3845252968 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe03c039060 0x7fe03c03b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe0480098c0 tx=0x7fe048006db0 comp rx=0 tx=0).stop 2024-09-12T03:20:19.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:19.022+0000 7fe05be12640 1 -- 172.21.15.64:0/3845252968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0541020d0 msgr2=0x7fe054196c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:19.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:19.022+0000 7fe05be12640 1 --2- 172.21.15.64:0/3845252968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0541020d0 0x7fe054196c10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe04c009230 tx=0x7fe04c0048f0 comp rx=0 tx=0).stop 2024-09-12T03:20:19.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:19.022+0000 7fe05be12640 1 -- 172.21.15.64:0/3845252968 shutdown_connections 2024-09-12T03:20:19.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:19.022+0000 7fe05be12640 1 --2- 172.21.15.64:0/3845252968 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe03c039060 0x7fe03c03b520 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:19.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:19.022+0000 7fe05be12640 1 --2- 172.21.15.64:0/3845252968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0541020d0 0x7fe054196c10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:19.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:19.022+0000 7fe05be12640 1 -- 172.21.15.64:0/3845252968 >> 172.21.15.64:0/3845252968 conn(0x7fe0540fbcd0 msgr2=0x7fe0540fc780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:19.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:19.022+0000 7fe05be12640 1 -- 172.21.15.64:0/3845252968 shutdown_connections 2024-09-12T03:20:19.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:19.023+0000 7fe05be12640 1 -- 172.21.15.64:0/3845252968 wait complete. 2024-09-12T03:20:19.732 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi115 2024-09-12T03:20:19.733 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-12T03:20:19.733 DEBUG:teuthology.orchestra.run.smithi115:> dd of=/etc/ceph/ceph.conf 2024-09-12T03:20:19.751 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-12T03:20:19.752 DEBUG:teuthology.orchestra.run.smithi115:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:20:19.810 INFO:tasks.cephadm:Adding host smithi115 to orchestrator... 2024-09-12T03:20:19.810 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph orch host add smithi115 2024-09-12T03:20:20.069 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:20.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:20 smithi064 ceph-mon[30952]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:20:20.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:20 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:20.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:20 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:20:20.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:20 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:20.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:20 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:20:20.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:20 smithi064 ceph-mon[30952]: Updating smithi064:/etc/ceph/ceph.conf 2024-09-12T03:20:20.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:20 smithi064 ceph-mon[30952]: Updating smithi064:/var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:20:20.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:20 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:20.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:20 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:20.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:20 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:20.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:20.487+0000 7f3454c3b640 1 -- 172.21.15.64:0/296328350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3450106560 msgr2=0x7f3450106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:20.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:20.487+0000 7f3454c3b640 1 --2- 172.21.15.64:0/296328350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3450106560 0x7f3450106940 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3440009880 tx=0x7f344002a5d0 comp rx=0 tx=0).stop 2024-09-12T03:20:20.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:20.487+0000 7f3454c3b640 1 -- 172.21.15.64:0/296328350 shutdown_connections 2024-09-12T03:20:20.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:20.487+0000 7f3454c3b640 1 --2- 172.21.15.64:0/296328350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3450106560 0x7f3450106940 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:20.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:20.487+0000 7f3454c3b640 1 -- 172.21.15.64:0/296328350 >> 172.21.15.64:0/296328350 conn(0x7f34500fc1b0 msgr2=0x7f34500fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:20.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:20.487+0000 7f3454c3b640 1 -- 172.21.15.64:0/296328350 shutdown_connections 2024-09-12T03:20:20.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:20.487+0000 7f3454c3b640 1 -- 172.21.15.64:0/296328350 wait complete. 2024-09-12T03:20:20.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:20.488+0000 7f3454c3b640 1 Processor -- start 2024-09-12T03:20:20.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:20.488+0000 7f3454c3b640 1 -- start start 2024-09-12T03:20:20.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:20.488+0000 7f3454c3b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3450106560 0x7f3450197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:20.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:20.488+0000 7f3454c3b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f345010aa40 con 0x7f3450106560 2024-09-12T03:20:20.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:20.488+0000 7f344e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3450106560 0x7f3450197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:20.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:20.488+0000 7f344e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3450106560 0x7f3450197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48990/0 (socket says 172.21.15.64:48990) 2024-09-12T03:20:20.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:20.488+0000 7f344e59c640 1 -- 172.21.15.64:0/1811704495 learned_addr learned my addr 172.21.15.64:0/1811704495 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:20:20.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:20.489+0000 7f344e59c640 1 -- 172.21.15.64:0/1811704495 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3450198390 con 0x7f3450106560 2024-09-12T03:20:20.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:20.489+0000 7f344e59c640 1 --2- 172.21.15.64:0/1811704495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3450106560 0x7f3450197e50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3440009230 tx=0x7f34400048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:20.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:20.489+0000 7f34477fe640 1 -- 172.21.15.64:0/1811704495 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f344003f070 con 0x7f3450106560 2024-09-12T03:20:20.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:20.489+0000 7f34477fe640 1 -- 172.21.15.64:0/1811704495 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3440004d10 con 0x7f3450106560 2024-09-12T03:20:20.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:20.489+0000 7f3454c3b640 1 -- 172.21.15.64:0/1811704495 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3450198620 con 0x7f3450106560 2024-09-12T03:20:20.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:20.489+0000 7f34477fe640 1 -- 172.21.15.64:0/1811704495 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3440035030 con 0x7f3450106560 2024-09-12T03:20:20.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:20.489+0000 7f3454c3b640 1 -- 172.21.15.64:0/1811704495 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3450198a80 con 0x7f3450106560 2024-09-12T03:20:20.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:20.490+0000 7f34477fe640 1 -- 172.21.15.64:0/1811704495 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f3440042070 con 0x7f3450106560 2024-09-12T03:20:20.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:20.490+0000 7f3454c3b640 1 -- 172.21.15.64:0/1811704495 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3450101bd0 con 0x7f3450106560 2024-09-12T03:20:20.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:20.490+0000 7f34477fe640 1 --2- 172.21.15.64:0/1811704495 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f343c03d4d0 0x7f343c03f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:20.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:20.490+0000 7f34477fe640 1 -- 172.21.15.64:0/1811704495 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f344006cce0 con 0x7f3450106560 2024-09-12T03:20:20.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:20.490+0000 7f3447fff640 1 --2- 172.21.15.64:0/1811704495 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f343c03d4d0 0x7f343c03f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:20.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:20.491+0000 7f3447fff640 1 --2- 172.21.15.64:0/1811704495 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f343c03d4d0 0x7f343c03f990 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f34380098c0 tx=0x7f3438006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:20.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:20.495+0000 7f34477fe640 1 -- 172.21.15.64:0/1811704495 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3440039350 con 0x7f3450106560 2024-09-12T03:20:20.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:20.673+0000 7f3454c3b640 1 -- 172.21.15.64:0/1811704495 --> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi115", "target": ["mon-mgr", ""]}) -- 0x7f34501069c0 con 0x7f343c03d4d0 2024-09-12T03:20:21.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:21 smithi064 ceph-mon[30952]: Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:20:21.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:21 smithi064 ceph-mon[30952]: Updating smithi064:/var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:20:22.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:22 smithi064 ceph-mon[30952]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi115", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:20:23.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:23 smithi064 ceph-mon[30952]: Deploying cephadm binary to smithi115 2024-09-12T03:20:24.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:24.580+0000 7f34477fe640 1 -- 172.21.15.64:0/1811704495 <== mgr.14150 v2:172.21.15.64:6800/759931848 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f34501069c0 con 0x7f343c03d4d0 2024-09-12T03:20:24.581 INFO:teuthology.orchestra.run.smithi064.stdout:Added host 'smithi115' with addr '172.21.15.115' 2024-09-12T03:20:24.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:24.583+0000 7f3454c3b640 1 -- 172.21.15.64:0/1811704495 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f343c03d4d0 msgr2=0x7f343c03f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:24.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:24.583+0000 7f3454c3b640 1 --2- 172.21.15.64:0/1811704495 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f343c03d4d0 0x7f343c03f990 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f34380098c0 tx=0x7f3438006db0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:24.583+0000 7f3454c3b640 1 -- 172.21.15.64:0/1811704495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3450106560 msgr2=0x7f3450197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:24.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:24.583+0000 7f3454c3b640 1 --2- 172.21.15.64:0/1811704495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3450106560 0x7f3450197e50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3440009230 tx=0x7f34400048f0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:24.583+0000 7f3454c3b640 1 -- 172.21.15.64:0/1811704495 shutdown_connections 2024-09-12T03:20:24.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:24.583+0000 7f3454c3b640 1 --2- 172.21.15.64:0/1811704495 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f343c03d4d0 0x7f343c03f990 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:24.583+0000 7f3454c3b640 1 --2- 172.21.15.64:0/1811704495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3450106560 0x7f3450197e50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:24.583+0000 7f3454c3b640 1 -- 172.21.15.64:0/1811704495 >> 172.21.15.64:0/1811704495 conn(0x7f34500fc1b0 msgr2=0x7f34500fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:24.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:24.583+0000 7f3454c3b640 1 -- 172.21.15.64:0/1811704495 shutdown_connections 2024-09-12T03:20:24.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:24.583+0000 7f3454c3b640 1 -- 172.21.15.64:0/1811704495 wait complete. 2024-09-12T03:20:25.195 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-12T03:20:25.444 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:25.842 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:25 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:25.842 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:25 smithi064 ceph-mon[30952]: Added host smithi115 2024-09-12T03:20:25.842 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:25 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:20:25.842 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:25 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:25.842 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:25 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:25.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:25.866+0000 7fb4b1815640 1 -- 172.21.15.64:0/609625182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4ac102730 msgr2=0x7fb4ac102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:25.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:25.866+0000 7fb4b1815640 1 --2- 172.21.15.64:0/609625182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4ac102730 0x7fb4ac102b10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4009810 tx=0x7fb4a402a540 comp rx=0 tx=0).stop 2024-09-12T03:20:25.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:25.866+0000 7fb4b1815640 1 -- 172.21.15.64:0/609625182 shutdown_connections 2024-09-12T03:20:25.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:25.866+0000 7fb4b1815640 1 --2- 172.21.15.64:0/609625182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4ac102730 0x7fb4ac102b10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:25.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:25.866+0000 7fb4b1815640 1 -- 172.21.15.64:0/609625182 >> 172.21.15.64:0/609625182 conn(0x7fb4ac077310 msgr2=0x7fb4ac0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:25.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:25.866+0000 7fb4b1815640 1 -- 172.21.15.64:0/609625182 shutdown_connections 2024-09-12T03:20:25.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:25.867+0000 7fb4b1815640 1 -- 172.21.15.64:0/609625182 wait complete. 2024-09-12T03:20:25.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:25.867+0000 7fb4b1815640 1 Processor -- start 2024-09-12T03:20:25.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:25.867+0000 7fb4b1815640 1 -- start start 2024-09-12T03:20:25.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:25.868+0000 7fb4b1815640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4ac102730 0x7fb4ac197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:25.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:25.868+0000 7fb4b1815640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac10bb30 con 0x7fb4ac102730 2024-09-12T03:20:25.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:25.868+0000 7fb4aaffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4ac102730 0x7fb4ac197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:25.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:25.868+0000 7fb4aaffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4ac102730 0x7fb4ac197060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49004/0 (socket says 172.21.15.64:49004) 2024-09-12T03:20:25.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:25.868+0000 7fb4aaffd640 1 -- 172.21.15.64:0/1662320962 learned_addr learned my addr 172.21.15.64:0/1662320962 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:20:25.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:25.868+0000 7fb4aaffd640 1 -- 172.21.15.64:0/1662320962 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4ac1975a0 con 0x7fb4ac102730 2024-09-12T03:20:25.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:25.868+0000 7fb4aaffd640 1 --2- 172.21.15.64:0/1662320962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4ac102730 0x7fb4ac197060 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4030b00 tx=0x7fb4a4030b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:25.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:25.868+0000 7fb4b0813640 1 -- 172.21.15.64:0/1662320962 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4a4040070 con 0x7fb4ac102730 2024-09-12T03:20:25.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:25.868+0000 7fb4b1815640 1 -- 172.21.15.64:0/1662320962 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4ac197830 con 0x7fb4ac102730 2024-09-12T03:20:25.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:25.869+0000 7fb4b0813640 1 -- 172.21.15.64:0/1662320962 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4a40318b0 con 0x7fb4ac102730 2024-09-12T03:20:25.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:25.869+0000 7fb4b0813640 1 -- 172.21.15.64:0/1662320962 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4a4003de0 con 0x7fb4ac102730 2024-09-12T03:20:25.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:25.869+0000 7fb4b1815640 1 -- 172.21.15.64:0/1662320962 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4ac197c90 con 0x7fb4ac102730 2024-09-12T03:20:25.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:25.869+0000 7fb4b0813640 1 -- 172.21.15.64:0/1662320962 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fb4a4039040 con 0x7fb4ac102730 2024-09-12T03:20:25.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:25.869+0000 7fb4b1815640 1 -- 172.21.15.64:0/1662320962 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4ac069f90 con 0x7fb4ac102730 2024-09-12T03:20:25.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:25.870+0000 7fb4b0813640 1 --2- 172.21.15.64:0/1662320962 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fb498039060 0x7fb49803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:25.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:25.870+0000 7fb4b0813640 1 -- 172.21.15.64:0/1662320962 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb4a406d620 con 0x7fb4ac102730 2024-09-12T03:20:25.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:25.870+0000 7fb4aa7fc640 1 --2- 172.21.15.64:0/1662320962 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fb498039060 0x7fb49803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:25.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:25.870+0000 7fb4aa7fc640 1 --2- 172.21.15.64:0/1662320962 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fb498039060 0x7fb49803b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb4940098c0 tx=0x7fb494006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:25.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:25.874+0000 7fb4b0813640 1 -- 172.21.15.64:0/1662320962 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb4a404fcf0 con 0x7fb4ac102730 2024-09-12T03:20:26.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:26.054+0000 7fb4b1815640 1 -- 172.21.15.64:0/1662320962 --> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb4ac1983c0 con 0x7fb498039060 2024-09-12T03:20:26.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:26.055+0000 7fb4b0813640 1 -- 172.21.15.64:0/1662320962 <== mgr.14150 v2:172.21.15.64:6800/759931848 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7fb4ac1983c0 con 0x7fb498039060 2024-09-12T03:20:26.055 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:20:26.055 INFO:teuthology.orchestra.run.smithi064.stdout:[{"addr": "172.21.15.64", "hostname": "smithi064", "labels": [], "status": ""}, {"addr": "172.21.15.115", "hostname": "smithi115", "labels": [], "status": ""}] 2024-09-12T03:20:26.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:26.057+0000 7fb4b1815640 1 -- 172.21.15.64:0/1662320962 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fb498039060 msgr2=0x7fb49803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:26.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:26.057+0000 7fb4b1815640 1 --2- 172.21.15.64:0/1662320962 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fb498039060 0x7fb49803b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb4940098c0 tx=0x7fb494006db0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:26.057+0000 7fb4b1815640 1 -- 172.21.15.64:0/1662320962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4ac102730 msgr2=0x7fb4ac197060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:26.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:26.057+0000 7fb4b1815640 1 --2- 172.21.15.64:0/1662320962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4ac102730 0x7fb4ac197060 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4030b00 tx=0x7fb4a4030b30 comp rx=0 tx=0).stop 2024-09-12T03:20:26.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:26.057+0000 7fb4b1815640 1 -- 172.21.15.64:0/1662320962 shutdown_connections 2024-09-12T03:20:26.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:26.057+0000 7fb4b1815640 1 --2- 172.21.15.64:0/1662320962 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fb498039060 0x7fb49803b520 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:26.057+0000 7fb4b1815640 1 --2- 172.21.15.64:0/1662320962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4ac102730 0x7fb4ac197060 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:26.057+0000 7fb4b1815640 1 -- 172.21.15.64:0/1662320962 >> 172.21.15.64:0/1662320962 conn(0x7fb4ac077310 msgr2=0x7fb4ac1034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:26.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:26.057+0000 7fb4b1815640 1 -- 172.21.15.64:0/1662320962 shutdown_connections 2024-09-12T03:20:26.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:26.057+0000 7fb4b1815640 1 -- 172.21.15.64:0/1662320962 wait complete. 2024-09-12T03:20:26.674 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi195 2024-09-12T03:20:26.674 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-12T03:20:26.674 DEBUG:teuthology.orchestra.run.smithi195:> dd of=/etc/ceph/ceph.conf 2024-09-12T03:20:26.694 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-12T03:20:26.694 DEBUG:teuthology.orchestra.run.smithi195:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:20:26.753 INFO:tasks.cephadm:Adding host smithi195 to orchestrator... 2024-09-12T03:20:26.753 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph orch host add smithi195 2024-09-12T03:20:27.004 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:27.240 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:26 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:27.240 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:26 smithi064 ceph-mon[30952]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:20:27.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:27.431+0000 7fe9ace79640 1 -- 172.21.15.64:0/2867247789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9a81027b0 msgr2=0x7fe9a8102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:27.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:27.431+0000 7fe9ace79640 1 --2- 172.21.15.64:0/2867247789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9a81027b0 0x7fe9a8102b90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe998009880 tx=0x7fe99802a670 comp rx=0 tx=0).stop 2024-09-12T03:20:27.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:27.431+0000 7fe9ace79640 1 -- 172.21.15.64:0/2867247789 shutdown_connections 2024-09-12T03:20:27.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:27.431+0000 7fe9ace79640 1 --2- 172.21.15.64:0/2867247789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9a81027b0 0x7fe9a8102b90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:27.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:27.431+0000 7fe9ace79640 1 -- 172.21.15.64:0/2867247789 >> 172.21.15.64:0/2867247789 conn(0x7fe9a80fc1b0 msgr2=0x7fe9a80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:27.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:27.431+0000 7fe9ace79640 1 -- 172.21.15.64:0/2867247789 shutdown_connections 2024-09-12T03:20:27.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:27.431+0000 7fe9ace79640 1 -- 172.21.15.64:0/2867247789 wait complete. 2024-09-12T03:20:27.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:27.432+0000 7fe9ace79640 1 Processor -- start 2024-09-12T03:20:27.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:27.432+0000 7fe9ace79640 1 -- start start 2024-09-12T03:20:27.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:27.433+0000 7fe9ace79640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9a81027b0 0x7fe9a8100410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:27.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:27.433+0000 7fe9ace79640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a810bb60 con 0x7fe9a81027b0 2024-09-12T03:20:27.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:27.433+0000 7fe9a659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9a81027b0 0x7fe9a8100410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:27.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:27.433+0000 7fe9a659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9a81027b0 0x7fe9a8100410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49022/0 (socket says 172.21.15.64:49022) 2024-09-12T03:20:27.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:27.433+0000 7fe9a659c640 1 -- 172.21.15.64:0/2477177542 learned_addr learned my addr 172.21.15.64:0/2477177542 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:20:27.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:27.433+0000 7fe9a659c640 1 -- 172.21.15.64:0/2477177542 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a8102300 con 0x7fe9a81027b0 2024-09-12T03:20:27.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:27.433+0000 7fe9a659c640 1 --2- 172.21.15.64:0/2477177542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9a81027b0 0x7fe9a8100410 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe998009230 tx=0x7fe9980048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:27.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:27.433+0000 7fe9977fe640 1 -- 172.21.15.64:0/2477177542 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe99803f070 con 0x7fe9a81027b0 2024-09-12T03:20:27.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:27.433+0000 7fe9ace79640 1 -- 172.21.15.64:0/2477177542 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9a8100950 con 0x7fe9a81027b0 2024-09-12T03:20:27.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:27.433+0000 7fe9977fe640 1 -- 172.21.15.64:0/2477177542 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe998004d10 con 0x7fe9a81027b0 2024-09-12T03:20:27.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:27.433+0000 7fe9977fe640 1 -- 172.21.15.64:0/2477177542 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe998035030 con 0x7fe9a81027b0 2024-09-12T03:20:27.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:27.433+0000 7fe9ace79640 1 -- 172.21.15.64:0/2477177542 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9a8100e10 con 0x7fe9a81027b0 2024-09-12T03:20:27.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:27.434+0000 7fe9977fe640 1 -- 172.21.15.64:0/2477177542 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fe998042070 con 0x7fe9a81027b0 2024-09-12T03:20:27.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:27.434+0000 7fe9ace79640 1 -- 172.21.15.64:0/2477177542 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9a806a750 con 0x7fe9a81027b0 2024-09-12T03:20:27.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:27.435+0000 7fe9977fe640 1 --2- 172.21.15.64:0/2477177542 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe990039060 0x7fe99003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:27.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:27.435+0000 7fe9977fe640 1 -- 172.21.15.64:0/2477177542 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe99806ce10 con 0x7fe9a81027b0 2024-09-12T03:20:27.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:27.435+0000 7fe9a5d9b640 1 --2- 172.21.15.64:0/2477177542 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe990039060 0x7fe99003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:27.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:27.435+0000 7fe9a5d9b640 1 --2- 172.21.15.64:0/2477177542 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe990039060 0x7fe99003b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe99c0098c0 tx=0x7fe99c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:27.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:27.439+0000 7fe9977fe640 1 -- 172.21.15.64:0/2477177542 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe9980037d0 con 0x7fe9a81027b0 2024-09-12T03:20:27.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:27.619+0000 7fe9ace79640 1 -- 172.21.15.64:0/2477177542 --> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi195", "target": ["mon-mgr", ""]}) -- 0x7fe9a8101940 con 0x7fe990039060 2024-09-12T03:20:29.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:28 smithi064 ceph-mon[30952]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:20:29.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:28 smithi064 ceph-mon[30952]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi195", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:20:29.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:28 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:29.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:28 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:29.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:28 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:29.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:28 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:29.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:28 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:20:29.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:28 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:29.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:28 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:20:30.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:29 smithi064 ceph-mon[30952]: Updating smithi115:/etc/ceph/ceph.conf 2024-09-12T03:20:30.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:29 smithi064 ceph-mon[30952]: Deploying cephadm binary to smithi195 2024-09-12T03:20:30.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:29 smithi064 ceph-mon[30952]: Updating smithi115:/var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:20:30.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:29 smithi064 ceph-mon[30952]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:20:30.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:29 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:30.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:29 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:30.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:29 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:31.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:30 smithi064 ceph-mon[30952]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:20:31.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:30 smithi064 ceph-mon[30952]: Updating smithi115:/var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:20:31.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:31.674+0000 7fe9977fe640 1 -- 172.21.15.64:0/2477177542 <== mgr.14150 v2:172.21.15.64:6800/759931848 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fe9a8101940 con 0x7fe990039060 2024-09-12T03:20:31.676 INFO:teuthology.orchestra.run.smithi064.stdout:Added host 'smithi195' with addr '172.21.15.195' 2024-09-12T03:20:31.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:31.676+0000 7fe9ace79640 1 -- 172.21.15.64:0/2477177542 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe990039060 msgr2=0x7fe99003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:31.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:31.676+0000 7fe9ace79640 1 --2- 172.21.15.64:0/2477177542 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe990039060 0x7fe99003b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe99c0098c0 tx=0x7fe99c006db0 comp rx=0 tx=0).stop 2024-09-12T03:20:31.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:31.676+0000 7fe9ace79640 1 -- 172.21.15.64:0/2477177542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9a81027b0 msgr2=0x7fe9a8100410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:31.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:31.676+0000 7fe9ace79640 1 --2- 172.21.15.64:0/2477177542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9a81027b0 0x7fe9a8100410 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe998009230 tx=0x7fe9980048f0 comp rx=0 tx=0).stop 2024-09-12T03:20:31.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:31.677+0000 7fe9ace79640 1 -- 172.21.15.64:0/2477177542 shutdown_connections 2024-09-12T03:20:31.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:31.677+0000 7fe9ace79640 1 --2- 172.21.15.64:0/2477177542 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe990039060 0x7fe99003b520 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:31.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:31.677+0000 7fe9ace79640 1 --2- 172.21.15.64:0/2477177542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9a81027b0 0x7fe9a8100410 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:31.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:31.677+0000 7fe9ace79640 1 -- 172.21.15.64:0/2477177542 >> 172.21.15.64:0/2477177542 conn(0x7fe9a80fc1b0 msgr2=0x7fe9a8103560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:31.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:31.677+0000 7fe9ace79640 1 -- 172.21.15.64:0/2477177542 shutdown_connections 2024-09-12T03:20:31.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:31.677+0000 7fe9ace79640 1 -- 172.21.15.64:0/2477177542 wait complete. 2024-09-12T03:20:32.282 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-12T03:20:32.532 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:32.914 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:32 smithi064 ceph-mon[30952]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:20:32.914 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:32 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:32.914 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:32 smithi064 ceph-mon[30952]: Added host smithi195 2024-09-12T03:20:32.914 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:32 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:20:32.914 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:32 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:32.914 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:32 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:32.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:32.920+0000 7f0ab5c53640 1 -- 172.21.15.64:0/474830610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ab0106560 msgr2=0x7f0ab0106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:32.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:32.920+0000 7f0ab5c53640 1 --2- 172.21.15.64:0/474830610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ab0106560 0x7f0ab0106940 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0009880 tx=0x7f0aa002a5d0 comp rx=0 tx=0).stop 2024-09-12T03:20:32.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:32.920+0000 7f0ab5c53640 1 -- 172.21.15.64:0/474830610 shutdown_connections 2024-09-12T03:20:32.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:32.920+0000 7f0ab5c53640 1 --2- 172.21.15.64:0/474830610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ab0106560 0x7f0ab0106940 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:32.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:32.920+0000 7f0ab5c53640 1 -- 172.21.15.64:0/474830610 >> 172.21.15.64:0/474830610 conn(0x7f0ab00fc1b0 msgr2=0x7f0ab00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:32.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:32.920+0000 7f0ab5c53640 1 -- 172.21.15.64:0/474830610 shutdown_connections 2024-09-12T03:20:32.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:32.920+0000 7f0ab5c53640 1 -- 172.21.15.64:0/474830610 wait complete. 2024-09-12T03:20:32.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:32.921+0000 7f0ab5c53640 1 Processor -- start 2024-09-12T03:20:32.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:32.921+0000 7f0ab5c53640 1 -- start start 2024-09-12T03:20:32.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:32.921+0000 7f0ab5c53640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ab0106560 0x7f0ab0197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:32.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:32.921+0000 7f0ab5c53640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab010aa40 con 0x7f0ab0106560 2024-09-12T03:20:32.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:32.921+0000 7f0aaf7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ab0106560 0x7f0ab0197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:32.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:32.921+0000 7f0aaf7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ab0106560 0x7f0ab0197010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37578/0 (socket says 172.21.15.64:37578) 2024-09-12T03:20:32.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:32.921+0000 7f0aaf7fe640 1 -- 172.21.15.64:0/2794968966 learned_addr learned my addr 172.21.15.64:0/2794968966 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:20:32.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:32.921+0000 7f0aaf7fe640 1 -- 172.21.15.64:0/2794968966 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ab0197550 con 0x7f0ab0106560 2024-09-12T03:20:32.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:32.922+0000 7f0aaf7fe640 1 --2- 172.21.15.64:0/2794968966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ab0106560 0x7f0ab0197010 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0009230 tx=0x7f0aa00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:32.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:32.922+0000 7f0aacff9640 1 -- 172.21.15.64:0/2794968966 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0aa003f070 con 0x7f0ab0106560 2024-09-12T03:20:32.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:32.922+0000 7f0aacff9640 1 -- 172.21.15.64:0/2794968966 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0aa0003aa0 con 0x7f0ab0106560 2024-09-12T03:20:32.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:32.922+0000 7f0ab5c53640 1 -- 172.21.15.64:0/2794968966 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ab01977e0 con 0x7f0ab0106560 2024-09-12T03:20:32.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:32.922+0000 7f0aacff9640 1 -- 172.21.15.64:0/2794968966 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0aa0035030 con 0x7f0ab0106560 2024-09-12T03:20:32.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:32.922+0000 7f0ab5c53640 1 -- 172.21.15.64:0/2794968966 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ab019b8e0 con 0x7f0ab0106560 2024-09-12T03:20:32.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:32.923+0000 7f0aacff9640 1 -- 172.21.15.64:0/2794968966 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f0aa0042070 con 0x7f0ab0106560 2024-09-12T03:20:32.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:32.923+0000 7f0ab5c53640 1 -- 172.21.15.64:0/2794968966 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ab0101bd0 con 0x7f0ab0106560 2024-09-12T03:20:32.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:32.923+0000 7f0aacff9640 1 --2- 172.21.15.64:0/2794968966 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f0a980390c0 0x7f0a9803b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:32.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:32.923+0000 7f0aacff9640 1 -- 172.21.15.64:0/2794968966 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0aa006cf00 con 0x7f0ab0106560 2024-09-12T03:20:32.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:32.923+0000 7f0aaeffd640 1 --2- 172.21.15.64:0/2794968966 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f0a980390c0 0x7f0a9803b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:32.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:32.924+0000 7f0aaeffd640 1 --2- 172.21.15.64:0/2794968966 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f0a980390c0 0x7f0a9803b580 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0aa40098c0 tx=0x7f0aa4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:32.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:32.928+0000 7f0aacff9640 1 -- 172.21.15.64:0/2794968966 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0aa00433b0 con 0x7f0ab0106560 2024-09-12T03:20:33.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:33.108+0000 7f0ab5c53640 1 -- 172.21.15.64:0/2794968966 --> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0ab01069c0 con 0x7f0a980390c0 2024-09-12T03:20:33.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:33.109+0000 7f0aacff9640 1 -- 172.21.15.64:0/2794968966 <== mgr.14150 v2:172.21.15.64:6800/759931848 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f0ab01069c0 con 0x7f0a980390c0 2024-09-12T03:20:33.110 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:20:33.110 INFO:teuthology.orchestra.run.smithi064.stdout:[{"addr": "172.21.15.64", "hostname": "smithi064", "labels": [], "status": ""}, {"addr": "172.21.15.115", "hostname": "smithi115", "labels": [], "status": ""}, {"addr": "172.21.15.195", "hostname": "smithi195", "labels": [], "status": ""}] 2024-09-12T03:20:33.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:33.111+0000 7f0ab5c53640 1 -- 172.21.15.64:0/2794968966 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f0a980390c0 msgr2=0x7f0a9803b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:33.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:33.111+0000 7f0ab5c53640 1 --2- 172.21.15.64:0/2794968966 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f0a980390c0 0x7f0a9803b580 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0aa40098c0 tx=0x7f0aa4006db0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:33.111+0000 7f0ab5c53640 1 -- 172.21.15.64:0/2794968966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ab0106560 msgr2=0x7f0ab0197010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:33.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:33.111+0000 7f0ab5c53640 1 --2- 172.21.15.64:0/2794968966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ab0106560 0x7f0ab0197010 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0009230 tx=0x7f0aa00048f0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:33.111+0000 7f0ab5c53640 1 -- 172.21.15.64:0/2794968966 shutdown_connections 2024-09-12T03:20:33.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:33.111+0000 7f0ab5c53640 1 --2- 172.21.15.64:0/2794968966 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f0a980390c0 0x7f0a9803b580 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:33.111+0000 7f0ab5c53640 1 --2- 172.21.15.64:0/2794968966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ab0106560 0x7f0ab0197010 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:33.111+0000 7f0ab5c53640 1 -- 172.21.15.64:0/2794968966 >> 172.21.15.64:0/2794968966 conn(0x7f0ab00fc1b0 msgr2=0x7f0ab00fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:33.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:33.111+0000 7f0ab5c53640 1 -- 172.21.15.64:0/2794968966 shutdown_connections 2024-09-12T03:20:33.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:33.111+0000 7f0ab5c53640 1 -- 172.21.15.64:0/2794968966 wait complete. 2024-09-12T03:20:33.683 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-12T03:20:33.683 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-12T03:20:33.936 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:34.344 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:34 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:34.345 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:34 smithi064 ceph-mon[30952]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:20:34.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:34.352+0000 7fe658607640 1 -- 172.21.15.64:0/2403280323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe650107660 msgr2=0x7fe650107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:34.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:34.352+0000 7fe658607640 1 --2- 172.21.15.64:0/2403280323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe650107660 0x7fe650107a40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe64c009880 tx=0x7fe64c02a5d0 comp rx=0 tx=0).stop 2024-09-12T03:20:34.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:34.352+0000 7fe658607640 1 -- 172.21.15.64:0/2403280323 shutdown_connections 2024-09-12T03:20:34.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:34.352+0000 7fe658607640 1 --2- 172.21.15.64:0/2403280323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe650107660 0x7fe650107a40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:34.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:34.352+0000 7fe658607640 1 -- 172.21.15.64:0/2403280323 >> 172.21.15.64:0/2403280323 conn(0x7fe6500fd290 msgr2=0x7fe6500ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:34.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:34.352+0000 7fe658607640 1 -- 172.21.15.64:0/2403280323 shutdown_connections 2024-09-12T03:20:34.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:34.352+0000 7fe658607640 1 -- 172.21.15.64:0/2403280323 wait complete. 2024-09-12T03:20:34.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:34.353+0000 7fe658607640 1 Processor -- start 2024-09-12T03:20:34.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:34.353+0000 7fe658607640 1 -- start start 2024-09-12T03:20:34.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:34.353+0000 7fe658607640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe650107660 0x7fe6500744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:34.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:34.353+0000 7fe658607640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe65010bb40 con 0x7fe650107660 2024-09-12T03:20:34.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:34.353+0000 7fe6563a3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe650107660 0x7fe6500744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:34.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:34.353+0000 7fe6563a3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe650107660 0x7fe6500744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37610/0 (socket says 172.21.15.64:37610) 2024-09-12T03:20:34.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:34.353+0000 7fe6563a3640 1 -- 172.21.15.64:0/444365947 learned_addr learned my addr 172.21.15.64:0/444365947 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:20:34.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:34.354+0000 7fe6563a3640 1 -- 172.21.15.64:0/444365947 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6500763e0 con 0x7fe650107660 2024-09-12T03:20:34.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:34.354+0000 7fe6563a3640 1 --2- 172.21.15.64:0/444365947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe650107660 0x7fe6500744f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe64c009230 tx=0x7fe64c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:34.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:34.354+0000 7fe6477fe640 1 -- 172.21.15.64:0/444365947 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe64c03f070 con 0x7fe650107660 2024-09-12T03:20:34.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:34.354+0000 7fe6477fe640 1 -- 172.21.15.64:0/444365947 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe64c004d10 con 0x7fe650107660 2024-09-12T03:20:34.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:34.354+0000 7fe658607640 1 -- 172.21.15.64:0/444365947 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe650074a30 con 0x7fe650107660 2024-09-12T03:20:34.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:34.354+0000 7fe6477fe640 1 -- 172.21.15.64:0/444365947 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe64c035030 con 0x7fe650107660 2024-09-12T03:20:34.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:34.354+0000 7fe658607640 1 -- 172.21.15.64:0/444365947 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe650074ef0 con 0x7fe650107660 2024-09-12T03:20:34.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:34.355+0000 7fe6477fe640 1 -- 172.21.15.64:0/444365947 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fe64c042070 con 0x7fe650107660 2024-09-12T03:20:34.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:34.355+0000 7fe658607640 1 -- 172.21.15.64:0/444365947 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe650102cd0 con 0x7fe650107660 2024-09-12T03:20:34.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:34.355+0000 7fe6477fe640 1 --2- 172.21.15.64:0/444365947 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe63c039060 0x7fe63c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:34.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:34.355+0000 7fe6477fe640 1 -- 172.21.15.64:0/444365947 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe64c06cd90 con 0x7fe650107660 2024-09-12T03:20:34.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:34.355+0000 7fe655ba2640 1 --2- 172.21.15.64:0/444365947 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe63c039060 0x7fe63c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:34.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:34.356+0000 7fe655ba2640 1 --2- 172.21.15.64:0/444365947 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe63c039060 0x7fe63c03b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe6400098c0 tx=0x7fe640006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:34.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:34.360+0000 7fe6477fe640 1 -- 172.21.15.64:0/444365947 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe64c042360 con 0x7fe650107660 2024-09-12T03:20:34.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:34.539+0000 7fe658607640 1 -- 172.21.15.64:0/444365947 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fe650075a20 con 0x7fe650107660 2024-09-12T03:20:35.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:35.039+0000 7fe6477fe640 1 -- 172.21.15.64:0/444365947 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fe64c02fdc0 con 0x7fe650107660 2024-09-12T03:20:35.041 INFO:teuthology.orchestra.run.smithi064.stderr:adjusted tunables profile to default 2024-09-12T03:20:35.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:35.042+0000 7fe658607640 1 -- 172.21.15.64:0/444365947 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe63c039060 msgr2=0x7fe63c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:35.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:35.042+0000 7fe658607640 1 --2- 172.21.15.64:0/444365947 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe63c039060 0x7fe63c03b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe6400098c0 tx=0x7fe640006db0 comp rx=0 tx=0).stop 2024-09-12T03:20:35.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:35.042+0000 7fe658607640 1 -- 172.21.15.64:0/444365947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe650107660 msgr2=0x7fe6500744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:35.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:35.042+0000 7fe658607640 1 --2- 172.21.15.64:0/444365947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe650107660 0x7fe6500744f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe64c009230 tx=0x7fe64c0048f0 comp rx=0 tx=0).stop 2024-09-12T03:20:35.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:35.042+0000 7fe658607640 1 -- 172.21.15.64:0/444365947 shutdown_connections 2024-09-12T03:20:35.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:35.042+0000 7fe658607640 1 --2- 172.21.15.64:0/444365947 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe63c039060 0x7fe63c03b520 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:35.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:35.042+0000 7fe658607640 1 --2- 172.21.15.64:0/444365947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe650107660 0x7fe6500744f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:35.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:35.042+0000 7fe658607640 1 -- 172.21.15.64:0/444365947 >> 172.21.15.64:0/444365947 conn(0x7fe6500fd290 msgr2=0x7fe6500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:35.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:35.042+0000 7fe658607640 1 -- 172.21.15.64:0/444365947 shutdown_connections 2024-09-12T03:20:35.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:35.042+0000 7fe658607640 1 -- 172.21.15.64:0/444365947 wait complete. 2024-09-12T03:20:35.323 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:35 smithi064 ceph-mon[30952]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:20:35.323 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:35 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/444365947' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-12T03:20:35.609 INFO:tasks.cephadm:Adding mon.a on smithi064 2024-09-12T03:20:35.609 INFO:tasks.cephadm:Adding mon.b on smithi115 2024-09-12T03:20:35.609 INFO:tasks.cephadm:Adding mon.c on smithi195 2024-09-12T03:20:35.609 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi064:172.21.15.64=a;smithi115:172.21.15.115=b;smithi195:172.21.15.195=c' 2024-09-12T03:20:35.879 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-12T03:20:35.925 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-12T03:20:36.435 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:36.411+0000 7ff7fc546640 1 -- 172.21.15.195:0/1557798557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7f4101620 msgr2=0x7ff7f4101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:36.435 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:36.411+0000 7ff7fc546640 1 --2- 172.21.15.195:0/1557798557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7f4101620 0x7ff7f4101a20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0009880 tx=0x7ff7f002a5d0 comp rx=0 tx=0).stop 2024-09-12T03:20:36.436 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:36.411+0000 7ff7fc546640 1 -- 172.21.15.195:0/1557798557 shutdown_connections 2024-09-12T03:20:36.436 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:36.411+0000 7ff7fc546640 1 --2- 172.21.15.195:0/1557798557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7f4101620 0x7ff7f4101a20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:36.436 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:36.411+0000 7ff7fc546640 1 -- 172.21.15.195:0/1557798557 >> 172.21.15.195:0/1557798557 conn(0x7ff7f40fcdb0 msgr2=0x7ff7f40ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:36.436 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:36.411+0000 7ff7fc546640 1 -- 172.21.15.195:0/1557798557 shutdown_connections 2024-09-12T03:20:36.436 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:36.411+0000 7ff7fc546640 1 -- 172.21.15.195:0/1557798557 wait complete. 2024-09-12T03:20:36.436 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:36.412+0000 7ff7fc546640 1 Processor -- start 2024-09-12T03:20:36.436 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:36.412+0000 7ff7fc546640 1 -- start start 2024-09-12T03:20:36.436 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:36.412+0000 7ff7fc546640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7f4101620 0x7ff7f4196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:36.436 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:36.412+0000 7ff7fc546640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f41075e0 con 0x7ff7f4101620 2024-09-12T03:20:36.436 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:36.412+0000 7ff7fa2e2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7f4101620 0x7ff7f4196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:36.436 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:36.412+0000 7ff7fa2e2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7f4101620 0x7ff7f4196d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.195:37036/0 (socket says 172.21.15.195:37036) 2024-09-12T03:20:36.436 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:36.412+0000 7ff7fa2e2640 1 -- 172.21.15.195:0/44860109 learned_addr learned my addr 172.21.15.195:0/44860109 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-09-12T03:20:36.437 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:36.413+0000 7ff7fa2e2640 1 -- 172.21.15.195:0/44860109 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f4197270 con 0x7ff7f4101620 2024-09-12T03:20:36.437 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:36.413+0000 7ff7fa2e2640 1 --2- 172.21.15.195:0/44860109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7f4101620 0x7ff7f4196d30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0005ab0 tx=0x7ff7f00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:36.437 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:36.413+0000 7ff7eb7fe640 1 -- 172.21.15.195:0/44860109 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7f003f070 con 0x7ff7f4101620 2024-09-12T03:20:36.437 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:36.413+0000 7ff7eb7fe640 1 -- 172.21.15.195:0/44860109 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7f0003aa0 con 0x7ff7f4101620 2024-09-12T03:20:36.437 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:36.413+0000 7ff7fc546640 1 -- 172.21.15.195:0/44860109 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7f4197500 con 0x7ff7f4101620 2024-09-12T03:20:36.437 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:36.413+0000 7ff7eb7fe640 1 -- 172.21.15.195:0/44860109 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7f0035030 con 0x7ff7f4101620 2024-09-12T03:20:36.437 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:36.413+0000 7ff7fc546640 1 -- 172.21.15.195:0/44860109 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7f419a1f0 con 0x7ff7f4101620 2024-09-12T03:20:36.437 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:36.414+0000 7ff7fc546640 1 -- 172.21.15.195:0/44860109 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7f4107d60 con 0x7ff7f4101620 2024-09-12T03:20:36.437 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:36.414+0000 7ff7eb7fe640 1 -- 172.21.15.195:0/44860109 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff7f0042070 con 0x7ff7f4101620 2024-09-12T03:20:36.438 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:36.414+0000 7ff7eb7fe640 1 --2- 172.21.15.195:0/44860109 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff7e0039060 0x7ff7e003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:36.438 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:36.414+0000 7ff7eb7fe640 1 -- 172.21.15.195:0/44860109 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff7f006cc90 con 0x7ff7f4101620 2024-09-12T03:20:36.438 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:36.415+0000 7ff7f9ae1640 1 --2- 172.21.15.195:0/44860109 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff7e0039060 0x7ff7e003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:36.438 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:36.415+0000 7ff7f9ae1640 1 --2- 172.21.15.195:0/44860109 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff7e0039060 0x7ff7e003b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff7e40098c0 tx=0x7ff7e4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:36.438 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:36.420+0000 7ff7eb7fe640 1 -- 172.21.15.195:0/44860109 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff7f00372d0 con 0x7ff7f4101620 2024-09-12T03:20:36.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:36 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/444365947' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-12T03:20:36.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:36 smithi064 ceph-mon[30952]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T03:20:36.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:36 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:36.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:36 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:36.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:36 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:36.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:36 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:36.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:36 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi195", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:20:36.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:36 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:36.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:36 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:20:36.612 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:36.611+0000 7ff7fc546640 1 -- 172.21.15.195:0/44860109 --> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi064:172.21.15.64=a;smithi115:172.21.15.115=b;smithi195:172.21.15.195=c", "target": ["mon-mgr", ""]}) -- 0x7ff7f402d830 con 0x7ff7e0039060 2024-09-12T03:20:36.617 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:36.617+0000 7ff7eb7fe640 1 -- 172.21.15.195:0/44860109 <== mgr.14150 v2:172.21.15.64:6800/759931848 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff7f402d830 con 0x7ff7e0039060 2024-09-12T03:20:36.617 INFO:teuthology.orchestra.run.smithi195.stdout:Scheduled mon update... 2024-09-12T03:20:36.619 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:36.619+0000 7ff7fc546640 1 -- 172.21.15.195:0/44860109 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff7e0039060 msgr2=0x7ff7e003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:36.619 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:36.619+0000 7ff7fc546640 1 --2- 172.21.15.195:0/44860109 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff7e0039060 0x7ff7e003b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff7e40098c0 tx=0x7ff7e4006db0 comp rx=0 tx=0).stop 2024-09-12T03:20:36.619 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:36.619+0000 7ff7fc546640 1 -- 172.21.15.195:0/44860109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7f4101620 msgr2=0x7ff7f4196d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:36.620 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:36.619+0000 7ff7fc546640 1 --2- 172.21.15.195:0/44860109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7f4101620 0x7ff7f4196d30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0005ab0 tx=0x7ff7f00048f0 comp rx=0 tx=0).stop 2024-09-12T03:20:36.620 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:36.619+0000 7ff7fc546640 1 -- 172.21.15.195:0/44860109 shutdown_connections 2024-09-12T03:20:36.620 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:36.619+0000 7ff7fc546640 1 --2- 172.21.15.195:0/44860109 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff7e0039060 0x7ff7e003b520 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:36.620 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:36.619+0000 7ff7fc546640 1 --2- 172.21.15.195:0/44860109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7f4101620 0x7ff7f4196d30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:36.620 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:36.619+0000 7ff7fc546640 1 -- 172.21.15.195:0/44860109 >> 172.21.15.195:0/44860109 conn(0x7ff7f40fcdb0 msgr2=0x7ff7f40fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:36.620 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:36.619+0000 7ff7fc546640 1 -- 172.21.15.195:0/44860109 shutdown_connections 2024-09-12T03:20:36.620 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:36.619+0000 7ff7fc546640 1 -- 172.21.15.195:0/44860109 wait complete. 2024-09-12T03:20:37.227 DEBUG:teuthology.orchestra.run.smithi115:mon.b> sudo journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@mon.b.service 2024-09-12T03:20:37.231 DEBUG:teuthology.orchestra.run.smithi195:mon.c> sudo journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@mon.c.service 2024-09-12T03:20:37.233 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T03:20:37.233 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T03:20:37.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:37 smithi064 ceph-mon[30952]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:20:37.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:37 smithi064 ceph-mon[30952]: Updating smithi195:/etc/ceph/ceph.conf 2024-09-12T03:20:37.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:37 smithi064 ceph-mon[30952]: Updating smithi195:/var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:20:37.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:37 smithi064 ceph-mon[30952]: Updating smithi195:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:20:37.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:37 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:37.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:37 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:37.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:37 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:37.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:37 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:37.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:37 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:20:37.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:37 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:37.847 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:20:38.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:38 smithi064 ceph-mon[30952]: Updating smithi195:/var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:20:38.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:38 smithi064 ceph-mon[30952]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi064:172.21.15.64=a;smithi115:172.21.15.115=b;smithi195:172.21.15.195=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:20:38.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:38 smithi064 ceph-mon[30952]: Saving service mon spec with placement smithi064:172.21.15.64=a;smithi115:172.21.15.115=b;smithi195:172.21.15.195=c;count:3 2024-09-12T03:20:38.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:38 smithi064 ceph-mon[30952]: Deploying daemon mon.c on smithi195 2024-09-12T03:20:39.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:39 smithi064 ceph-mon[30952]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:20:39.833 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:39.801+0000 7f24debd6640 1 -- 172.21.15.195:0/1026997061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24d8102790 msgr2=0x7f24d8102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:39.834 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:39.801+0000 7f24debd6640 1 --2- 172.21.15.195:0/1026997061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24d8102790 0x7f24d8102b70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f24cc009810 tx=0x7f24cc02a540 comp rx=0 tx=0).stop 2024-09-12T03:20:39.834 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:39.801+0000 7f24debd6640 1 -- 172.21.15.195:0/1026997061 shutdown_connections 2024-09-12T03:20:39.834 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:39.801+0000 7f24debd6640 1 --2- 172.21.15.195:0/1026997061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24d8102790 0x7f24d8102b70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:39.834 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:39.801+0000 7f24debd6640 1 -- 172.21.15.195:0/1026997061 >> 172.21.15.195:0/1026997061 conn(0x7f24d8077310 msgr2=0x7f24d80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:39.834 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:39.801+0000 7f24debd6640 1 -- 172.21.15.195:0/1026997061 shutdown_connections 2024-09-12T03:20:39.834 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:39.801+0000 7f24debd6640 1 -- 172.21.15.195:0/1026997061 wait complete. 2024-09-12T03:20:39.834 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:39.801+0000 7f24debd6640 1 Processor -- start 2024-09-12T03:20:39.834 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:39.801+0000 7f24debd6640 1 -- start start 2024-09-12T03:20:39.834 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:39.802+0000 7f24debd6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24d8102790 0x7f24d81003f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:39.834 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:39.802+0000 7f24debd6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f24d810bb30 con 0x7f24d8102790 2024-09-12T03:20:39.835 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:39.802+0000 7f24dc972640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24d8102790 0x7f24d81003f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:39.835 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:39.802+0000 7f24dc972640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24d8102790 0x7f24d81003f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.195:37062/0 (socket says 172.21.15.195:37062) 2024-09-12T03:20:39.835 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:39.802+0000 7f24dc972640 1 -- 172.21.15.195:0/3991310212 learned_addr learned my addr 172.21.15.195:0/3991310212 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-09-12T03:20:39.835 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:39.802+0000 7f24dc972640 1 -- 172.21.15.195:0/3991310212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d81022e0 con 0x7f24d8102790 2024-09-12T03:20:39.835 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:39.802+0000 7f24dc972640 1 --2- 172.21.15.195:0/3991310212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24d8102790 0x7f24d81003f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f24cc030b00 tx=0x7f24cc030b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:39.835 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:39.803+0000 7f24d5ffb640 1 -- 172.21.15.195:0/3991310212 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24cc040070 con 0x7f24d8102790 2024-09-12T03:20:39.835 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:39.803+0000 7f24debd6640 1 -- 172.21.15.195:0/3991310212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24d8100930 con 0x7f24d8102790 2024-09-12T03:20:39.835 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:39.803+0000 7f24d5ffb640 1 -- 172.21.15.195:0/3991310212 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24cc0318b0 con 0x7f24d8102790 2024-09-12T03:20:39.835 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:39.803+0000 7f24d5ffb640 1 -- 172.21.15.195:0/3991310212 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24cc003de0 con 0x7f24d8102790 2024-09-12T03:20:39.835 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:39.803+0000 7f24debd6640 1 -- 172.21.15.195:0/3991310212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24d8100df0 con 0x7f24d8102790 2024-09-12T03:20:39.835 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:39.804+0000 7f24debd6640 1 -- 172.21.15.195:0/3991310212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24d8069f30 con 0x7f24d8102790 2024-09-12T03:20:39.835 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:39.804+0000 7f24d5ffb640 1 -- 172.21.15.195:0/3991310212 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f24cc039040 con 0x7f24d8102790 2024-09-12T03:20:39.835 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:39.804+0000 7f24d5ffb640 1 --2- 172.21.15.195:0/3991310212 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f24c8039060 0x7f24c803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:39.835 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:39.804+0000 7f24d5ffb640 1 -- 172.21.15.195:0/3991310212 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f24cc06d650 con 0x7f24d8102790 2024-09-12T03:20:39.836 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:39.804+0000 7f24d7fff640 1 --2- 172.21.15.195:0/3991310212 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f24c8039060 0x7f24c803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:39.836 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:39.805+0000 7f24d7fff640 1 --2- 172.21.15.195:0/3991310212 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f24c8039060 0x7f24c803b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f24c40098c0 tx=0x7f24c4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:39.836 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:39.809+0000 7f24d5ffb640 1 -- 172.21.15.195:0/3991310212 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f24cc0443c0 con 0x7f24d8102790 2024-09-12T03:20:40.061 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:40.061+0000 7f24debd6640 1 -- 172.21.15.195:0/3991310212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f24d8101920 con 0x7f24d8102790 2024-09-12T03:20:40.062 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:40.062+0000 7f24d5ffb640 1 -- 172.21.15.195:0/3991310212 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f24cc03a570 con 0x7f24d8102790 2024-09-12T03:20:40.062 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-12T03:20:40.062 INFO:teuthology.orchestra.run.smithi195.stdout:{"epoch":1,"fsid":"b0812460-70b5-11ef-bcea-c7b262605968","modified":"2024-09-12T03:19:13.042174Z","created":"2024-09-12T03:19:13.042174Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-12T03:20:40.062 INFO:teuthology.orchestra.run.smithi195.stderr:dumped monmap epoch 1 2024-09-12T03:20:40.064 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:40.064+0000 7f24debd6640 1 -- 172.21.15.195:0/3991310212 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f24c8039060 msgr2=0x7f24c803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:40.064 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:40.064+0000 7f24debd6640 1 --2- 172.21.15.195:0/3991310212 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f24c8039060 0x7f24c803b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f24c40098c0 tx=0x7f24c4006db0 comp rx=0 tx=0).stop 2024-09-12T03:20:40.065 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:40.064+0000 7f24debd6640 1 -- 172.21.15.195:0/3991310212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24d8102790 msgr2=0x7f24d81003f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:40.065 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:40.064+0000 7f24debd6640 1 --2- 172.21.15.195:0/3991310212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24d8102790 0x7f24d81003f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f24cc030b00 tx=0x7f24cc030b30 comp rx=0 tx=0).stop 2024-09-12T03:20:40.065 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:40.064+0000 7f24debd6640 1 -- 172.21.15.195:0/3991310212 shutdown_connections 2024-09-12T03:20:40.065 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:40.064+0000 7f24debd6640 1 --2- 172.21.15.195:0/3991310212 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f24c8039060 0x7f24c803b520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:40.065 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:40.064+0000 7f24debd6640 1 --2- 172.21.15.195:0/3991310212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24d8102790 0x7f24d81003f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:40.065 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:40.064+0000 7f24debd6640 1 -- 172.21.15.195:0/3991310212 >> 172.21.15.195:0/3991310212 conn(0x7f24d8077310 msgr2=0x7f24d8105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:40.065 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:40.064+0000 7f24debd6640 1 -- 172.21.15.195:0/3991310212 shutdown_connections 2024-09-12T03:20:40.065 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:40.064+0000 7f24debd6640 1 -- 172.21.15.195:0/3991310212 wait complete. 2024-09-12T03:20:40.380 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:40 smithi195 systemd[1]: Starting Ceph mon.c for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:20:40.983 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:40 smithi195 podman[34767]: 2024-09-12 03:20:40.839689954 +0000 UTC m=+0.100072239 container create 7ddb2cba258e6b2e2653a6039f4e1d3e2013edd2e5b8b0d55d89725396940207 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mon-c, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True) 2024-09-12T03:20:40.983 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:40 smithi195 podman[34767]: 2024-09-12 03:20:40.75561918 +0000 UTC m=+0.016001432 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:20:40.983 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:40 smithi195 podman[34767]: 2024-09-12 03:20:40.98380063 +0000 UTC m=+0.244182871 container init 7ddb2cba258e6b2e2653a6039f4e1d3e2013edd2e5b8b0d55d89725396940207 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mon-c, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:20:41.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:40 smithi195 podman[34767]: 2024-09-12 03:20:40.987924681 +0000 UTC m=+0.248306922 container start 7ddb2cba258e6b2e2653a6039f4e1d3e2013edd2e5b8b0d55d89725396940207 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mon-c, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True) 2024-09-12T03:20:41.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: set uid:gid to 167:167 (ceph:ceph) 2024-09-12T03:20:41.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable), process ceph-mon, pid 2 2024-09-12T03:20:41.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: pidfile_write: ignore empty --pid-file 2024-09-12T03:20:41.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: load: jerasure load: lrc 2024-09-12T03:20:41.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: RocksDB version: 7.9.2 2024-09-12T03:20:41.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Git sha 0 2024-09-12T03:20:41.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Compile date 2024-09-09 16:23:28 2024-09-12T03:20:41.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: DB SUMMARY 2024-09-12T03:20:41.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: DB Session ID: OIOD79PYOMUG5LZNEKXN 2024-09-12T03:20:41.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: CURRENT file: CURRENT 2024-09-12T03:20:41.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: IDENTITY file: IDENTITY 2024-09-12T03:20:41.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-12T03:20:41.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-12T03:20:41.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-12T03:20:41.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.error_if_exists: 0 2024-09-12T03:20:41.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.create_if_missing: 0 2024-09-12T03:20:41.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.paranoid_checks: 1 2024-09-12T03:20:41.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-12T03:20:41.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-12T03:20:41.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-12T03:20:41.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.env: 0x55b721a59940 2024-09-12T03:20:41.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.fs: PosixFileSystem 2024-09-12T03:20:41.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.info_log: 0x55b722a256e0 2024-09-12T03:20:41.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.max_file_opening_threads: 16 2024-09-12T03:20:41.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.statistics: (nil) 2024-09-12T03:20:41.459 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.use_fsync: 0 2024-09-12T03:20:41.459 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.max_log_file_size: 0 2024-09-12T03:20:41.459 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-12T03:20:41.459 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-12T03:20:41.459 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.keep_log_file_num: 1000 2024-09-12T03:20:41.459 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.recycle_log_file_num: 0 2024-09-12T03:20:41.459 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.allow_fallocate: 1 2024-09-12T03:20:41.460 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.allow_mmap_reads: 0 2024-09-12T03:20:41.461 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.allow_mmap_writes: 0 2024-09-12T03:20:41.461 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.use_direct_reads: 0 2024-09-12T03:20:41.461 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-12T03:20:41.461 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.create_missing_column_families: 0 2024-09-12T03:20:41.461 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.db_log_dir: 2024-09-12T03:20:41.461 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.wal_dir: 2024-09-12T03:20:41.461 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-12T03:20:41.461 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-12T03:20:41.461 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-12T03:20:41.461 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-12T03:20:41.461 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-12T03:20:41.462 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-12T03:20:41.462 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.advise_random_on_open: 1 2024-09-12T03:20:41.462 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.db_write_buffer_size: 0 2024-09-12T03:20:41.462 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.write_buffer_manager: 0x55b722a34dc0 2024-09-12T03:20:41.462 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-12T03:20:41.462 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-12T03:20:41.462 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-12T03:20:41.462 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.rate_limiter: (nil) 2024-09-12T03:20:41.462 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-12T03:20:41.462 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.wal_recovery_mode: 2 2024-09-12T03:20:41.462 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.enable_thread_tracking: 0 2024-09-12T03:20:41.462 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.enable_pipelined_write: 0 2024-09-12T03:20:41.462 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.unordered_write: 0 2024-09-12T03:20:41.462 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-12T03:20:41.463 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-12T03:20:41.463 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-12T03:20:41.463 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-12T03:20:41.463 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.row_cache: None 2024-09-12T03:20:41.463 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.wal_filter: None 2024-09-12T03:20:41.463 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-12T03:20:41.463 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.allow_ingest_behind: 0 2024-09-12T03:20:41.463 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.two_write_queues: 0 2024-09-12T03:20:41.463 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.manual_wal_flush: 0 2024-09-12T03:20:41.463 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.wal_compression: 0 2024-09-12T03:20:41.463 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.atomic_flush: 0 2024-09-12T03:20:41.463 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-12T03:20:41.464 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-12T03:20:41.464 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-12T03:20:41.464 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.log_readahead_size: 0 2024-09-12T03:20:41.464 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-12T03:20:41.464 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.best_efforts_recovery: 0 2024-09-12T03:20:41.464 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-12T03:20:41.464 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-12T03:20:41.464 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.allow_data_in_errors: 0 2024-09-12T03:20:41.464 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.db_host_id: __hostname__ 2024-09-12T03:20:41.464 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-12T03:20:41.464 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.max_background_jobs: 2 2024-09-12T03:20:41.464 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.max_background_compactions: -1 2024-09-12T03:20:41.464 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.max_subcompactions: 1 2024-09-12T03:20:41.464 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-12T03:20:41.465 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-12T03:20:41.465 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-12T03:20:41.465 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.max_total_wal_size: 0 2024-09-12T03:20:41.465 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-12T03:20:41.465 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-12T03:20:41.465 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-12T03:20:41.465 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-12T03:20:41.465 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.max_open_files: -1 2024-09-12T03:20:41.465 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.bytes_per_sync: 0 2024-09-12T03:20:41.465 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-12T03:20:41.465 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-12T03:20:41.465 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.compaction_readahead_size: 0 2024-09-12T03:20:41.465 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.max_background_flushes: -1 2024-09-12T03:20:41.466 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Compression algorithms supported: 2024-09-12T03:20:41.466 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: kZSTD supported: 0 2024-09-12T03:20:41.466 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: kXpressCompression supported: 0 2024-09-12T03:20:41.466 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: kBZip2Compression supported: 0 2024-09-12T03:20:41.466 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-12T03:20:41.466 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: kLZ4Compression supported: 1 2024-09-12T03:20:41.466 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: kZlibCompression supported: 1 2024-09-12T03:20:41.466 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: kLZ4HCCompression supported: 1 2024-09-12T03:20:41.466 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: kSnappyCompression supported: 1 2024-09-12T03:20:41.466 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-12T03:20:41.466 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-12T03:20:41.466 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-12T03:20:41.466 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-12T03:20:41.466 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-12T03:20:41.467 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.merge_operator: 2024-09-12T03:20:41.467 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.compaction_filter: None 2024-09-12T03:20:41.467 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.compaction_filter_factory: None 2024-09-12T03:20:41.467 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.sst_partitioner_factory: None 2024-09-12T03:20:41.467 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-12T03:20:41.467 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-12T03:20:41.467 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b722a25320) 2024-09-12T03:20:41.467 INFO:journalctl@ceph.mon.c.smithi195.stdout: cache_index_and_filter_blocks: 1 2024-09-12T03:20:41.467 INFO:journalctl@ceph.mon.c.smithi195.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-12T03:20:41.467 INFO:journalctl@ceph.mon.c.smithi195.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-12T03:20:41.467 INFO:journalctl@ceph.mon.c.smithi195.stdout: pin_top_level_index_and_filter: 1 2024-09-12T03:20:41.467 INFO:journalctl@ceph.mon.c.smithi195.stdout: index_type: 0 2024-09-12T03:20:41.467 INFO:journalctl@ceph.mon.c.smithi195.stdout: data_block_index_type: 0 2024-09-12T03:20:41.467 INFO:journalctl@ceph.mon.c.smithi195.stdout: index_shortening: 1 2024-09-12T03:20:41.468 INFO:journalctl@ceph.mon.c.smithi195.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-12T03:20:41.468 INFO:journalctl@ceph.mon.c.smithi195.stdout: checksum: 4 2024-09-12T03:20:41.468 INFO:journalctl@ceph.mon.c.smithi195.stdout: no_block_cache: 0 2024-09-12T03:20:41.468 INFO:journalctl@ceph.mon.c.smithi195.stdout: block_cache: 0x55b722a191f0 2024-09-12T03:20:41.468 INFO:journalctl@ceph.mon.c.smithi195.stdout: block_cache_name: BinnedLRUCache 2024-09-12T03:20:41.468 INFO:journalctl@ceph.mon.c.smithi195.stdout: block_cache_options: 2024-09-12T03:20:41.468 INFO:journalctl@ceph.mon.c.smithi195.stdout: capacity : 536870912 2024-09-12T03:20:41.468 INFO:journalctl@ceph.mon.c.smithi195.stdout: num_shard_bits : 4 2024-09-12T03:20:41.468 INFO:journalctl@ceph.mon.c.smithi195.stdout: strict_capacity_limit : 0 2024-09-12T03:20:41.468 INFO:journalctl@ceph.mon.c.smithi195.stdout: high_pri_pool_ratio: 0.000 2024-09-12T03:20:41.468 INFO:journalctl@ceph.mon.c.smithi195.stdout: block_cache_compressed: (nil) 2024-09-12T03:20:41.468 INFO:journalctl@ceph.mon.c.smithi195.stdout: persistent_cache: (nil) 2024-09-12T03:20:41.468 INFO:journalctl@ceph.mon.c.smithi195.stdout: block_size: 4096 2024-09-12T03:20:41.468 INFO:journalctl@ceph.mon.c.smithi195.stdout: block_size_deviation: 10 2024-09-12T03:20:41.469 INFO:journalctl@ceph.mon.c.smithi195.stdout: block_restart_interval: 16 2024-09-12T03:20:41.469 INFO:journalctl@ceph.mon.c.smithi195.stdout: index_block_restart_interval: 1 2024-09-12T03:20:41.469 INFO:journalctl@ceph.mon.c.smithi195.stdout: metadata_block_size: 4096 2024-09-12T03:20:41.469 INFO:journalctl@ceph.mon.c.smithi195.stdout: partition_filters: 0 2024-09-12T03:20:41.469 INFO:journalctl@ceph.mon.c.smithi195.stdout: use_delta_encoding: 1 2024-09-12T03:20:41.469 INFO:journalctl@ceph.mon.c.smithi195.stdout: filter_policy: bloomfilter 2024-09-12T03:20:41.469 INFO:journalctl@ceph.mon.c.smithi195.stdout: whole_key_filtering: 1 2024-09-12T03:20:41.469 INFO:journalctl@ceph.mon.c.smithi195.stdout: verify_compression: 0 2024-09-12T03:20:41.469 INFO:journalctl@ceph.mon.c.smithi195.stdout: read_amp_bytes_per_bit: 0 2024-09-12T03:20:41.469 INFO:journalctl@ceph.mon.c.smithi195.stdout: format_version: 5 2024-09-12T03:20:41.469 INFO:journalctl@ceph.mon.c.smithi195.stdout: enable_index_compression: 1 2024-09-12T03:20:41.469 INFO:journalctl@ceph.mon.c.smithi195.stdout: block_align: 0 2024-09-12T03:20:41.469 INFO:journalctl@ceph.mon.c.smithi195.stdout: max_auto_readahead_size: 262144 2024-09-12T03:20:41.469 INFO:journalctl@ceph.mon.c.smithi195.stdout: prepopulate_block_cache: 0 2024-09-12T03:20:41.470 INFO:journalctl@ceph.mon.c.smithi195.stdout: initial_auto_readahead_size: 8192 2024-09-12T03:20:41.470 INFO:journalctl@ceph.mon.c.smithi195.stdout: num_file_reads_for_auto_readahead: 2 2024-09-12T03:20:41.470 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.write_buffer_size: 33554432 2024-09-12T03:20:41.470 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.max_write_buffer_number: 2 2024-09-12T03:20:41.470 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.compression: NoCompression 2024-09-12T03:20:41.470 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.bottommost_compression: Disabled 2024-09-12T03:20:41.470 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.prefix_extractor: nullptr 2024-09-12T03:20:41.470 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-12T03:20:41.470 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.num_levels: 7 2024-09-12T03:20:41.471 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-12T03:20:41.471 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-12T03:20:41.471 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-12T03:20:41.471 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-12T03:20:41.471 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-12T03:20:41.471 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-12T03:20:41.471 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-12T03:20:41.471 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-12T03:20:41.471 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-12T03:20:41.471 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-12T03:20:41.471 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-12T03:20:41.471 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-12T03:20:41.471 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-12T03:20:41.471 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.compression_opts.level: 32767 2024-09-12T03:20:41.472 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.compression_opts.strategy: 0 2024-09-12T03:20:41.472 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-12T03:20:41.472 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-12T03:20:41.472 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-12T03:20:41.472 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-12T03:20:41.472 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.compression_opts.enabled: false 2024-09-12T03:20:41.472 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-12T03:20:41.472 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-12T03:20:41.472 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-12T03:20:41.472 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-12T03:20:41.472 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.target_file_size_base: 67108864 2024-09-12T03:20:41.472 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-12T03:20:41.472 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-12T03:20:41.473 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-12T03:20:41.473 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-12T03:20:41.473 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-12T03:20:41.473 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-12T03:20:41.473 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-12T03:20:41.473 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-12T03:20:41.473 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-12T03:20:41.473 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-12T03:20:41.473 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-12T03:20:41.473 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-12T03:20:41.473 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-12T03:20:41.473 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-12T03:20:41.473 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.arena_block_size: 1048576 2024-09-12T03:20:41.473 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-12T03:20:41.474 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-12T03:20:41.474 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.disable_auto_compactions: 0 2024-09-12T03:20:41.474 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-12T03:20:41.474 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-12T03:20:41.474 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-12T03:20:41.474 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-12T03:20:41.474 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-12T03:20:41.474 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-12T03:20:41.474 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-12T03:20:41.474 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-12T03:20:41.474 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-12T03:20:41.474 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-12T03:20:41.474 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-12T03:20:41.475 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.inplace_update_support: 0 2024-09-12T03:20:41.475 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-12T03:20:41.475 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-12T03:20:41.475 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-12T03:20:41.475 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-12T03:20:41.475 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.bloom_locality: 0 2024-09-12T03:20:41.475 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.max_successive_merges: 0 2024-09-12T03:20:41.475 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-12T03:20:41.475 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.paranoid_file_checks: 0 2024-09-12T03:20:41.475 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.force_consistency_checks: 1 2024-09-12T03:20:41.475 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.report_bg_io_stats: 0 2024-09-12T03:20:41.475 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.ttl: 2592000 2024-09-12T03:20:41.476 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-12T03:20:41.476 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-12T03:20:41.476 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-12T03:20:41.476 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.enable_blob_files: false 2024-09-12T03:20:41.476 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.min_blob_size: 0 2024-09-12T03:20:41.476 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.blob_file_size: 268435456 2024-09-12T03:20:41.476 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-12T03:20:41.476 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-12T03:20:41.476 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-12T03:20:41.476 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-12T03:20:41.476 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-12T03:20:41.476 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.blob_file_starting_level: 0 2024-09-12T03:20:41.477 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-12T03:20:41.477 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-12T03:20:41.477 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-12T03:20:41.477 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 6ab1a920-3f73-4d7d-83c8-511d958ce515 2024-09-12T03:20:41.477 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726111241015664, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-12T03:20:41.477 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-12T03:20:41.477 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726111241016224, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726111241, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "6ab1a920-3f73-4d7d-83c8-511d958ce515", "db_session_id": "OIOD79PYOMUG5LZNEKXN", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-12T03:20:41.477 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726111241016350, "job": 1, "event": "recovery_finished"} 2024-09-12T03:20:41.477 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-12T03:20:41.477 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-12T03:20:41.477 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55b722a46e00 2024-09-12T03:20:41.477 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: DB pointer 0x55b722b42000 2024-09-12T03:20:41.477 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-12T03:20:41.477 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-12T03:20:41.478 INFO:journalctl@ceph.mon.c.smithi195.stdout: ** DB Stats ** 2024-09-12T03:20:41.478 INFO:journalctl@ceph.mon.c.smithi195.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T03:20:41.478 INFO:journalctl@ceph.mon.c.smithi195.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-12T03:20:41.478 INFO:journalctl@ceph.mon.c.smithi195.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T03:20:41.478 INFO:journalctl@ceph.mon.c.smithi195.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T03:20:41.478 INFO:journalctl@ceph.mon.c.smithi195.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-12T03:20:41.478 INFO:journalctl@ceph.mon.c.smithi195.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T03:20:41.478 INFO:journalctl@ceph.mon.c.smithi195.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T03:20:41.478 INFO:journalctl@ceph.mon.c.smithi195.stdout: 2024-09-12T03:20:41.478 INFO:journalctl@ceph.mon.c.smithi195.stdout: ** Compaction Stats [default] ** 2024-09-12T03:20:41.479 INFO:journalctl@ceph.mon.c.smithi195.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T03:20:41.479 INFO:journalctl@ceph.mon.c.smithi195.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-12T03:20:41.479 INFO:journalctl@ceph.mon.c.smithi195.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T03:20:41.479 INFO:journalctl@ceph.mon.c.smithi195.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T03:20:41.479 INFO:journalctl@ceph.mon.c.smithi195.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T03:20:41.479 INFO:journalctl@ceph.mon.c.smithi195.stdout: 2024-09-12T03:20:41.479 INFO:journalctl@ceph.mon.c.smithi195.stdout: ** Compaction Stats [default] ** 2024-09-12T03:20:41.479 INFO:journalctl@ceph.mon.c.smithi195.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T03:20:41.479 INFO:journalctl@ceph.mon.c.smithi195.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-12T03:20:41.479 INFO:journalctl@ceph.mon.c.smithi195.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T03:20:41.479 INFO:journalctl@ceph.mon.c.smithi195.stdout: 2024-09-12T03:20:41.479 INFO:journalctl@ceph.mon.c.smithi195.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-12T03:20:41.479 INFO:journalctl@ceph.mon.c.smithi195.stdout: 2024-09-12T03:20:41.480 INFO:journalctl@ceph.mon.c.smithi195.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T03:20:41.480 INFO:journalctl@ceph.mon.c.smithi195.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-12T03:20:41.480 INFO:journalctl@ceph.mon.c.smithi195.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-12T03:20:41.480 INFO:journalctl@ceph.mon.c.smithi195.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-12T03:20:41.480 INFO:journalctl@ceph.mon.c.smithi195.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-12T03:20:41.480 INFO:journalctl@ceph.mon.c.smithi195.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-12T03:20:41.480 INFO:journalctl@ceph.mon.c.smithi195.stdout: Cumulative compaction: 0.00 GB write, 0.43 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T03:20:41.480 INFO:journalctl@ceph.mon.c.smithi195.stdout: Interval compaction: 0.00 GB write, 0.43 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T03:20:41.480 INFO:journalctl@ceph.mon.c.smithi195.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-12T03:20:41.480 INFO:journalctl@ceph.mon.c.smithi195.stdout: Block cache BinnedLRUCache@0x55b722a191f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-12T03:20:41.480 INFO:journalctl@ceph.mon.c.smithi195.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-12T03:20:41.480 INFO:journalctl@ceph.mon.c.smithi195.stdout: 2024-09-12T03:20:41.481 INFO:journalctl@ceph.mon.c.smithi195.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-12T03:20:41.481 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-12T03:20:41.481 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: using public_addr v2:172.21.15.195:0/0 -> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-09-12T03:20:41.481 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: starting mon.c rank -1 at public addrs [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] at bind addrs [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid b0812460-70b5-11ef-bcea-c7b262605968 2024-09-12T03:20:41.481 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: mon.c@-1(???) e0 preinit fsid b0812460-70b5-11ef-bcea-c7b262605968 2024-09-12T03:20:41.481 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: mon.c@-1(synchronizing).mds e1 new map 2024-09-12T03:20:41.481 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-12T03:20:41.481 INFO:journalctl@ceph.mon.c.smithi195.stdout: e1 2024-09-12T03:20:41.481 INFO:journalctl@ceph.mon.c.smithi195.stdout: btime 2024-09-12T03:19:15:900094+0000 2024-09-12T03:20:41.481 INFO:journalctl@ceph.mon.c.smithi195.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T03:20:41.481 INFO:journalctl@ceph.mon.c.smithi195.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T03:20:41.481 INFO:journalctl@ceph.mon.c.smithi195.stdout: legacy client fscid: -1 2024-09-12T03:20:41.481 INFO:journalctl@ceph.mon.c.smithi195.stdout: 2024-09-12T03:20:41.482 INFO:journalctl@ceph.mon.c.smithi195.stdout: No filesystems configured 2024-09-12T03:20:41.482 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-12T03:20:41.482 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-12T03:20:41.482 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-12T03:20:41.482 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-12T03:20:41.482 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-12T03:20:41.482 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-12T03:20:41.482 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-12T03:20:41.482 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T03:20:41.482 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T03:20:41.482 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T03:20:41.482 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:41.482 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:20:41.482 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:20:41.483 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/444365947' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-12T03:20:41.483 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/444365947' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-12T03:20:41.483 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T03:20:41.483 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:41.483 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:41.483 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:41.483 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:41.483 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi195", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:20:41.483 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:41.483 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:20:41.483 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:20:41.483 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: Updating smithi195:/etc/ceph/ceph.conf 2024-09-12T03:20:41.483 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: Updating smithi195:/var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:20:41.484 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: Updating smithi195:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:20:41.484 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:41.484 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:41.484 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:41.484 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:41.484 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:20:41.484 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:41.484 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: Updating smithi195:/var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:20:41.484 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi064:172.21.15.64=a;smithi115:172.21.15.115=b;smithi195:172.21.15.195=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:20:41.484 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: Saving service mon spec with placement smithi064:172.21.15.64=a;smithi115:172.21.15.115=b;smithi195:172.21.15.195=c;count:3 2024-09-12T03:20:41.484 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: Deploying daemon mon.c on smithi195 2024-09-12T03:20:41.484 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:20:41.484 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 ceph-mon[34786]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-12T03:20:41.485 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 bash[34767]: 7ddb2cba258e6b2e2653a6039f4e1d3e2013edd2e5b8b0d55d89725396940207 2024-09-12T03:20:41.485 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:41 smithi195 systemd[1]: Started Ceph mon.c for b0812460-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:20:41.626 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T03:20:41.626 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T03:20:41.872 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:20:46.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:46 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:20:46.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:46 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:20:46.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:46 smithi195 ceph-mon[34786]: mon.a calling monitor election 2024-09-12T03:20:46.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:46 smithi195 ceph-mon[34786]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:20:46.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:46 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:20:46.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:46 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:20:46.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:46 smithi195 ceph-mon[34786]: mon.c calling monitor election 2024-09-12T03:20:46.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:46 smithi195 ceph-mon[34786]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:20:46.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:46 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:20:46.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:46 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:20:46.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:46 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:20:46.459 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:46 smithi195 ceph-mon[34786]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T03:20:46.459 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:46 smithi195 ceph-mon[34786]: monmap epoch 2 2024-09-12T03:20:46.459 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:46 smithi195 ceph-mon[34786]: fsid b0812460-70b5-11ef-bcea-c7b262605968 2024-09-12T03:20:46.459 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:46 smithi195 ceph-mon[34786]: last_changed 2024-09-12T03:20:41.042383+0000 2024-09-12T03:20:46.459 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:46 smithi195 ceph-mon[34786]: created 2024-09-12T03:19:13.042174+0000 2024-09-12T03:20:46.459 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:46 smithi195 ceph-mon[34786]: min_mon_release 19 (squid) 2024-09-12T03:20:46.459 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:46 smithi195 ceph-mon[34786]: election_strategy: 1 2024-09-12T03:20:46.459 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:46 smithi195 ceph-mon[34786]: 0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2024-09-12T03:20:46.460 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:46 smithi195 ceph-mon[34786]: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.c 2024-09-12T03:20:46.460 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:46 smithi195 ceph-mon[34786]: fsmap 2024-09-12T03:20:46.460 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:46 smithi195 ceph-mon[34786]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T03:20:46.460 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:46 smithi195 ceph-mon[34786]: mgrmap e13: x(active, since 38s) 2024-09-12T03:20:46.460 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:46 smithi195 ceph-mon[34786]: overall HEALTH_OK 2024-09-12T03:20:46.460 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:46 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:46.460 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:46 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:46.460 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:46 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:46.461 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:46 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:20:46.461 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:46 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:46.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:46 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:20:46.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:46 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:20:46.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:46 smithi064 ceph-mon[30952]: mon.a calling monitor election 2024-09-12T03:20:46.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:46 smithi064 ceph-mon[30952]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:20:46.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:46 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:20:46.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:46 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:20:46.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:46 smithi064 ceph-mon[30952]: mon.c calling monitor election 2024-09-12T03:20:46.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:46 smithi064 ceph-mon[30952]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:20:46.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:46 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:20:46.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:46 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:20:46.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:46 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:20:46.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:46 smithi064 ceph-mon[30952]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T03:20:46.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:46 smithi064 ceph-mon[30952]: monmap epoch 2 2024-09-12T03:20:46.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:46 smithi064 ceph-mon[30952]: fsid b0812460-70b5-11ef-bcea-c7b262605968 2024-09-12T03:20:46.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:46 smithi064 ceph-mon[30952]: last_changed 2024-09-12T03:20:41.042383+0000 2024-09-12T03:20:46.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:46 smithi064 ceph-mon[30952]: created 2024-09-12T03:19:13.042174+0000 2024-09-12T03:20:46.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:46 smithi064 ceph-mon[30952]: min_mon_release 19 (squid) 2024-09-12T03:20:46.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:46 smithi064 ceph-mon[30952]: election_strategy: 1 2024-09-12T03:20:46.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:46 smithi064 ceph-mon[30952]: 0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2024-09-12T03:20:46.525 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:46 smithi064 ceph-mon[30952]: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.c 2024-09-12T03:20:46.525 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:46 smithi064 ceph-mon[30952]: fsmap 2024-09-12T03:20:46.525 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:46 smithi064 ceph-mon[30952]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T03:20:46.525 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:46 smithi064 ceph-mon[30952]: mgrmap e13: x(active, since 38s) 2024-09-12T03:20:46.525 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:46 smithi064 ceph-mon[30952]: overall HEALTH_OK 2024-09-12T03:20:46.525 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:46 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:46.525 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:46 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:46.525 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:46 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:46.525 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:46 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:20:46.525 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:46 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:46.706 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.704+0000 7fab3079c640 1 -- 172.21.15.195:0/4113146705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab14003530 msgr2=0x7fab140059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:46.707 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.704+0000 7fab3079c640 1 --2- 172.21.15.195:0/4113146705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab14003530 0x7fab140059c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fab281016c0 tx=0x7fab2402c490 comp rx=0 tx=0).stop 2024-09-12T03:20:46.707 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.705+0000 7fab3079c640 1 -- 172.21.15.195:0/4113146705 shutdown_connections 2024-09-12T03:20:46.707 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.705+0000 7fab3079c640 1 --2- 172.21.15.195:0/4113146705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab14003530 0x7fab140059c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.707 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.705+0000 7fab3079c640 1 -- 172.21.15.195:0/4113146705 >> 172.21.15.195:0/4113146705 conn(0x7fab280fc280 msgr2=0x7fab280fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:46.707 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.705+0000 7fab3079c640 1 -- 172.21.15.195:0/4113146705 shutdown_connections 2024-09-12T03:20:46.707 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.705+0000 7fab3079c640 1 -- 172.21.15.195:0/4113146705 wait complete. 2024-09-12T03:20:46.707 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.705+0000 7fab3079c640 1 Processor -- start 2024-09-12T03:20:46.707 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.706+0000 7fab3079c640 1 -- start start 2024-09-12T03:20:46.707 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.706+0000 7fab3079c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab14003530 0x7fab28198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:46.707 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.706+0000 7fab3079c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab281985b0 0x7fab28198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:46.707 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.706+0000 7fab3079c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fab28108540 con 0x7fab281985b0 2024-09-12T03:20:46.708 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.706+0000 7fab3079c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fab281083c0 con 0x7fab14003530 2024-09-12T03:20:46.708 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.706+0000 7fab2e538640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab14003530 0x7fab28198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:46.708 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.706+0000 7fab2e538640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab14003530 0x7fab28198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:33250/0 (socket says 172.21.15.195:33250) 2024-09-12T03:20:46.708 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.706+0000 7fab2e538640 1 -- 172.21.15.195:0/2099652820 learned_addr learned my addr 172.21.15.195:0/2099652820 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-09-12T03:20:46.708 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.706+0000 7fab2e538640 1 -- 172.21.15.195:0/2099652820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab14003530 msgr2=0x7fab28198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-12T03:20:46.708 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.706+0000 7fab2e538640 1 -- 172.21.15.195:0/2099652820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab14003530 msgr2=0x7fab28198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T03:20:46.708 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.706+0000 7fab2e538640 1 --2- 172.21.15.195:0/2099652820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab14003530 0x7fab28198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T03:20:46.708 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.706+0000 7fab2dd37640 1 --2- 172.21.15.195:0/2099652820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab281985b0 0x7fab28198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:46.708 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.706+0000 7fab2e538640 1 --2- 172.21.15.195:0/2099652820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab14003530 0x7fab28198070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T03:20:46.709 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.707+0000 7fab2dd37640 1 -- 172.21.15.195:0/2099652820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab14003530 msgr2=0x7fab28198070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:20:46.709 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.707+0000 7fab2dd37640 1 --2- 172.21.15.195:0/2099652820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab14003530 0x7fab28198070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.709 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.707+0000 7fab2dd37640 1 -- 172.21.15.195:0/2099652820 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab281a10e0 con 0x7fab281985b0 2024-09-12T03:20:46.710 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.707+0000 7fab2dd37640 1 --2- 172.21.15.195:0/2099652820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab281985b0 0x7fab28198a10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fab1800b400 tx=0x7fab1800b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:46.710 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.707+0000 7fab1f7fe640 1 -- 172.21.15.195:0/2099652820 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fab18011630 con 0x7fab281985b0 2024-09-12T03:20:46.710 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.707+0000 7fab1f7fe640 1 -- 172.21.15.195:0/2099652820 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab18011c80 con 0x7fab281985b0 2024-09-12T03:20:46.711 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.707+0000 7fab3079c640 1 -- 172.21.15.195:0/2099652820 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab281a12b0 con 0x7fab281985b0 2024-09-12T03:20:46.711 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.707+0000 7fab1f7fe640 1 -- 172.21.15.195:0/2099652820 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fab18010d10 con 0x7fab281985b0 2024-09-12T03:20:46.711 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.707+0000 7fab3079c640 1 -- 172.21.15.195:0/2099652820 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab281a1810 con 0x7fab281985b0 2024-09-12T03:20:46.711 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.708+0000 7fab1f7fe640 1 -- 172.21.15.195:0/2099652820 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fab18018020 con 0x7fab281985b0 2024-09-12T03:20:46.711 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.708+0000 7fab1f7fe640 1 --2- 172.21.15.195:0/2099652820 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fab0c038e00 0x7fab0c03b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:46.711 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.708+0000 7fab1f7fe640 1 -- 172.21.15.195:0/2099652820 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fab1804cc50 con 0x7fab281985b0 2024-09-12T03:20:46.711 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.709+0000 7fab3079c640 1 -- 172.21.15.195:0/2099652820 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab28101cc0 con 0x7fab281985b0 2024-09-12T03:20:46.711 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.709+0000 7fab2e538640 1 --2- 172.21.15.195:0/2099652820 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fab0c038e00 0x7fab0c03b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:46.711 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.709+0000 7fab2e538640 1 --2- 172.21.15.195:0/2099652820 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fab0c038e00 0x7fab0c03b2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fab24012c10 tx=0x7fab240323a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:46.714 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.714+0000 7fab1f7fe640 1 -- 172.21.15.195:0/2099652820 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fab1801f020 con 0x7fab281985b0 2024-09-12T03:20:46.947 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.946+0000 7fab3079c640 1 -- 172.21.15.195:0/2099652820 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fab2818fcd0 con 0x7fab281985b0 2024-09-12T03:20:46.947 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.947+0000 7fab1f7fe640 1 -- 172.21.15.195:0/2099652820 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fab18018310 con 0x7fab281985b0 2024-09-12T03:20:46.948 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-12T03:20:46.948 INFO:teuthology.orchestra.run.smithi195.stdout:{"epoch":2,"fsid":"b0812460-70b5-11ef-bcea-c7b262605968","modified":"2024-09-12T03:20:41.042383Z","created":"2024-09-12T03:19:13.042174Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3300","nonce":0},{"type":"v1","addr":"172.21.15.195:6789","nonce":0}]},"addr":"172.21.15.195:6789/0","public_addr":"172.21.15.195:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-12T03:20:46.948 INFO:teuthology.orchestra.run.smithi195.stderr:dumped monmap epoch 2 2024-09-12T03:20:46.949 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.949+0000 7fab3079c640 1 -- 172.21.15.195:0/2099652820 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fab0c038e00 msgr2=0x7fab0c03b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:46.950 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.949+0000 7fab3079c640 1 --2- 172.21.15.195:0/2099652820 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fab0c038e00 0x7fab0c03b2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fab24012c10 tx=0x7fab240323a0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.950 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.949+0000 7fab3079c640 1 -- 172.21.15.195:0/2099652820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab281985b0 msgr2=0x7fab28198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:46.950 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.949+0000 7fab3079c640 1 --2- 172.21.15.195:0/2099652820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab281985b0 0x7fab28198a10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fab1800b400 tx=0x7fab1800b8c0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.950 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.949+0000 7fab3079c640 1 -- 172.21.15.195:0/2099652820 shutdown_connections 2024-09-12T03:20:46.950 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.949+0000 7fab3079c640 1 --2- 172.21.15.195:0/2099652820 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fab0c038e00 0x7fab0c03b2c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.950 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.949+0000 7fab3079c640 1 --2- 172.21.15.195:0/2099652820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab281985b0 0x7fab28198a10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.950 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.949+0000 7fab3079c640 1 --2- 172.21.15.195:0/2099652820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab14003530 0x7fab28198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.950 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.949+0000 7fab3079c640 1 -- 172.21.15.195:0/2099652820 >> 172.21.15.195:0/2099652820 conn(0x7fab280fc280 msgr2=0x7fab280fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:46.951 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.949+0000 7fab3079c640 1 -- 172.21.15.195:0/2099652820 shutdown_connections 2024-09-12T03:20:46.951 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:46.949+0000 7fab3079c640 1 -- 172.21.15.195:0/2099652820 wait complete. 2024-09-12T03:20:47.297 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:47 smithi195 ceph-mon[34786]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:20:47.297 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:47 smithi195 ceph-mon[34786]: Deploying daemon mon.b on smithi115 2024-09-12T03:20:47.297 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:47 smithi195 ceph-mon[34786]: from='client.? 172.21.15.195:0/2099652820' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T03:20:47.298 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:47 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:20:47.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:47 smithi064 ceph-mon[30952]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:20:47.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:47 smithi064 ceph-mon[30952]: Deploying daemon mon.b on smithi115 2024-09-12T03:20:47.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:47 smithi064 ceph-mon[30952]: from='client.? 172.21.15.195:0/2099652820' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T03:20:47.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:47 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:20:48.552 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T03:20:48.552 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T03:20:48.804 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:20:49.202 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:49 smithi195 ceph-mon[34786]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:20:49.207 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.206+0000 7f1962bcb640 1 -- 172.21.15.195:0/2587372817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f195c107670 msgr2=0x7f195c107a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:49.207 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.206+0000 7f1962bcb640 1 --2- 172.21.15.195:0/2587372817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f195c107670 0x7f195c107a50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1950009880 tx=0x7f195002a5d0 comp rx=0 tx=0).stop 2024-09-12T03:20:49.207 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.207+0000 7f1962bcb640 1 -- 172.21.15.195:0/2587372817 shutdown_connections 2024-09-12T03:20:49.207 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.207+0000 7f1962bcb640 1 --2- 172.21.15.195:0/2587372817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f195c107670 0x7f195c107a50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:49.207 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.207+0000 7f1962bcb640 1 -- 172.21.15.195:0/2587372817 >> 172.21.15.195:0/2587372817 conn(0x7f195c0fd350 msgr2=0x7f195c0ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:49.207 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.207+0000 7f1962bcb640 1 -- 172.21.15.195:0/2587372817 shutdown_connections 2024-09-12T03:20:49.207 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.207+0000 7f1962bcb640 1 -- 172.21.15.195:0/2587372817 wait complete. 2024-09-12T03:20:49.209 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.207+0000 7f1962bcb640 1 Processor -- start 2024-09-12T03:20:49.209 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.207+0000 7f1962bcb640 1 -- start start 2024-09-12T03:20:49.209 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.208+0000 7f1962bcb640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f195c107670 0x7f195c1972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:49.209 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.208+0000 7f1962bcb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f195c197810 0x7f195c19bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:49.209 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.208+0000 7f1962bcb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f195c10bbe0 con 0x7f195c197810 2024-09-12T03:20:49.210 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.208+0000 7f1962bcb640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f195c10ba60 con 0x7f195c107670 2024-09-12T03:20:49.210 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.208+0000 7f1960967640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f195c107670 0x7f195c1972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:49.210 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.208+0000 7f1960967640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f195c107670 0x7f195c1972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:33260/0 (socket says 172.21.15.195:33260) 2024-09-12T03:20:49.210 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.208+0000 7f1960967640 1 -- 172.21.15.195:0/3165153831 learned_addr learned my addr 172.21.15.195:0/3165153831 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-09-12T03:20:49.210 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.208+0000 7f195bfff640 1 --2- 172.21.15.195:0/3165153831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f195c197810 0x7f195c19bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:49.210 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.208+0000 7f1960967640 1 -- 172.21.15.195:0/3165153831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f195c107670 msgr2=0x7f195c1972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-12T03:20:49.210 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.208+0000 7f1960967640 1 -- 172.21.15.195:0/3165153831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f195c107670 msgr2=0x7f195c1972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T03:20:49.210 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.208+0000 7f1960967640 1 --2- 172.21.15.195:0/3165153831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f195c107670 0x7f195c1972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T03:20:49.210 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.208+0000 7f1960967640 1 --2- 172.21.15.195:0/3165153831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f195c107670 0x7f195c1972d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T03:20:49.210 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.208+0000 7f195bfff640 1 -- 172.21.15.195:0/3165153831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f195c107670 msgr2=0x7f195c1972d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:20:49.210 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.208+0000 7f195bfff640 1 --2- 172.21.15.195:0/3165153831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f195c107670 0x7f195c1972d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:49.210 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.208+0000 7f195bfff640 1 -- 172.21.15.195:0/3165153831 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f195c197e30 con 0x7f195c197810 2024-09-12T03:20:49.211 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.208+0000 7f195bfff640 1 --2- 172.21.15.195:0/3165153831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f195c197810 0x7f195c19bb50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f194800b3d0 tx=0x7f194800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:49.211 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.209+0000 7f1959ffb640 1 -- 172.21.15.195:0/3165153831 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f1948011630 con 0x7f195c197810 2024-09-12T03:20:49.211 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.209+0000 7f1959ffb640 1 -- 172.21.15.195:0/3165153831 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1948011c80 con 0x7f195c197810 2024-09-12T03:20:49.211 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.209+0000 7f1962bcb640 1 -- 172.21.15.195:0/3165153831 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f195c19c090 con 0x7f195c197810 2024-09-12T03:20:49.211 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.209+0000 7f1959ffb640 1 -- 172.21.15.195:0/3165153831 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f1948010d80 con 0x7f195c197810 2024-09-12T03:20:49.211 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.209+0000 7f1962bcb640 1 -- 172.21.15.195:0/3165153831 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f195c19c5d0 con 0x7f195c197810 2024-09-12T03:20:49.212 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.210+0000 7f1959ffb640 1 -- 172.21.15.195:0/3165153831 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f1948018020 con 0x7f195c197810 2024-09-12T03:20:49.212 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.210+0000 7f1959ffb640 1 --2- 172.21.15.195:0/3165153831 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f194c038df0 0x7f194c03b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:49.212 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.210+0000 7f1959ffb640 1 -- 172.21.15.195:0/3165153831 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f194804bc20 con 0x7f195c197810 2024-09-12T03:20:49.212 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.210+0000 7f1962bcb640 1 -- 172.21.15.195:0/3165153831 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f195c102db0 con 0x7f195c197810 2024-09-12T03:20:49.212 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.210+0000 7f1960967640 1 --2- 172.21.15.195:0/3165153831 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f194c038df0 0x7f194c03b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:49.212 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.211+0000 7f1960967640 1 --2- 172.21.15.195:0/3165153831 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f194c038df0 0x7f194c03b2b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f195002abc0 tx=0x7f195002aad0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:49.215 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.215+0000 7f1959ffb640 1 -- 172.21.15.195:0/3165153831 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f194801f020 con 0x7f195c197810 2024-09-12T03:20:49.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:49 smithi064 ceph-mon[30952]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:20:49.441 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.441+0000 7f1962bcb640 1 -- 172.21.15.195:0/3165153831 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f195c19c8e0 con 0x7f195c197810 2024-09-12T03:20:49.442 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.442+0000 7f1959ffb640 1 -- 172.21.15.195:0/3165153831 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f194801a920 con 0x7f195c197810 2024-09-12T03:20:49.443 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-12T03:20:49.443 INFO:teuthology.orchestra.run.smithi195.stdout:{"epoch":2,"fsid":"b0812460-70b5-11ef-bcea-c7b262605968","modified":"2024-09-12T03:20:41.042383Z","created":"2024-09-12T03:19:13.042174Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3300","nonce":0},{"type":"v1","addr":"172.21.15.195:6789","nonce":0}]},"addr":"172.21.15.195:6789/0","public_addr":"172.21.15.195:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-12T03:20:49.443 INFO:teuthology.orchestra.run.smithi195.stderr:dumped monmap epoch 2 2024-09-12T03:20:49.445 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.444+0000 7f1962bcb640 1 -- 172.21.15.195:0/3165153831 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f194c038df0 msgr2=0x7f194c03b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:49.445 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.444+0000 7f1962bcb640 1 --2- 172.21.15.195:0/3165153831 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f194c038df0 0x7f194c03b2b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f195002abc0 tx=0x7f195002aad0 comp rx=0 tx=0).stop 2024-09-12T03:20:49.445 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.444+0000 7f1962bcb640 1 -- 172.21.15.195:0/3165153831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f195c197810 msgr2=0x7f195c19bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:49.445 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.444+0000 7f1962bcb640 1 --2- 172.21.15.195:0/3165153831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f195c197810 0x7f195c19bb50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f194800b3d0 tx=0x7f194800b890 comp rx=0 tx=0).stop 2024-09-12T03:20:49.445 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.444+0000 7f1962bcb640 1 -- 172.21.15.195:0/3165153831 shutdown_connections 2024-09-12T03:20:49.445 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.444+0000 7f1962bcb640 1 --2- 172.21.15.195:0/3165153831 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f194c038df0 0x7f194c03b2b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:49.445 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.444+0000 7f1962bcb640 1 --2- 172.21.15.195:0/3165153831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f195c197810 0x7f195c19bb50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:49.445 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.444+0000 7f1962bcb640 1 --2- 172.21.15.195:0/3165153831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f195c107670 0x7f195c1972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:49.445 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.444+0000 7f1962bcb640 1 -- 172.21.15.195:0/3165153831 >> 172.21.15.195:0/3165153831 conn(0x7f195c0fd350 msgr2=0x7f195c0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:49.445 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.444+0000 7f1962bcb640 1 -- 172.21.15.195:0/3165153831 shutdown_connections 2024-09-12T03:20:49.445 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:49.444+0000 7f1962bcb640 1 -- 172.21.15.195:0/3165153831 wait complete. 2024-09-12T03:20:50.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:50 smithi115 systemd[1]: Started Ceph mon.b for b0812460-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:20:51.066 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T03:20:51.066 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T03:20:51.315 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:20:55.332 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.330+0000 7fdc9bdb8640 1 -- 172.21.15.195:0/2801638061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc80003530 msgr2=0x7fdc800059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:55.332 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.330+0000 7fdc9bdb8640 1 --2- 172.21.15.195:0/2801638061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc80003530 0x7fdc800059c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdc941016a0 tx=0x7fdc9002c3f0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.332 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.330+0000 7fdc9bdb8640 1 -- 172.21.15.195:0/2801638061 shutdown_connections 2024-09-12T03:20:55.332 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.330+0000 7fdc9bdb8640 1 --2- 172.21.15.195:0/2801638061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc80003530 0x7fdc800059c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.332 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.330+0000 7fdc9bdb8640 1 -- 172.21.15.195:0/2801638061 >> 172.21.15.195:0/2801638061 conn(0x7fdc940fc220 msgr2=0x7fdc940fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:55.332 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.330+0000 7fdc9bdb8640 1 -- 172.21.15.195:0/2801638061 shutdown_connections 2024-09-12T03:20:55.332 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.330+0000 7fdc9bdb8640 1 -- 172.21.15.195:0/2801638061 wait complete. 2024-09-12T03:20:55.334 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.331+0000 7fdc9bdb8640 1 Processor -- start 2024-09-12T03:20:55.334 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.331+0000 7fdc9bdb8640 1 -- start start 2024-09-12T03:20:55.335 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.331+0000 7fdc9bdb8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc80003530 0x7fdc94197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:55.335 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.331+0000 7fdc9bdb8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc94197750 0x7fdc9419bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:55.335 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.331+0000 7fdc9bdb8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc9419c020 0x7fdc9419c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:55.335 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.331+0000 7fdc9bdb8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdc94108510 con 0x7fdc80003530 2024-09-12T03:20:55.335 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.331+0000 7fdc9bdb8640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdc94108390 con 0x7fdc94197750 2024-09-12T03:20:55.335 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.331+0000 7fdc9bdb8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdc94108690 con 0x7fdc9419c020 2024-09-12T03:20:55.335 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.331+0000 7fdc9a355640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc9419c020 0x7fdc9419c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:55.335 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.331+0000 7fdc9a355640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc9419c020 0x7fdc9419c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:33268/0 (socket says 172.21.15.195:33268) 2024-09-12T03:20:55.335 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.331+0000 7fdc9a355640 1 -- 172.21.15.195:0/2576378946 learned_addr learned my addr 172.21.15.195:0/2576378946 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-09-12T03:20:55.335 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.331+0000 7fdc99b54640 1 --2- 172.21.15.195:0/2576378946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc80003530 0x7fdc94197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:55.336 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.331+0000 7fdc99353640 1 --2- 172.21.15.195:0/2576378946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc94197750 0x7fdc9419bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:55.336 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.331+0000 7fdc9a355640 1 -- 172.21.15.195:0/2576378946 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc9419c020 msgr2=0x7fdc9419c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-12T03:20:55.336 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.331+0000 7fdc9a355640 1 -- 172.21.15.195:0/2576378946 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc9419c020 msgr2=0x7fdc9419c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T03:20:55.336 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.331+0000 7fdc9a355640 1 --2- 172.21.15.195:0/2576378946 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc9419c020 0x7fdc9419c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T03:20:55.336 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.331+0000 7fdc9a355640 1 --2- 172.21.15.195:0/2576378946 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc9419c020 0x7fdc9419c4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T03:20:55.336 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.332+0000 7fdc99353640 1 -- 172.21.15.195:0/2576378946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc94197750 msgr2=0x7fdc9419bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-12T03:20:55.336 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.332+0000 7fdc99353640 1 -- 172.21.15.195:0/2576378946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc94197750 msgr2=0x7fdc9419bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T03:20:55.336 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.332+0000 7fdc99353640 1 --2- 172.21.15.195:0/2576378946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc94197750 0x7fdc9419bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T03:20:55.336 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.332+0000 7fdc99353640 1 --2- 172.21.15.195:0/2576378946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc94197750 0x7fdc9419bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T03:20:55.336 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.332+0000 7fdc99b54640 1 -- 172.21.15.195:0/2576378946 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc9419c020 msgr2=0x7fdc9419c4d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:20:55.336 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.332+0000 7fdc99b54640 1 --2- 172.21.15.195:0/2576378946 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc9419c020 0x7fdc9419c4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.336 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.332+0000 7fdc99b54640 1 -- 172.21.15.195:0/2576378946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc94197750 msgr2=0x7fdc9419bae0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:20:55.337 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.332+0000 7fdc99b54640 1 --2- 172.21.15.195:0/2576378946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc94197750 0x7fdc9419bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.337 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.332+0000 7fdc99b54640 1 -- 172.21.15.195:0/2576378946 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc941a3d90 con 0x7fdc80003530 2024-09-12T03:20:55.337 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.332+0000 7fdc99b54640 1 --2- 172.21.15.195:0/2576378946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc80003530 0x7fdc94197210 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdc90036040 tx=0x7fdc90034a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:55.338 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.332+0000 7fdc8affd640 1 -- 172.21.15.195:0/2576378946 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc90042070 con 0x7fdc80003530 2024-09-12T03:20:55.338 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.332+0000 7fdc9bdb8640 1 -- 172.21.15.195:0/2576378946 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc941a3f60 con 0x7fdc80003530 2024-09-12T03:20:55.338 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.332+0000 7fdc8affd640 1 -- 172.21.15.195:0/2576378946 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc90005310 con 0x7fdc80003530 2024-09-12T03:20:55.338 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.332+0000 7fdc9bdb8640 1 -- 172.21.15.195:0/2576378946 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc941a4440 con 0x7fdc80003530 2024-09-12T03:20:55.338 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.332+0000 7fdc8affd640 1 -- 172.21.15.195:0/2576378946 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc90012650 con 0x7fdc80003530 2024-09-12T03:20:55.338 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.333+0000 7fdc8affd640 1 -- 172.21.15.195:0/2576378946 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fdc90045070 con 0x7fdc80003530 2024-09-12T03:20:55.338 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.333+0000 7fdc9bdb8640 1 -- 172.21.15.195:0/2576378946 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc94109580 con 0x7fdc80003530 2024-09-12T03:20:55.338 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.333+0000 7fdc8affd640 1 --2- 172.21.15.195:0/2576378946 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fdc780394f0 0x7fdc7803b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:55.338 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.334+0000 7fdc8affd640 1 -- 172.21.15.195:0/2576378946 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fdc9006dcf0 con 0x7fdc80003530 2024-09-12T03:20:55.339 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.334+0000 7fdc99353640 1 --2- 172.21.15.195:0/2576378946 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fdc780394f0 0x7fdc7803b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:55.339 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.334+0000 7fdc99353640 1 --2- 172.21.15.195:0/2576378946 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fdc780394f0 0x7fdc7803b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdc84006fd0 tx=0x7fdc84008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:55.339 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.338+0000 7fdc8affd640 1 -- 172.21.15.195:0/2576378946 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdc900381c0 con 0x7fdc80003530 2024-09-12T03:20:55.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:55 smithi195 ceph-mon[34786]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:20:55.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:55 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:20:55.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:55 smithi195 ceph-mon[34786]: mon.c calling monitor election 2024-09-12T03:20:55.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:55 smithi195 ceph-mon[34786]: mon.a calling monitor election 2024-09-12T03:20:55.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:55 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:20:55.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:55 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:20:55.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:55 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:20:55.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:55 smithi195 ceph-mon[34786]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:20:55.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:55 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:20:55.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:55 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:20:55.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:55 smithi195 ceph-mon[34786]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:20:55.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:55 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:20:55.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:55 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:20:55.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:55 smithi195 ceph-mon[34786]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T03:20:55.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:55 smithi195 ceph-mon[34786]: monmap epoch 3 2024-09-12T03:20:55.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:55 smithi195 ceph-mon[34786]: fsid b0812460-70b5-11ef-bcea-c7b262605968 2024-09-12T03:20:55.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:55 smithi195 ceph-mon[34786]: last_changed 2024-09-12T03:20:50.048351+0000 2024-09-12T03:20:55.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:55 smithi195 ceph-mon[34786]: created 2024-09-12T03:19:13.042174+0000 2024-09-12T03:20:55.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:55 smithi195 ceph-mon[34786]: min_mon_release 19 (squid) 2024-09-12T03:20:55.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:55 smithi195 ceph-mon[34786]: election_strategy: 1 2024-09-12T03:20:55.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:55 smithi195 ceph-mon[34786]: 0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2024-09-12T03:20:55.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:55 smithi195 ceph-mon[34786]: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.c 2024-09-12T03:20:55.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:55 smithi195 ceph-mon[34786]: 2: [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon.b 2024-09-12T03:20:55.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:55 smithi195 ceph-mon[34786]: fsmap 2024-09-12T03:20:55.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:55 smithi195 ceph-mon[34786]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T03:20:55.459 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:55 smithi195 ceph-mon[34786]: mgrmap e13: x(active, since 47s) 2024-09-12T03:20:55.459 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:55 smithi195 ceph-mon[34786]: overall HEALTH_OK 2024-09-12T03:20:55.459 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:55 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:55.459 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:55 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:55.459 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:55 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:55.459 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:55 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:55.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:55 smithi064 ceph-mon[30952]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:20:55.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:55 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:20:55.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:55 smithi064 ceph-mon[30952]: mon.c calling monitor election 2024-09-12T03:20:55.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:55 smithi064 ceph-mon[30952]: mon.a calling monitor election 2024-09-12T03:20:55.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:55 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:20:55.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:55 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:20:55.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:55 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:20:55.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:55 smithi064 ceph-mon[30952]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:20:55.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:55 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:20:55.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:55 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:20:55.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:55 smithi064 ceph-mon[30952]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:20:55.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:55 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:20:55.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:55 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:20:55.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:55 smithi064 ceph-mon[30952]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T03:20:55.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:55 smithi064 ceph-mon[30952]: monmap epoch 3 2024-09-12T03:20:55.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:55 smithi064 ceph-mon[30952]: fsid b0812460-70b5-11ef-bcea-c7b262605968 2024-09-12T03:20:55.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:55 smithi064 ceph-mon[30952]: last_changed 2024-09-12T03:20:50.048351+0000 2024-09-12T03:20:55.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:55 smithi064 ceph-mon[30952]: created 2024-09-12T03:19:13.042174+0000 2024-09-12T03:20:55.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:55 smithi064 ceph-mon[30952]: min_mon_release 19 (squid) 2024-09-12T03:20:55.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:55 smithi064 ceph-mon[30952]: election_strategy: 1 2024-09-12T03:20:55.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:55 smithi064 ceph-mon[30952]: 0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2024-09-12T03:20:55.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:55 smithi064 ceph-mon[30952]: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.c 2024-09-12T03:20:55.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:55 smithi064 ceph-mon[30952]: 2: [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon.b 2024-09-12T03:20:55.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:55 smithi064 ceph-mon[30952]: fsmap 2024-09-12T03:20:55.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:55 smithi064 ceph-mon[30952]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T03:20:55.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:55 smithi064 ceph-mon[30952]: mgrmap e13: x(active, since 47s) 2024-09-12T03:20:55.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:55 smithi064 ceph-mon[30952]: overall HEALTH_OK 2024-09-12T03:20:55.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:55 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:55.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:55 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:55.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:55 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:55.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:55 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:55.592 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.591+0000 7fdc9bdb8640 1 -- 172.21.15.195:0/2576378946 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fdc941a4820 con 0x7fdc80003530 2024-09-12T03:20:55.593 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.592+0000 7fdc8affd640 1 -- 172.21.15.195:0/2576378946 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7fdc90012cc0 con 0x7fdc80003530 2024-09-12T03:20:55.594 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-12T03:20:55.594 INFO:teuthology.orchestra.run.smithi195.stdout:{"epoch":3,"fsid":"b0812460-70b5-11ef-bcea-c7b262605968","modified":"2024-09-12T03:20:50.048351Z","created":"2024-09-12T03:19:13.042174Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3300","nonce":0},{"type":"v1","addr":"172.21.15.195:6789","nonce":0}]},"addr":"172.21.15.195:6789/0","public_addr":"172.21.15.195:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:3300","nonce":0},{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-12T03:20:55.594 INFO:teuthology.orchestra.run.smithi195.stderr:dumped monmap epoch 3 2024-09-12T03:20:55.596 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.594+0000 7fdc9bdb8640 1 -- 172.21.15.195:0/2576378946 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fdc780394f0 msgr2=0x7fdc7803b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:55.596 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.594+0000 7fdc9bdb8640 1 --2- 172.21.15.195:0/2576378946 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fdc780394f0 0x7fdc7803b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdc84006fd0 tx=0x7fdc84008040 comp rx=0 tx=0).stop 2024-09-12T03:20:55.596 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.594+0000 7fdc9bdb8640 1 -- 172.21.15.195:0/2576378946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc80003530 msgr2=0x7fdc94197210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:55.596 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.594+0000 7fdc9bdb8640 1 --2- 172.21.15.195:0/2576378946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc80003530 0x7fdc94197210 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdc90036040 tx=0x7fdc90034a70 comp rx=0 tx=0).stop 2024-09-12T03:20:55.596 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.594+0000 7fdc9bdb8640 1 -- 172.21.15.195:0/2576378946 shutdown_connections 2024-09-12T03:20:55.596 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.594+0000 7fdc9bdb8640 1 --2- 172.21.15.195:0/2576378946 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fdc780394f0 0x7fdc7803b9b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.596 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.594+0000 7fdc9bdb8640 1 --2- 172.21.15.195:0/2576378946 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc9419c020 0x7fdc9419c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.596 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.594+0000 7fdc9bdb8640 1 --2- 172.21.15.195:0/2576378946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc94197750 0x7fdc9419bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.596 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.594+0000 7fdc9bdb8640 1 --2- 172.21.15.195:0/2576378946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc80003530 0x7fdc94197210 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.596 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.594+0000 7fdc9bdb8640 1 -- 172.21.15.195:0/2576378946 >> 172.21.15.195:0/2576378946 conn(0x7fdc940fc220 msgr2=0x7fdc94105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:55.597 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.594+0000 7fdc9bdb8640 1 -- 172.21.15.195:0/2576378946 shutdown_connections 2024-09-12T03:20:55.597 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:55.595+0000 7fdc9bdb8640 1 -- 172.21.15.195:0/2576378946 wait complete. 2024-09-12T03:20:56.412 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:56 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:20:56.412 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:56 smithi195 ceph-mon[34786]: from='client.? 172.21.15.195:0/2576378946' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T03:20:56.412 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:56 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:20:56.439 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-12T03:20:56.440 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-12T03:20:56.468 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:56 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:20:56.469 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:56 smithi064 ceph-mon[30952]: from='client.? 172.21.15.195:0/2576378946' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T03:20:56.469 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:56 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:20:56.709 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:57.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.207+0000 7ff910443640 1 -- 172.21.15.64:0/3255521690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9081097a0 msgr2=0x7ff908109b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:57.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.207+0000 7ff910443640 1 --2- 172.21.15.64:0/3255521690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9081097a0 0x7ff908109b80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff904009880 tx=0x7ff90402a5d0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.207+0000 7ff910443640 1 -- 172.21.15.64:0/3255521690 shutdown_connections 2024-09-12T03:20:57.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.207+0000 7ff910443640 1 --2- 172.21.15.64:0/3255521690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9081097a0 0x7ff908109b80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.207+0000 7ff910443640 1 -- 172.21.15.64:0/3255521690 >> 172.21.15.64:0/3255521690 conn(0x7ff9080ff480 msgr2=0x7ff9081018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:57.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.207+0000 7ff910443640 1 -- 172.21.15.64:0/3255521690 shutdown_connections 2024-09-12T03:20:57.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.207+0000 7ff910443640 1 -- 172.21.15.64:0/3255521690 wait complete. 2024-09-12T03:20:57.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.208+0000 7ff910443640 1 Processor -- start 2024-09-12T03:20:57.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.208+0000 7ff910443640 1 -- start start 2024-09-12T03:20:57.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.208+0000 7ff910443640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9081097a0 0x7ff908199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:57.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.208+0000 7ff910443640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff908199a60 0x7ff90819dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:57.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.208+0000 7ff910443640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff90819e2e0 0x7ff90819e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:57.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.208+0000 7ff910443640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff90810dcb0 con 0x7ff908199a60 2024-09-12T03:20:57.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.208+0000 7ff910443640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff90810db30 con 0x7ff9081097a0 2024-09-12T03:20:57.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.208+0000 7ff910443640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff90810de30 con 0x7ff90819e2e0 2024-09-12T03:20:57.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.208+0000 7ff90d9de640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff908199a60 0x7ff90819dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:57.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.208+0000 7ff90d9de640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff908199a60 0x7ff90819dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34624/0 (socket says 172.21.15.64:34624) 2024-09-12T03:20:57.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.208+0000 7ff90d9de640 1 -- 172.21.15.64:0/373040541 learned_addr learned my addr 172.21.15.64:0/373040541 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:20:57.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.208+0000 7ff90e1df640 1 --2- 172.21.15.64:0/373040541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9081097a0 0x7ff908199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:57.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.209+0000 7ff90e9e0640 1 --2- 172.21.15.64:0/373040541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff90819e2e0 0x7ff90819e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:57.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.209+0000 7ff90e1df640 1 -- 172.21.15.64:0/373040541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff90819e2e0 msgr2=0x7ff90819e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:57.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.209+0000 7ff90e1df640 1 --2- 172.21.15.64:0/373040541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff90819e2e0 0x7ff90819e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.209+0000 7ff90e1df640 1 -- 172.21.15.64:0/373040541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff908199a60 msgr2=0x7ff90819dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:57.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.209+0000 7ff90e1df640 1 --2- 172.21.15.64:0/373040541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff908199a60 0x7ff90819dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.209+0000 7ff90e1df640 1 -- 172.21.15.64:0/373040541 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9081a6050 con 0x7ff9081097a0 2024-09-12T03:20:57.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.209+0000 7ff90e9e0640 1 --2- 172.21.15.64:0/373040541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff90819e2e0 0x7ff90819e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:57.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.209+0000 7ff90e1df640 1 --2- 172.21.15.64:0/373040541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9081097a0 0x7ff908199520 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff904000c00 tx=0x7ff9040042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:57.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.209+0000 7ff8ff7fe640 1 -- 172.21.15.64:0/373040541 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90403f030 con 0x7ff9081097a0 2024-09-12T03:20:57.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.209+0000 7ff8ff7fe640 1 -- 172.21.15.64:0/373040541 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff904003cb0 con 0x7ff9081097a0 2024-09-12T03:20:57.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.209+0000 7ff910443640 1 -- 172.21.15.64:0/373040541 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9081a6220 con 0x7ff9081097a0 2024-09-12T03:20:57.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.209+0000 7ff8ff7fe640 1 -- 172.21.15.64:0/373040541 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff904035070 con 0x7ff9081097a0 2024-09-12T03:20:57.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.209+0000 7ff910443640 1 -- 172.21.15.64:0/373040541 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9081a6730 con 0x7ff9081097a0 2024-09-12T03:20:57.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.210+0000 7ff8ff7fe640 1 -- 172.21.15.64:0/373040541 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff904042030 con 0x7ff9081097a0 2024-09-12T03:20:57.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.210+0000 7ff8ff7fe640 1 --2- 172.21.15.64:0/373040541 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff8f40394f0 0x7ff8f403b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:57.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.210+0000 7ff8ff7fe640 1 -- 172.21.15.64:0/373040541 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff90406dbe0 con 0x7ff9081097a0 2024-09-12T03:20:57.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.211+0000 7ff90d9de640 1 --2- 172.21.15.64:0/373040541 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff8f40394f0 0x7ff8f403b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:57.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.211+0000 7ff910443640 1 -- 172.21.15.64:0/373040541 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8ec005290 con 0x7ff9081097a0 2024-09-12T03:20:57.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.211+0000 7ff90d9de640 1 --2- 172.21.15.64:0/373040541 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff8f40394f0 0x7ff8f403b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8009b80 tx=0x7ff8f8009240 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:57.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.216+0000 7ff8ff7fe640 1 -- 172.21.15.64:0/373040541 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff90404dc30 con 0x7ff9081097a0 2024-09-12T03:20:57.321 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:20:57.321 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:20:57.321 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: mon.c calling monitor election 2024-09-12T03:20:57.321 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: mon.a calling monitor election 2024-09-12T03:20:57.321 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:20:57.321 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:20:57.321 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:20:57.321 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:20:57.322 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:20:57.322 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:20:57.322 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:20:57.322 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:20:57.322 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:20:57.322 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T03:20:57.322 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: monmap epoch 3 2024-09-12T03:20:57.322 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: fsid b0812460-70b5-11ef-bcea-c7b262605968 2024-09-12T03:20:57.322 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: last_changed 2024-09-12T03:20:50.048351+0000 2024-09-12T03:20:57.322 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: created 2024-09-12T03:19:13.042174+0000 2024-09-12T03:20:57.322 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: min_mon_release 19 (squid) 2024-09-12T03:20:57.322 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: election_strategy: 1 2024-09-12T03:20:57.322 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: 0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2024-09-12T03:20:57.322 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.c 2024-09-12T03:20:57.323 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: 2: [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon.b 2024-09-12T03:20:57.323 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: fsmap 2024-09-12T03:20:57.323 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T03:20:57.323 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: mgrmap e13: x(active, since 47s) 2024-09-12T03:20:57.323 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: overall HEALTH_OK 2024-09-12T03:20:57.323 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:57.323 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:57.323 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:57.323 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:57.323 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:20:57.323 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: from='client.? 172.21.15.195:0/2576378946' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T03:20:57.323 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:20:57.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.397+0000 7ff910443640 1 -- 172.21.15.64:0/373040541 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7ff8ec005580 con 0x7ff9081097a0 2024-09-12T03:20:57.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.397+0000 7ff8ff7fe640 1 -- 172.21.15.64:0/373040541 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7ff90403b460 con 0x7ff9081097a0 2024-09-12T03:20:57.398 INFO:teuthology.orchestra.run.smithi064.stdout:# minimal ceph.conf for b0812460-70b5-11ef-bcea-c7b262605968 2024-09-12T03:20:57.398 INFO:teuthology.orchestra.run.smithi064.stdout:[global] 2024-09-12T03:20:57.398 INFO:teuthology.orchestra.run.smithi064.stdout: fsid = b0812460-70b5-11ef-bcea-c7b262605968 2024-09-12T03:20:57.398 INFO:teuthology.orchestra.run.smithi064.stdout: mon_host = [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-09-12T03:20:57.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.400+0000 7ff910443640 1 -- 172.21.15.64:0/373040541 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff8f40394f0 msgr2=0x7ff8f403b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:57.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.400+0000 7ff910443640 1 --2- 172.21.15.64:0/373040541 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff8f40394f0 0x7ff8f403b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8009b80 tx=0x7ff8f8009240 comp rx=0 tx=0).stop 2024-09-12T03:20:57.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.400+0000 7ff910443640 1 -- 172.21.15.64:0/373040541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9081097a0 msgr2=0x7ff908199520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:57.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.400+0000 7ff910443640 1 --2- 172.21.15.64:0/373040541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9081097a0 0x7ff908199520 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff904000c00 tx=0x7ff9040042f0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.401+0000 7ff910443640 1 -- 172.21.15.64:0/373040541 shutdown_connections 2024-09-12T03:20:57.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.401+0000 7ff910443640 1 --2- 172.21.15.64:0/373040541 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff8f40394f0 0x7ff8f403b9b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.401+0000 7ff910443640 1 --2- 172.21.15.64:0/373040541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff90819e2e0 0x7ff90819e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.401+0000 7ff910443640 1 --2- 172.21.15.64:0/373040541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff908199a60 0x7ff90819dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.401+0000 7ff910443640 1 --2- 172.21.15.64:0/373040541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9081097a0 0x7ff908199520 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.401+0000 7ff910443640 1 -- 172.21.15.64:0/373040541 >> 172.21.15.64:0/373040541 conn(0x7ff9080ff480 msgr2=0x7ff908100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:57.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.401+0000 7ff910443640 1 -- 172.21.15.64:0/373040541 shutdown_connections 2024-09-12T03:20:57.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:20:57.401+0000 7ff910443640 1 -- 172.21.15.64:0/373040541 wait complete. 2024-09-12T03:20:57.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: mon.b calling monitor election 2024-09-12T03:20:57.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: Updating smithi064:/etc/ceph/ceph.conf 2024-09-12T03:20:57.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: Updating smithi115:/etc/ceph/ceph.conf 2024-09-12T03:20:57.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: Updating smithi195:/etc/ceph/ceph.conf 2024-09-12T03:20:57.958 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: Updating smithi195:/var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:20:57.958 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: Updating smithi115:/var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:20:57.958 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: Updating smithi064:/var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:20:57.958 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: mon.b calling monitor election 2024-09-12T03:20:57.958 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: mon.c calling monitor election 2024-09-12T03:20:57.959 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: mon.a calling monitor election 2024-09-12T03:20:57.959 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T03:20:57.959 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: monmap epoch 3 2024-09-12T03:20:57.959 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: fsid b0812460-70b5-11ef-bcea-c7b262605968 2024-09-12T03:20:57.959 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: last_changed 2024-09-12T03:20:50.048351+0000 2024-09-12T03:20:57.959 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: created 2024-09-12T03:19:13.042174+0000 2024-09-12T03:20:57.960 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: min_mon_release 19 (squid) 2024-09-12T03:20:57.960 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: election_strategy: 1 2024-09-12T03:20:57.960 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: 0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2024-09-12T03:20:57.960 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.c 2024-09-12T03:20:57.960 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: 2: [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon.b 2024-09-12T03:20:57.961 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: fsmap 2024-09-12T03:20:57.961 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T03:20:57.961 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: mgrmap e13: x(active, since 49s) 2024-09-12T03:20:57.961 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: overall HEALTH_OK 2024-09-12T03:20:57.961 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:57.961 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:57.962 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:57.962 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:57.962 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:57.962 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:57.962 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:57.962 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:57.962 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:57.963 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:57.963 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:57.963 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: Reconfiguring mon.a (unknown last config time)... 2024-09-12T03:20:57.963 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:20:57.963 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T03:20:57.963 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:57.964 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: Reconfiguring daemon mon.a on smithi064 2024-09-12T03:20:57.964 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/373040541' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:57.964 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:57 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:57.965 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: mon.b calling monitor election 2024-09-12T03:20:57.965 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: Updating smithi064:/etc/ceph/ceph.conf 2024-09-12T03:20:57.965 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: Updating smithi115:/etc/ceph/ceph.conf 2024-09-12T03:20:57.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: Updating smithi195:/etc/ceph/ceph.conf 2024-09-12T03:20:57.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: Updating smithi195:/var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:20:57.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: Updating smithi115:/var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:20:57.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: Updating smithi064:/var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:20:57.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: mon.b calling monitor election 2024-09-12T03:20:57.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: mon.c calling monitor election 2024-09-12T03:20:57.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: mon.a calling monitor election 2024-09-12T03:20:57.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T03:20:57.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: monmap epoch 3 2024-09-12T03:20:57.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: fsid b0812460-70b5-11ef-bcea-c7b262605968 2024-09-12T03:20:57.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: last_changed 2024-09-12T03:20:50.048351+0000 2024-09-12T03:20:57.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: created 2024-09-12T03:19:13.042174+0000 2024-09-12T03:20:57.966 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: min_mon_release 19 (squid) 2024-09-12T03:20:57.967 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: election_strategy: 1 2024-09-12T03:20:57.967 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: 0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2024-09-12T03:20:57.967 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.c 2024-09-12T03:20:57.967 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: 2: [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon.b 2024-09-12T03:20:57.967 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: fsmap 2024-09-12T03:20:57.967 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T03:20:57.967 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: mgrmap e13: x(active, since 49s) 2024-09-12T03:20:57.967 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: overall HEALTH_OK 2024-09-12T03:20:57.967 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:57.967 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:57.967 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:57.967 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:57.967 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:57.968 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:57.968 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:57.968 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:57.968 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:57.968 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:57.968 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:57.968 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: Reconfiguring mon.a (unknown last config time)... 2024-09-12T03:20:57.968 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:20:57.968 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T03:20:57.968 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:57.968 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: Reconfiguring daemon mon.a on smithi064 2024-09-12T03:20:57.968 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/373040541' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:57.968 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:57 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:57.992 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-12T03:20:57.993 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-12T03:20:57.993 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T03:20:58.063 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-12T03:20:58.063 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:20:58.129 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-12T03:20:58.129 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T03:20:58.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: mon.b calling monitor election 2024-09-12T03:20:58.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: Updating smithi064:/etc/ceph/ceph.conf 2024-09-12T03:20:58.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: Updating smithi115:/etc/ceph/ceph.conf 2024-09-12T03:20:58.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: Updating smithi195:/etc/ceph/ceph.conf 2024-09-12T03:20:58.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: Updating smithi195:/var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:20:58.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: Updating smithi115:/var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:20:58.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: Updating smithi064:/var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:20:58.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: mon.b calling monitor election 2024-09-12T03:20:58.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: mon.c calling monitor election 2024-09-12T03:20:58.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: mon.a calling monitor election 2024-09-12T03:20:58.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T03:20:58.144 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: monmap epoch 3 2024-09-12T03:20:58.144 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: fsid b0812460-70b5-11ef-bcea-c7b262605968 2024-09-12T03:20:58.144 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: last_changed 2024-09-12T03:20:50.048351+0000 2024-09-12T03:20:58.144 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: created 2024-09-12T03:19:13.042174+0000 2024-09-12T03:20:58.144 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: min_mon_release 19 (squid) 2024-09-12T03:20:58.144 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: election_strategy: 1 2024-09-12T03:20:58.144 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: 0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2024-09-12T03:20:58.144 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.c 2024-09-12T03:20:58.144 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: 2: [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon.b 2024-09-12T03:20:58.144 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: fsmap 2024-09-12T03:20:58.144 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T03:20:58.144 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: mgrmap e13: x(active, since 49s) 2024-09-12T03:20:58.144 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: overall HEALTH_OK 2024-09-12T03:20:58.145 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:58.145 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:58.145 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:58.145 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:58.145 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:58.145 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:58.145 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:58.145 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:58.146 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:58.146 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:58.146 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:58.146 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: Reconfiguring mon.a (unknown last config time)... 2024-09-12T03:20:58.146 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:20:58.146 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T03:20:58.146 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:58.146 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: Reconfiguring daemon mon.a on smithi064 2024-09-12T03:20:58.146 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/373040541' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:58.146 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:57 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:20:58.157 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-12T03:20:58.157 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:20:58.223 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-12T03:20:58.224 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T03:20:58.253 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-12T03:20:58.253 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:20:58.320 INFO:tasks.cephadm:Adding mgr.x on smithi064 2024-09-12T03:20:58.320 INFO:tasks.cephadm:Adding mgr.y on smithi115 2024-09-12T03:20:58.320 INFO:tasks.cephadm:Adding mgr.z on smithi195 2024-09-12T03:20:58.321 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph orch apply mgr '3;smithi064=x;smithi115=y;smithi195=z' 2024-09-12T03:20:58.599 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:20:58.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:58 smithi195 ceph-mon[34786]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:20:58.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:20:58 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:20:58.996 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:58 smithi064 ceph-mon[30952]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:20:58.996 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:20:58 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:20:59.049 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.048+0000 7ff23e364640 1 -- 172.21.15.195:0/3301132517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2381005e0 msgr2=0x7ff2381009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:59.049 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.048+0000 7ff23e364640 1 --2- 172.21.15.195:0/3301132517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2381005e0 0x7ff2381009e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff228009880 tx=0x7ff22802a5d0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.050 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.048+0000 7ff23e364640 1 -- 172.21.15.195:0/3301132517 shutdown_connections 2024-09-12T03:20:59.050 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.048+0000 7ff23e364640 1 --2- 172.21.15.195:0/3301132517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2381005e0 0x7ff2381009e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.050 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.048+0000 7ff23e364640 1 -- 172.21.15.195:0/3301132517 >> 172.21.15.195:0/3301132517 conn(0x7ff2380fbdb0 msgr2=0x7ff2380fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:59.050 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.048+0000 7ff23e364640 1 -- 172.21.15.195:0/3301132517 shutdown_connections 2024-09-12T03:20:59.050 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.048+0000 7ff23e364640 1 -- 172.21.15.195:0/3301132517 wait complete. 2024-09-12T03:20:59.051 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.049+0000 7ff23e364640 1 Processor -- start 2024-09-12T03:20:59.051 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.049+0000 7ff23e364640 1 -- start start 2024-09-12T03:20:59.051 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.049+0000 7ff23e364640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2381005e0 0x7ff238196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:59.051 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.049+0000 7ff23e364640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff238197460 0x7ff23819c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:59.051 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.049+0000 7ff23e364640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2381978e0 0x7ff23819ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:59.051 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.049+0000 7ff23e364640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff238106580 con 0x7ff2381978e0 2024-09-12T03:20:59.051 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.049+0000 7ff23e364640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff238106400 con 0x7ff238197460 2024-09-12T03:20:59.051 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.049+0000 7ff23e364640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff238106700 con 0x7ff2381005e0 2024-09-12T03:20:59.052 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.049+0000 7ff23c901640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2381978e0 0x7ff23819ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:59.052 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.049+0000 7ff23c901640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2381978e0 0x7ff23819ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.195:45564/0 (socket says 172.21.15.195:45564) 2024-09-12T03:20:59.052 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.049+0000 7ff23c901640 1 -- 172.21.15.195:0/2300907239 learned_addr learned my addr 172.21.15.195:0/2300907239 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-09-12T03:20:59.052 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.049+0000 7ff22ffff640 1 --2- 172.21.15.195:0/2300907239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff238197460 0x7ff23819c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:59.052 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.049+0000 7ff237fff640 1 --2- 172.21.15.195:0/2300907239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2381005e0 0x7ff238196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:59.052 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.049+0000 7ff23c901640 1 -- 172.21.15.195:0/2300907239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2381005e0 msgr2=0x7ff238196f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:59.052 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.049+0000 7ff23c901640 1 --2- 172.21.15.195:0/2300907239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2381005e0 0x7ff238196f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.052 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.049+0000 7ff23c901640 1 -- 172.21.15.195:0/2300907239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff238197460 msgr2=0x7ff23819c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:59.052 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.049+0000 7ff23c901640 1 --2- 172.21.15.195:0/2300907239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff238197460 0x7ff23819c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.052 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.049+0000 7ff23c901640 1 -- 172.21.15.195:0/2300907239 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2380fc190 con 0x7ff2381978e0 2024-09-12T03:20:59.052 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.050+0000 7ff23c901640 1 --2- 172.21.15.195:0/2300907239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2381978e0 0x7ff23819ea60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff23000ba50 tx=0x7ff23000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:59.053 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.052+0000 7ff235ffb640 1 -- 172.21.15.195:0/2300907239 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff230019070 con 0x7ff2381978e0 2024-09-12T03:20:59.053 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.052+0000 7ff23e364640 1 -- 172.21.15.195:0/2300907239 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2380fc480 con 0x7ff2381978e0 2024-09-12T03:20:59.053 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.052+0000 7ff23e364640 1 -- 172.21.15.195:0/2300907239 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2380fca40 con 0x7ff2381978e0 2024-09-12T03:20:59.053 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.052+0000 7ff235ffb640 1 -- 172.21.15.195:0/2300907239 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff23000cbe0 con 0x7ff2381978e0 2024-09-12T03:20:59.053 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.052+0000 7ff235ffb640 1 -- 172.21.15.195:0/2300907239 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff230023060 con 0x7ff2381978e0 2024-09-12T03:20:59.054 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.052+0000 7ff235ffb640 1 -- 172.21.15.195:0/2300907239 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff23001c070 con 0x7ff2381978e0 2024-09-12T03:20:59.054 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.053+0000 7ff23e364640 1 -- 172.21.15.195:0/2300907239 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2180052c0 con 0x7ff2381978e0 2024-09-12T03:20:59.054 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.053+0000 7ff235ffb640 1 --2- 172.21.15.195:0/2300907239 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff2240394f0 0x7ff22403b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:59.054 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.053+0000 7ff235ffb640 1 -- 172.21.15.195:0/2300907239 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff23004caf0 con 0x7ff2381978e0 2024-09-12T03:20:59.054 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.053+0000 7ff237fff640 1 --2- 172.21.15.195:0/2300907239 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff2240394f0 0x7ff22403b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:59.055 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.053+0000 7ff237fff640 1 --2- 172.21.15.195:0/2300907239 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff2240394f0 0x7ff22403b9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff2280314e0 tx=0x7ff228005bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:59.059 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.058+0000 7ff235ffb640 1 -- 172.21.15.195:0/2300907239 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff23001bae0 con 0x7ff2381978e0 2024-09-12T03:20:59.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:58 smithi115 ceph-mon[34801]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:20:59.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:20:58 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:20:59.239 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.237+0000 7ff23e364640 1 -- 172.21.15.195:0/2300907239 --> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi064=x;smithi115=y;smithi195=z", "target": ["mon-mgr", ""]}) -- 0x7ff218000bf0 con 0x7ff2240394f0 2024-09-12T03:20:59.244 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.243+0000 7ff235ffb640 1 -- 172.21.15.195:0/2300907239 <== mgr.14150 v2:172.21.15.64:6800/759931848 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff218000bf0 con 0x7ff2240394f0 2024-09-12T03:20:59.244 INFO:teuthology.orchestra.run.smithi195.stdout:Scheduled mgr update... 2024-09-12T03:20:59.246 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.245+0000 7ff23e364640 1 -- 172.21.15.195:0/2300907239 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff2240394f0 msgr2=0x7ff22403b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:59.247 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.245+0000 7ff23e364640 1 --2- 172.21.15.195:0/2300907239 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff2240394f0 0x7ff22403b9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff2280314e0 tx=0x7ff228005bd0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.247 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.245+0000 7ff23e364640 1 -- 172.21.15.195:0/2300907239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2381978e0 msgr2=0x7ff23819ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:59.247 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.245+0000 7ff23e364640 1 --2- 172.21.15.195:0/2300907239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2381978e0 0x7ff23819ea60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff23000ba50 tx=0x7ff23000bf10 comp rx=0 tx=0).stop 2024-09-12T03:20:59.247 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.245+0000 7ff23e364640 1 -- 172.21.15.195:0/2300907239 shutdown_connections 2024-09-12T03:20:59.247 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.245+0000 7ff23e364640 1 --2- 172.21.15.195:0/2300907239 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff2240394f0 0x7ff22403b9b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.247 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.245+0000 7ff23e364640 1 --2- 172.21.15.195:0/2300907239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2381978e0 0x7ff23819ea60 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.247 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.245+0000 7ff23e364640 1 --2- 172.21.15.195:0/2300907239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff238197460 0x7ff23819c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.247 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.245+0000 7ff23e364640 1 --2- 172.21.15.195:0/2300907239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2381005e0 0x7ff238196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.247 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.245+0000 7ff23e364640 1 -- 172.21.15.195:0/2300907239 >> 172.21.15.195:0/2300907239 conn(0x7ff2380fbdb0 msgr2=0x7ff23806a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:59.247 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.245+0000 7ff23e364640 1 -- 172.21.15.195:0/2300907239 shutdown_connections 2024-09-12T03:20:59.247 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:20:59.245+0000 7ff23e364640 1 -- 172.21.15.195:0/2300907239 wait complete. 2024-09-12T03:20:59.815 DEBUG:teuthology.orchestra.run.smithi115:mgr.y> sudo journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@mgr.y.service 2024-09-12T03:20:59.818 DEBUG:teuthology.orchestra.run.smithi195:mgr.z> sudo journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@mgr.z.service 2024-09-12T03:20:59.821 INFO:tasks.cephadm:Deploying OSDs... 2024-09-12T03:20:59.821 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-12T03:20:59.821 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T03:20:59.841 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T03:20:59.842 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_1 2024-09-12T03:20:59.901 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T03:20:59.901 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:20:59.901 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-12T03:20:59.901 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:20:59.901 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:20:59.902 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-09-12 03:20:17.603237815 +0000 2024-09-12T03:20:59.902 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-09-12 03:16:20.847930539 +0000 2024-09-12T03:20:59.902 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-09-12 03:16:20.847930539 +0000 2024-09-12T03:20:59.902 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-09-12 03:16:20.847930539 +0000 2024-09-12T03:20:59.902 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T03:20:59.972 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-09-12T03:20:59.972 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-09-12T03:20:59.972 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000291154 s, 1.8 MB/s 2024-09-12T03:20:59.973 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T03:21:00.034 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_2 2024-09-12T03:21:00.092 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T03:21:00.093 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:21:00.093 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-12T03:21:00.093 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:21:00.093 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:21:00.093 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-09-12 03:20:17.635237200 +0000 2024-09-12T03:21:00.093 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-09-12 03:16:21.188923853 +0000 2024-09-12T03:21:00.093 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-09-12 03:16:21.188923853 +0000 2024-09-12T03:21:00.093 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-09-12 03:16:21.188923853 +0000 2024-09-12T03:21:00.093 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T03:21:00.160 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-09-12T03:21:00.161 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-09-12T03:21:00.161 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000223517 s, 2.3 MB/s 2024-09-12T03:21:00.162 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T03:21:00.220 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_3 2024-09-12T03:21:00.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:00 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:00.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:00 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:00.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:00 smithi064 ceph-mon[30952]: Reconfiguring mon.b (monmap changed)... 2024-09-12T03:21:00.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:00 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:21:00.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:00 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T03:21:00.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:00 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:00.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:00 smithi064 ceph-mon[30952]: Reconfiguring daemon mon.b on smithi115 2024-09-12T03:21:00.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:00 smithi064 ceph-mon[30952]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi064=x;smithi115=y;smithi195=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:21:00.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:00 smithi064 ceph-mon[30952]: Saving service mgr spec with placement smithi064=x;smithi115=y;smithi195=z;count:3 2024-09-12T03:21:00.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:00 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:00.278 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T03:21:00.279 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:21:00.279 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-12T03:21:00.279 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:21:00.279 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:21:00.279 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-09-12 03:20:17.667236585 +0000 2024-09-12T03:21:00.279 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-09-12 03:16:21.511917519 +0000 2024-09-12T03:21:00.279 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-09-12 03:16:21.511917519 +0000 2024-09-12T03:21:00.279 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-09-12 03:16:21.511917519 +0000 2024-09-12T03:21:00.279 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T03:21:00.345 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-09-12T03:21:00.345 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-09-12T03:21:00.345 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000237781 s, 2.2 MB/s 2024-09-12T03:21:00.346 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T03:21:00.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:00 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:00.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:00 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:00.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:00 smithi115 ceph-mon[34801]: Reconfiguring mon.b (monmap changed)... 2024-09-12T03:21:00.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:00 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:21:00.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:00 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T03:21:00.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:00 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:00.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:00 smithi115 ceph-mon[34801]: Reconfiguring daemon mon.b on smithi115 2024-09-12T03:21:00.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:00 smithi115 ceph-mon[34801]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi064=x;smithi115=y;smithi195=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:21:00.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:00 smithi115 ceph-mon[34801]: Saving service mgr spec with placement smithi064=x;smithi115=y;smithi195=z;count:3 2024-09-12T03:21:00.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:00 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:00.405 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_4 2024-09-12T03:21:00.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:00 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:00.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:00 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:00.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:00 smithi195 ceph-mon[34786]: Reconfiguring mon.b (monmap changed)... 2024-09-12T03:21:00.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:00 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:21:00.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:00 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T03:21:00.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:00 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:00.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:00 smithi195 ceph-mon[34786]: Reconfiguring daemon mon.b on smithi115 2024-09-12T03:21:00.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:00 smithi195 ceph-mon[34786]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi064=x;smithi115=y;smithi195=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:21:00.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:00 smithi195 ceph-mon[34786]: Saving service mgr spec with placement smithi064=x;smithi115=y;smithi195=z;count:3 2024-09-12T03:21:00.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:00 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:00.461 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T03:21:00.461 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:21:00.462 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-12T03:21:00.462 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:21:00.462 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:21:00.462 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-09-12 03:20:17.701235932 +0000 2024-09-12T03:21:00.462 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-09-12 03:16:21.845910971 +0000 2024-09-12T03:21:00.462 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-09-12 03:16:21.845910971 +0000 2024-09-12T03:21:00.462 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-09-12 03:16:21.845910971 +0000 2024-09-12T03:21:00.462 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T03:21:00.527 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-09-12T03:21:00.527 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-09-12T03:21:00.527 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000222584 s, 2.3 MB/s 2024-09-12T03:21:00.528 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T03:21:00.586 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-12T03:21:00.586 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T03:21:00.602 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T03:21:00.603 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_1 2024-09-12T03:21:00.660 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T03:21:00.660 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:21:00.660 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-12T03:21:00.661 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:21:00.661 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:21:00.661 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-09-12 03:20:49.426298805 +0000 2024-09-12T03:21:00.661 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-09-12 03:16:14.504104213 +0000 2024-09-12T03:21:00.661 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-09-12 03:16:14.504104213 +0000 2024-09-12T03:21:00.661 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-09-12 03:16:14.504104213 +0000 2024-09-12T03:21:00.661 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T03:21:00.727 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-09-12T03:21:00.727 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-09-12T03:21:00.727 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000196841 s, 2.6 MB/s 2024-09-12T03:21:00.729 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T03:21:00.786 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_2 2024-09-12T03:21:00.842 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T03:21:00.842 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:21:00.842 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-12T03:21:00.842 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:21:00.843 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:21:00.843 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-09-12 03:20:49.426298805 +0000 2024-09-12T03:21:00.843 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-09-12 03:16:14.839098276 +0000 2024-09-12T03:21:00.843 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-09-12 03:16:14.839098276 +0000 2024-09-12T03:21:00.843 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-09-12 03:16:14.839098276 +0000 2024-09-12T03:21:00.843 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T03:21:00.906 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-09-12T03:21:00.906 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-09-12T03:21:00.907 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000207139 s, 2.5 MB/s 2024-09-12T03:21:00.907 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T03:21:00.964 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_3 2024-09-12T03:21:01.019 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T03:21:01.019 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:21:01.019 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-12T03:21:01.019 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:21:01.019 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:21:01.019 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-09-12 03:20:49.427298787 +0000 2024-09-12T03:21:01.020 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-09-12 03:16:15.136093014 +0000 2024-09-12T03:21:01.020 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-09-12 03:16:15.136093014 +0000 2024-09-12T03:21:01.020 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-09-12 03:16:15.136093014 +0000 2024-09-12T03:21:01.020 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T03:21:01.083 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-09-12T03:21:01.083 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-09-12T03:21:01.083 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000220897 s, 2.3 MB/s 2024-09-12T03:21:01.084 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T03:21:01.141 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_4 2024-09-12T03:21:01.197 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T03:21:01.197 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:21:01.197 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-12T03:21:01.197 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:21:01.197 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:21:01.198 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-09-12 03:20:49.427298787 +0000 2024-09-12T03:21:01.198 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-09-12 03:16:15.478086954 +0000 2024-09-12T03:21:01.198 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-09-12 03:16:15.478086954 +0000 2024-09-12T03:21:01.198 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-09-12 03:16:15.478086954 +0000 2024-09-12T03:21:01.198 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T03:21:01.261 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-09-12T03:21:01.261 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-09-12T03:21:01.261 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000221737 s, 2.3 MB/s 2024-09-12T03:21:01.262 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T03:21:01.319 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-12T03:21:01.319 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T03:21:01.336 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T03:21:01.336 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_1 2024-09-12T03:21:01.392 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T03:21:01.392 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:21:01.392 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-12T03:21:01.393 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:21:01.393 INFO:teuthology.orchestra.run.smithi195.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:21:01.393 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-09-12 03:20:40.222275285 +0000 2024-09-12T03:21:01.393 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-09-12 03:16:14.488222392 +0000 2024-09-12T03:21:01.393 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-09-12 03:16:14.488222392 +0000 2024-09-12T03:21:01.393 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: 2024-09-12 03:16:14.488222392 +0000 2024-09-12T03:21:01.393 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T03:21:01.457 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-09-12T03:21:01.457 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-09-12T03:21:01.457 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 9.4613e-05 s, 5.4 MB/s 2024-09-12T03:21:01.459 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T03:21:01.515 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_2 2024-09-12T03:21:01.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:01 smithi064 ceph-mon[30952]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:01.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:01 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:01.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:01 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:01.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:01 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:21:01.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:01 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T03:21:01.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:01 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:01.572 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T03:21:01.572 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:21:01.572 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-12T03:21:01.572 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:21:01.572 INFO:teuthology.orchestra.run.smithi195.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:21:01.572 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-09-12 03:20:40.222275285 +0000 2024-09-12T03:21:01.572 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-09-12 03:16:14.811216190 +0000 2024-09-12T03:21:01.572 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-09-12 03:16:14.811216190 +0000 2024-09-12T03:21:01.572 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: 2024-09-12 03:16:14.811216190 +0000 2024-09-12T03:21:01.573 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T03:21:01.633 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:01 smithi195 ceph-mon[34786]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:01.633 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:01 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:01.633 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:01 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:01.633 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:01 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:21:01.633 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:01 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T03:21:01.633 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:01 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:01.635 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-09-12T03:21:01.635 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-09-12T03:21:01.635 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 8.6785e-05 s, 5.9 MB/s 2024-09-12T03:21:01.636 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T03:21:01.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:01 smithi115 ceph-mon[34801]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:01.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:01 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:01.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:01 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:01.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:01 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:21:01.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:01 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T03:21:01.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:01 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:01.693 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_3 2024-09-12T03:21:01.748 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T03:21:01.748 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:21:01.748 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-12T03:21:01.748 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:21:01.748 INFO:teuthology.orchestra.run.smithi195.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:21:01.748 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-09-12 03:20:40.223275266 +0000 2024-09-12T03:21:01.748 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-09-12 03:16:15.136209950 +0000 2024-09-12T03:21:01.749 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-09-12 03:16:15.136209950 +0000 2024-09-12T03:21:01.749 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: 2024-09-12 03:16:15.136209950 +0000 2024-09-12T03:21:01.749 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T03:21:01.812 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-09-12T03:21:01.812 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-09-12T03:21:01.812 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 8.7679e-05 s, 5.8 MB/s 2024-09-12T03:21:01.813 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T03:21:01.870 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_4 2024-09-12T03:21:01.926 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T03:21:01.927 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:21:01.927 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-12T03:21:01.927 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:21:01.927 INFO:teuthology.orchestra.run.smithi195.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:21:01.927 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-09-12 03:20:40.223275266 +0000 2024-09-12T03:21:01.927 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-09-12 03:16:15.437204170 +0000 2024-09-12T03:21:01.927 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-09-12 03:16:15.437204170 +0000 2024-09-12T03:21:01.927 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: 2024-09-12 03:16:15.437204170 +0000 2024-09-12T03:21:01.927 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T03:21:01.991 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-09-12T03:21:01.991 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-09-12T03:21:01.991 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.00011581 s, 4.4 MB/s 2024-09-12T03:21:01.992 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T03:21:02.052 INFO:tasks.cephadm:Deploying osd.0 on smithi064 with /dev/vg_nvme/lv_4... 2024-09-12T03:21:02.052 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T03:21:02.317 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:21:02.439 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:02 smithi064 ceph-mon[30952]: Reconfiguring mon.c (monmap changed)... 2024-09-12T03:21:02.439 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:02 smithi064 ceph-mon[30952]: Reconfiguring daemon mon.c on smithi195 2024-09-12T03:21:02.439 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:02 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:02.439 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:02 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:02.440 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:02 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:21:02.440 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:02 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:02.440 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:02 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:21:02.440 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:02 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:02.440 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:02 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T03:21:02.440 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:02 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T03:21:02.440 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:02 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T03:21:02.440 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:02 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:02.582 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:02 smithi195 ceph-mon[34786]: Reconfiguring mon.c (monmap changed)... 2024-09-12T03:21:02.582 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:02 smithi195 ceph-mon[34786]: Reconfiguring daemon mon.c on smithi195 2024-09-12T03:21:02.582 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:02 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:02.582 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:02 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:02.582 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:02 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:21:02.582 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:02 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:02.582 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:02 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:21:02.582 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:02 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:02.582 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:02 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T03:21:02.582 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:02 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T03:21:02.582 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:02 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T03:21:02.582 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:02 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:02.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:02 smithi115 ceph-mon[34801]: Reconfiguring mon.c (monmap changed)... 2024-09-12T03:21:02.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:02 smithi115 ceph-mon[34801]: Reconfiguring daemon mon.c on smithi195 2024-09-12T03:21:02.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:02 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:02.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:02 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:02.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:02 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:21:02.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:02 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:02.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:02 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:21:02.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:02 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:02.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:02 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T03:21:02.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:02 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T03:21:02.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:02 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T03:21:02.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:02 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:03.483 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:03 smithi195 ceph-mon[34786]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:03.483 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:03 smithi195 ceph-mon[34786]: Deploying daemon mgr.z on smithi195 2024-09-12T03:21:03.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:03 smithi064 ceph-mon[30952]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:03.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:03 smithi064 ceph-mon[30952]: Deploying daemon mgr.z on smithi195 2024-09-12T03:21:03.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:03 smithi115 ceph-mon[34801]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:03.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:03 smithi115 ceph-mon[34801]: Deploying daemon mgr.z on smithi195 2024-09-12T03:21:03.846 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:21:03 smithi195 systemd[1]: Starting Ceph mgr.z for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:21:04.140 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:21:03 smithi195 podman[36576]: 2024-09-12 03:21:03.845977159 +0000 UTC m=+0.100609312 container create b3e44952077302e9879c26179fb2c81b696303eff2d9afa320226ba61037c47d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3) 2024-09-12T03:21:04.140 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:21:03 smithi195 podman[36576]: 2024-09-12 03:21:03.761184502 +0000 UTC m=+0.015816651 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:21:04.140 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:21:04 smithi195 podman[36576]: 2024-09-12 03:21:04.04040263 +0000 UTC m=+0.295034779 container init b3e44952077302e9879c26179fb2c81b696303eff2d9afa320226ba61037c47d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-12T03:21:04.140 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:21:04 smithi195 podman[36576]: 2024-09-12 03:21:04.043277938 +0000 UTC m=+0.297910088 container start b3e44952077302e9879c26179fb2c81b696303eff2d9afa320226ba61037c47d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-12T03:21:04.140 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:21:04 smithi195 ceph-mgr[36595]: -- 172.21.15.195:0/2155165616 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560fd671d0e0 con 0x560fd66fb400 2024-09-12T03:21:04.140 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:21:04 smithi195 bash[36576]: b3e44952077302e9879c26179fb2c81b696303eff2d9afa320226ba61037c47d 2024-09-12T03:21:04.445 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:21:04 smithi195 systemd[1]: Started Ceph mgr.z for b0812460-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:21:04.445 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:21:04 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:21:04.163+0000 7f74226eb040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T03:21:04.445 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:21:04 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:21:04.228+0000 7f74226eb040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T03:21:04.445 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:21:04 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:21:04.378+0000 7f74226eb040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T03:21:04.446 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:21:04 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:21:04.445+0000 7f74226eb040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T03:21:04.512 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:21:04.538 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi064:vg_nvme/lv_4 2024-09-12T03:21:04.706 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:21:04 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:21:04.592+0000 7f74226eb040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:21:04.789 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:21:05.049 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:21:04 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:21:04.732+0000 7f74226eb040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T03:21:05.049 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:21:05 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:21:05.048+0000 7f74226eb040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T03:21:05.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.179+0000 7f7b411f0640 1 -- 172.21.15.64:0/113589977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b3c100b10 msgr2=0x7f7b3c100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:05.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.179+0000 7f7b411f0640 1 --2- 172.21.15.64:0/113589977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b3c100b10 0x7f7b3c100f10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c009900 tx=0x7f7b2c02a410 comp rx=0 tx=0).stop 2024-09-12T03:21:05.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.179+0000 7f7b411f0640 1 -- 172.21.15.64:0/113589977 shutdown_connections 2024-09-12T03:21:05.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.179+0000 7f7b411f0640 1 --2- 172.21.15.64:0/113589977 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b3c1026d0 0x7f7b3c108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.179+0000 7f7b411f0640 1 --2- 172.21.15.64:0/113589977 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b3c101d10 0x7f7b3c102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.179+0000 7f7b411f0640 1 --2- 172.21.15.64:0/113589977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b3c100b10 0x7f7b3c100f10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.179+0000 7f7b411f0640 1 -- 172.21.15.64:0/113589977 >> 172.21.15.64:0/113589977 conn(0x7f7b3c0fc2a0 msgr2=0x7f7b3c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:05.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.179+0000 7f7b411f0640 1 -- 172.21.15.64:0/113589977 shutdown_connections 2024-09-12T03:21:05.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.179+0000 7f7b411f0640 1 -- 172.21.15.64:0/113589977 wait complete. 2024-09-12T03:21:05.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.180+0000 7f7b411f0640 1 Processor -- start 2024-09-12T03:21:05.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.180+0000 7f7b411f0640 1 -- start start 2024-09-12T03:21:05.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.180+0000 7f7b411f0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b3c100b10 0x7f7b3c104aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:05.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.180+0000 7f7b411f0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b3c101d10 0x7f7b3c1030f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:05.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.180+0000 7f7b411f0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b3c1026d0 0x7f7b3c103630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:05.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.181+0000 7f7b411f0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3c10c620 con 0x7f7b3c1026d0 2024-09-12T03:21:05.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.181+0000 7f7b411f0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3c10c4a0 con 0x7f7b3c100b10 2024-09-12T03:21:05.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.181+0000 7f7b411f0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3c10c7a0 con 0x7f7b3c101d10 2024-09-12T03:21:05.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.181+0000 7f7b3ad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b3c100b10 0x7f7b3c104aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:05.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.181+0000 7f7b3ad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b3c100b10 0x7f7b3c104aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.64:46510/0 (socket says 172.21.15.64:46510) 2024-09-12T03:21:05.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.181+0000 7f7b3ad9d640 1 -- 172.21.15.64:0/3560529549 learned_addr learned my addr 172.21.15.64:0/3560529549 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:21:05.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.181+0000 7f7b3a59c640 1 --2- 172.21.15.64:0/3560529549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b3c101d10 0x7f7b3c1030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:05.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.181+0000 7f7b3b59e640 1 --2- 172.21.15.64:0/3560529549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b3c1026d0 0x7f7b3c103630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:05.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.181+0000 7f7b3b59e640 1 -- 172.21.15.64:0/3560529549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b3c101d10 msgr2=0x7f7b3c1030f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:05.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.181+0000 7f7b3b59e640 1 --2- 172.21.15.64:0/3560529549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b3c101d10 0x7f7b3c1030f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.181+0000 7f7b3b59e640 1 -- 172.21.15.64:0/3560529549 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b3c100b10 msgr2=0x7f7b3c104aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:05.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.181+0000 7f7b3b59e640 1 --2- 172.21.15.64:0/3560529549 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b3c100b10 0x7f7b3c104aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.181+0000 7f7b3b59e640 1 -- 172.21.15.64:0/3560529549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b3c103ef0 con 0x7f7b3c1026d0 2024-09-12T03:21:05.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.181+0000 7f7b3ad9d640 1 --2- 172.21.15.64:0/3560529549 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b3c100b10 0x7f7b3c104aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:05.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.181+0000 7f7b3b59e640 1 --2- 172.21.15.64:0/3560529549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b3c1026d0 0x7f7b3c103630 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7b3400e940 tx=0x7f7b3400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:05.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.181+0000 7f7b23fff640 1 -- 172.21.15.64:0/3560529549 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b34004d20 con 0x7f7b3c1026d0 2024-09-12T03:21:05.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.181+0000 7f7b23fff640 1 -- 172.21.15.64:0/3560529549 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b34005020 con 0x7f7b3c1026d0 2024-09-12T03:21:05.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.181+0000 7f7b411f0640 1 -- 172.21.15.64:0/3560529549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b3c1a1fa0 con 0x7f7b3c1026d0 2024-09-12T03:21:05.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.181+0000 7f7b23fff640 1 -- 172.21.15.64:0/3560529549 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b340116d0 con 0x7f7b3c1026d0 2024-09-12T03:21:05.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.181+0000 7f7b411f0640 1 -- 172.21.15.64:0/3560529549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b3c1a24b0 con 0x7f7b3c1026d0 2024-09-12T03:21:05.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.182+0000 7f7b23fff640 1 -- 172.21.15.64:0/3560529549 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f7b34011870 con 0x7f7b3c1026d0 2024-09-12T03:21:05.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.182+0000 7f7b411f0640 1 -- 172.21.15.64:0/3560529549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b3c100f10 con 0x7f7b3c1026d0 2024-09-12T03:21:05.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.183+0000 7f7b23fff640 1 --2- 172.21.15.64:0/3560529549 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f7b24039290 0x7f7b2403b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:05.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.183+0000 7f7b23fff640 1 -- 172.21.15.64:0/3560529549 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7b3404c480 con 0x7f7b3c1026d0 2024-09-12T03:21:05.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.183+0000 7f7b3ad9d640 1 --2- 172.21.15.64:0/3560529549 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f7b24039290 0x7f7b2403b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:05.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.183+0000 7f7b3ad9d640 1 --2- 172.21.15.64:0/3560529549 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f7b24039290 0x7f7b2403b750 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c032fd0 tx=0x7f7b2c005dd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:05.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.187+0000 7f7b23fff640 1 -- 172.21.15.64:0/3560529549 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7b34016030 con 0x7f7b3c1026d0 2024-09-12T03:21:05.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:05.372+0000 7f7b411f0640 1 -- 172.21.15.64:0/3560529549 --> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f7b3c06af90 con 0x7f7b24039290 2024-09-12T03:21:05.456 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:21:05 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:21:05.109+0000 7f74226eb040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T03:21:05.456 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:21:05 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:21:05.296+0000 7f74226eb040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T03:21:05.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:05 smithi195 ceph-mon[34786]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:05.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:05 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:05.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:05 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:05.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:05 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:05.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:05 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T03:21:05.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:05 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T03:21:05.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:05 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T03:21:05.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:05 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:05.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:05 smithi195 ceph-mon[34786]: Deploying daemon mgr.y on smithi115 2024-09-12T03:21:05.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:05 smithi064 ceph-mon[30952]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:05.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:05 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:05.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:05 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:05.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:05 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:05.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:05 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T03:21:05.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:05 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T03:21:05.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:05 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T03:21:05.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:05 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:05.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:05 smithi064 ceph-mon[30952]: Deploying daemon mgr.y on smithi115 2024-09-12T03:21:05.600 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:05 smithi115 ceph-mon[34801]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:05.600 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:05 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:05.600 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:05 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:05.600 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:05 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:05.600 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:05 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T03:21:05.600 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:05 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T03:21:05.600 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:05 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T03:21:05.601 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:05 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:05.601 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:05 smithi115 ceph-mon[34801]: Deploying daemon mgr.y on smithi115 2024-09-12T03:21:06.143 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:21:06 smithi115 systemd[1]: Starting Ceph mgr.y for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:21:06.309 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:06 smithi064 ceph-mon[30952]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:21:06.309 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:06 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:21:06.309 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:06 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:21:06.309 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:06 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:06.456 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:21:06 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:21:06.184+0000 7f74226eb040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T03:21:06.456 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:21:06 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:21:06.253+0000 7f74226eb040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T03:21:06.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:06 smithi195 ceph-mon[34786]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:21:06.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:06 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:21:06.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:06 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:21:06.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:06 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:06.465 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:06 smithi115 ceph-mon[34801]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:21:06.465 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:06 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:21:06.465 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:06 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:21:06.465 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:06 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:06.465 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:21:06 smithi115 podman[36153]: 2024-09-12 03:21:06.166471025 +0000 UTC m=+0.015627349 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:21:06.465 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:21:06 smithi115 podman[36153]: 2024-09-12 03:21:06.269010457 +0000 UTC m=+0.118166826 container create 4d9efb9334255324927f77dcf5ca7993488fd087ec41564f942947ee751ecdc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:21:06.466 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:21:06 smithi115 podman[36153]: 2024-09-12 03:21:06.463107763 +0000 UTC m=+0.312264100 container init 4d9efb9334255324927f77dcf5ca7993488fd087ec41564f942947ee751ecdc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-12T03:21:06.466 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:21:06 smithi115 podman[36153]: 2024-09-12 03:21:06.465869025 +0000 UTC m=+0.315025352 container start 4d9efb9334255324927f77dcf5ca7993488fd087ec41564f942947ee751ecdc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3) 2024-09-12T03:21:06.809 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:21:06 smithi115 ceph-mgr[36172]: -- 172.21.15.115:0/2747603163 <== mon.2 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fd671b90e0 con 0x55fd67197400 2024-09-12T03:21:06.809 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:21:06 smithi115 bash[36153]: 4d9efb9334255324927f77dcf5ca7993488fd087ec41564f942947ee751ecdc4 2024-09-12T03:21:06.809 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:21:06 smithi115 systemd[1]: Started Ceph mgr.y for b0812460-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:21:06.809 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:21:06 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:21:06.589+0000 7f91d5663040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T03:21:06.809 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:21:06 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:21:06.653+0000 7f91d5663040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T03:21:06.809 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:21:06 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:21:06.807+0000 7f91d5663040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T03:21:07.142 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:21:06 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:21:06.877+0000 7f91d5663040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T03:21:07.143 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:21:07 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:21:07.034+0000 7f91d5663040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:21:07.282 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:21:06 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:21:06.997+0000 7f74226eb040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T03:21:07.282 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:21:07 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:21:07.136+0000 7f74226eb040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T03:21:07.282 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:21:07 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:21:07.281+0000 7f74226eb040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:21:07.282 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:07 smithi195 ceph-mon[34786]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:07.282 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:07 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:07.282 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:07 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:07.282 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:07 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:07.283 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:07 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:07.283 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:07 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:21:07.446 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:21:07 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:21:07.190+0000 7f91d5663040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T03:21:07.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:07 smithi115 ceph-mon[34801]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:07.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:07 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:07.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:07 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:07.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:07 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:07.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:07 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:07.447 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:07 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:21:07.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:07 smithi064 ceph-mon[30952]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:07.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:07 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:07.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:07 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:07.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:07 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:07.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:07 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:07.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:07 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:21:07.567 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:21:07 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:21:07.344+0000 7f74226eb040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T03:21:07.567 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:21:07 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:21:07.502+0000 7f74226eb040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T03:21:07.784 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:21:07 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:21:07.525+0000 7f91d5663040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T03:21:07.784 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:21:07 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:21:07.588+0000 7f91d5663040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T03:21:07.868 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:21:07 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:21:07.573+0000 7f74226eb040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T03:21:07.868 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:21:07 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:21:07.635+0000 7f74226eb040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T03:21:07.868 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:21:07 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:21:07.796+0000 7f74226eb040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T03:21:07.868 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:21:07 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:21:07.867+0000 7f74226eb040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T03:21:08.142 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:21:07 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:21:07.935+0000 7f74226eb040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T03:21:08.143 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:21:08 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:21:08.022+0000 7f74226eb040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T03:21:08.143 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:21:08 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:21:08.081+0000 7f74226eb040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T03:21:08.143 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:21:08 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:21:08.141+0000 7f74226eb040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T03:21:08.143 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:21:07 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:21:07.783+0000 7f91d5663040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T03:21:08.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:08 smithi195 ceph-mon[34786]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:08.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:08 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:08.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:08 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/1063005157' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "326271c9-1f28-4ce6-b3be-fcc57479d8b7"}]: dispatch 2024-09-12T03:21:08.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:08 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/1063005157' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "326271c9-1f28-4ce6-b3be-fcc57479d8b7"}]': finished 2024-09-12T03:21:08.708 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:08 smithi195 ceph-mon[34786]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T03:21:08.708 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:08 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:21:08.708 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:08 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:08.708 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:08 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:08.708 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:08 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:08.709 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:08 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:08.709 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:08 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:08.709 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:08 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:21:08.709 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:08 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:08.709 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:08 smithi195 ceph-mon[34786]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T03:21:08.710 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:08 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T03:21:08.710 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:08 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T03:21:08.710 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:08 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:08.710 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:08 smithi195 ceph-mon[34786]: Reconfiguring daemon mgr.x on smithi064 2024-09-12T03:21:08.710 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:08 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/3113348862' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:21:08.736 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:21:08 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:21:08.670+0000 7f91d5663040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T03:21:08.737 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:21:08 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:21:08.735+0000 7f91d5663040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T03:21:08.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:08 smithi115 ceph-mon[34801]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:08.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:08 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:08.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:08 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/1063005157' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "326271c9-1f28-4ce6-b3be-fcc57479d8b7"}]: dispatch 2024-09-12T03:21:08.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:08 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/1063005157' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "326271c9-1f28-4ce6-b3be-fcc57479d8b7"}]': finished 2024-09-12T03:21:08.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:08 smithi115 ceph-mon[34801]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T03:21:08.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:08 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:21:08.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:08 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:08.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:08 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:08.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:08 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:08.739 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:08 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:08.739 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:08 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:08.739 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:08 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:21:08.739 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:08 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:08.739 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:08 smithi115 ceph-mon[34801]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T03:21:08.739 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:08 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T03:21:08.739 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:08 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T03:21:08.740 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:08 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:08.740 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:08 smithi115 ceph-mon[34801]: Reconfiguring daemon mgr.x on smithi064 2024-09-12T03:21:08.740 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:08 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/3113348862' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:21:08.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:08 smithi064 ceph-mon[30952]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:08.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:08 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:08.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:08 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/1063005157' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "326271c9-1f28-4ce6-b3be-fcc57479d8b7"}]: dispatch 2024-09-12T03:21:08.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:08 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/1063005157' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "326271c9-1f28-4ce6-b3be-fcc57479d8b7"}]': finished 2024-09-12T03:21:08.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:08 smithi064 ceph-mon[30952]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T03:21:08.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:08 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:21:08.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:08 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:08.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:08 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:08.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:08 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:08.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:08 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:08.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:08 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:08.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:08 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:21:08.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:08 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:08.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:08 smithi064 ceph-mon[30952]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T03:21:08.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:08 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T03:21:08.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:08 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T03:21:08.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:08 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:08.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:08 smithi064 ceph-mon[30952]: Reconfiguring daemon mgr.x on smithi064 2024-09-12T03:21:08.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:08 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/3113348862' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:21:09.206 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:21:08 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:21:08.932+0000 7f74226eb040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T03:21:09.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:09.469+0000 7f7b23fff640 1 -- 172.21.15.64:0/3560529549 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7f7b340108f0 con 0x7f7b3c1026d0 2024-09-12T03:21:09.701 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:09 smithi115 ceph-mon[34801]: Standby manager daemon z started 2024-09-12T03:21:09.701 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:09 smithi115 ceph-mon[34801]: from='mgr.? 172.21.15.195:0/750929331' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T03:21:09.702 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:09 smithi115 ceph-mon[34801]: from='mgr.? 172.21.15.195:0/750929331' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:21:09.702 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:09 smithi115 ceph-mon[34801]: from='mgr.? 172.21.15.195:0/750929331' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T03:21:09.702 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:09 smithi115 ceph-mon[34801]: from='mgr.? 172.21.15.195:0/750929331' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:21:09.702 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:21:09 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:21:09.443+0000 7f91d5663040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T03:21:09.702 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:21:09 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:21:09.559+0000 7f91d5663040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T03:21:09.702 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:21:09 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:21:09.700+0000 7f91d5663040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:21:09.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:09 smithi064 ceph-mon[30952]: Standby manager daemon z started 2024-09-12T03:21:09.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:09 smithi064 ceph-mon[30952]: from='mgr.? 172.21.15.195:0/750929331' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T03:21:09.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:09 smithi064 ceph-mon[30952]: from='mgr.? 172.21.15.195:0/750929331' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:21:09.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:09 smithi064 ceph-mon[30952]: from='mgr.? 172.21.15.195:0/750929331' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T03:21:09.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:09 smithi064 ceph-mon[30952]: from='mgr.? 172.21.15.195:0/750929331' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:21:09.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:09 smithi195 ceph-mon[34786]: Standby manager daemon z started 2024-09-12T03:21:09.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:09 smithi195 ceph-mon[34786]: from='mgr.? 172.21.15.195:0/750929331' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T03:21:09.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:09 smithi195 ceph-mon[34786]: from='mgr.? 172.21.15.195:0/750929331' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:21:09.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:09 smithi195 ceph-mon[34786]: from='mgr.? 172.21.15.195:0/750929331' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T03:21:09.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:09 smithi195 ceph-mon[34786]: from='mgr.? 172.21.15.195:0/750929331' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:21:09.974 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:21:09 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:21:09.761+0000 7f91d5663040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T03:21:09.974 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:21:09 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:21:09.907+0000 7f91d5663040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T03:21:09.975 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:21:09 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:21:09.973+0000 7f91d5663040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T03:21:10.257 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:21:10 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:21:10.034+0000 7f91d5663040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T03:21:10.257 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:21:10 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:21:10.191+0000 7f91d5663040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T03:21:10.257 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:21:10 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:21:10.255+0000 7f91d5663040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T03:21:10.525 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:10 smithi115 ceph-mon[34801]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:10.526 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:10 smithi115 ceph-mon[34801]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-12T03:21:10.526 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:10 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T03:21:10.526 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:21:10 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:21:10.323+0000 7f91d5663040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T03:21:10.526 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:21:10 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:21:10.405+0000 7f91d5663040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T03:21:10.526 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:21:10 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:21:10.464+0000 7f91d5663040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T03:21:10.526 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:21:10 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:21:10.524+0000 7f91d5663040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T03:21:10.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:10 smithi064 ceph-mon[30952]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:10.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:10 smithi064 ceph-mon[30952]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-12T03:21:10.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:10 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T03:21:10.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:10 smithi195 ceph-mon[34786]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:10.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:10 smithi195 ceph-mon[34786]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-12T03:21:10.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:10 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T03:21:11.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:11.477+0000 7f7b23fff640 1 -- 172.21.15.64:0/3560529549 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f7b340172b0 con 0x7f7b3c1026d0 2024-09-12T03:21:11.642 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:21:11 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:21:11.306+0000 7f91d5663040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T03:21:11.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:11 smithi115 ceph-mon[34801]: Standby manager daemon y started 2024-09-12T03:21:11.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:11 smithi115 ceph-mon[34801]: from='mgr.? 172.21.15.115:0/4243194548' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T03:21:11.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:11 smithi115 ceph-mon[34801]: from='mgr.? 172.21.15.115:0/4243194548' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:21:11.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:11 smithi115 ceph-mon[34801]: from='mgr.? 172.21.15.115:0/4243194548' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T03:21:11.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:11 smithi115 ceph-mon[34801]: from='mgr.? 172.21.15.115:0/4243194548' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:21:11.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:11 smithi064 ceph-mon[30952]: Standby manager daemon y started 2024-09-12T03:21:11.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:11 smithi064 ceph-mon[30952]: from='mgr.? 172.21.15.115:0/4243194548' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T03:21:11.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:11 smithi064 ceph-mon[30952]: from='mgr.? 172.21.15.115:0/4243194548' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:21:11.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:11 smithi064 ceph-mon[30952]: from='mgr.? 172.21.15.115:0/4243194548' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T03:21:11.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:11 smithi064 ceph-mon[30952]: from='mgr.? 172.21.15.115:0/4243194548' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:21:11.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:11 smithi195 ceph-mon[34786]: Standby manager daemon y started 2024-09-12T03:21:11.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:11 smithi195 ceph-mon[34786]: from='mgr.? 172.21.15.115:0/4243194548' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T03:21:11.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:11 smithi195 ceph-mon[34786]: from='mgr.? 172.21.15.115:0/4243194548' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:21:11.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:11 smithi195 ceph-mon[34786]: from='mgr.? 172.21.15.115:0/4243194548' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T03:21:11.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:11 smithi195 ceph-mon[34786]: from='mgr.? 172.21.15.115:0/4243194548' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:21:12.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:12 smithi064 ceph-mon[30952]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:12.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:12 smithi064 ceph-mon[30952]: mgrmap e15: x(active, since 64s), standbys: z, y 2024-09-12T03:21:12.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:12 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T03:21:12.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:12 smithi115 ceph-mon[34801]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:12.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:12 smithi115 ceph-mon[34801]: mgrmap e15: x(active, since 64s), standbys: z, y 2024-09-12T03:21:12.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:12 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T03:21:12.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:12 smithi195 ceph-mon[34786]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:12.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:12 smithi195 ceph-mon[34786]: mgrmap e15: x(active, since 64s), standbys: z, y 2024-09-12T03:21:12.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:12 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T03:21:15.391 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:15 smithi064 ceph-mon[30952]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:15.392 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:15 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:15.392 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:15 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:15.392 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:15 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:21:15.392 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:15 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:15.392 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:15 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:21:15.392 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:15 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:15.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:15 smithi115 ceph-mon[34801]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:15.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:15 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:15.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:15 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:15.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:15 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:21:15.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:15 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:15.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:15 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:21:15.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:15 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:15.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:15 smithi195 ceph-mon[34786]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:15.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:15 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:15.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:15 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:15.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:15 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:21:15.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:15 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:15.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:15 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:21:15.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:15 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:17.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:17 smithi064 ceph-mon[30952]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:17.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:17 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T03:21:17.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:17 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:17.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:17 smithi064 ceph-mon[30952]: Deploying daemon osd.0 on smithi064 2024-09-12T03:21:17.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:17 smithi115 ceph-mon[34801]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:17.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:17 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T03:21:17.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:17 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:17.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:17 smithi115 ceph-mon[34801]: Deploying daemon osd.0 on smithi064 2024-09-12T03:21:17.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:17 smithi195 ceph-mon[34786]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:17.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:17 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T03:21:17.459 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:17 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:17.459 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:17 smithi195 ceph-mon[34786]: Deploying daemon osd.0 on smithi064 2024-09-12T03:21:19.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:19 smithi064 ceph-mon[30952]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:19.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:19 smithi115 ceph-mon[34801]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:19.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:19 smithi195 ceph-mon[34786]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:21.260 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:21 smithi064 ceph-mon[30952]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:21.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:21 smithi115 ceph-mon[34801]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:21.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:21 smithi195 ceph-mon[34786]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:23.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:23 smithi115 ceph-mon[34801]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:23.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:23 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:21:23.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:23 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:23.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:23 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:23.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:23 smithi195 ceph-mon[34786]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:23.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:23 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:21:23.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:23 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:23.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:23 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:23.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:23 smithi064 ceph-mon[30952]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:23.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:23 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:21:23.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:23 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:23.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:23 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:25.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:25 smithi115 ceph-mon[34801]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:25.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:25 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:25.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:25 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:25.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:25 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:25.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:25 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:21:25.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:25 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:25.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:25 smithi195 ceph-mon[34786]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:25.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:25 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:25.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:25 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:25.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:25 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:25.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:25 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:21:25.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:25 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:25.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:25 smithi064 ceph-mon[30952]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:25.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:25 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:25.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:25 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:25.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:25 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:25.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:25 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:21:25.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:25 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:26.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:26.266+0000 7f7b23fff640 1 -- 172.21.15.64:0/3560529549 <== mgr.14150 v2:172.21.15.64:6800/759931848 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7b3c06af90 con 0x7f7b24039290 2024-09-12T03:21:26.267 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 0 on host 'smithi064' 2024-09-12T03:21:26.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:26.269+0000 7f7b411f0640 1 -- 172.21.15.64:0/3560529549 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f7b24039290 msgr2=0x7f7b2403b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:26.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:26.269+0000 7f7b411f0640 1 --2- 172.21.15.64:0/3560529549 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f7b24039290 0x7f7b2403b750 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c032fd0 tx=0x7f7b2c005dd0 comp rx=0 tx=0).stop 2024-09-12T03:21:26.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:26.269+0000 7f7b411f0640 1 -- 172.21.15.64:0/3560529549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b3c1026d0 msgr2=0x7f7b3c103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:26.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:26.269+0000 7f7b411f0640 1 --2- 172.21.15.64:0/3560529549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b3c1026d0 0x7f7b3c103630 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7b3400e940 tx=0x7f7b3400ee00 comp rx=0 tx=0).stop 2024-09-12T03:21:26.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:26.269+0000 7f7b411f0640 1 -- 172.21.15.64:0/3560529549 shutdown_connections 2024-09-12T03:21:26.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:26.269+0000 7f7b411f0640 1 --2- 172.21.15.64:0/3560529549 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f7b24039290 0x7f7b2403b750 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:26.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:26.269+0000 7f7b411f0640 1 --2- 172.21.15.64:0/3560529549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b3c1026d0 0x7f7b3c103630 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:26.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:26.269+0000 7f7b411f0640 1 --2- 172.21.15.64:0/3560529549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b3c101d10 0x7f7b3c1030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:26.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:26.269+0000 7f7b411f0640 1 --2- 172.21.15.64:0/3560529549 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b3c100b10 0x7f7b3c104aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:26.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:26.269+0000 7f7b411f0640 1 -- 172.21.15.64:0/3560529549 >> 172.21.15.64:0/3560529549 conn(0x7f7b3c0fc2a0 msgr2=0x7f7b3c06a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:26.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:26.270+0000 7f7b411f0640 1 -- 172.21.15.64:0/3560529549 shutdown_connections 2024-09-12T03:21:26.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:21:26.270+0000 7f7b411f0640 1 -- 172.21.15.64:0/3560529549 wait complete. 2024-09-12T03:21:26.890 DEBUG:teuthology.orchestra.run.smithi064:osd.0> sudo journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@osd.0.service 2024-09-12T03:21:26.893 INFO:tasks.cephadm:Deploying osd.1 on smithi115 with /dev/vg_nvme/lv_4... 2024-09-12T03:21:26.893 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T03:21:27.132 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:21:27.190 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:27 smithi064 ceph-mon[30952]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:27.191 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:27 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:21:27.191 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:27 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:27.191 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:27 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:27.253 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:27 smithi115 ceph-mon[34801]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:27.253 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:27 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:21:27.253 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:27 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:27.253 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:27 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:27.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:27 smithi195 ceph-mon[34786]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:27.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:27 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:21:27.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:27 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:27.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:27 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:28.247 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 12 03:21:28 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-0[45113]: 2024-09-12T03:21:28.141+0000 7faa07ca5640 -1 osd.0 0 log_to_monitors true 2024-09-12T03:21:29.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:29 smithi064 ceph-mon[30952]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:29.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:29 smithi064 ceph-mon[30952]: from='osd.0 [v2:172.21.15.64:6802/1307832590,v1:172.21.15.64:6803/1307832590]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T03:21:29.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:29 smithi115 ceph-mon[34801]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:29.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:29 smithi115 ceph-mon[34801]: from='osd.0 [v2:172.21.15.64:6802/1307832590,v1:172.21.15.64:6803/1307832590]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T03:21:29.420 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T03:21:29.446 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi115:vg_nvme/lv_4 2024-09-12T03:21:29.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:29 smithi195 ceph-mon[34786]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:29.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:29 smithi195 ceph-mon[34786]: from='osd.0 [v2:172.21.15.64:6802/1307832590,v1:172.21.15.64:6803/1307832590]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T03:21:29.694 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:21:30.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:30.130+0000 7fd46a661640 1 -- 172.21.15.115:0/1031212269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd464074c50 msgr2=0x7fd4640750b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:30.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:30.130+0000 7fd46a661640 1 --2- 172.21.15.115:0/1031212269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd464074c50 0x7fd4640750b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd458009900 tx=0x7fd45802a410 comp rx=0 tx=0).stop 2024-09-12T03:21:30.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:30.130+0000 7fd46a661640 1 -- 172.21.15.115:0/1031212269 shutdown_connections 2024-09-12T03:21:30.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:30.130+0000 7fd46a661640 1 --2- 172.21.15.115:0/1031212269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd46410e1e0 0x7fd4641105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:30.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:30.130+0000 7fd46a661640 1 --2- 172.21.15.115:0/1031212269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd464074c50 0x7fd4640750b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:30.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:30.130+0000 7fd46a661640 1 --2- 172.21.15.115:0/1031212269 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd464076220 0x7fd464074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:30.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:30.130+0000 7fd46a661640 1 -- 172.21.15.115:0/1031212269 >> 172.21.15.115:0/1031212269 conn(0x7fd4640fd390 msgr2=0x7fd4640ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:30.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:30.130+0000 7fd46a661640 1 -- 172.21.15.115:0/1031212269 shutdown_connections 2024-09-12T03:21:30.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:30.130+0000 7fd46a661640 1 -- 172.21.15.115:0/1031212269 wait complete. 2024-09-12T03:21:30.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:30.130+0000 7fd46a661640 1 Processor -- start 2024-09-12T03:21:30.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:30.130+0000 7fd46a661640 1 -- start start 2024-09-12T03:21:30.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:30.131+0000 7fd46a661640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd464074c50 0x7fd46419e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:30.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:30.131+0000 7fd46a661640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd464076220 0x7fd46419e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:30.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:30.131+0000 7fd46a661640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd46410e1e0 0x7fd4641a5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:30.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:30.131+0000 7fd46a661640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd464113d50 con 0x7fd46410e1e0 2024-09-12T03:21:30.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:30.131+0000 7fd46a661640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd464113bd0 con 0x7fd464076220 2024-09-12T03:21:30.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:30.131+0000 7fd46a661640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd464113ed0 con 0x7fd464074c50 2024-09-12T03:21:30.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:30.131+0000 7fd4637fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd464076220 0x7fd46419e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:30.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:30.131+0000 7fd4637fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd464076220 0x7fd46419e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58774/0 (socket says 172.21.15.115:58774) 2024-09-12T03:21:30.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:30.131+0000 7fd4637fe640 1 -- 172.21.15.115:0/4276047252 learned_addr learned my addr 172.21.15.115:0/4276047252 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-09-12T03:21:30.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:30.131+0000 7fd468bfe640 1 --2- 172.21.15.115:0/4276047252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd46410e1e0 0x7fd4641a5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:30.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:30.131+0000 7fd463fff640 1 --2- 172.21.15.115:0/4276047252 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd464074c50 0x7fd46419e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:30.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:30.131+0000 7fd4637fe640 1 -- 172.21.15.115:0/4276047252 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd464074c50 msgr2=0x7fd46419e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:30.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:30.131+0000 7fd4637fe640 1 --2- 172.21.15.115:0/4276047252 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd464074c50 0x7fd46419e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:30.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:30.131+0000 7fd4637fe640 1 -- 172.21.15.115:0/4276047252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd46410e1e0 msgr2=0x7fd4641a5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:30.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:30.131+0000 7fd4637fe640 1 --2- 172.21.15.115:0/4276047252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd46410e1e0 0x7fd4641a5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:30.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:30.131+0000 7fd4637fe640 1 -- 172.21.15.115:0/4276047252 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4641a60e0 con 0x7fd464076220 2024-09-12T03:21:30.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:30.131+0000 7fd4637fe640 1 --2- 172.21.15.115:0/4276047252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd464076220 0x7fd46419e9f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd458004770 tx=0x7fd458004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:30.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:30.132+0000 7fd4617fa640 1 -- 172.21.15.115:0/4276047252 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45803f030 con 0x7fd464076220 2024-09-12T03:21:30.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:30.132+0000 7fd4617fa640 1 -- 172.21.15.115:0/4276047252 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd458003aa0 con 0x7fd464076220 2024-09-12T03:21:30.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:30.132+0000 7fd46a661640 1 -- 172.21.15.115:0/4276047252 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4641a6370 con 0x7fd464076220 2024-09-12T03:21:30.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:30.132+0000 7fd4617fa640 1 -- 172.21.15.115:0/4276047252 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd458035070 con 0x7fd464076220 2024-09-12T03:21:30.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:30.132+0000 7fd46a661640 1 -- 172.21.15.115:0/4276047252 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4641a68a0 con 0x7fd464076220 2024-09-12T03:21:30.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:30.133+0000 7fd46a661640 1 -- 172.21.15.115:0/4276047252 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4641043b0 con 0x7fd464076220 2024-09-12T03:21:30.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:30.134+0000 7fd4617fa640 1 -- 172.21.15.115:0/4276047252 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fd458042030 con 0x7fd464076220 2024-09-12T03:21:30.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:30.135+0000 7fd4617fa640 1 --2- 172.21.15.115:0/4276047252 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fd44c0a2ba0 0x7fd44c0a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:30.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:30.135+0000 7fd4617fa640 1 -- 172.21.15.115:0/4276047252 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7fd4580edad0 con 0x7fd464076220 2024-09-12T03:21:30.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:30.135+0000 7fd463fff640 1 --2- 172.21.15.115:0/4276047252 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fd44c0a2ba0 0x7fd44c0a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:30.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:30.135+0000 7fd463fff640 1 --2- 172.21.15.115:0/4276047252 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fd44c0a2ba0 0x7fd44c0a5060 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd454005d00 tx=0x7fd45400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:30.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:30.139+0000 7fd4617fa640 1 -- 172.21.15.115:0/4276047252 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd4580bbf70 con 0x7fd464076220 2024-09-12T03:21:30.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:30.311+0000 7fd46a661640 1 -- 172.21.15.115:0/4276047252 --> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fd464062630 con 0x7fd44c0a2ba0 2024-09-12T03:21:30.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:30 smithi115 ceph-mon[34801]: from='osd.0 [v2:172.21.15.64:6802/1307832590,v1:172.21.15.64:6803/1307832590]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T03:21:30.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:30 smithi115 ceph-mon[34801]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T03:21:30.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:30 smithi115 ceph-mon[34801]: from='osd.0 [v2:172.21.15.64:6802/1307832590,v1:172.21.15.64:6803/1307832590]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-12T03:21:30.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:30 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:21:30.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:30 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:30.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:30 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:30.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:30 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:21:30.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:30 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:30.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:30 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:30.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:30 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:21:30.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:30 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:30.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:30 smithi195 ceph-mon[34786]: from='osd.0 [v2:172.21.15.64:6802/1307832590,v1:172.21.15.64:6803/1307832590]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T03:21:30.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:30 smithi195 ceph-mon[34786]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T03:21:30.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:30 smithi195 ceph-mon[34786]: from='osd.0 [v2:172.21.15.64:6802/1307832590,v1:172.21.15.64:6803/1307832590]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-12T03:21:30.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:30 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:21:30.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:30 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:30.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:30 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:30.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:30 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:21:30.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:30 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:30.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:30 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:30.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:30 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:21:30.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:30 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:30.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:30 smithi064 ceph-mon[30952]: from='osd.0 [v2:172.21.15.64:6802/1307832590,v1:172.21.15.64:6803/1307832590]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T03:21:30.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:30 smithi064 ceph-mon[30952]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T03:21:30.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:30 smithi064 ceph-mon[30952]: from='osd.0 [v2:172.21.15.64:6802/1307832590,v1:172.21.15.64:6803/1307832590]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-12T03:21:30.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:30 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:21:30.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:30 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:30.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:30 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:30.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:30 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:21:30.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:30 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:30.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:30 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:30.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:30 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:21:30.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:30 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:31.022 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 12 03:21:30 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-0[45113]: 2024-09-12T03:21:30.540+0000 7faa0443c640 -1 osd.0 0 waiting for initial osdmap 2024-09-12T03:21:31.022 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 12 03:21:30 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-0[45113]: 2024-09-12T03:21:30.543+0000 7fa9fba48640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:21:31.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:31 smithi115 ceph-mon[34801]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:31.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:31 smithi115 ceph-mon[34801]: Detected new or changed devices on smithi064 2024-09-12T03:21:31.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:31 smithi115 ceph-mon[34801]: Adjusting osd_memory_target on smithi064 to 17088M 2024-09-12T03:21:31.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:31 smithi115 ceph-mon[34801]: from='osd.0 [v2:172.21.15.64:6802/1307832590,v1:172.21.15.64:6803/1307832590]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-12T03:21:31.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:31 smithi115 ceph-mon[34801]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T03:21:31.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:31 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:21:31.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:31 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:21:31.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:31 smithi115 ceph-mon[34801]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:21:31.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:31 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:21:31.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:31 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:21:31.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:31 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:31.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:31 smithi115 ceph-mon[34801]: from='osd.0 [v2:172.21.15.64:6802/1307832590,v1:172.21.15.64:6803/1307832590]' entity='osd.0' 2024-09-12T03:21:31.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:31 smithi195 ceph-mon[34786]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:31.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:31 smithi195 ceph-mon[34786]: Detected new or changed devices on smithi064 2024-09-12T03:21:31.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:31 smithi195 ceph-mon[34786]: Adjusting osd_memory_target on smithi064 to 17088M 2024-09-12T03:21:31.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:31 smithi195 ceph-mon[34786]: from='osd.0 [v2:172.21.15.64:6802/1307832590,v1:172.21.15.64:6803/1307832590]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-12T03:21:31.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:31 smithi195 ceph-mon[34786]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T03:21:31.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:31 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:21:31.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:31 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:21:31.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:31 smithi195 ceph-mon[34786]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:21:31.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:31 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:21:31.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:31 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:21:31.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:31 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:31.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:31 smithi195 ceph-mon[34786]: from='osd.0 [v2:172.21.15.64:6802/1307832590,v1:172.21.15.64:6803/1307832590]' entity='osd.0' 2024-09-12T03:21:31.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:31 smithi064 ceph-mon[30952]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:31.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:31 smithi064 ceph-mon[30952]: Detected new or changed devices on smithi064 2024-09-12T03:21:31.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:31 smithi064 ceph-mon[30952]: Adjusting osd_memory_target on smithi064 to 17088M 2024-09-12T03:21:31.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:31 smithi064 ceph-mon[30952]: from='osd.0 [v2:172.21.15.64:6802/1307832590,v1:172.21.15.64:6803/1307832590]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-12T03:21:31.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:31 smithi064 ceph-mon[30952]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T03:21:31.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:31 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:21:31.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:31 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:21:31.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:31 smithi064 ceph-mon[30952]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:21:31.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:31 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:21:31.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:31 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:21:31.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:31 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:31.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:31 smithi064 ceph-mon[30952]: from='osd.0 [v2:172.21.15.64:6802/1307832590,v1:172.21.15.64:6803/1307832590]' entity='osd.0' 2024-09-12T03:21:32.389 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:32 smithi115 ceph-mon[34801]: purged_snaps scrub starts 2024-09-12T03:21:32.389 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:32 smithi115 ceph-mon[34801]: purged_snaps scrub ok 2024-09-12T03:21:32.389 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:32 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:21:32.389 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:32 smithi115 ceph-mon[34801]: osd.0 [v2:172.21.15.64:6802/1307832590,v1:172.21.15.64:6803/1307832590] boot 2024-09-12T03:21:32.390 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:32 smithi115 ceph-mon[34801]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T03:21:32.390 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:32 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:21:32.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:32 smithi195 ceph-mon[34786]: purged_snaps scrub starts 2024-09-12T03:21:32.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:32 smithi195 ceph-mon[34786]: purged_snaps scrub ok 2024-09-12T03:21:32.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:32 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:21:32.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:32 smithi195 ceph-mon[34786]: osd.0 [v2:172.21.15.64:6802/1307832590,v1:172.21.15.64:6803/1307832590] boot 2024-09-12T03:21:32.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:32 smithi195 ceph-mon[34786]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T03:21:32.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:32 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:21:32.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:32 smithi064 ceph-mon[30952]: purged_snaps scrub starts 2024-09-12T03:21:32.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:32 smithi064 ceph-mon[30952]: purged_snaps scrub ok 2024-09-12T03:21:32.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:32 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:21:32.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:32 smithi064 ceph-mon[30952]: osd.0 [v2:172.21.15.64:6802/1307832590,v1:172.21.15.64:6803/1307832590] boot 2024-09-12T03:21:32.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:32 smithi064 ceph-mon[30952]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T03:21:32.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:32 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:21:33.245 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:33 smithi115 ceph-mon[34801]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:33.245 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:33 smithi115 ceph-mon[34801]: osdmap e9: 1 total, 1 up, 1 in 2024-09-12T03:21:33.245 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:33 smithi115 ceph-mon[34801]: from='client.? 172.21.15.115:0/444171527' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7dca0c28-c234-423c-936e-2351d63f918e"}]: dispatch 2024-09-12T03:21:33.245 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:33 smithi115 ceph-mon[34801]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7dca0c28-c234-423c-936e-2351d63f918e"}]: dispatch 2024-09-12T03:21:33.245 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:33 smithi115 ceph-mon[34801]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7dca0c28-c234-423c-936e-2351d63f918e"}]': finished 2024-09-12T03:21:33.245 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:33 smithi115 ceph-mon[34801]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T03:21:33.245 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:33 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:21:33.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:33 smithi195 ceph-mon[34786]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:33.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:33 smithi195 ceph-mon[34786]: osdmap e9: 1 total, 1 up, 1 in 2024-09-12T03:21:33.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:33 smithi195 ceph-mon[34786]: from='client.? 172.21.15.115:0/444171527' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7dca0c28-c234-423c-936e-2351d63f918e"}]: dispatch 2024-09-12T03:21:33.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:33 smithi195 ceph-mon[34786]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7dca0c28-c234-423c-936e-2351d63f918e"}]: dispatch 2024-09-12T03:21:33.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:33 smithi195 ceph-mon[34786]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7dca0c28-c234-423c-936e-2351d63f918e"}]': finished 2024-09-12T03:21:33.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:33 smithi195 ceph-mon[34786]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T03:21:33.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:33 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:21:33.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:33 smithi064 ceph-mon[30952]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:21:33.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:33 smithi064 ceph-mon[30952]: osdmap e9: 1 total, 1 up, 1 in 2024-09-12T03:21:33.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:33 smithi064 ceph-mon[30952]: from='client.? 172.21.15.115:0/444171527' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7dca0c28-c234-423c-936e-2351d63f918e"}]: dispatch 2024-09-12T03:21:33.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:33 smithi064 ceph-mon[30952]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7dca0c28-c234-423c-936e-2351d63f918e"}]: dispatch 2024-09-12T03:21:33.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:33 smithi064 ceph-mon[30952]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7dca0c28-c234-423c-936e-2351d63f918e"}]': finished 2024-09-12T03:21:33.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:33 smithi064 ceph-mon[30952]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T03:21:33.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:33 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:21:34.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:34 smithi115 ceph-mon[34801]: from='client.? 172.21.15.115:0/1078028044' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:21:34.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:34 smithi195 ceph-mon[34786]: from='client.? 172.21.15.115:0/1078028044' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:21:34.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:34 smithi064 ceph-mon[30952]: from='client.? 172.21.15.115:0/1078028044' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:21:35.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:35 smithi115 ceph-mon[34801]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:21:35.423 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:35 smithi064 ceph-mon[30952]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:21:35.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:35 smithi195 ceph-mon[34786]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:21:37.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:37 smithi115 ceph-mon[34801]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:21:37.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:37 smithi195 ceph-mon[34786]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:21:37.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:37 smithi064 ceph-mon[30952]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:21:39.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:39 smithi195 ceph-mon[34786]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:21:39.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:39 smithi064 ceph-mon[30952]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:21:39.531 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:39 smithi115 ceph-mon[34801]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:21:41.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:41 smithi195 ceph-mon[34786]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:21:41.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:41 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T03:21:41.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:41 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:41.475 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:41 smithi115 ceph-mon[34801]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:21:41.475 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:41 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T03:21:41.475 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:41 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:41.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:41 smithi064 ceph-mon[30952]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:21:41.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:41 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T03:21:41.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:41 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:42.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:42 smithi195 ceph-mon[34786]: Deploying daemon osd.1 on smithi115 2024-09-12T03:21:42.468 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:42 smithi115 ceph-mon[34801]: Deploying daemon osd.1 on smithi115 2024-09-12T03:21:42.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:42 smithi064 ceph-mon[30952]: Deploying daemon osd.1 on smithi115 2024-09-12T03:21:43.433 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:43 smithi115 ceph-mon[34801]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:21:43.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:43 smithi195 ceph-mon[34786]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:21:43.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:43 smithi064 ceph-mon[30952]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:21:45.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:45 smithi195 ceph-mon[34786]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:21:45.477 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:45 smithi064 ceph-mon[30952]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:21:45.593 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:45 smithi115 ceph-mon[34801]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:21:47.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:47 smithi115 ceph-mon[34801]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:21:47.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:47 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:21:47.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:47 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:47.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:47 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:47.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:47 smithi195 ceph-mon[34786]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:21:47.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:47 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:21:47.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:47 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:47.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:47 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:47.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:47 smithi064 ceph-mon[30952]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:21:47.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:47 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:21:47.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:47 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:47.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:47 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:49.374 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:49 smithi115 ceph-mon[34801]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:21:49.375 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:49 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:49.375 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:49 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:49.375 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:49 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:49.375 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:49 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:21:49.375 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:49 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:49.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:49 smithi195 ceph-mon[34786]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:21:49.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:49 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:49.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:49 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:49.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:49 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:49.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:49 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:21:49.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:49 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:49.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:49 smithi064 ceph-mon[30952]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:21:49.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:49 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:49.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:49 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:49.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:49 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:49.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:49 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:21:49.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:49 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:51.268 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:51 smithi115 ceph-mon[34801]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:21:51.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:51.307+0000 7fd4617fa640 1 -- 172.21.15.115:0/4276047252 <== mgr.14150 v2:172.21.15.64:6800/759931848 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd464062630 con 0x7fd44c0a2ba0 2024-09-12T03:21:51.308 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 1 on host 'smithi115' 2024-09-12T03:21:51.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:51.309+0000 7fd46a661640 1 -- 172.21.15.115:0/4276047252 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fd44c0a2ba0 msgr2=0x7fd44c0a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:51.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:51.309+0000 7fd46a661640 1 --2- 172.21.15.115:0/4276047252 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fd44c0a2ba0 0x7fd44c0a5060 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd454005d00 tx=0x7fd45400b410 comp rx=0 tx=0).stop 2024-09-12T03:21:51.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:51.309+0000 7fd46a661640 1 -- 172.21.15.115:0/4276047252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd464076220 msgr2=0x7fd46419e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:51.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:51.309+0000 7fd46a661640 1 --2- 172.21.15.115:0/4276047252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd464076220 0x7fd46419e9f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd458004770 tx=0x7fd458004270 comp rx=0 tx=0).stop 2024-09-12T03:21:51.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:51.309+0000 7fd46a661640 1 -- 172.21.15.115:0/4276047252 shutdown_connections 2024-09-12T03:21:51.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:51.309+0000 7fd46a661640 1 --2- 172.21.15.115:0/4276047252 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fd44c0a2ba0 0x7fd44c0a5060 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:51.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:51.309+0000 7fd46a661640 1 --2- 172.21.15.115:0/4276047252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd46410e1e0 0x7fd4641a5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:51.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:51.309+0000 7fd46a661640 1 --2- 172.21.15.115:0/4276047252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd464076220 0x7fd46419e9f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:51.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:51.309+0000 7fd46a661640 1 --2- 172.21.15.115:0/4276047252 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd464074c50 0x7fd46419e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:51.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:51.309+0000 7fd46a661640 1 -- 172.21.15.115:0/4276047252 >> 172.21.15.115:0/4276047252 conn(0x7fd4640fd390 msgr2=0x7fd464101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:51.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:51.309+0000 7fd46a661640 1 -- 172.21.15.115:0/4276047252 shutdown_connections 2024-09-12T03:21:51.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:21:51.310+0000 7fd46a661640 1 -- 172.21.15.115:0/4276047252 wait complete. 2024-09-12T03:21:51.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:51 smithi195 ceph-mon[34786]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:21:51.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:51 smithi064 ceph-mon[30952]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:21:51.898 DEBUG:teuthology.orchestra.run.smithi115:osd.1> sudo journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@osd.1.service 2024-09-12T03:21:51.901 INFO:tasks.cephadm:Deploying osd.2 on smithi195 with /dev/vg_nvme/lv_4... 2024-09-12T03:21:51.901 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T03:21:52.150 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:21:52.176 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:52 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:21:52.176 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:52 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:52.176 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:52 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:52.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:52 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:21:52.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:52 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:52.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:52 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:52.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:52 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:21:52.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:52 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:52.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:52 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:53.440 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:53 smithi195 ceph-mon[34786]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:21:53.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:53 smithi115 ceph-mon[34801]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:21:53.643 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 12 03:21:53 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-1[39262]: 2024-09-12T03:21:53.305+0000 7f4fc9199640 -1 osd.1 0 log_to_monitors true 2024-09-12T03:21:53.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:53 smithi064 ceph-mon[30952]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:21:54.514 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-12T03:21:54.537 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:54 smithi195 ceph-mon[34786]: from='osd.1 [v2:172.21.15.115:6800/3123728622,v1:172.21.15.115:6801/3123728622]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T03:21:54.537 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:54 smithi195 ceph-mon[34786]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T03:21:54.539 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi195:vg_nvme/lv_4 2024-09-12T03:21:54.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:54 smithi115 ceph-mon[34801]: from='osd.1 [v2:172.21.15.115:6800/3123728622,v1:172.21.15.115:6801/3123728622]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T03:21:54.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:54 smithi115 ceph-mon[34801]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T03:21:54.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:54 smithi064 ceph-mon[30952]: from='osd.1 [v2:172.21.15.115:6800/3123728622,v1:172.21.15.115:6801/3123728622]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T03:21:54.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:54 smithi064 ceph-mon[30952]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T03:21:54.788 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:21:55.243 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.242+0000 7f2da2525640 1 -- 172.21.15.195:0/264927461 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d9c100ad0 msgr2=0x7f2d9c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:55.243 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.242+0000 7f2da2525640 1 --2- 172.21.15.195:0/264927461 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d9c100ad0 0x7f2d9c100ed0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c009860 tx=0x7f2d8c02a340 comp rx=0 tx=0).stop 2024-09-12T03:21:55.243 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.242+0000 7f2da2525640 1 -- 172.21.15.195:0/264927461 shutdown_connections 2024-09-12T03:21:55.243 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.242+0000 7f2da2525640 1 --2- 172.21.15.195:0/264927461 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d9c102690 0x7f2d9c108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:55.244 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.242+0000 7f2da2525640 1 --2- 172.21.15.195:0/264927461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d9c101cd0 0x7f2d9c102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:55.244 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.242+0000 7f2da2525640 1 --2- 172.21.15.195:0/264927461 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d9c100ad0 0x7f2d9c100ed0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:55.244 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.242+0000 7f2da2525640 1 -- 172.21.15.195:0/264927461 >> 172.21.15.195:0/264927461 conn(0x7f2d9c0fc280 msgr2=0x7f2d9c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:55.244 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.242+0000 7f2da2525640 1 -- 172.21.15.195:0/264927461 shutdown_connections 2024-09-12T03:21:55.244 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.242+0000 7f2da2525640 1 -- 172.21.15.195:0/264927461 wait complete. 2024-09-12T03:21:55.246 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.242+0000 7f2da2525640 1 Processor -- start 2024-09-12T03:21:55.246 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.242+0000 7f2da2525640 1 -- start start 2024-09-12T03:21:55.246 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.243+0000 7f2da2525640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d9c100ad0 0x7f2d9c197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:55.246 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.243+0000 7f2da2525640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d9c101cd0 0x7f2d9c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:55.246 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.243+0000 7f2da2525640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d9c102690 0x7f2d9c19f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:55.246 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.243+0000 7f2da2525640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9c10c610 con 0x7f2d9c101cd0 2024-09-12T03:21:55.246 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.243+0000 7f2da2525640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9c10c490 con 0x7f2d9c100ad0 2024-09-12T03:21:55.246 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.243+0000 7f2da2525640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9c10c790 con 0x7f2d9c102690 2024-09-12T03:21:55.246 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.243+0000 7f2da0ac2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d9c102690 0x7f2d9c19f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:55.246 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.243+0000 7f2da0ac2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d9c102690 0x7f2d9c19f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:52636/0 (socket says 172.21.15.195:52636) 2024-09-12T03:21:55.247 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.243+0000 7f2da0ac2640 1 -- 172.21.15.195:0/511057838 learned_addr learned my addr 172.21.15.195:0/511057838 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-09-12T03:21:55.247 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.243+0000 7f2d9bfff640 1 --2- 172.21.15.195:0/511057838 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d9c100ad0 0x7f2d9c197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:55.247 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.243+0000 7f2d9b7fe640 1 --2- 172.21.15.195:0/511057838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d9c101cd0 0x7f2d9c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:55.247 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.243+0000 7f2da0ac2640 1 -- 172.21.15.195:0/511057838 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d9c100ad0 msgr2=0x7f2d9c197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:55.247 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.243+0000 7f2da0ac2640 1 --2- 172.21.15.195:0/511057838 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d9c100ad0 0x7f2d9c197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:55.247 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.243+0000 7f2da0ac2640 1 -- 172.21.15.195:0/511057838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d9c101cd0 msgr2=0x7f2d9c198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:55.247 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.243+0000 7f2da0ac2640 1 --2- 172.21.15.195:0/511057838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d9c101cd0 0x7f2d9c198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:55.247 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.243+0000 7f2da0ac2640 1 -- 172.21.15.195:0/511057838 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d9c19fa80 con 0x7f2d9c102690 2024-09-12T03:21:55.247 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.243+0000 7f2d9bfff640 1 --2- 172.21.15.195:0/511057838 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d9c100ad0 0x7f2d9c197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:55.247 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.243+0000 7f2da0ac2640 1 --2- 172.21.15.195:0/511057838 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d9c102690 0x7f2d9c19f410 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2d94009fd0 tx=0x7f2d9400c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:55.247 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.243+0000 7f2d997fa640 1 -- 172.21.15.195:0/511057838 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d94004b20 con 0x7f2d9c102690 2024-09-12T03:21:55.247 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.244+0000 7f2d997fa640 1 -- 172.21.15.195:0/511057838 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d94003e20 con 0x7f2d9c102690 2024-09-12T03:21:55.247 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.244+0000 7f2da2525640 1 -- 172.21.15.195:0/511057838 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d9c19fd70 con 0x7f2d9c102690 2024-09-12T03:21:55.248 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.244+0000 7f2d997fa640 1 -- 172.21.15.195:0/511057838 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d94004820 con 0x7f2d9c102690 2024-09-12T03:21:55.248 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.244+0000 7f2da2525640 1 -- 172.21.15.195:0/511057838 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d9c1a0250 con 0x7f2d9c102690 2024-09-12T03:21:55.248 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.245+0000 7f2da2525640 1 -- 172.21.15.195:0/511057838 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d9c1a0480 con 0x7f2d9c102690 2024-09-12T03:21:55.248 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.246+0000 7f2d997fa640 1 -- 172.21.15.195:0/511057838 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f2d9401d020 con 0x7f2d9c102690 2024-09-12T03:21:55.249 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.246+0000 7f2d997fa640 1 --2- 172.21.15.195:0/511057838 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f2d840a2b20 0x7f2d840a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:55.249 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.246+0000 7f2d997fa640 1 -- 172.21.15.195:0/511057838 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 1903+0+0 (secure 0 0 0) 0x7f2d940cd640 con 0x7f2d9c102690 2024-09-12T03:21:55.249 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.247+0000 7f2d9bfff640 1 --2- 172.21.15.195:0/511057838 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f2d840a2b20 0x7f2d840a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:55.249 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.247+0000 7f2d9bfff640 1 --2- 172.21.15.195:0/511057838 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f2d840a2b20 0x7f2d840a4fe0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c009830 tx=0x7f2d8c005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:55.252 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.251+0000 7f2d997fa640 1 -- 172.21.15.195:0/511057838 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2d9409bf90 con 0x7f2d9c102690 2024-09-12T03:21:55.451 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:21:55.450+0000 7f2da2525640 1 -- 172.21.15.195:0/511057838 --> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f2d9c06a6f0 con 0x7f2d840a2b20 2024-09-12T03:21:55.452 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:55 smithi195 ceph-mon[34786]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:21:55.452 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:55 smithi195 ceph-mon[34786]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T03:21:55.452 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:55 smithi195 ceph-mon[34786]: from='osd.1 [v2:172.21.15.115:6800/3123728622,v1:172.21.15.115:6801/3123728622]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-12T03:21:55.452 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:55 smithi195 ceph-mon[34786]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T03:21:55.452 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:55 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:21:55.452 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:55 smithi195 ceph-mon[34786]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-12T03:21:55.452 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:55 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:55.452 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:55 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:55.452 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:55 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:21:55.452 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:55 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:55.452 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:55 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:55.452 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:55 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:21:55.453 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:55 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:55.642 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 12 03:21:55 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-1[39262]: 2024-09-12T03:21:55.321+0000 7f4fc511c640 -1 osd.1 0 waiting for initial osdmap 2024-09-12T03:21:55.643 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 12 03:21:55 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-1[39262]: 2024-09-12T03:21:55.325+0000 7f4fc0f44640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:21:55.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:55 smithi115 ceph-mon[34801]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:21:55.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:55 smithi115 ceph-mon[34801]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T03:21:55.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:55 smithi115 ceph-mon[34801]: from='osd.1 [v2:172.21.15.115:6800/3123728622,v1:172.21.15.115:6801/3123728622]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-12T03:21:55.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:55 smithi115 ceph-mon[34801]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T03:21:55.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:55 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:21:55.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:55 smithi115 ceph-mon[34801]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-12T03:21:55.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:55 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:55.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:55 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:55.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:55 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:21:55.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:55 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:55.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:55 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:55.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:55 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:21:55.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:55 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:55.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:55 smithi064 ceph-mon[30952]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:21:55.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:55 smithi064 ceph-mon[30952]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T03:21:55.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:55 smithi064 ceph-mon[30952]: from='osd.1 [v2:172.21.15.115:6800/3123728622,v1:172.21.15.115:6801/3123728622]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-12T03:21:55.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:55 smithi064 ceph-mon[30952]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T03:21:55.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:55 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:21:55.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:55 smithi064 ceph-mon[30952]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-12T03:21:55.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:55 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:55.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:55 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:55.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:55 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:21:55.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:55 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:55.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:55 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:55.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:55 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:21:55.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:55 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:21:56.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:56 smithi195 ceph-mon[34786]: purged_snaps scrub starts 2024-09-12T03:21:56.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:56 smithi195 ceph-mon[34786]: purged_snaps scrub ok 2024-09-12T03:21:56.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:56 smithi195 ceph-mon[34786]: Detected new or changed devices on smithi115 2024-09-12T03:21:56.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:56 smithi195 ceph-mon[34786]: Adjusting osd_memory_target on smithi115 to 17088M 2024-09-12T03:21:56.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:56 smithi195 ceph-mon[34786]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-12T03:21:56.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:56 smithi195 ceph-mon[34786]: osdmap e12: 2 total, 1 up, 2 in 2024-09-12T03:21:56.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:56 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:21:56.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:56 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:21:56.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:56 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:21:56.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:56 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:21:56.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:56 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:56.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:56 smithi115 ceph-mon[34801]: purged_snaps scrub starts 2024-09-12T03:21:56.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:56 smithi115 ceph-mon[34801]: purged_snaps scrub ok 2024-09-12T03:21:56.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:56 smithi115 ceph-mon[34801]: Detected new or changed devices on smithi115 2024-09-12T03:21:56.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:56 smithi115 ceph-mon[34801]: Adjusting osd_memory_target on smithi115 to 17088M 2024-09-12T03:21:56.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:56 smithi115 ceph-mon[34801]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-12T03:21:56.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:56 smithi115 ceph-mon[34801]: osdmap e12: 2 total, 1 up, 2 in 2024-09-12T03:21:56.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:56 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:21:56.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:56 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:21:56.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:56 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:21:56.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:56 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:21:56.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:56 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:56.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:56 smithi064 ceph-mon[30952]: purged_snaps scrub starts 2024-09-12T03:21:56.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:56 smithi064 ceph-mon[30952]: purged_snaps scrub ok 2024-09-12T03:21:56.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:56 smithi064 ceph-mon[30952]: Detected new or changed devices on smithi115 2024-09-12T03:21:56.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:56 smithi064 ceph-mon[30952]: Adjusting osd_memory_target on smithi115 to 17088M 2024-09-12T03:21:56.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:56 smithi064 ceph-mon[30952]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-12T03:21:56.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:56 smithi064 ceph-mon[30952]: osdmap e12: 2 total, 1 up, 2 in 2024-09-12T03:21:56.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:56 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:21:56.774 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:56 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:21:56.774 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:56 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:21:56.774 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:56 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:21:56.774 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:56 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:57.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:57 smithi115 ceph-mon[34801]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:21:57.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:57 smithi115 ceph-mon[34801]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:21:57.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:57 smithi115 ceph-mon[34801]: osd.1 [v2:172.21.15.115:6800/3123728622,v1:172.21.15.115:6801/3123728622] boot 2024-09-12T03:21:57.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:57 smithi115 ceph-mon[34801]: osdmap e13: 2 total, 2 up, 2 in 2024-09-12T03:21:57.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:57 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:21:57.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:57 smithi195 ceph-mon[34786]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:21:57.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:57 smithi195 ceph-mon[34786]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:21:57.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:57 smithi195 ceph-mon[34786]: osd.1 [v2:172.21.15.115:6800/3123728622,v1:172.21.15.115:6801/3123728622] boot 2024-09-12T03:21:57.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:57 smithi195 ceph-mon[34786]: osdmap e13: 2 total, 2 up, 2 in 2024-09-12T03:21:57.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:57 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:21:57.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:57 smithi064 ceph-mon[30952]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:21:57.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:57 smithi064 ceph-mon[30952]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:21:57.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:57 smithi064 ceph-mon[30952]: osd.1 [v2:172.21.15.115:6800/3123728622,v1:172.21.15.115:6801/3123728622] boot 2024-09-12T03:21:57.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:57 smithi064 ceph-mon[30952]: osdmap e13: 2 total, 2 up, 2 in 2024-09-12T03:21:57.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:57 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:21:58.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:58 smithi195 ceph-mon[34786]: osdmap e14: 2 total, 2 up, 2 in 2024-09-12T03:21:58.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:58 smithi195 ceph-mon[34786]: from='client.? 172.21.15.195:0/2070192063' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b3a3c7a-aec6-455a-9a9a-1ac9cc900f24"}]: dispatch 2024-09-12T03:21:58.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:58 smithi195 ceph-mon[34786]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b3a3c7a-aec6-455a-9a9a-1ac9cc900f24"}]: dispatch 2024-09-12T03:21:58.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:58 smithi195 ceph-mon[34786]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b3a3c7a-aec6-455a-9a9a-1ac9cc900f24"}]': finished 2024-09-12T03:21:58.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:58 smithi195 ceph-mon[34786]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T03:21:58.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:58 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:21:58.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:58 smithi195 ceph-mon[34786]: from='client.? 172.21.15.195:0/2698882080' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:21:58.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:58 smithi115 ceph-mon[34801]: osdmap e14: 2 total, 2 up, 2 in 2024-09-12T03:21:58.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:58 smithi115 ceph-mon[34801]: from='client.? 172.21.15.195:0/2070192063' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b3a3c7a-aec6-455a-9a9a-1ac9cc900f24"}]: dispatch 2024-09-12T03:21:58.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:58 smithi115 ceph-mon[34801]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b3a3c7a-aec6-455a-9a9a-1ac9cc900f24"}]: dispatch 2024-09-12T03:21:58.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:58 smithi115 ceph-mon[34801]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b3a3c7a-aec6-455a-9a9a-1ac9cc900f24"}]': finished 2024-09-12T03:21:58.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:58 smithi115 ceph-mon[34801]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T03:21:58.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:58 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:21:58.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:58 smithi115 ceph-mon[34801]: from='client.? 172.21.15.195:0/2698882080' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:21:58.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:58 smithi064 ceph-mon[30952]: osdmap e14: 2 total, 2 up, 2 in 2024-09-12T03:21:58.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:58 smithi064 ceph-mon[30952]: from='client.? 172.21.15.195:0/2070192063' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b3a3c7a-aec6-455a-9a9a-1ac9cc900f24"}]: dispatch 2024-09-12T03:21:58.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:58 smithi064 ceph-mon[30952]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b3a3c7a-aec6-455a-9a9a-1ac9cc900f24"}]: dispatch 2024-09-12T03:21:58.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:58 smithi064 ceph-mon[30952]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b3a3c7a-aec6-455a-9a9a-1ac9cc900f24"}]': finished 2024-09-12T03:21:58.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:58 smithi064 ceph-mon[30952]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T03:21:58.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:58 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:21:58.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:58 smithi064 ceph-mon[30952]: from='client.? 172.21.15.195:0/2698882080' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:21:59.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:21:59 smithi115 ceph-mon[34801]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:21:59.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:21:59 smithi195 ceph-mon[34786]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:21:59.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:21:59 smithi064 ceph-mon[30952]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:22:01.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:01 smithi115 ceph-mon[34801]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:22:01.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:01 smithi195 ceph-mon[34786]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:22:01.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:01 smithi064 ceph-mon[30952]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:22:03.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:03 smithi115 ceph-mon[34801]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:22:03.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:03 smithi195 ceph-mon[34786]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:22:03.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:03 smithi064 ceph-mon[30952]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:22:05.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:05 smithi195 ceph-mon[34786]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:22:05.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:05 smithi115 ceph-mon[34801]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:22:05.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:05 smithi064 ceph-mon[30952]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:22:06.629 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:06 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T03:22:06.629 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:06 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:06.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:06 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T03:22:06.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:06 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:06.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:06 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T03:22:06.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:06 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:07.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:07 smithi195 ceph-mon[34786]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:22:07.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:07 smithi195 ceph-mon[34786]: Deploying daemon osd.2 on smithi195 2024-09-12T03:22:07.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:07 smithi115 ceph-mon[34801]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:22:07.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:07 smithi115 ceph-mon[34801]: Deploying daemon osd.2 on smithi195 2024-09-12T03:22:07.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:07 smithi064 ceph-mon[30952]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:22:07.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:07 smithi064 ceph-mon[30952]: Deploying daemon osd.2 on smithi195 2024-09-12T03:22:09.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:09 smithi115 ceph-mon[34801]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:22:09.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:09 smithi195 ceph-mon[34786]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:22:09.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:09 smithi064 ceph-mon[30952]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:22:11.531 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:11 smithi195 ceph-mon[34786]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:22:11.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:11 smithi115 ceph-mon[34801]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:22:11.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:11 smithi064 ceph-mon[30952]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:22:12.630 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:12 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:22:12.630 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:12 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:12.630 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:12 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:12.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:12 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:22:12.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:12 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:12.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:12 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:12.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:12 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:22:12.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:12 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:12.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:12 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:13.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:13 smithi195 ceph-mon[34786]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:22:13.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:13 smithi115 ceph-mon[34801]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:22:13.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:13 smithi064 ceph-mon[30952]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:22:15.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:15 smithi115 ceph-mon[34801]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:22:15.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:15 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:15.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:15 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:15.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:15 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:15.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:15 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:22:15.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:15 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:15.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:15 smithi195 ceph-mon[34786]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:22:15.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:15 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:15.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:15 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:15.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:15 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:15.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:15 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:22:15.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:15 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:15.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:15 smithi064 ceph-mon[30952]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:22:15.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:15 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:15.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:15 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:15.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:15 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:15.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:15 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:22:15.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:15 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:16.087 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:22:16.086+0000 7f2d997fa640 1 -- 172.21.15.195:0/511057838 <== mgr.14150 v2:172.21.15.64:6800/759931848 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2d9c06a6f0 con 0x7f2d840a2b20 2024-09-12T03:22:16.087 INFO:teuthology.orchestra.run.smithi195.stdout:Created osd(s) 2 on host 'smithi195' 2024-09-12T03:22:16.089 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:22:16.089+0000 7f2da2525640 1 -- 172.21.15.195:0/511057838 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f2d840a2b20 msgr2=0x7f2d840a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:16.089 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:22:16.089+0000 7f2da2525640 1 --2- 172.21.15.195:0/511057838 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f2d840a2b20 0x7f2d840a4fe0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c009830 tx=0x7f2d8c005d20 comp rx=0 tx=0).stop 2024-09-12T03:22:16.090 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:22:16.089+0000 7f2da2525640 1 -- 172.21.15.195:0/511057838 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d9c102690 msgr2=0x7f2d9c19f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:16.090 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:22:16.089+0000 7f2da2525640 1 --2- 172.21.15.195:0/511057838 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d9c102690 0x7f2d9c19f410 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2d94009fd0 tx=0x7f2d9400c800 comp rx=0 tx=0).stop 2024-09-12T03:22:16.090 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:22:16.089+0000 7f2da2525640 1 -- 172.21.15.195:0/511057838 shutdown_connections 2024-09-12T03:22:16.090 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:22:16.089+0000 7f2da2525640 1 --2- 172.21.15.195:0/511057838 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f2d840a2b20 0x7f2d840a4fe0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:16.090 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:22:16.089+0000 7f2da2525640 1 --2- 172.21.15.195:0/511057838 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d9c102690 0x7f2d9c19f410 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:16.090 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:22:16.089+0000 7f2da2525640 1 --2- 172.21.15.195:0/511057838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d9c101cd0 0x7f2d9c198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:16.090 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:22:16.089+0000 7f2da2525640 1 --2- 172.21.15.195:0/511057838 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d9c100ad0 0x7f2d9c197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:16.090 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:22:16.089+0000 7f2da2525640 1 -- 172.21.15.195:0/511057838 >> 172.21.15.195:0/511057838 conn(0x7f2d9c0fc280 msgr2=0x7f2d9c069fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:16.090 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:22:16.089+0000 7f2da2525640 1 -- 172.21.15.195:0/511057838 shutdown_connections 2024-09-12T03:22:16.091 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:22:16.089+0000 7f2da2525640 1 -- 172.21.15.195:0/511057838 wait complete. 2024-09-12T03:22:16.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:16 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:22:16.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:16 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:16.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:16 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:16.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:16 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:22:16.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:16 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:16.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:16 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:16.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:16 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:22:16.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:16 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:16.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:16 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:16.632 DEBUG:teuthology.orchestra.run.smithi195:osd.2> sudo journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@osd.2.service 2024-09-12T03:22:16.635 INFO:tasks.cephadm:Deploying osd.3 on smithi064 with /dev/vg_nvme/lv_3... 2024-09-12T03:22:16.635 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T03:22:16.886 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:22:17.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:17 smithi195 ceph-mon[34786]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:22:17.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:17 smithi064 ceph-mon[30952]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:22:17.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:17 smithi115 ceph-mon[34801]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:22:17.746 INFO:journalctl@ceph.osd.2.smithi195.stdout:Sep 12 03:22:17 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-2[39737]: 2024-09-12T03:22:17.679+0000 7fdc31a34640 -1 osd.2 0 log_to_monitors true 2024-09-12T03:22:18.303 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:18 smithi064 ceph-mon[30952]: from='osd.2 [v2:172.21.15.195:6800/1664125451,v1:172.21.15.195:6801/1664125451]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T03:22:18.303 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:18 smithi064 ceph-mon[30952]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T03:22:18.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:18 smithi115 ceph-mon[34801]: from='osd.2 [v2:172.21.15.195:6800/1664125451,v1:172.21.15.195:6801/1664125451]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T03:22:18.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:18 smithi115 ceph-mon[34801]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T03:22:18.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:18 smithi195 ceph-mon[34786]: from='osd.2 [v2:172.21.15.195:6800/1664125451,v1:172.21.15.195:6801/1664125451]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T03:22:18.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:18 smithi195 ceph-mon[34786]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T03:22:19.354 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:19 smithi195 ceph-mon[34786]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:22:19.354 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:19 smithi195 ceph-mon[34786]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T03:22:19.354 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:19 smithi195 ceph-mon[34786]: from='osd.2 [v2:172.21.15.195:6800/1664125451,v1:172.21.15.195:6801/1664125451]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-09-12T03:22:19.354 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:19 smithi195 ceph-mon[34786]: osdmap e16: 3 total, 2 up, 3 in 2024-09-12T03:22:19.354 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:19 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:22:19.354 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:19 smithi195 ceph-mon[34786]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-09-12T03:22:19.426 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:19 smithi064 ceph-mon[30952]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:22:19.426 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:19 smithi064 ceph-mon[30952]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T03:22:19.426 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:19 smithi064 ceph-mon[30952]: from='osd.2 [v2:172.21.15.195:6800/1664125451,v1:172.21.15.195:6801/1664125451]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-09-12T03:22:19.426 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:19 smithi064 ceph-mon[30952]: osdmap e16: 3 total, 2 up, 3 in 2024-09-12T03:22:19.426 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:19 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:22:19.426 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:19 smithi064 ceph-mon[30952]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-09-12T03:22:19.538 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:19 smithi115 ceph-mon[34801]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:22:19.539 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:19 smithi115 ceph-mon[34801]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T03:22:19.539 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:19 smithi115 ceph-mon[34801]: from='osd.2 [v2:172.21.15.195:6800/1664125451,v1:172.21.15.195:6801/1664125451]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-09-12T03:22:19.539 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:19 smithi115 ceph-mon[34801]: osdmap e16: 3 total, 2 up, 3 in 2024-09-12T03:22:19.539 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:19 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:22:19.539 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:19 smithi115 ceph-mon[34801]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-09-12T03:22:19.586 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:22:19.623 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi064:vg_nvme/lv_3 2024-09-12T03:22:19.684 INFO:journalctl@ceph.osd.2.smithi195.stdout:Sep 12 03:22:19 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-2[39737]: 2024-09-12T03:22:19.589+0000 7fdc2d9b7640 -1 osd.2 0 waiting for initial osdmap 2024-09-12T03:22:19.684 INFO:journalctl@ceph.osd.2.smithi195.stdout:Sep 12 03:22:19 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-2[39737]: 2024-09-12T03:22:19.595+0000 7fdc257d7640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:22:19.890 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:22:20.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:20 smithi064 ceph-mon[30952]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-09-12T03:22:20.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:20 smithi064 ceph-mon[30952]: osdmap e17: 3 total, 2 up, 3 in 2024-09-12T03:22:20.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:20 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:22:20.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:20 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:22:20.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:20 smithi064 ceph-mon[30952]: Detected new or changed devices on smithi195 2024-09-12T03:22:20.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:20 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:20.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:20 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:20.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:20 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:22:20.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:20 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:20.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:20 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:20.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:20 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:22:20.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:20 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:20.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:20 smithi064 ceph-mon[30952]: from='osd.2 ' entity='osd.2' 2024-09-12T03:22:20.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:20 smithi195 ceph-mon[34786]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-09-12T03:22:20.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:20 smithi195 ceph-mon[34786]: osdmap e17: 3 total, 2 up, 3 in 2024-09-12T03:22:20.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:20 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:22:20.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:20 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:22:20.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:20 smithi195 ceph-mon[34786]: Detected new or changed devices on smithi195 2024-09-12T03:22:20.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:20 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:20.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:20 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:20.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:20 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:22:20.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:20 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:20.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:20 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:20.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:20 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:22:20.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:20 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:20.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:20 smithi195 ceph-mon[34786]: from='osd.2 ' entity='osd.2' 2024-09-12T03:22:20.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.580+0000 7efe7a55b640 1 -- 172.21.15.64:0/4268204416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe74100ad0 msgr2=0x7efe74100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:20.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.580+0000 7efe7a55b640 1 --2- 172.21.15.64:0/4268204416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe74100ad0 0x7efe74100ed0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7efe64009900 tx=0x7efe6402a410 comp rx=0 tx=0).stop 2024-09-12T03:22:20.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.580+0000 7efe7a55b640 1 -- 172.21.15.64:0/4268204416 shutdown_connections 2024-09-12T03:22:20.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.580+0000 7efe7a55b640 1 --2- 172.21.15.64:0/4268204416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe74102690 0x7efe74108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:20.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.580+0000 7efe7a55b640 1 --2- 172.21.15.64:0/4268204416 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe74101cd0 0x7efe74102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:20.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.580+0000 7efe7a55b640 1 --2- 172.21.15.64:0/4268204416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe74100ad0 0x7efe74100ed0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:20.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.580+0000 7efe7a55b640 1 -- 172.21.15.64:0/4268204416 >> 172.21.15.64:0/4268204416 conn(0x7efe740fc280 msgr2=0x7efe740fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:20.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.580+0000 7efe7a55b640 1 -- 172.21.15.64:0/4268204416 shutdown_connections 2024-09-12T03:22:20.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.580+0000 7efe7a55b640 1 -- 172.21.15.64:0/4268204416 wait complete. 2024-09-12T03:22:20.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.580+0000 7efe7a55b640 1 Processor -- start 2024-09-12T03:22:20.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.580+0000 7efe7a55b640 1 -- start start 2024-09-12T03:22:20.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.581+0000 7efe7a55b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe74100ad0 0x7efe7406aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:20.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.581+0000 7efe7a55b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe74101cd0 0x7efe7419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:20.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.581+0000 7efe7a55b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe74102690 0x7efe7419f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:20.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.581+0000 7efe7a55b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efe7410c5e0 con 0x7efe74101cd0 2024-09-12T03:22:20.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.581+0000 7efe7a55b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efe7410c460 con 0x7efe74102690 2024-09-12T03:22:20.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.581+0000 7efe7a55b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efe7410c760 con 0x7efe74100ad0 2024-09-12T03:22:20.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.581+0000 7efe737fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe74101cd0 0x7efe7419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:20.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.581+0000 7efe73fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe74100ad0 0x7efe7406aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:20.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.581+0000 7efe737fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe74101cd0 0x7efe7419cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33224/0 (socket says 172.21.15.64:33224) 2024-09-12T03:22:20.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.581+0000 7efe737fe640 1 -- 172.21.15.64:0/1983249139 learned_addr learned my addr 172.21.15.64:0/1983249139 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:22:20.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.581+0000 7efe78af8640 1 --2- 172.21.15.64:0/1983249139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe74102690 0x7efe7419f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:20.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.581+0000 7efe737fe640 1 -- 172.21.15.64:0/1983249139 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe74100ad0 msgr2=0x7efe7406aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:20.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.581+0000 7efe737fe640 1 --2- 172.21.15.64:0/1983249139 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe74100ad0 0x7efe7406aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:20.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.581+0000 7efe737fe640 1 -- 172.21.15.64:0/1983249139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe74102690 msgr2=0x7efe7419f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:20.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.581+0000 7efe737fe640 1 --2- 172.21.15.64:0/1983249139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe74102690 0x7efe7419f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:20.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.581+0000 7efe737fe640 1 -- 172.21.15.64:0/1983249139 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe74196a00 con 0x7efe74101cd0 2024-09-12T03:22:20.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.581+0000 7efe78af8640 1 --2- 172.21.15.64:0/1983249139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe74102690 0x7efe7419f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:20.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.582+0000 7efe737fe640 1 --2- 172.21.15.64:0/1983249139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe74101cd0 0x7efe7419cbb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efe6800e9f0 tx=0x7efe6800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:20.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.582+0000 7efe717fa640 1 -- 172.21.15.64:0/1983249139 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6800cd80 con 0x7efe74101cd0 2024-09-12T03:22:20.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.582+0000 7efe7a55b640 1 -- 172.21.15.64:0/1983249139 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe74196c90 con 0x7efe74101cd0 2024-09-12T03:22:20.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.582+0000 7efe717fa640 1 -- 172.21.15.64:0/1983249139 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe680108e0 con 0x7efe74101cd0 2024-09-12T03:22:20.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.582+0000 7efe717fa640 1 -- 172.21.15.64:0/1983249139 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe68018920 con 0x7efe74101cd0 2024-09-12T03:22:20.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.582+0000 7efe7a55b640 1 -- 172.21.15.64:0/1983249139 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe741971d0 con 0x7efe74101cd0 2024-09-12T03:22:20.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.583+0000 7efe7a55b640 1 -- 172.21.15.64:0/1983249139 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe74198670 con 0x7efe74101cd0 2024-09-12T03:22:20.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.584+0000 7efe717fa640 1 -- 172.21.15.64:0/1983249139 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7efe68010a90 con 0x7efe74101cd0 2024-09-12T03:22:20.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.584+0000 7efe717fa640 1 --2- 172.21.15.64:0/1983249139 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7efe5c0a2af0 0x7efe5c0a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:20.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.585+0000 7efe73fff640 1 --2- 172.21.15.64:0/1983249139 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7efe5c0a2af0 0x7efe5c0a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:20.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.585+0000 7efe717fa640 1 -- 172.21.15.64:0/1983249139 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2571+0+0 (secure 0 0 0) 0x7efe680cdc40 con 0x7efe74101cd0 2024-09-12T03:22:20.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.585+0000 7efe73fff640 1 --2- 172.21.15.64:0/1983249139 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7efe5c0a2af0 0x7efe5c0a4fb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efe64004770 tx=0x7efe64000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:20.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.589+0000 7efe717fa640 1 -- 172.21.15.64:0/1983249139 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efe680d4080 con 0x7efe74101cd0 2024-09-12T03:22:20.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:20 smithi115 ceph-mon[34801]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-09-12T03:22:20.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:20 smithi115 ceph-mon[34801]: osdmap e17: 3 total, 2 up, 3 in 2024-09-12T03:22:20.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:20 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:22:20.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:20 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:22:20.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:20 smithi115 ceph-mon[34801]: Detected new or changed devices on smithi195 2024-09-12T03:22:20.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:20 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:20.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:20 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:20.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:20 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:22:20.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:20 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:20.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:20 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:20.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:20 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:22:20.645 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:20 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:20.645 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:20 smithi115 ceph-mon[34801]: from='osd.2 ' entity='osd.2' 2024-09-12T03:22:20.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:20.777+0000 7efe7a55b640 1 -- 172.21.15.64:0/1983249139 --> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7efe74107b10 con 0x7efe5c0a2af0 2024-09-12T03:22:21.447 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:21 smithi064 ceph-mon[30952]: purged_snaps scrub starts 2024-09-12T03:22:21.447 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:21 smithi064 ceph-mon[30952]: purged_snaps scrub ok 2024-09-12T03:22:21.447 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:21 smithi064 ceph-mon[30952]: Adjusting osd_memory_target on smithi195 to 17088M 2024-09-12T03:22:21.447 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:21 smithi064 ceph-mon[30952]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:22:21.447 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:21 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:22:21.448 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:21 smithi064 ceph-mon[30952]: osd.2 [v2:172.21.15.195:6800/1664125451,v1:172.21.15.195:6801/1664125451] boot 2024-09-12T03:22:21.448 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:21 smithi064 ceph-mon[30952]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T03:22:21.448 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:21 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:22:21.448 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:21 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:22:21.448 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:21 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:22:21.448 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:21 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:21.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:21 smithi195 ceph-mon[34786]: purged_snaps scrub starts 2024-09-12T03:22:21.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:21 smithi195 ceph-mon[34786]: purged_snaps scrub ok 2024-09-12T03:22:21.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:21 smithi195 ceph-mon[34786]: Adjusting osd_memory_target on smithi195 to 17088M 2024-09-12T03:22:21.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:21 smithi195 ceph-mon[34786]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:22:21.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:21 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:22:21.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:21 smithi195 ceph-mon[34786]: osd.2 [v2:172.21.15.195:6800/1664125451,v1:172.21.15.195:6801/1664125451] boot 2024-09-12T03:22:21.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:21 smithi195 ceph-mon[34786]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T03:22:21.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:21 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:22:21.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:21 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:22:21.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:21 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:22:21.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:21 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:21.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:21 smithi115 ceph-mon[34801]: purged_snaps scrub starts 2024-09-12T03:22:21.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:21 smithi115 ceph-mon[34801]: purged_snaps scrub ok 2024-09-12T03:22:21.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:21 smithi115 ceph-mon[34801]: Adjusting osd_memory_target on smithi195 to 17088M 2024-09-12T03:22:21.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:21 smithi115 ceph-mon[34801]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:22:21.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:21 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:22:21.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:21 smithi115 ceph-mon[34801]: osd.2 [v2:172.21.15.195:6800/1664125451,v1:172.21.15.195:6801/1664125451] boot 2024-09-12T03:22:21.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:21 smithi115 ceph-mon[34801]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T03:22:21.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:21 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:22:21.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:21 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:22:21.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:21 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:22:21.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:21 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:22.421 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:22 smithi064 ceph-mon[30952]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:22:22.422 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:22 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:22:22.422 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:22 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T03:22:22.422 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:22 smithi064 ceph-mon[30952]: osdmap e19: 3 total, 3 up, 3 in 2024-09-12T03:22:22.422 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:22 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:22:22.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:22 smithi195 ceph-mon[34786]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:22:22.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:22 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:22:22.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:22 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T03:22:22.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:22 smithi195 ceph-mon[34786]: osdmap e19: 3 total, 3 up, 3 in 2024-09-12T03:22:22.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:22 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:22:22.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:22 smithi115 ceph-mon[34801]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:22:22.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:22 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:22:22.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:22 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T03:22:22.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:22 smithi115 ceph-mon[34801]: osdmap e19: 3 total, 3 up, 3 in 2024-09-12T03:22:22.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:22 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:22:23.409 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:23 smithi064 ceph-mon[30952]: pgmap v75: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:22:23.409 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:23 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T03:22:23.410 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:23 smithi064 ceph-mon[30952]: osdmap e20: 3 total, 3 up, 3 in 2024-09-12T03:22:23.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:23 smithi195 ceph-mon[34786]: pgmap v75: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:22:23.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:23 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T03:22:23.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:23 smithi195 ceph-mon[34786]: osdmap e20: 3 total, 3 up, 3 in 2024-09-12T03:22:23.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:23 smithi115 ceph-mon[34801]: pgmap v75: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:22:23.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:23 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T03:22:23.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:23 smithi115 ceph-mon[34801]: osdmap e20: 3 total, 3 up, 3 in 2024-09-12T03:22:23.759 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 12 03:22:23 smithi064 sudo[49663]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T03:22:23.759 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 12 03:22:23 smithi064 sudo[49663]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:22:23.759 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 12 03:22:23 smithi064 sudo[49663]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:22:23.759 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 12 03:22:23 smithi064 sudo[49663]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:22:24.022 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 12 03:22:23 smithi064 sudo[49707]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T03:22:24.022 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 12 03:22:23 smithi064 sudo[49707]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:22:24.022 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 12 03:22:23 smithi064 sudo[49707]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:22:24.023 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 12 03:22:23 smithi064 sudo[49707]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:22:24.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:24.181+0000 7efe717fa640 1 -- 172.21.15.64:0/1983249139 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7efe68098260 con 0x7efe74101cd0 2024-09-12T03:22:24.445 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 12 03:22:24 smithi115 sudo[43210]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T03:22:24.446 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 12 03:22:24 smithi115 sudo[43210]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:22:24.446 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 12 03:22:24 smithi115 sudo[43210]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:22:24.446 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 12 03:22:24 smithi115 sudo[43210]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:22:24.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:24 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/3277486330' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8768c339-2c1e-4294-8b78-3c2adab6750f"}]: dispatch 2024-09-12T03:22:24.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:24 smithi115 ceph-mon[34801]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8768c339-2c1e-4294-8b78-3c2adab6750f"}]: dispatch 2024-09-12T03:22:24.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:24 smithi115 ceph-mon[34801]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8768c339-2c1e-4294-8b78-3c2adab6750f"}]': finished 2024-09-12T03:22:24.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:24 smithi115 ceph-mon[34801]: osdmap e21: 4 total, 3 up, 4 in 2024-09-12T03:22:24.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:24 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:22:24.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:24 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/565814695' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:22:24.446 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:24 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:24.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:24 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/3277486330' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8768c339-2c1e-4294-8b78-3c2adab6750f"}]: dispatch 2024-09-12T03:22:24.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:24 smithi195 ceph-mon[34786]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8768c339-2c1e-4294-8b78-3c2adab6750f"}]: dispatch 2024-09-12T03:22:24.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:24 smithi195 ceph-mon[34786]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8768c339-2c1e-4294-8b78-3c2adab6750f"}]': finished 2024-09-12T03:22:24.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:24 smithi195 ceph-mon[34786]: osdmap e21: 4 total, 3 up, 4 in 2024-09-12T03:22:24.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:24 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:22:24.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:24 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/565814695' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:22:24.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:24 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:24.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:24 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/3277486330' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8768c339-2c1e-4294-8b78-3c2adab6750f"}]: dispatch 2024-09-12T03:22:24.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:24 smithi064 ceph-mon[30952]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8768c339-2c1e-4294-8b78-3c2adab6750f"}]: dispatch 2024-09-12T03:22:24.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:24 smithi064 ceph-mon[30952]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8768c339-2c1e-4294-8b78-3c2adab6750f"}]': finished 2024-09-12T03:22:24.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:24 smithi064 ceph-mon[30952]: osdmap e21: 4 total, 3 up, 4 in 2024-09-12T03:22:24.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:24 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:22:24.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:24 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/565814695' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:22:24.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:24 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:24.892 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 12 03:22:24 smithi115 sudo[43213]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T03:22:24.893 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 12 03:22:24 smithi115 sudo[43213]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:22:24.893 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 12 03:22:24 smithi115 sudo[43213]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:22:24.893 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 12 03:22:24 smithi115 sudo[43213]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:22:24.973 INFO:journalctl@ceph.osd.2.smithi195.stdout:Sep 12 03:22:24 smithi195 sudo[43348]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T03:22:24.973 INFO:journalctl@ceph.osd.2.smithi195.stdout:Sep 12 03:22:24 smithi195 sudo[43348]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:22:24.973 INFO:journalctl@ceph.osd.2.smithi195.stdout:Sep 12 03:22:24 smithi195 sudo[43348]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:22:24.973 INFO:journalctl@ceph.osd.2.smithi195.stdout:Sep 12 03:22:24 smithi195 sudo[43348]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:22:25.456 INFO:journalctl@ceph.osd.2.smithi195.stdout:Sep 12 03:22:24 smithi195 sudo[43351]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T03:22:25.456 INFO:journalctl@ceph.osd.2.smithi195.stdout:Sep 12 03:22:24 smithi195 sudo[43351]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:22:25.456 INFO:journalctl@ceph.osd.2.smithi195.stdout:Sep 12 03:22:24 smithi195 sudo[43351]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:22:25.456 INFO:journalctl@ceph.osd.2.smithi195.stdout:Sep 12 03:22:24 smithi195 sudo[43351]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:22:25.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:25 smithi064 sudo[50104]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T03:22:25.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:25 smithi064 sudo[50104]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:22:25.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:25 smithi064 sudo[50104]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:22:25.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:25 smithi064 sudo[50104]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:22:26.169 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:25 smithi195 ceph-mon[34786]: pgmap v79: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:22:26.169 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:25 smithi195 ceph-mon[34786]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T03:22:26.169 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:25 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:26.169 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:25 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:26.169 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:25 smithi195 ceph-mon[34786]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T03:22:26.169 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:26 smithi195 sudo[43354]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T03:22:26.169 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:26 smithi195 sudo[43354]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:22:26.169 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:26 smithi195 sudo[43354]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:22:26.170 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:26 smithi195 sudo[43354]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:22:26.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:25 smithi064 sudo[50111]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T03:22:26.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:25 smithi064 sudo[50111]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:22:26.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:25 smithi064 sudo[50111]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:22:26.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:25 smithi064 sudo[50111]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:22:26.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:25 smithi064 ceph-mon[30952]: pgmap v79: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:22:26.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:25 smithi064 ceph-mon[30952]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T03:22:26.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:25 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:26.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:25 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:26.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:25 smithi064 ceph-mon[30952]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T03:22:26.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:25 smithi115 ceph-mon[34801]: pgmap v79: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:22:26.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:25 smithi115 ceph-mon[34801]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T03:22:26.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:25 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:26.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:25 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:26.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:25 smithi115 ceph-mon[34801]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T03:22:26.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:26 smithi195 sudo[43357]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T03:22:26.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:26 smithi195 sudo[43357]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:22:26.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:26 smithi195 sudo[43357]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:22:26.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:26 smithi195 sudo[43357]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:22:26.905 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:26 smithi115 sudo[43216]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T03:22:26.906 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:26 smithi115 sudo[43216]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:22:26.906 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:26 smithi115 sudo[43216]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:22:26.906 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:26 smithi115 sudo[43216]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:22:27.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:26 smithi195 ceph-mon[34786]: pgmap v80: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:22:27.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:26 smithi195 ceph-mon[34786]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T03:22:27.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:26 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:22:27.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:26 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:22:27.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:26 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:22:27.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:26 smithi195 ceph-mon[34786]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T03:22:27.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:26 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:22:27.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:26 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:22:27.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:26 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:22:27.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:26 smithi195 ceph-mon[34786]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T03:22:27.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:26 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:22:27.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:26 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:22:27.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:26 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:22:27.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:26 smithi064 ceph-mon[30952]: pgmap v80: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:22:27.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:26 smithi064 ceph-mon[30952]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T03:22:27.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:26 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:22:27.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:26 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:22:27.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:26 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:22:27.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:26 smithi064 ceph-mon[30952]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T03:22:27.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:26 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:22:27.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:26 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:22:27.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:26 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:22:27.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:26 smithi064 ceph-mon[30952]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T03:22:27.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:26 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:22:27.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:26 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:22:27.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:26 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:22:27.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:26 smithi115 sudo[43219]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T03:22:27.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:26 smithi115 sudo[43219]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:22:27.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:26 smithi115 sudo[43219]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:22:27.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:26 smithi115 sudo[43219]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:22:27.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:26 smithi115 ceph-mon[34801]: pgmap v80: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:22:27.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:26 smithi115 ceph-mon[34801]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T03:22:27.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:26 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:22:27.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:26 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:22:27.394 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:26 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:22:27.394 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:26 smithi115 ceph-mon[34801]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T03:22:27.394 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:26 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:22:27.394 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:26 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:22:27.394 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:26 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:22:27.394 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:26 smithi115 ceph-mon[34801]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T03:22:27.394 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:26 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:22:27.394 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:26 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:22:27.394 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:26 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:22:28.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:27 smithi195 ceph-mon[34786]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T03:22:28.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:27 smithi195 ceph-mon[34786]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T03:22:28.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:27 smithi064 ceph-mon[30952]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T03:22:28.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:27 smithi064 ceph-mon[30952]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T03:22:28.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:27 smithi115 ceph-mon[34801]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T03:22:28.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:27 smithi115 ceph-mon[34801]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T03:22:29.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:28 smithi195 ceph-mon[34786]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:22:29.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:28 smithi064 ceph-mon[30952]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:22:29.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:28 smithi115 ceph-mon[34801]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:22:31.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:30 smithi195 ceph-mon[34786]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:22:31.232 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:30 smithi064 ceph-mon[30952]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:22:31.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:30 smithi115 ceph-mon[34801]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:22:32.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:31 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T03:22:32.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:31 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:32.210 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:31 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T03:22:32.210 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:31 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:32.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:31 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T03:22:32.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:31 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:33.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:32 smithi064 ceph-mon[30952]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:22:33.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:32 smithi064 ceph-mon[30952]: Deploying daemon osd.3 on smithi064 2024-09-12T03:22:33.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:32 smithi195 ceph-mon[34786]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:22:33.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:32 smithi195 ceph-mon[34786]: Deploying daemon osd.3 on smithi064 2024-09-12T03:22:33.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:32 smithi115 ceph-mon[34801]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:22:33.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:32 smithi115 ceph-mon[34801]: Deploying daemon osd.3 on smithi064 2024-09-12T03:22:34.951 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:34 smithi064 ceph-mon[30952]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:22:35.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:34 smithi195 ceph-mon[34786]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:22:35.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:34 smithi115 ceph-mon[34801]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:22:37.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:36 smithi064 ceph-mon[30952]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:22:37.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:36 smithi195 ceph-mon[34786]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:22:37.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:36 smithi115 ceph-mon[34801]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:22:38.196 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:37 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:22:38.196 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:37 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:38.196 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:37 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:38.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:37 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:22:38.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:37 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:38.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:37 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:38.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:37 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:22:38.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:37 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:38.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:37 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:39.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:38 smithi195 ceph-mon[34786]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:22:39.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:38 smithi064 ceph-mon[30952]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:22:39.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:38 smithi115 ceph-mon[34801]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:22:41.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:41 smithi064 ceph-mon[30952]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:22:41.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:41 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:41.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:41 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:41.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:41 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:41.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:41 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:22:41.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:41 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:41.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:41.591+0000 7efe717fa640 1 -- 172.21.15.64:0/1983249139 <== mgr.14150 v2:172.21.15.64:6800/759931848 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7efe74107b10 con 0x7efe5c0a2af0 2024-09-12T03:22:41.592 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 3 on host 'smithi064' 2024-09-12T03:22:41.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:41.594+0000 7efe7a55b640 1 -- 172.21.15.64:0/1983249139 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7efe5c0a2af0 msgr2=0x7efe5c0a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:41.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:41.594+0000 7efe7a55b640 1 --2- 172.21.15.64:0/1983249139 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7efe5c0a2af0 0x7efe5c0a4fb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efe64004770 tx=0x7efe64000bc0 comp rx=0 tx=0).stop 2024-09-12T03:22:41.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:41.594+0000 7efe7a55b640 1 -- 172.21.15.64:0/1983249139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe74101cd0 msgr2=0x7efe7419cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:41.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:41.594+0000 7efe7a55b640 1 --2- 172.21.15.64:0/1983249139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe74101cd0 0x7efe7419cbb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efe6800e9f0 tx=0x7efe6800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:22:41.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:41.594+0000 7efe7a55b640 1 -- 172.21.15.64:0/1983249139 shutdown_connections 2024-09-12T03:22:41.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:41.594+0000 7efe7a55b640 1 --2- 172.21.15.64:0/1983249139 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7efe5c0a2af0 0x7efe5c0a4fb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:41.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:41.594+0000 7efe7a55b640 1 --2- 172.21.15.64:0/1983249139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe74102690 0x7efe7419f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:41.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:41.594+0000 7efe7a55b640 1 --2- 172.21.15.64:0/1983249139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe74101cd0 0x7efe7419cbb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:41.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:41.594+0000 7efe7a55b640 1 --2- 172.21.15.64:0/1983249139 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe74100ad0 0x7efe7406aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:41.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:41.594+0000 7efe7a55b640 1 -- 172.21.15.64:0/1983249139 >> 172.21.15.64:0/1983249139 conn(0x7efe740fc280 msgr2=0x7efe74105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:41.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:41.594+0000 7efe7a55b640 1 -- 172.21.15.64:0/1983249139 shutdown_connections 2024-09-12T03:22:41.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:22:41.594+0000 7efe7a55b640 1 -- 172.21.15.64:0/1983249139 wait complete. 2024-09-12T03:22:41.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:41 smithi115 ceph-mon[34801]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:22:41.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:41 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:41.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:41 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:41.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:41 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:41.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:41 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:22:41.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:41 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:41.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:41 smithi195 ceph-mon[34786]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:22:41.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:41 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:41.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:41 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:41.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:41 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:41.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:41 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:22:41.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:41 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:42.200 DEBUG:teuthology.orchestra.run.smithi064:osd.3> sudo journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@osd.3.service 2024-09-12T03:22:42.203 INFO:tasks.cephadm:Deploying osd.4 on smithi115 with /dev/vg_nvme/lv_3... 2024-09-12T03:22:42.203 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T03:22:42.422 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:42 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:22:42.422 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:42 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:42.422 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:42 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:42.457 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:22:42.578 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:42 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:22:42.578 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:42 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:42.578 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:42 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:42.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:42 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:22:42.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:42 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:42.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:42 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:43.362 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:43 smithi064 ceph-mon[30952]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:22:43.577 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:43 smithi115 ceph-mon[34801]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:22:43.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:43 smithi195 ceph-mon[34786]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:22:44.022 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 12 03:22:43 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-3[51715]: 2024-09-12T03:22:43.677+0000 7f178239a640 -1 osd.3 0 log_to_monitors true 2024-09-12T03:22:44.422 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:44 smithi115 ceph-mon[34801]: from='osd.3 [v2:172.21.15.64:6810/3935517185,v1:172.21.15.64:6811/3935517185]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T03:22:44.685 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:44 smithi064 ceph-mon[30952]: from='osd.3 [v2:172.21.15.64:6810/3935517185,v1:172.21.15.64:6811/3935517185]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T03:22:44.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:44 smithi195 ceph-mon[34786]: from='osd.3 [v2:172.21.15.64:6810/3935517185,v1:172.21.15.64:6811/3935517185]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T03:22:44.962 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T03:22:44.987 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi115:vg_nvme/lv_3 2024-09-12T03:22:45.239 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:22:45.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:45 smithi115 ceph-mon[34801]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:22:45.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:45 smithi115 ceph-mon[34801]: from='osd.3 [v2:172.21.15.64:6810/3935517185,v1:172.21.15.64:6811/3935517185]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T03:22:45.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:45 smithi115 ceph-mon[34801]: osdmap e22: 4 total, 3 up, 4 in 2024-09-12T03:22:45.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:45 smithi115 ceph-mon[34801]: from='osd.3 [v2:172.21.15.64:6810/3935517185,v1:172.21.15.64:6811/3935517185]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-12T03:22:45.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:45 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:22:45.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:45 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:45.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:45 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:45.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:45 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:22:45.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:45 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:22:45.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:45 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:45.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:45 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:45.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:45 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:22:45.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:45 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:45.691 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:45 smithi064 ceph-mon[30952]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:22:45.692 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:45 smithi064 ceph-mon[30952]: from='osd.3 [v2:172.21.15.64:6810/3935517185,v1:172.21.15.64:6811/3935517185]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T03:22:45.692 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:45 smithi064 ceph-mon[30952]: osdmap e22: 4 total, 3 up, 4 in 2024-09-12T03:22:45.692 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:45 smithi064 ceph-mon[30952]: from='osd.3 [v2:172.21.15.64:6810/3935517185,v1:172.21.15.64:6811/3935517185]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-12T03:22:45.692 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:45 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:22:45.692 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:45 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:45.692 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:45 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:45.692 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:45 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:22:45.692 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:45 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:22:45.692 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:45 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:45.692 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:45 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:45.692 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:45 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:22:45.693 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:45 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:45.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:45 smithi195 ceph-mon[34786]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:22:45.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:45 smithi195 ceph-mon[34786]: from='osd.3 [v2:172.21.15.64:6810/3935517185,v1:172.21.15.64:6811/3935517185]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T03:22:45.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:45 smithi195 ceph-mon[34786]: osdmap e22: 4 total, 3 up, 4 in 2024-09-12T03:22:45.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:45 smithi195 ceph-mon[34786]: from='osd.3 [v2:172.21.15.64:6810/3935517185,v1:172.21.15.64:6811/3935517185]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-12T03:22:45.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:45 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:22:45.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:45 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:45.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:45 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:45.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:45 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:22:45.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:45 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:22:45.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:45 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:45.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:45 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:45.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:45 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:22:45.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:45 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:22:45.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:22:45.728+0000 7fd21d4f4640 1 -- 172.21.15.115:0/3505856446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd218101c00 msgr2=0x7fd218102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:45.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:22:45.728+0000 7fd21d4f4640 1 --2- 172.21.15.115:0/3505856446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd218101c00 0x7fd218102000 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd208009900 tx=0x7fd20802a410 comp rx=0 tx=0).stop 2024-09-12T03:22:45.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:22:45.729+0000 7fd21d4f4640 1 -- 172.21.15.115:0/3505856446 shutdown_connections 2024-09-12T03:22:45.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:22:45.729+0000 7fd21d4f4640 1 --2- 172.21.15.115:0/3505856446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2181037c0 0x7fd21810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:45.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:22:45.729+0000 7fd21d4f4640 1 --2- 172.21.15.115:0/3505856446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd218102e00 0x7fd218103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:45.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:22:45.729+0000 7fd21d4f4640 1 --2- 172.21.15.115:0/3505856446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd218101c00 0x7fd218102000 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:45.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:22:45.729+0000 7fd21d4f4640 1 -- 172.21.15.115:0/3505856446 >> 172.21.15.115:0/3505856446 conn(0x7fd2180fd390 msgr2=0x7fd2180ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:45.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:22:45.729+0000 7fd21d4f4640 1 -- 172.21.15.115:0/3505856446 shutdown_connections 2024-09-12T03:22:45.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:22:45.729+0000 7fd21d4f4640 1 -- 172.21.15.115:0/3505856446 wait complete. 2024-09-12T03:22:45.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:22:45.729+0000 7fd21d4f4640 1 Processor -- start 2024-09-12T03:22:45.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:22:45.729+0000 7fd21d4f4640 1 -- start start 2024-09-12T03:22:45.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:22:45.729+0000 7fd21d4f4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd218101c00 0x7fd218197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:45.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:22:45.729+0000 7fd21d4f4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd218102e00 0x7fd2181983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:45.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:22:45.729+0000 7fd21d4f4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2181037c0 0x7fd21819f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:45.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:22:45.729+0000 7fd21d4f4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd21810d790 con 0x7fd218101c00 2024-09-12T03:22:45.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:22:45.729+0000 7fd21d4f4640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd21810d610 con 0x7fd218102e00 2024-09-12T03:22:45.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:22:45.729+0000 7fd21d4f4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd21810d910 con 0x7fd2181037c0 2024-09-12T03:22:45.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:22:45.729+0000 7fd20ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd218102e00 0x7fd2181983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:45.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:22:45.729+0000 7fd20ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd218102e00 0x7fd2181983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48704/0 (socket says 172.21.15.115:48704) 2024-09-12T03:22:45.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:22:45.729+0000 7fd20ffff640 1 -- 172.21.15.115:0/392019161 learned_addr learned my addr 172.21.15.115:0/392019161 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-09-12T03:22:45.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:22:45.729+0000 7fd216ffd640 1 --2- 172.21.15.115:0/392019161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd218101c00 0x7fd218197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:45.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:22:45.730+0000 7fd20ffff640 1 -- 172.21.15.115:0/392019161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2181037c0 msgr2=0x7fd21819f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:45.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:22:45.730+0000 7fd20ffff640 1 --2- 172.21.15.115:0/392019161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2181037c0 0x7fd21819f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:45.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:22:45.730+0000 7fd20ffff640 1 -- 172.21.15.115:0/392019161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd218101c00 msgr2=0x7fd218197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:45.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:22:45.730+0000 7fd20ffff640 1 --2- 172.21.15.115:0/392019161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd218101c00 0x7fd218197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:45.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:22:45.730+0000 7fd20ffff640 1 -- 172.21.15.115:0/392019161 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd21819faa0 con 0x7fd218102e00 2024-09-12T03:22:45.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:22:45.730+0000 7fd20ffff640 1 --2- 172.21.15.115:0/392019161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd218102e00 0x7fd2181983b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd20000d950 tx=0x7fd20000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:45.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:22:45.730+0000 7fd214ff9640 1 -- 172.21.15.115:0/392019161 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd200017030 con 0x7fd218102e00 2024-09-12T03:22:45.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:22:45.730+0000 7fd21d4f4640 1 -- 172.21.15.115:0/392019161 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd21819fd90 con 0x7fd218102e00 2024-09-12T03:22:45.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:22:45.730+0000 7fd214ff9640 1 -- 172.21.15.115:0/392019161 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd200004020 con 0x7fd218102e00 2024-09-12T03:22:45.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:22:45.730+0000 7fd214ff9640 1 -- 172.21.15.115:0/392019161 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2000047a0 con 0x7fd218102e00 2024-09-12T03:22:45.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:22:45.730+0000 7fd21d4f4640 1 -- 172.21.15.115:0/392019161 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2181a02a0 con 0x7fd218102e00 2024-09-12T03:22:45.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:22:45.731+0000 7fd21d4f4640 1 -- 172.21.15.115:0/392019161 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2181090a0 con 0x7fd218102e00 2024-09-12T03:22:45.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:22:45.732+0000 7fd214ff9640 1 -- 172.21.15.115:0/392019161 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd2000041d0 con 0x7fd218102e00 2024-09-12T03:22:45.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:22:45.733+0000 7fd214ff9640 1 --2- 172.21.15.115:0/392019161 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fd2040a2bb0 0x7fd2040a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:45.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:22:45.733+0000 7fd216ffd640 1 --2- 172.21.15.115:0/392019161 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fd2040a2bb0 0x7fd2040a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:45.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:22:45.733+0000 7fd214ff9640 1 -- 172.21.15.115:0/392019161 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 3389+0+0 (secure 0 0 0) 0x7fd2000cd7a0 con 0x7fd218102e00 2024-09-12T03:22:45.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:22:45.734+0000 7fd216ffd640 1 --2- 172.21.15.115:0/392019161 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fd2040a2bb0 0x7fd2040a5070 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd208004770 tx=0x7fd208000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:45.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:22:45.738+0000 7fd214ff9640 1 -- 172.21.15.115:0/392019161 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd20009ba40 con 0x7fd218102e00 2024-09-12T03:22:45.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:22:45.911+0000 7fd21d4f4640 1 -- 172.21.15.115:0/392019161 --> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fd218062630 con 0x7fd2040a2bb0 2024-09-12T03:22:46.022 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 12 03:22:45 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-3[51715]: 2024-09-12T03:22:45.690+0000 7f177e31d640 -1 osd.3 0 waiting for initial osdmap 2024-09-12T03:22:46.022 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 12 03:22:45 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-3[51715]: 2024-09-12T03:22:45.695+0000 7f177613d640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:22:46.557 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:46 smithi115 ceph-mon[34801]: Detected new or changed devices on smithi064 2024-09-12T03:22:46.557 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:46 smithi115 ceph-mon[34801]: Adjusting osd_memory_target on smithi064 to 8544M 2024-09-12T03:22:46.557 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:46 smithi115 ceph-mon[34801]: from='osd.3 [v2:172.21.15.64:6810/3935517185,v1:172.21.15.64:6811/3935517185]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-12T03:22:46.557 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:46 smithi115 ceph-mon[34801]: osdmap e23: 4 total, 3 up, 4 in 2024-09-12T03:22:46.557 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:46 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:22:46.557 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:46 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:22:46.558 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:46 smithi115 ceph-mon[34801]: from='osd.3 [v2:172.21.15.64:6810/3935517185,v1:172.21.15.64:6811/3935517185]' entity='osd.3' 2024-09-12T03:22:46.558 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:46 smithi115 ceph-mon[34801]: osd.3 [v2:172.21.15.64:6810/3935517185,v1:172.21.15.64:6811/3935517185] boot 2024-09-12T03:22:46.558 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:46 smithi115 ceph-mon[34801]: osdmap e24: 4 total, 4 up, 4 in 2024-09-12T03:22:46.558 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:46 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:22:46.558 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:46 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:22:46.558 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:46 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:22:46.558 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:46 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:46.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:46 smithi195 ceph-mon[34786]: Detected new or changed devices on smithi064 2024-09-12T03:22:46.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:46 smithi195 ceph-mon[34786]: Adjusting osd_memory_target on smithi064 to 8544M 2024-09-12T03:22:46.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:46 smithi195 ceph-mon[34786]: from='osd.3 [v2:172.21.15.64:6810/3935517185,v1:172.21.15.64:6811/3935517185]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-12T03:22:46.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:46 smithi195 ceph-mon[34786]: osdmap e23: 4 total, 3 up, 4 in 2024-09-12T03:22:46.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:46 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:22:46.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:46 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:22:46.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:46 smithi195 ceph-mon[34786]: from='osd.3 [v2:172.21.15.64:6810/3935517185,v1:172.21.15.64:6811/3935517185]' entity='osd.3' 2024-09-12T03:22:46.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:46 smithi195 ceph-mon[34786]: osd.3 [v2:172.21.15.64:6810/3935517185,v1:172.21.15.64:6811/3935517185] boot 2024-09-12T03:22:46.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:46 smithi195 ceph-mon[34786]: osdmap e24: 4 total, 4 up, 4 in 2024-09-12T03:22:46.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:46 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:22:46.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:46 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:22:46.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:46 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:22:46.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:46 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:46.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:46 smithi064 ceph-mon[30952]: Detected new or changed devices on smithi064 2024-09-12T03:22:46.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:46 smithi064 ceph-mon[30952]: Adjusting osd_memory_target on smithi064 to 8544M 2024-09-12T03:22:46.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:46 smithi064 ceph-mon[30952]: from='osd.3 [v2:172.21.15.64:6810/3935517185,v1:172.21.15.64:6811/3935517185]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-12T03:22:46.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:46 smithi064 ceph-mon[30952]: osdmap e23: 4 total, 3 up, 4 in 2024-09-12T03:22:46.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:46 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:22:46.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:46 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:22:46.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:46 smithi064 ceph-mon[30952]: from='osd.3 [v2:172.21.15.64:6810/3935517185,v1:172.21.15.64:6811/3935517185]' entity='osd.3' 2024-09-12T03:22:46.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:46 smithi064 ceph-mon[30952]: osd.3 [v2:172.21.15.64:6810/3935517185,v1:172.21.15.64:6811/3935517185] boot 2024-09-12T03:22:46.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:46 smithi064 ceph-mon[30952]: osdmap e24: 4 total, 4 up, 4 in 2024-09-12T03:22:46.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:46 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:22:46.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:46 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:22:46.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:46 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:22:46.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:46 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:47.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:47 smithi064 ceph-mon[30952]: purged_snaps scrub starts 2024-09-12T03:22:47.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:47 smithi064 ceph-mon[30952]: purged_snaps scrub ok 2024-09-12T03:22:47.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:47 smithi064 ceph-mon[30952]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:22:47.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:47 smithi064 ceph-mon[30952]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:22:47.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:47 smithi064 ceph-mon[30952]: osdmap e25: 4 total, 4 up, 4 in 2024-09-12T03:22:47.563 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:47 smithi115 ceph-mon[34801]: purged_snaps scrub starts 2024-09-12T03:22:47.563 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:47 smithi115 ceph-mon[34801]: purged_snaps scrub ok 2024-09-12T03:22:47.563 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:47 smithi115 ceph-mon[34801]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:22:47.564 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:47 smithi115 ceph-mon[34801]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:22:47.564 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:47 smithi115 ceph-mon[34801]: osdmap e25: 4 total, 4 up, 4 in 2024-09-12T03:22:47.607 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:47 smithi195 ceph-mon[34786]: purged_snaps scrub starts 2024-09-12T03:22:47.607 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:47 smithi195 ceph-mon[34786]: purged_snaps scrub ok 2024-09-12T03:22:47.608 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:47 smithi195 ceph-mon[34786]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:22:47.608 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:47 smithi195 ceph-mon[34786]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:22:47.608 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:47 smithi195 ceph-mon[34786]: osdmap e25: 4 total, 4 up, 4 in 2024-09-12T03:22:49.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:48 smithi115 ceph-mon[34801]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:22:49.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:48 smithi115 ceph-mon[34801]: osdmap e26: 4 total, 4 up, 4 in 2024-09-12T03:22:49.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:48 smithi115 ceph-mon[34801]: from='client.? 172.21.15.115:0/1755784913' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "123aac47-a8ae-47c7-93a7-0dd293356322"}]: dispatch 2024-09-12T03:22:49.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:48 smithi115 ceph-mon[34801]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "123aac47-a8ae-47c7-93a7-0dd293356322"}]: dispatch 2024-09-12T03:22:49.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:48 smithi115 ceph-mon[34801]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "123aac47-a8ae-47c7-93a7-0dd293356322"}]': finished 2024-09-12T03:22:49.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:48 smithi115 ceph-mon[34801]: osdmap e27: 5 total, 4 up, 5 in 2024-09-12T03:22:49.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:48 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:22:49.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:48 smithi115 ceph-mon[34801]: from='client.? 172.21.15.115:0/92299257' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:22:49.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:48 smithi195 ceph-mon[34786]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:22:49.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:48 smithi195 ceph-mon[34786]: osdmap e26: 4 total, 4 up, 4 in 2024-09-12T03:22:49.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:48 smithi195 ceph-mon[34786]: from='client.? 172.21.15.115:0/1755784913' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "123aac47-a8ae-47c7-93a7-0dd293356322"}]: dispatch 2024-09-12T03:22:49.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:48 smithi195 ceph-mon[34786]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "123aac47-a8ae-47c7-93a7-0dd293356322"}]: dispatch 2024-09-12T03:22:49.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:48 smithi195 ceph-mon[34786]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "123aac47-a8ae-47c7-93a7-0dd293356322"}]': finished 2024-09-12T03:22:49.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:48 smithi195 ceph-mon[34786]: osdmap e27: 5 total, 4 up, 5 in 2024-09-12T03:22:49.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:48 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:22:49.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:48 smithi195 ceph-mon[34786]: from='client.? 172.21.15.115:0/92299257' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:22:49.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:48 smithi064 ceph-mon[30952]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:22:49.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:48 smithi064 ceph-mon[30952]: osdmap e26: 4 total, 4 up, 4 in 2024-09-12T03:22:49.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:48 smithi064 ceph-mon[30952]: from='client.? 172.21.15.115:0/1755784913' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "123aac47-a8ae-47c7-93a7-0dd293356322"}]: dispatch 2024-09-12T03:22:49.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:48 smithi064 ceph-mon[30952]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "123aac47-a8ae-47c7-93a7-0dd293356322"}]: dispatch 2024-09-12T03:22:49.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:48 smithi064 ceph-mon[30952]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "123aac47-a8ae-47c7-93a7-0dd293356322"}]': finished 2024-09-12T03:22:49.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:48 smithi064 ceph-mon[30952]: osdmap e27: 5 total, 4 up, 5 in 2024-09-12T03:22:49.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:48 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:22:49.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:48 smithi064 ceph-mon[30952]: from='client.? 172.21.15.115:0/92299257' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:22:50.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:50 smithi115 ceph-mon[34801]: osdmap e28: 5 total, 4 up, 5 in 2024-09-12T03:22:50.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:50 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:22:50.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:50 smithi195 ceph-mon[34786]: osdmap e28: 5 total, 4 up, 5 in 2024-09-12T03:22:50.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:50 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:22:50.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:50 smithi064 ceph-mon[30952]: osdmap e28: 5 total, 4 up, 5 in 2024-09-12T03:22:50.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:50 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:22:51.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:51 smithi115 ceph-mon[34801]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:22:51.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:51 smithi115 ceph-mon[34801]: osdmap e29: 5 total, 4 up, 5 in 2024-09-12T03:22:51.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:51 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:22:51.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:51 smithi195 ceph-mon[34786]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:22:51.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:51 smithi195 ceph-mon[34786]: osdmap e29: 5 total, 4 up, 5 in 2024-09-12T03:22:51.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:51 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:22:51.785 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:51 smithi064 ceph-mon[30952]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:22:51.785 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:51 smithi064 ceph-mon[30952]: osdmap e29: 5 total, 4 up, 5 in 2024-09-12T03:22:51.785 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:51 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:22:53.635 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:53 smithi115 ceph-mon[34801]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:22:53.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:53 smithi195 ceph-mon[34786]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:22:53.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:53 smithi064 ceph-mon[30952]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:22:55.588 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:55 smithi115 ceph-mon[34801]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-09-12T03:22:55.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:55 smithi195 ceph-mon[34786]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-09-12T03:22:55.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:55 smithi064 ceph-mon[30952]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-09-12T03:22:57.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:57 smithi115 ceph-mon[34801]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-12T03:22:57.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:57 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T03:22:57.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:57 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:57.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:57 smithi195 ceph-mon[34786]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-12T03:22:57.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:57 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T03:22:57.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:57 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:57.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:57 smithi064 ceph-mon[30952]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-12T03:22:57.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:57 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T03:22:57.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:57 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:58.575 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:58 smithi115 ceph-mon[34801]: Deploying daemon osd.4 on smithi115 2024-09-12T03:22:58.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:58 smithi195 ceph-mon[34786]: Deploying daemon osd.4 on smithi115 2024-09-12T03:22:58.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:58 smithi064 ceph-mon[30952]: Deploying daemon osd.4 on smithi115 2024-09-12T03:22:59.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:22:59 smithi115 ceph-mon[34801]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-12T03:22:59.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:22:59 smithi195 ceph-mon[34786]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-12T03:22:59.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:22:59 smithi064 ceph-mon[30952]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-12T03:23:01.565 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:01 smithi115 ceph-mon[34801]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T03:23:01.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:01 smithi195 ceph-mon[34786]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T03:23:01.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:01 smithi064 ceph-mon[30952]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T03:23:03.623 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:03 smithi115 ceph-mon[34801]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-12T03:23:03.624 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:03 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:23:03.624 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:03 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:03.624 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:03 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:03.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:03 smithi195 ceph-mon[34786]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-12T03:23:03.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:03 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:23:03.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:03 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:03.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:03 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:03.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:03 smithi064 ceph-mon[30952]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-12T03:23:03.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:03 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:23:03.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:03 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:03.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:03 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:05.443 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:05 smithi115 ceph-mon[34801]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T03:23:05.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:05 smithi195 ceph-mon[34786]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T03:23:05.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:05 smithi064 ceph-mon[30952]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T03:23:06.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:06 smithi064 ceph-mon[30952]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:23:06.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:06 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:06.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:06 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:06.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:06 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:06.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:06 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:23:06.900 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:06 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:06.930 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:06 smithi115 ceph-mon[34801]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:23:06.930 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:06 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:06.930 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:06 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:06.930 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:06 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:06.930 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:06 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:23:06.930 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:06 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:06.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:06 smithi195 ceph-mon[34786]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:23:06.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:06 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:06.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:06 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:06.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:06 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:06.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:06 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:23:06.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:06 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:06.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:23:06.967+0000 7fd214ff9640 1 -- 172.21.15.115:0/392019161 <== mgr.14150 v2:172.21.15.64:6800/759931848 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd218062630 con 0x7fd2040a2bb0 2024-09-12T03:23:06.968 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 4 on host 'smithi115' 2024-09-12T03:23:06.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:23:06.969+0000 7fd21d4f4640 1 -- 172.21.15.115:0/392019161 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fd2040a2bb0 msgr2=0x7fd2040a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:06.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:23:06.969+0000 7fd21d4f4640 1 --2- 172.21.15.115:0/392019161 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fd2040a2bb0 0x7fd2040a5070 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd208004770 tx=0x7fd208000bc0 comp rx=0 tx=0).stop 2024-09-12T03:23:06.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:23:06.969+0000 7fd21d4f4640 1 -- 172.21.15.115:0/392019161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd218102e00 msgr2=0x7fd2181983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:06.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:23:06.969+0000 7fd21d4f4640 1 --2- 172.21.15.115:0/392019161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd218102e00 0x7fd2181983b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd20000d950 tx=0x7fd20000de10 comp rx=0 tx=0).stop 2024-09-12T03:23:06.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:23:06.969+0000 7fd21d4f4640 1 -- 172.21.15.115:0/392019161 shutdown_connections 2024-09-12T03:23:06.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:23:06.969+0000 7fd21d4f4640 1 --2- 172.21.15.115:0/392019161 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fd2040a2bb0 0x7fd2040a5070 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:06.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:23:06.969+0000 7fd21d4f4640 1 --2- 172.21.15.115:0/392019161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2181037c0 0x7fd21819f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:06.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:23:06.969+0000 7fd21d4f4640 1 --2- 172.21.15.115:0/392019161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd218102e00 0x7fd2181983b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:06.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:23:06.969+0000 7fd21d4f4640 1 --2- 172.21.15.115:0/392019161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd218101c00 0x7fd218197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:06.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:23:06.969+0000 7fd21d4f4640 1 -- 172.21.15.115:0/392019161 >> 172.21.15.115:0/392019161 conn(0x7fd2180fd390 msgr2=0x7fd218106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:06.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:23:06.969+0000 7fd21d4f4640 1 -- 172.21.15.115:0/392019161 shutdown_connections 2024-09-12T03:23:06.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:23:06.969+0000 7fd21d4f4640 1 -- 172.21.15.115:0/392019161 wait complete. 2024-09-12T03:23:07.612 DEBUG:teuthology.orchestra.run.smithi115:osd.4> sudo journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@osd.4.service 2024-09-12T03:23:07.615 INFO:tasks.cephadm:Deploying osd.5 on smithi195 with /dev/vg_nvme/lv_3... 2024-09-12T03:23:07.615 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T03:23:07.834 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:07 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:23:07.834 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:07 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:07.834 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:07 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:07.866 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:23:07.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:07 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:23:07.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:07 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:07.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:07 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:08.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:07 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:23:08.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:07 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:08.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:07 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:09.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:08 smithi115 ceph-mon[34801]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:23:09.237 INFO:journalctl@ceph.osd.4.smithi115.stdout:Sep 12 03:23:09 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-4[45965]: 2024-09-12T03:23:09.144+0000 7f34594e1640 -1 osd.4 0 log_to_monitors true 2024-09-12T03:23:09.249 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:08 smithi195 ceph-mon[34786]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:23:09.271 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:08 smithi064 ceph-mon[30952]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:23:10.155 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:09 smithi195 ceph-mon[34786]: from='osd.4 [v2:172.21.15.115:6808/446665227,v1:172.21.15.115:6809/446665227]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T03:23:10.156 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:09 smithi195 ceph-mon[34786]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T03:23:10.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:09 smithi064 ceph-mon[30952]: from='osd.4 [v2:172.21.15.115:6808/446665227,v1:172.21.15.115:6809/446665227]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T03:23:10.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:09 smithi064 ceph-mon[30952]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T03:23:10.323 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:09 smithi115 ceph-mon[34801]: from='osd.4 [v2:172.21.15.115:6808/446665227,v1:172.21.15.115:6809/446665227]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T03:23:10.324 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:09 smithi115 ceph-mon[34801]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T03:23:10.443 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-12T03:23:10.470 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi195:vg_nvme/lv_3 2024-09-12T03:23:10.719 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:23:11.000 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:10 smithi195 ceph-mon[34786]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:23:11.000 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:10 smithi195 ceph-mon[34786]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T03:23:11.000 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:10 smithi195 ceph-mon[34786]: from='osd.4 [v2:172.21.15.115:6808/446665227,v1:172.21.15.115:6809/446665227]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-12T03:23:11.001 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:10 smithi195 ceph-mon[34786]: osdmap e30: 5 total, 4 up, 5 in 2024-09-12T03:23:11.001 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:10 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:23:11.001 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:10 smithi195 ceph-mon[34786]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-12T03:23:11.001 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:10 smithi195 ceph-mon[34786]: Detected new or changed devices on smithi115 2024-09-12T03:23:11.001 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:10 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:11.001 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:10 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:11.001 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:10 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:23:11.001 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:10 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:23:11.002 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:10 smithi195 ceph-mon[34786]: Adjusting osd_memory_target on smithi115 to 8544M 2024-09-12T03:23:11.002 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:10 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:11.002 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:10 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:11.002 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:10 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:23:11.002 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:10 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:11.202 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:11.188+0000 7ff63761c640 1 -- 172.21.15.195:0/303029771 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff630101cd0 msgr2=0x7ff630102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:11.202 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:11.188+0000 7ff63761c640 1 --2- 172.21.15.195:0/303029771 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff630101cd0 0x7ff630102150 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009900 tx=0x7ff62c02a410 comp rx=0 tx=0).stop 2024-09-12T03:23:11.203 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:11.188+0000 7ff63761c640 1 -- 172.21.15.195:0/303029771 shutdown_connections 2024-09-12T03:23:11.203 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:11.188+0000 7ff63761c640 1 --2- 172.21.15.195:0/303029771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff630102690 0x7ff630108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:11.203 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:11.188+0000 7ff63761c640 1 --2- 172.21.15.195:0/303029771 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff630101cd0 0x7ff630102150 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:11.203 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:11.188+0000 7ff63761c640 1 --2- 172.21.15.195:0/303029771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff630100ad0 0x7ff630100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:11.203 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:11.188+0000 7ff63761c640 1 -- 172.21.15.195:0/303029771 >> 172.21.15.195:0/303029771 conn(0x7ff6300fc280 msgr2=0x7ff6300fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:11.203 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:11.188+0000 7ff63761c640 1 -- 172.21.15.195:0/303029771 shutdown_connections 2024-09-12T03:23:11.203 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:11.188+0000 7ff63761c640 1 -- 172.21.15.195:0/303029771 wait complete. 2024-09-12T03:23:11.204 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:11.189+0000 7ff63761c640 1 Processor -- start 2024-09-12T03:23:11.204 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:11.189+0000 7ff63761c640 1 -- start start 2024-09-12T03:23:11.204 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:11.189+0000 7ff63761c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff630100ad0 0x7ff630197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:11.204 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:11.189+0000 7ff63761c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff630101cd0 0x7ff630198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:11.204 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:11.189+0000 7ff63761c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff630102690 0x7ff63019f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:11.204 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:11.189+0000 7ff63761c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff63010c600 con 0x7ff630102690 2024-09-12T03:23:11.204 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:11.189+0000 7ff63761c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff63010c480 con 0x7ff630101cd0 2024-09-12T03:23:11.204 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:11.189+0000 7ff63761c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff63010c780 con 0x7ff630100ad0 2024-09-12T03:23:11.204 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:11.189+0000 7ff6353b8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff630100ad0 0x7ff630197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:11.204 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:11.189+0000 7ff6353b8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff630100ad0 0x7ff630197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:58876/0 (socket says 172.21.15.195:58876) 2024-09-12T03:23:11.204 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:11.189+0000 7ff6353b8640 1 -- 172.21.15.195:0/3763913603 learned_addr learned my addr 172.21.15.195:0/3763913603 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-09-12T03:23:11.205 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:11.189+0000 7ff634bb7640 1 --2- 172.21.15.195:0/3763913603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff630101cd0 0x7ff630198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:11.205 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:11.189+0000 7ff635bb9640 1 --2- 172.21.15.195:0/3763913603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff630102690 0x7ff63019f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:11.205 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:11.189+0000 7ff6353b8640 1 -- 172.21.15.195:0/3763913603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff630101cd0 msgr2=0x7ff630198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:11.205 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:11.189+0000 7ff6353b8640 1 --2- 172.21.15.195:0/3763913603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff630101cd0 0x7ff630198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:11.205 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:11.189+0000 7ff6353b8640 1 -- 172.21.15.195:0/3763913603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff630102690 msgr2=0x7ff63019f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:11.205 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:11.189+0000 7ff6353b8640 1 --2- 172.21.15.195:0/3763913603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff630102690 0x7ff63019f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:11.205 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:11.189+0000 7ff6353b8640 1 -- 172.21.15.195:0/3763913603 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff63019fa40 con 0x7ff630100ad0 2024-09-12T03:23:11.205 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:11.189+0000 7ff6353b8640 1 --2- 172.21.15.195:0/3763913603 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff630100ad0 0x7ff630197e10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff62000e9c0 tx=0x7ff62000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:11.205 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:11.190+0000 7ff6267fc640 1 -- 172.21.15.195:0/3763913603 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62000cd80 con 0x7ff630100ad0 2024-09-12T03:23:11.205 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:11.190+0000 7ff6267fc640 1 -- 172.21.15.195:0/3763913603 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6200035f0 con 0x7ff630100ad0 2024-09-12T03:23:11.205 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:11.190+0000 7ff63761c640 1 -- 172.21.15.195:0/3763913603 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff63019fd30 con 0x7ff630100ad0 2024-09-12T03:23:11.205 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:11.190+0000 7ff6267fc640 1 -- 172.21.15.195:0/3763913603 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff620010690 con 0x7ff630100ad0 2024-09-12T03:23:11.205 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:11.190+0000 7ff63761c640 1 -- 172.21.15.195:0/3763913603 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6301a0190 con 0x7ff630100ad0 2024-09-12T03:23:11.206 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:11.191+0000 7ff63761c640 1 -- 172.21.15.195:0/3763913603 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6301a0420 con 0x7ff630100ad0 2024-09-12T03:23:11.206 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:11.192+0000 7ff6267fc640 1 -- 172.21.15.195:0/3763913603 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff6200037a0 con 0x7ff630100ad0 2024-09-12T03:23:11.206 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:11.192+0000 7ff6267fc640 1 --2- 172.21.15.195:0/3763913603 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff61c0a2bb0 0x7ff61c0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:11.206 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:11.193+0000 7ff6267fc640 1 -- 172.21.15.195:0/3763913603 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 3821+0+0 (secure 0 0 0) 0x7ff6200ce320 con 0x7ff630100ad0 2024-09-12T03:23:11.206 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:11.193+0000 7ff634bb7640 1 --2- 172.21.15.195:0/3763913603 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff61c0a2bb0 0x7ff61c0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:11.206 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:11.193+0000 7ff634bb7640 1 --2- 172.21.15.195:0/3763913603 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff61c0a2bb0 0x7ff61c0a5070 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff62c006010 tx=0x7ff62c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:11.206 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:11.196+0000 7ff6267fc640 1 -- 172.21.15.195:0/3763913603 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff62009c410 con 0x7ff630100ad0 2024-09-12T03:23:11.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:10 smithi064 ceph-mon[30952]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:23:11.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:10 smithi064 ceph-mon[30952]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T03:23:11.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:10 smithi064 ceph-mon[30952]: from='osd.4 [v2:172.21.15.115:6808/446665227,v1:172.21.15.115:6809/446665227]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-12T03:23:11.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:10 smithi064 ceph-mon[30952]: osdmap e30: 5 total, 4 up, 5 in 2024-09-12T03:23:11.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:10 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:23:11.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:10 smithi064 ceph-mon[30952]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-12T03:23:11.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:10 smithi064 ceph-mon[30952]: Detected new or changed devices on smithi115 2024-09-12T03:23:11.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:10 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:11.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:10 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:11.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:10 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:23:11.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:10 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:23:11.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:10 smithi064 ceph-mon[30952]: Adjusting osd_memory_target on smithi115 to 8544M 2024-09-12T03:23:11.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:10 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:11.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:10 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:11.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:10 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:23:11.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:10 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:11.378 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:11.378+0000 7ff63761c640 1 -- 172.21.15.195:0/3763913603 --> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ff63006a6f0 con 0x7ff61c0a2bb0 2024-09-12T03:23:11.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:10 smithi115 ceph-mon[34801]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:23:11.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:10 smithi115 ceph-mon[34801]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T03:23:11.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:10 smithi115 ceph-mon[34801]: from='osd.4 [v2:172.21.15.115:6808/446665227,v1:172.21.15.115:6809/446665227]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-12T03:23:11.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:10 smithi115 ceph-mon[34801]: osdmap e30: 5 total, 4 up, 5 in 2024-09-12T03:23:11.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:10 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:23:11.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:10 smithi115 ceph-mon[34801]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-12T03:23:11.394 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:10 smithi115 ceph-mon[34801]: Detected new or changed devices on smithi115 2024-09-12T03:23:11.394 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:10 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:11.394 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:10 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:11.394 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:10 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:23:11.394 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:10 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:23:11.394 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:10 smithi115 ceph-mon[34801]: Adjusting osd_memory_target on smithi115 to 8544M 2024-09-12T03:23:11.394 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:10 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:11.395 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:10 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:11.395 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:10 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:23:11.395 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:10 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:11.892 INFO:journalctl@ceph.osd.4.smithi115.stdout:Sep 12 03:23:11 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-4[45965]: 2024-09-12T03:23:11.392+0000 7f3455c78640 -1 osd.4 0 waiting for initial osdmap 2024-09-12T03:23:11.893 INFO:journalctl@ceph.osd.4.smithi115.stdout:Sep 12 03:23:11 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-4[45965]: 2024-09-12T03:23:11.396+0000 7f344d284640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:23:12.097 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:11 smithi195 ceph-mon[34786]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-12T03:23:12.097 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:11 smithi195 ceph-mon[34786]: osdmap e31: 5 total, 4 up, 5 in 2024-09-12T03:23:12.097 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:11 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:23:12.097 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:11 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:23:12.097 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:11 smithi195 ceph-mon[34786]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:23:12.097 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:11 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:23:12.097 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:11 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:23:12.098 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:11 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:12.098 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:11 smithi195 ceph-mon[34786]: from='osd.4 ' entity='osd.4' 2024-09-12T03:23:12.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:11 smithi064 ceph-mon[30952]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-12T03:23:12.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:11 smithi064 ceph-mon[30952]: osdmap e31: 5 total, 4 up, 5 in 2024-09-12T03:23:12.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:11 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:23:12.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:11 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:23:12.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:11 smithi064 ceph-mon[30952]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:23:12.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:11 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:23:12.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:11 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:23:12.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:11 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:12.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:11 smithi064 ceph-mon[30952]: from='osd.4 ' entity='osd.4' 2024-09-12T03:23:12.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:11 smithi115 ceph-mon[34801]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-12T03:23:12.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:11 smithi115 ceph-mon[34801]: osdmap e31: 5 total, 4 up, 5 in 2024-09-12T03:23:12.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:11 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:23:12.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:11 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:23:12.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:11 smithi115 ceph-mon[34801]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:23:12.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:11 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:23:12.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:11 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:23:12.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:11 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:12.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:11 smithi115 ceph-mon[34801]: from='osd.4 ' entity='osd.4' 2024-09-12T03:23:13.245 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:12 smithi195 ceph-mon[34786]: purged_snaps scrub starts 2024-09-12T03:23:13.246 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:12 smithi195 ceph-mon[34786]: purged_snaps scrub ok 2024-09-12T03:23:13.246 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:12 smithi195 ceph-mon[34786]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:23:13.246 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:12 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:23:13.246 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:12 smithi195 ceph-mon[34786]: osd.4 [v2:172.21.15.115:6808/446665227,v1:172.21.15.115:6809/446665227] boot 2024-09-12T03:23:13.246 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:12 smithi195 ceph-mon[34786]: osdmap e32: 5 total, 5 up, 5 in 2024-09-12T03:23:13.246 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:12 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:23:13.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:12 smithi064 ceph-mon[30952]: purged_snaps scrub starts 2024-09-12T03:23:13.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:12 smithi064 ceph-mon[30952]: purged_snaps scrub ok 2024-09-12T03:23:13.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:12 smithi064 ceph-mon[30952]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:23:13.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:12 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:23:13.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:12 smithi064 ceph-mon[30952]: osd.4 [v2:172.21.15.115:6808/446665227,v1:172.21.15.115:6809/446665227] boot 2024-09-12T03:23:13.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:12 smithi064 ceph-mon[30952]: osdmap e32: 5 total, 5 up, 5 in 2024-09-12T03:23:13.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:12 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:23:13.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:12 smithi115 ceph-mon[34801]: purged_snaps scrub starts 2024-09-12T03:23:13.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:12 smithi115 ceph-mon[34801]: purged_snaps scrub ok 2024-09-12T03:23:13.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:12 smithi115 ceph-mon[34801]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:23:13.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:12 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:23:13.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:12 smithi115 ceph-mon[34801]: osd.4 [v2:172.21.15.115:6808/446665227,v1:172.21.15.115:6809/446665227] boot 2024-09-12T03:23:13.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:12 smithi115 ceph-mon[34801]: osdmap e32: 5 total, 5 up, 5 in 2024-09-12T03:23:13.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:12 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:23:14.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:13 smithi064 ceph-mon[30952]: from='client.? 172.21.15.195:0/4001149037' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c90bcaed-e7a8-4852-8677-b01dc73ea44e"}]: dispatch 2024-09-12T03:23:14.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:13 smithi064 ceph-mon[30952]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c90bcaed-e7a8-4852-8677-b01dc73ea44e"}]: dispatch 2024-09-12T03:23:14.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:13 smithi064 ceph-mon[30952]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c90bcaed-e7a8-4852-8677-b01dc73ea44e"}]': finished 2024-09-12T03:23:14.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:13 smithi064 ceph-mon[30952]: osdmap e33: 6 total, 5 up, 6 in 2024-09-12T03:23:14.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:13 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:23:14.282 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:13 smithi195 ceph-mon[34786]: from='client.? 172.21.15.195:0/4001149037' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c90bcaed-e7a8-4852-8677-b01dc73ea44e"}]: dispatch 2024-09-12T03:23:14.282 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:13 smithi195 ceph-mon[34786]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c90bcaed-e7a8-4852-8677-b01dc73ea44e"}]: dispatch 2024-09-12T03:23:14.282 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:13 smithi195 ceph-mon[34786]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c90bcaed-e7a8-4852-8677-b01dc73ea44e"}]': finished 2024-09-12T03:23:14.282 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:13 smithi195 ceph-mon[34786]: osdmap e33: 6 total, 5 up, 6 in 2024-09-12T03:23:14.282 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:13 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:23:14.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:13 smithi115 ceph-mon[34801]: from='client.? 172.21.15.195:0/4001149037' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c90bcaed-e7a8-4852-8677-b01dc73ea44e"}]: dispatch 2024-09-12T03:23:14.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:13 smithi115 ceph-mon[34801]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c90bcaed-e7a8-4852-8677-b01dc73ea44e"}]: dispatch 2024-09-12T03:23:14.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:13 smithi115 ceph-mon[34801]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c90bcaed-e7a8-4852-8677-b01dc73ea44e"}]': finished 2024-09-12T03:23:14.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:13 smithi115 ceph-mon[34801]: osdmap e33: 6 total, 5 up, 6 in 2024-09-12T03:23:14.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:13 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:23:15.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:14 smithi064 ceph-mon[30952]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:23:15.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:14 smithi064 ceph-mon[30952]: from='client.? 172.21.15.195:0/1761636461' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:23:15.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:14 smithi115 ceph-mon[34801]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:23:15.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:14 smithi115 ceph-mon[34801]: from='client.? 172.21.15.195:0/1761636461' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:23:15.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:14 smithi195 ceph-mon[34786]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:23:15.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:14 smithi195 ceph-mon[34786]: from='client.? 172.21.15.195:0/1761636461' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:23:17.258 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:16 smithi195 ceph-mon[34786]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:23:17.271 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:16 smithi064 ceph-mon[30952]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:23:17.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:16 smithi115 ceph-mon[34801]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:23:19.175 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:18 smithi195 ceph-mon[34786]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:23:19.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:18 smithi064 ceph-mon[30952]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:23:19.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:18 smithi115 ceph-mon[34801]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:23:21.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:21 smithi064 ceph-mon[30952]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:23:21.286 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:21 smithi195 ceph-mon[34786]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:23:21.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:21 smithi115 ceph-mon[34801]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:23:23.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:23 smithi195 ceph-mon[34786]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:23:23.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:23 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T03:23:23.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:23 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:23.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:23 smithi195 ceph-mon[34786]: Deploying daemon osd.5 on smithi195 2024-09-12T03:23:23.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:23 smithi064 ceph-mon[30952]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:23:23.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:23 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T03:23:23.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:23 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:23.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:23 smithi064 ceph-mon[30952]: Deploying daemon osd.5 on smithi195 2024-09-12T03:23:23.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:23 smithi115 ceph-mon[34801]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:23:23.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:23 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T03:23:23.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:23 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:23.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:23 smithi115 ceph-mon[34801]: Deploying daemon osd.5 on smithi195 2024-09-12T03:23:25.134 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:25 smithi195 ceph-mon[34786]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:23:25.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:25 smithi064 ceph-mon[30952]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:23:25.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:25 smithi115 ceph-mon[34801]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:23:27.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:27 smithi064 ceph-mon[30952]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:23:27.287 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:27 smithi195 ceph-mon[34786]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:23:27.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:27 smithi115 ceph-mon[34801]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:23:29.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:29 smithi195 ceph-mon[34786]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:23:29.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:29 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:23:29.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:29 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:29.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:29 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:29.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:29 smithi064 ceph-mon[30952]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:23:29.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:29 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:23:29.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:29 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:29.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:29 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:29.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:29 smithi115 ceph-mon[34801]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:23:29.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:29 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:23:29.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:29 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:29.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:29 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:31.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:31 smithi064 ceph-mon[30952]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:23:31.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:31 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:31.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:31 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:31.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:31 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:31.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:31 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:23:31.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:31 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:31.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:31 smithi115 ceph-mon[34801]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:23:31.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:31 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:31.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:31 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:31.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:31 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:31.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:31 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:23:31.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:31 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:31.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:31 smithi195 ceph-mon[34786]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:23:31.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:31 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:31.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:31 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:31.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:31 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:31.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:31 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:23:31.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:31 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:32.117 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:32.117+0000 7ff6267fc640 1 -- 172.21.15.195:0/3763913603 <== mgr.14150 v2:172.21.15.64:6800/759931848 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff63006a6f0 con 0x7ff61c0a2bb0 2024-09-12T03:23:32.117 INFO:teuthology.orchestra.run.smithi195.stdout:Created osd(s) 5 on host 'smithi195' 2024-09-12T03:23:32.120 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:32.120+0000 7ff63761c640 1 -- 172.21.15.195:0/3763913603 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff61c0a2bb0 msgr2=0x7ff61c0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:32.120 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:32.120+0000 7ff63761c640 1 --2- 172.21.15.195:0/3763913603 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff61c0a2bb0 0x7ff61c0a5070 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff62c006010 tx=0x7ff62c000bc0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.120 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:32.120+0000 7ff63761c640 1 -- 172.21.15.195:0/3763913603 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff630100ad0 msgr2=0x7ff630197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:32.120 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:32.120+0000 7ff63761c640 1 --2- 172.21.15.195:0/3763913603 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff630100ad0 0x7ff630197e10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff62000e9c0 tx=0x7ff62000ee80 comp rx=0 tx=0).stop 2024-09-12T03:23:32.120 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:32.120+0000 7ff63761c640 1 -- 172.21.15.195:0/3763913603 shutdown_connections 2024-09-12T03:23:32.120 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:32.120+0000 7ff63761c640 1 --2- 172.21.15.195:0/3763913603 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff61c0a2bb0 0x7ff61c0a5070 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.120 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:32.120+0000 7ff63761c640 1 --2- 172.21.15.195:0/3763913603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff630102690 0x7ff63019f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.121 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:32.120+0000 7ff63761c640 1 --2- 172.21.15.195:0/3763913603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff630101cd0 0x7ff630198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.121 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:32.120+0000 7ff63761c640 1 --2- 172.21.15.195:0/3763913603 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff630100ad0 0x7ff630197e10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.121 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:32.120+0000 7ff63761c640 1 -- 172.21.15.195:0/3763913603 >> 172.21.15.195:0/3763913603 conn(0x7ff6300fc280 msgr2=0x7ff630069fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:32.121 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:32.120+0000 7ff63761c640 1 -- 172.21.15.195:0/3763913603 shutdown_connections 2024-09-12T03:23:32.121 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:23:32.120+0000 7ff63761c640 1 -- 172.21.15.195:0/3763913603 wait complete. 2024-09-12T03:23:32.436 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:32 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:23:32.436 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:32 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:32.436 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:32 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:32.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:32 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:23:32.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:32 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:32.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:32 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:32.731 DEBUG:teuthology.orchestra.run.smithi195:osd.5> sudo journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@osd.5.service 2024-09-12T03:23:32.734 INFO:tasks.cephadm:Deploying osd.6 on smithi064 with /dev/vg_nvme/lv_2... 2024-09-12T03:23:32.734 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T03:23:32.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:32 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:23:32.763 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:32 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:32.763 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:32 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:33.000 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:23:33.473 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:33 smithi064 ceph-mon[30952]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:23:33.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:33 smithi115 ceph-mon[34801]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:23:33.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:33 smithi195 ceph-mon[34786]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:23:34.087 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:23:34 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5[46203]: 2024-09-12T03:23:34.029+0000 7f6c65468640 -1 osd.5 0 log_to_monitors true 2024-09-12T03:23:34.388 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:34 smithi064 ceph-mon[30952]: from='osd.5 [v2:172.21.15.195:6808/1187293668,v1:172.21.15.195:6809/1187293668]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T03:23:34.388 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:34 smithi064 ceph-mon[30952]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T03:23:34.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:34 smithi195 ceph-mon[34786]: from='osd.5 [v2:172.21.15.195:6808/1187293668,v1:172.21.15.195:6809/1187293668]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T03:23:34.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:34 smithi195 ceph-mon[34786]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T03:23:34.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:34 smithi115 ceph-mon[34801]: from='osd.5 [v2:172.21.15.195:6808/1187293668,v1:172.21.15.195:6809/1187293668]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T03:23:34.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:34 smithi115 ceph-mon[34801]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T03:23:35.401 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:23:35 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5[46203]: 2024-09-12T03:23:35.286+0000 7f6c613eb640 -1 osd.5 0 waiting for initial osdmap 2024-09-12T03:23:35.402 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:23:35 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5[46203]: 2024-09-12T03:23:35.295+0000 7f6c5d213640 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:23:35.402 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:35 smithi195 ceph-mon[34786]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:23:35.402 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:35 smithi195 ceph-mon[34786]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T03:23:35.402 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:35 smithi195 ceph-mon[34786]: osdmap e34: 6 total, 5 up, 6 in 2024-09-12T03:23:35.402 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:35 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:23:35.402 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:35 smithi195 ceph-mon[34786]: from='osd.5 [v2:172.21.15.195:6808/1187293668,v1:172.21.15.195:6809/1187293668]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-09-12T03:23:35.402 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:35 smithi195 ceph-mon[34786]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-09-12T03:23:35.419 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:23:35.450 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi064:vg_nvme/lv_2 2024-09-12T03:23:35.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:35 smithi115 ceph-mon[34801]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:23:35.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:35 smithi115 ceph-mon[34801]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T03:23:35.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:35 smithi115 ceph-mon[34801]: osdmap e34: 6 total, 5 up, 6 in 2024-09-12T03:23:35.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:35 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:23:35.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:35 smithi115 ceph-mon[34801]: from='osd.5 [v2:172.21.15.195:6808/1187293668,v1:172.21.15.195:6809/1187293668]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-09-12T03:23:35.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:35 smithi115 ceph-mon[34801]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-09-12T03:23:35.762 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:23:35.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:35 smithi064 ceph-mon[30952]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:23:35.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:35 smithi064 ceph-mon[30952]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T03:23:35.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:35 smithi064 ceph-mon[30952]: osdmap e34: 6 total, 5 up, 6 in 2024-09-12T03:23:35.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:35 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:23:35.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:35 smithi064 ceph-mon[30952]: from='osd.5 [v2:172.21.15.195:6808/1187293668,v1:172.21.15.195:6809/1187293668]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-09-12T03:23:35.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:35 smithi064 ceph-mon[30952]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-09-12T03:23:36.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:36.232+0000 7fb8bc696640 1 -- 172.21.15.64:0/3346312797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8b4102690 msgr2=0x7fb8b4108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:36.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:36.232+0000 7fb8bc696640 1 --2- 172.21.15.64:0/3346312797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8b4102690 0x7fb8b4108f20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac009ca0 tx=0x7fb8ac02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:23:36.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:36.232+0000 7fb8bc696640 1 -- 172.21.15.64:0/3346312797 shutdown_connections 2024-09-12T03:23:36.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:36.232+0000 7fb8bc696640 1 --2- 172.21.15.64:0/3346312797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8b4102690 0x7fb8b4108f20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:36.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:36.232+0000 7fb8bc696640 1 --2- 172.21.15.64:0/3346312797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8b4101cd0 0x7fb8b4102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:36.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:36.232+0000 7fb8bc696640 1 --2- 172.21.15.64:0/3346312797 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8b4100ad0 0x7fb8b4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:36.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:36.232+0000 7fb8bc696640 1 -- 172.21.15.64:0/3346312797 >> 172.21.15.64:0/3346312797 conn(0x7fb8b40fc280 msgr2=0x7fb8b40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:36.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:36.232+0000 7fb8bc696640 1 -- 172.21.15.64:0/3346312797 shutdown_connections 2024-09-12T03:23:36.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:36.233+0000 7fb8bc696640 1 -- 172.21.15.64:0/3346312797 wait complete. 2024-09-12T03:23:36.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:36.233+0000 7fb8bc696640 1 Processor -- start 2024-09-12T03:23:36.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:36.233+0000 7fb8bc696640 1 -- start start 2024-09-12T03:23:36.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:36.234+0000 7fb8bc696640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8b4100ad0 0x7fb8b4195ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:36.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:36.234+0000 7fb8bc696640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8b4101cd0 0x7fb8b41961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:36.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:36.234+0000 7fb8bc696640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8b4102690 0x7fb8b419d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:36.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:36.234+0000 7fb8bc696640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b410b5f0 con 0x7fb8b4101cd0 2024-09-12T03:23:36.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:36.234+0000 7fb8bc696640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b410b470 con 0x7fb8b4102690 2024-09-12T03:23:36.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:36.234+0000 7fb8bc696640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b410b770 con 0x7fb8b4100ad0 2024-09-12T03:23:36.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:36.234+0000 7fb8b9c31640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8b4101cd0 0x7fb8b41961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:36.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:36.234+0000 7fb8b9c31640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8b4101cd0 0x7fb8b41961e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46310/0 (socket says 172.21.15.64:46310) 2024-09-12T03:23:36.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:36.234+0000 7fb8b9c31640 1 -- 172.21.15.64:0/3931804786 learned_addr learned my addr 172.21.15.64:0/3931804786 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:23:36.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:36.234+0000 7fb8ba432640 1 --2- 172.21.15.64:0/3931804786 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8b4100ad0 0x7fb8b4195ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:36.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:36.234+0000 7fb8b9c31640 1 -- 172.21.15.64:0/3931804786 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8b4100ad0 msgr2=0x7fb8b4195ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:36.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:36.234+0000 7fb8b9c31640 1 --2- 172.21.15.64:0/3931804786 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8b4100ad0 0x7fb8b4195ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:36.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:36.234+0000 7fb8bac33640 1 --2- 172.21.15.64:0/3931804786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8b4102690 0x7fb8b419d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:36.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:36.234+0000 7fb8b9c31640 1 -- 172.21.15.64:0/3931804786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8b4102690 msgr2=0x7fb8b419d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:36.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:36.234+0000 7fb8b9c31640 1 --2- 172.21.15.64:0/3931804786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8b4102690 0x7fb8b419d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:36.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:36.234+0000 7fb8b9c31640 1 -- 172.21.15.64:0/3931804786 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8b419d8c0 con 0x7fb8b4101cd0 2024-09-12T03:23:36.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:36.234+0000 7fb8b9c31640 1 --2- 172.21.15.64:0/3931804786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8b4101cd0 0x7fb8b41961e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb8a400b4a0 tx=0x7fb8a400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:36.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:36.235+0000 7fb8ab7fe640 1 -- 172.21.15.64:0/3931804786 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a4015020 con 0x7fb8b4101cd0 2024-09-12T03:23:36.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:36.235+0000 7fb8ab7fe640 1 -- 172.21.15.64:0/3931804786 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8a4011ce0 con 0x7fb8b4101cd0 2024-09-12T03:23:36.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:36.235+0000 7fb8bc696640 1 -- 172.21.15.64:0/3931804786 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8b419dbb0 con 0x7fb8b4101cd0 2024-09-12T03:23:36.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:36.235+0000 7fb8ab7fe640 1 -- 172.21.15.64:0/3931804786 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a4010dc0 con 0x7fb8b4101cd0 2024-09-12T03:23:36.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:36.235+0000 7fb8bc696640 1 -- 172.21.15.64:0/3931804786 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8b419e0f0 con 0x7fb8b4101cd0 2024-09-12T03:23:36.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:36.236+0000 7fb8bc696640 1 -- 172.21.15.64:0/3931804786 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8b4107f20 con 0x7fb8b4101cd0 2024-09-12T03:23:36.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:36.237+0000 7fb8ab7fe640 1 -- 172.21.15.64:0/3931804786 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb8a4018020 con 0x7fb8b4101cd0 2024-09-12T03:23:36.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:36.238+0000 7fb8ab7fe640 1 --2- 172.21.15.64:0/3931804786 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fb8a00a4d90 0x7fb8a00a7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:36.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:36.238+0000 7fb8ba432640 1 --2- 172.21.15.64:0/3931804786 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fb8a00a4d90 0x7fb8a00a7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:36.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:36.238+0000 7fb8ab7fe640 1 -- 172.21.15.64:0/3931804786 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4566+0+0 (secure 0 0 0) 0x7fb8a40cdad0 con 0x7fb8b4101cd0 2024-09-12T03:23:36.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:36.238+0000 7fb8ba432640 1 --2- 172.21.15.64:0/3931804786 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fb8a00a4d90 0x7fb8a00a7250 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4196b80 tx=0x7fb8b0009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:36.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:36.242+0000 7fb8ab7fe640 1 -- 172.21.15.64:0/3931804786 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb8a401f020 con 0x7fb8b4101cd0 2024-09-12T03:23:36.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:36.448+0000 7fb8bc696640 1 -- 172.21.15.64:0/3931804786 --> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fb8b4062630 con 0x7fb8a00a4d90 2024-09-12T03:23:36.449 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:36 smithi064 ceph-mon[30952]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-09-12T03:23:36.449 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:36 smithi064 ceph-mon[30952]: osdmap e35: 6 total, 5 up, 6 in 2024-09-12T03:23:36.449 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:36 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:23:36.449 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:36 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:23:36.449 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:36 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:36.449 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:36 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:36.449 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:36 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:23:36.449 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:36 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:23:36.450 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:36 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:36.450 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:36 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:36.450 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:36 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:23:36.450 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:36 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:36.450 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:36 smithi064 ceph-mon[30952]: osd.5 [v2:172.21.15.195:6808/1187293668,v1:172.21.15.195:6809/1187293668] boot 2024-09-12T03:23:36.450 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:36 smithi064 ceph-mon[30952]: osdmap e36: 6 total, 6 up, 6 in 2024-09-12T03:23:36.450 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:36 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:23:36.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:36 smithi115 ceph-mon[34801]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-09-12T03:23:36.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:36 smithi115 ceph-mon[34801]: osdmap e35: 6 total, 5 up, 6 in 2024-09-12T03:23:36.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:36 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:23:36.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:36 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:23:36.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:36 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:36.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:36 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:36.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:36 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:23:36.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:36 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:23:36.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:36 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:36.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:36 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:36.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:36 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:23:36.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:36 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:36.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:36 smithi115 ceph-mon[34801]: osd.5 [v2:172.21.15.195:6808/1187293668,v1:172.21.15.195:6809/1187293668] boot 2024-09-12T03:23:36.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:36 smithi115 ceph-mon[34801]: osdmap e36: 6 total, 6 up, 6 in 2024-09-12T03:23:36.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:36 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:23:36.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:36 smithi195 ceph-mon[34786]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-09-12T03:23:36.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:36 smithi195 ceph-mon[34786]: osdmap e35: 6 total, 5 up, 6 in 2024-09-12T03:23:36.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:36 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:23:36.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:36 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:23:36.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:36 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:36.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:36 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:36.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:36 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:23:36.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:36 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:23:36.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:36 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:36.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:36 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:36.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:36 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:23:36.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:36 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:36.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:36 smithi195 ceph-mon[34786]: osd.5 [v2:172.21.15.195:6808/1187293668,v1:172.21.15.195:6809/1187293668] boot 2024-09-12T03:23:36.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:36 smithi195 ceph-mon[34786]: osdmap e36: 6 total, 6 up, 6 in 2024-09-12T03:23:36.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:36 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:23:37.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:37 smithi064 ceph-mon[30952]: purged_snaps scrub starts 2024-09-12T03:23:37.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:37 smithi064 ceph-mon[30952]: purged_snaps scrub ok 2024-09-12T03:23:37.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:37 smithi064 ceph-mon[30952]: Detected new or changed devices on smithi195 2024-09-12T03:23:37.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:37 smithi064 ceph-mon[30952]: Adjusting osd_memory_target on smithi195 to 8544M 2024-09-12T03:23:37.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:37 smithi064 ceph-mon[30952]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:23:37.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:37 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:23:37.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:37 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:23:37.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:37 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:37.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:37 smithi064 ceph-mon[30952]: osdmap e37: 6 total, 6 up, 6 in 2024-09-12T03:23:37.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:37 smithi115 ceph-mon[34801]: purged_snaps scrub starts 2024-09-12T03:23:37.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:37 smithi115 ceph-mon[34801]: purged_snaps scrub ok 2024-09-12T03:23:37.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:37 smithi115 ceph-mon[34801]: Detected new or changed devices on smithi195 2024-09-12T03:23:37.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:37 smithi115 ceph-mon[34801]: Adjusting osd_memory_target on smithi195 to 8544M 2024-09-12T03:23:37.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:37 smithi115 ceph-mon[34801]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:23:37.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:37 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:23:37.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:37 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:23:37.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:37 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:37.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:37 smithi115 ceph-mon[34801]: osdmap e37: 6 total, 6 up, 6 in 2024-09-12T03:23:37.709 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:37 smithi195 ceph-mon[34786]: purged_snaps scrub starts 2024-09-12T03:23:37.709 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:37 smithi195 ceph-mon[34786]: purged_snaps scrub ok 2024-09-12T03:23:37.709 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:37 smithi195 ceph-mon[34786]: Detected new or changed devices on smithi195 2024-09-12T03:23:37.709 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:37 smithi195 ceph-mon[34786]: Adjusting osd_memory_target on smithi195 to 8544M 2024-09-12T03:23:37.709 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:37 smithi195 ceph-mon[34786]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:23:37.709 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:37 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:23:37.709 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:37 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:23:37.709 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:37 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:37.709 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:37 smithi195 ceph-mon[34786]: osdmap e37: 6 total, 6 up, 6 in 2024-09-12T03:23:38.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:38 smithi064 ceph-mon[30952]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:23:38.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:38 smithi064 ceph-mon[30952]: osdmap e38: 6 total, 6 up, 6 in 2024-09-12T03:23:38.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:38 smithi115 ceph-mon[34801]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:23:38.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:38 smithi115 ceph-mon[34801]: osdmap e38: 6 total, 6 up, 6 in 2024-09-12T03:23:38.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:38 smithi195 ceph-mon[34786]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:23:38.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:38 smithi195 ceph-mon[34786]: osdmap e38: 6 total, 6 up, 6 in 2024-09-12T03:23:39.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:39 smithi064 ceph-mon[30952]: pgmap v132: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:23:39.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:39 smithi064 ceph-mon[30952]: osdmap e39: 6 total, 6 up, 6 in 2024-09-12T03:23:39.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:39 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/4066331163' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd292547-6695-4889-8867-e74318bbb00f"}]: dispatch 2024-09-12T03:23:39.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:39 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/4066331163' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd292547-6695-4889-8867-e74318bbb00f"}]': finished 2024-09-12T03:23:39.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:39 smithi064 ceph-mon[30952]: osdmap e40: 7 total, 6 up, 7 in 2024-09-12T03:23:39.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:39 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:23:39.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:39 smithi115 ceph-mon[34801]: pgmap v132: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:23:39.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:39 smithi115 ceph-mon[34801]: osdmap e39: 6 total, 6 up, 6 in 2024-09-12T03:23:39.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:39 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/4066331163' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd292547-6695-4889-8867-e74318bbb00f"}]: dispatch 2024-09-12T03:23:39.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:39 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/4066331163' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd292547-6695-4889-8867-e74318bbb00f"}]': finished 2024-09-12T03:23:39.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:39 smithi115 ceph-mon[34801]: osdmap e40: 7 total, 6 up, 7 in 2024-09-12T03:23:39.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:39 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:23:39.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:39 smithi195 ceph-mon[34786]: pgmap v132: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:23:39.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:39 smithi195 ceph-mon[34786]: osdmap e39: 6 total, 6 up, 6 in 2024-09-12T03:23:39.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:39 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/4066331163' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd292547-6695-4889-8867-e74318bbb00f"}]: dispatch 2024-09-12T03:23:39.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:39 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/4066331163' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd292547-6695-4889-8867-e74318bbb00f"}]': finished 2024-09-12T03:23:39.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:39 smithi195 ceph-mon[34786]: osdmap e40: 7 total, 6 up, 7 in 2024-09-12T03:23:39.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:39 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:23:40.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:40 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/3030459579' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:23:40.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:40 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/3030459579' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:23:40.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:40 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/3030459579' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:23:41.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:41 smithi115 ceph-mon[34801]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:23:41.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:41 smithi195 ceph-mon[34786]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:23:41.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:41 smithi064 ceph-mon[30952]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:23:43.565 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:43 smithi064 ceph-mon[30952]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:23:43.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:43 smithi115 ceph-mon[34801]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:23:43.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:43 smithi195 ceph-mon[34786]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:23:45.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:45 smithi115 ceph-mon[34801]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T03:23:45.666 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:45 smithi064 ceph-mon[30952]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T03:23:45.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:45 smithi195 ceph-mon[34786]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T03:23:47.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:47 smithi064 ceph-mon[30952]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-12T03:23:47.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:47 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T03:23:47.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:47 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:47.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:47 smithi115 ceph-mon[34801]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-12T03:23:47.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:47 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T03:23:47.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:47 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:47.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:47 smithi195 ceph-mon[34786]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-12T03:23:47.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:47 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T03:23:47.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:47 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:48.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:48 smithi115 ceph-mon[34801]: Deploying daemon osd.6 on smithi064 2024-09-12T03:23:48.698 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:48 smithi064 ceph-mon[30952]: Deploying daemon osd.6 on smithi064 2024-09-12T03:23:48.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:48 smithi195 ceph-mon[34786]: Deploying daemon osd.6 on smithi064 2024-09-12T03:23:49.638 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:49 smithi064 ceph-mon[30952]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-12T03:23:49.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:49 smithi115 ceph-mon[34801]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-12T03:23:49.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:49 smithi195 ceph-mon[34786]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-12T03:23:51.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:51 smithi115 ceph-mon[34801]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-12T03:23:51.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:51 smithi195 ceph-mon[34786]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-12T03:23:51.752 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:51 smithi064 ceph-mon[30952]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-12T03:23:53.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:53 smithi115 ceph-mon[34801]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T03:23:53.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:53 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:23:53.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:53 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:53.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:53 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:53.661 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:53 smithi064 ceph-mon[30952]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T03:23:53.661 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:53 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:23:53.661 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:53 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:53.662 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:53 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:53.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:53 smithi195 ceph-mon[34786]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T03:23:53.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:53 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:23:53.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:53 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:53.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:53 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:55.483 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:55 smithi064 ceph-mon[30952]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T03:23:55.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:55 smithi115 ceph-mon[34801]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T03:23:55.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:55 smithi195 ceph-mon[34786]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T03:23:56.765 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 6 on host 'smithi064' 2024-09-12T03:23:56.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:56.765+0000 7fb8ab7fe640 1 -- 172.21.15.64:0/3931804786 <== mgr.14150 v2:172.21.15.64:6800/759931848 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb8b4062630 con 0x7fb8a00a4d90 2024-09-12T03:23:56.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:56.767+0000 7fb8bc696640 1 -- 172.21.15.64:0/3931804786 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fb8a00a4d90 msgr2=0x7fb8a00a7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:56.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:56.767+0000 7fb8bc696640 1 --2- 172.21.15.64:0/3931804786 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fb8a00a4d90 0x7fb8a00a7250 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4196b80 tx=0x7fb8b0009110 comp rx=0 tx=0).stop 2024-09-12T03:23:56.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:56.767+0000 7fb8bc696640 1 -- 172.21.15.64:0/3931804786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8b4101cd0 msgr2=0x7fb8b41961e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:56.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:56.767+0000 7fb8bc696640 1 --2- 172.21.15.64:0/3931804786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8b4101cd0 0x7fb8b41961e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb8a400b4a0 tx=0x7fb8a400b8a0 comp rx=0 tx=0).stop 2024-09-12T03:23:56.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:56.767+0000 7fb8bc696640 1 -- 172.21.15.64:0/3931804786 shutdown_connections 2024-09-12T03:23:56.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:56.767+0000 7fb8bc696640 1 --2- 172.21.15.64:0/3931804786 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fb8a00a4d90 0x7fb8a00a7250 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:56.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:56.767+0000 7fb8bc696640 1 --2- 172.21.15.64:0/3931804786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8b4102690 0x7fb8b419d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:56.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:56.767+0000 7fb8bc696640 1 --2- 172.21.15.64:0/3931804786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8b4101cd0 0x7fb8b41961e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:56.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:56.767+0000 7fb8bc696640 1 --2- 172.21.15.64:0/3931804786 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8b4100ad0 0x7fb8b4195ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:56.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:56.767+0000 7fb8bc696640 1 -- 172.21.15.64:0/3931804786 >> 172.21.15.64:0/3931804786 conn(0x7fb8b40fc280 msgr2=0x7fb8b41050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:56.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:56.767+0000 7fb8bc696640 1 -- 172.21.15.64:0/3931804786 shutdown_connections 2024-09-12T03:23:56.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:23:56.767+0000 7fb8bc696640 1 -- 172.21.15.64:0/3931804786 wait complete. 2024-09-12T03:23:57.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:56 smithi064 ceph-mon[30952]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:23:57.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:56 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:57.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:56 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:57.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:56 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:57.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:56 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:23:57.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:56 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:57.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:56 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:23:57.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:56 smithi115 ceph-mon[34801]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:23:57.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:56 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:57.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:56 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:57.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:56 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:57.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:56 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:23:57.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:56 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:57.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:56 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:23:57.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:56 smithi195 ceph-mon[34786]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:23:57.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:56 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:57.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:56 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:57.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:56 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:57.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:56 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:23:57.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:56 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:57.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:56 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:23:57.416 DEBUG:teuthology.orchestra.run.smithi064:osd.6> sudo journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@osd.6.service 2024-09-12T03:23:57.419 INFO:tasks.cephadm:Deploying osd.7 on smithi115 with /dev/vg_nvme/lv_2... 2024-09-12T03:23:57.419 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T03:23:57.672 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:23:57.758 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:57 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:57.758 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:57 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:58.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:57 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:58.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:57 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:58.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:57 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:58.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:57 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:23:58.962 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:58 smithi064 ceph-mon[30952]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:23:59.091 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:58 smithi115 ceph-mon[34801]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:23:59.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:58 smithi195 ceph-mon[34786]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:23:59.772 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:23:59 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:23:59.280+0000 7f40ca464640 -1 osd.6 0 log_to_monitors true 2024-09-12T03:24:00.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:23:59 smithi064 ceph-mon[30952]: from='osd.6 [v2:172.21.15.64:6818/560343274,v1:172.21.15.64:6819/560343274]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T03:24:00.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:23:59 smithi115 ceph-mon[34801]: from='osd.6 [v2:172.21.15.64:6818/560343274,v1:172.21.15.64:6819/560343274]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T03:24:00.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:23:59 smithi195 ceph-mon[34786]: from='osd.6 [v2:172.21.15.64:6818/560343274,v1:172.21.15.64:6819/560343274]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T03:24:00.252 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T03:24:00.277 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi115:vg_nvme/lv_2 2024-09-12T03:24:00.520 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:24:00.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:00 smithi115 ceph-mon[34801]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:24:00.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:00 smithi115 ceph-mon[34801]: from='osd.6 [v2:172.21.15.64:6818/560343274,v1:172.21.15.64:6819/560343274]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T03:24:00.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:00 smithi115 ceph-mon[34801]: osdmap e41: 7 total, 6 up, 7 in 2024-09-12T03:24:00.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:00 smithi115 ceph-mon[34801]: from='osd.6 [v2:172.21.15.64:6818/560343274,v1:172.21.15.64:6819/560343274]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-12T03:24:00.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:00 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:24:00.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:00 smithi115 ceph-mon[34801]: Detected new or changed devices on smithi064 2024-09-12T03:24:00.894 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:00 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:00.894 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:00 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:00.894 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:00 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:24:00.894 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:00 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:24:00.894 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:00 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:24:00.894 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:00 smithi115 ceph-mon[34801]: Adjusting osd_memory_target on smithi064 to 5696M 2024-09-12T03:24:00.894 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:00 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:00.894 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:00 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:00.894 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:00 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:24:00.894 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:00 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:00.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:00.944+0000 7fd948f5f640 1 -- 172.21.15.115:0/875831449 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd944100ad0 msgr2=0x7fd944100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:00.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:00.944+0000 7fd948f5f640 1 --2- 172.21.15.115:0/875831449 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd944100ad0 0x7fd944100ed0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd934009900 tx=0x7fd93402a410 comp rx=0 tx=0).stop 2024-09-12T03:24:00.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:00.944+0000 7fd948f5f640 1 -- 172.21.15.115:0/875831449 shutdown_connections 2024-09-12T03:24:00.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:00.944+0000 7fd948f5f640 1 --2- 172.21.15.115:0/875831449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd944102690 0x7fd944108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:00.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:00.944+0000 7fd948f5f640 1 --2- 172.21.15.115:0/875831449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd944101cd0 0x7fd944102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:00.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:00.944+0000 7fd948f5f640 1 --2- 172.21.15.115:0/875831449 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd944100ad0 0x7fd944100ed0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:00.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:00.944+0000 7fd948f5f640 1 -- 172.21.15.115:0/875831449 >> 172.21.15.115:0/875831449 conn(0x7fd9440fc280 msgr2=0x7fd9440fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:00.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:00.944+0000 7fd948f5f640 1 -- 172.21.15.115:0/875831449 shutdown_connections 2024-09-12T03:24:00.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:00.944+0000 7fd948f5f640 1 -- 172.21.15.115:0/875831449 wait complete. 2024-09-12T03:24:00.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:00.945+0000 7fd948f5f640 1 Processor -- start 2024-09-12T03:24:00.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:00.945+0000 7fd948f5f640 1 -- start start 2024-09-12T03:24:00.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:00.945+0000 7fd948f5f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd944100ad0 0x7fd944197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:00.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:00.945+0000 7fd948f5f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd944101cd0 0x7fd944198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:00.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:00.945+0000 7fd948f5f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd944102690 0x7fd94419f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:00.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:00.945+0000 7fd948f5f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd94410c600 con 0x7fd944101cd0 2024-09-12T03:24:00.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:00.945+0000 7fd948f5f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd94410c480 con 0x7fd944100ad0 2024-09-12T03:24:00.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:00.945+0000 7fd948f5f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd94410c780 con 0x7fd944102690 2024-09-12T03:24:00.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:00.945+0000 7fd94259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd944100ad0 0x7fd944197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:00.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:00.945+0000 7fd94259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd944100ad0 0x7fd944197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56148/0 (socket says 172.21.15.115:56148) 2024-09-12T03:24:00.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:00.945+0000 7fd941d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd944101cd0 0x7fd944198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:00.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:00.945+0000 7fd94259c640 1 -- 172.21.15.115:0/3648525695 learned_addr learned my addr 172.21.15.115:0/3648525695 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-09-12T03:24:00.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:00.945+0000 7fd942d9d640 1 --2- 172.21.15.115:0/3648525695 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd944102690 0x7fd94419f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:00.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:00.946+0000 7fd94259c640 1 -- 172.21.15.115:0/3648525695 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd944102690 msgr2=0x7fd94419f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:00.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:00.946+0000 7fd94259c640 1 --2- 172.21.15.115:0/3648525695 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd944102690 0x7fd94419f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:00.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:00.946+0000 7fd94259c640 1 -- 172.21.15.115:0/3648525695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd944101cd0 msgr2=0x7fd944198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:00.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:00.946+0000 7fd94259c640 1 --2- 172.21.15.115:0/3648525695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd944101cd0 0x7fd944198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:00.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:00.946+0000 7fd94259c640 1 -- 172.21.15.115:0/3648525695 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd94419fa70 con 0x7fd944100ad0 2024-09-12T03:24:00.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:00.946+0000 7fd941d9b640 1 --2- 172.21.15.115:0/3648525695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd944101cd0 0x7fd944198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:24:00.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:00.946+0000 7fd94259c640 1 --2- 172.21.15.115:0/3648525695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd944100ad0 0x7fd944197e40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd9340314e0 tx=0x7fd934031a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:00.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:00.946+0000 7fd9337fe640 1 -- 172.21.15.115:0/3648525695 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93403f030 con 0x7fd944100ad0 2024-09-12T03:24:00.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:00.946+0000 7fd9337fe640 1 -- 172.21.15.115:0/3648525695 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9340044b0 con 0x7fd944100ad0 2024-09-12T03:24:00.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:00.946+0000 7fd948f5f640 1 -- 172.21.15.115:0/3648525695 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd94419fd00 con 0x7fd944100ad0 2024-09-12T03:24:00.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:00.946+0000 7fd9337fe640 1 -- 172.21.15.115:0/3648525695 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd934035070 con 0x7fd944100ad0 2024-09-12T03:24:00.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:00.946+0000 7fd948f5f640 1 -- 172.21.15.115:0/3648525695 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9441a0190 con 0x7fd944100ad0 2024-09-12T03:24:00.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:00.947+0000 7fd948f5f640 1 -- 172.21.15.115:0/3648525695 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9441a0360 con 0x7fd944100ad0 2024-09-12T03:24:00.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:00.948+0000 7fd9337fe640 1 -- 172.21.15.115:0/3648525695 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd934042030 con 0x7fd944100ad0 2024-09-12T03:24:00.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:00.949+0000 7fd9337fe640 1 --2- 172.21.15.115:0/3648525695 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fd92c0a2bb0 0x7fd92c0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:00.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:00.949+0000 7fd9337fe640 1 -- 172.21.15.115:0/3648525695 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4685+0+0 (secure 0 0 0) 0x7fd9340ee730 con 0x7fd944100ad0 2024-09-12T03:24:00.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:00.949+0000 7fd941d9b640 1 --2- 172.21.15.115:0/3648525695 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fd92c0a2bb0 0x7fd92c0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:00.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:00.949+0000 7fd941d9b640 1 --2- 172.21.15.115:0/3648525695 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fd92c0a2bb0 0x7fd92c0a5070 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd944199360 tx=0x7fd93800a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:00.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:00.953+0000 7fd9337fe640 1 -- 172.21.15.115:0/3648525695 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd9340bc4c0 con 0x7fd944100ad0 2024-09-12T03:24:01.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:01.127+0000 7fd948f5f640 1 -- 172.21.15.115:0/3648525695 --> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fd94406a730 con 0x7fd92c0a2bb0 2024-09-12T03:24:01.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:00 smithi195 ceph-mon[34786]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:24:01.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:00 smithi195 ceph-mon[34786]: from='osd.6 [v2:172.21.15.64:6818/560343274,v1:172.21.15.64:6819/560343274]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T03:24:01.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:00 smithi195 ceph-mon[34786]: osdmap e41: 7 total, 6 up, 7 in 2024-09-12T03:24:01.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:00 smithi195 ceph-mon[34786]: from='osd.6 [v2:172.21.15.64:6818/560343274,v1:172.21.15.64:6819/560343274]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-12T03:24:01.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:00 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:24:01.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:00 smithi195 ceph-mon[34786]: Detected new or changed devices on smithi064 2024-09-12T03:24:01.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:00 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:01.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:00 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:01.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:00 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:24:01.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:00 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:24:01.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:00 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:24:01.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:00 smithi195 ceph-mon[34786]: Adjusting osd_memory_target on smithi064 to 5696M 2024-09-12T03:24:01.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:00 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:01.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:00 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:01.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:00 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:24:01.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:00 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:01.208 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:00 smithi064 ceph-mon[30952]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:24:01.208 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:00 smithi064 ceph-mon[30952]: from='osd.6 [v2:172.21.15.64:6818/560343274,v1:172.21.15.64:6819/560343274]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T03:24:01.208 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:00 smithi064 ceph-mon[30952]: osdmap e41: 7 total, 6 up, 7 in 2024-09-12T03:24:01.208 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:00 smithi064 ceph-mon[30952]: from='osd.6 [v2:172.21.15.64:6818/560343274,v1:172.21.15.64:6819/560343274]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-12T03:24:01.208 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:00 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:24:01.208 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:00 smithi064 ceph-mon[30952]: Detected new or changed devices on smithi064 2024-09-12T03:24:01.208 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:00 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:01.208 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:00 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:01.209 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:00 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:24:01.209 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:00 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:24:01.209 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:00 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:24:01.209 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:00 smithi064 ceph-mon[30952]: Adjusting osd_memory_target on smithi064 to 5696M 2024-09-12T03:24:01.209 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:00 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:01.209 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:00 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:01.209 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:00 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:24:01.209 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:00 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:01.522 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:24:01 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:24:01.205+0000 7f40c6bfb640 -1 osd.6 0 waiting for initial osdmap 2024-09-12T03:24:01.522 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:24:01 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:24:01.215+0000 7f40be207640 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:24:02.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:01 smithi115 ceph-mon[34801]: from='osd.6 [v2:172.21.15.64:6818/560343274,v1:172.21.15.64:6819/560343274]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-12T03:24:02.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:01 smithi115 ceph-mon[34801]: osdmap e42: 7 total, 6 up, 7 in 2024-09-12T03:24:02.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:01 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:24:02.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:01 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:24:02.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:01 smithi115 ceph-mon[34801]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:24:02.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:01 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:24:02.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:01 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:24:02.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:01 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:02.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:01 smithi115 ceph-mon[34801]: from='osd.6 [v2:172.21.15.64:6818/560343274,v1:172.21.15.64:6819/560343274]' entity='osd.6' 2024-09-12T03:24:02.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:01 smithi195 ceph-mon[34786]: from='osd.6 [v2:172.21.15.64:6818/560343274,v1:172.21.15.64:6819/560343274]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-12T03:24:02.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:01 smithi195 ceph-mon[34786]: osdmap e42: 7 total, 6 up, 7 in 2024-09-12T03:24:02.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:01 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:24:02.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:01 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:24:02.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:01 smithi195 ceph-mon[34786]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:24:02.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:01 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:24:02.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:01 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:24:02.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:01 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:02.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:01 smithi195 ceph-mon[34786]: from='osd.6 [v2:172.21.15.64:6818/560343274,v1:172.21.15.64:6819/560343274]' entity='osd.6' 2024-09-12T03:24:02.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:01 smithi064 ceph-mon[30952]: from='osd.6 [v2:172.21.15.64:6818/560343274,v1:172.21.15.64:6819/560343274]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-12T03:24:02.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:01 smithi064 ceph-mon[30952]: osdmap e42: 7 total, 6 up, 7 in 2024-09-12T03:24:02.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:01 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:24:02.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:01 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:24:02.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:01 smithi064 ceph-mon[30952]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:24:02.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:01 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:24:02.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:01 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:24:02.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:01 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:02.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:01 smithi064 ceph-mon[30952]: from='osd.6 [v2:172.21.15.64:6818/560343274,v1:172.21.15.64:6819/560343274]' entity='osd.6' 2024-09-12T03:24:03.106 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:02 smithi115 ceph-mon[34801]: purged_snaps scrub starts 2024-09-12T03:24:03.106 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:02 smithi115 ceph-mon[34801]: purged_snaps scrub ok 2024-09-12T03:24:03.106 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:02 smithi115 ceph-mon[34801]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:24:03.106 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:02 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:24:03.106 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:02 smithi115 ceph-mon[34801]: osd.6 [v2:172.21.15.64:6818/560343274,v1:172.21.15.64:6819/560343274] boot 2024-09-12T03:24:03.107 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:02 smithi115 ceph-mon[34801]: osdmap e43: 7 total, 7 up, 7 in 2024-09-12T03:24:03.107 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:02 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:24:03.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:02 smithi195 ceph-mon[34786]: purged_snaps scrub starts 2024-09-12T03:24:03.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:02 smithi195 ceph-mon[34786]: purged_snaps scrub ok 2024-09-12T03:24:03.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:02 smithi195 ceph-mon[34786]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:24:03.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:02 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:24:03.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:02 smithi195 ceph-mon[34786]: osd.6 [v2:172.21.15.64:6818/560343274,v1:172.21.15.64:6819/560343274] boot 2024-09-12T03:24:03.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:02 smithi195 ceph-mon[34786]: osdmap e43: 7 total, 7 up, 7 in 2024-09-12T03:24:03.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:02 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:24:03.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:02 smithi064 ceph-mon[30952]: purged_snaps scrub starts 2024-09-12T03:24:03.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:02 smithi064 ceph-mon[30952]: purged_snaps scrub ok 2024-09-12T03:24:03.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:02 smithi064 ceph-mon[30952]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:24:03.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:02 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:24:03.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:02 smithi064 ceph-mon[30952]: osd.6 [v2:172.21.15.64:6818/560343274,v1:172.21.15.64:6819/560343274] boot 2024-09-12T03:24:03.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:02 smithi064 ceph-mon[30952]: osdmap e43: 7 total, 7 up, 7 in 2024-09-12T03:24:03.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:02 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:24:04.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:04 smithi115 ceph-mon[34801]: osdmap e44: 7 total, 7 up, 7 in 2024-09-12T03:24:04.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:04 smithi115 ceph-mon[34801]: from='client.? 172.21.15.115:0/3537392417' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad7c4151-cd98-4113-879b-ca2b7c83c7cc"}]: dispatch 2024-09-12T03:24:04.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:04 smithi115 ceph-mon[34801]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad7c4151-cd98-4113-879b-ca2b7c83c7cc"}]: dispatch 2024-09-12T03:24:04.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:04 smithi115 ceph-mon[34801]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad7c4151-cd98-4113-879b-ca2b7c83c7cc"}]': finished 2024-09-12T03:24:04.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:04 smithi115 ceph-mon[34801]: osdmap e45: 8 total, 7 up, 8 in 2024-09-12T03:24:04.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:04 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:24:04.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:04 smithi115 ceph-mon[34801]: from='client.? 172.21.15.115:0/1678423578' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:24:04.525 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:04 smithi064 ceph-mon[30952]: osdmap e44: 7 total, 7 up, 7 in 2024-09-12T03:24:04.525 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:04 smithi064 ceph-mon[30952]: from='client.? 172.21.15.115:0/3537392417' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad7c4151-cd98-4113-879b-ca2b7c83c7cc"}]: dispatch 2024-09-12T03:24:04.525 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:04 smithi064 ceph-mon[30952]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad7c4151-cd98-4113-879b-ca2b7c83c7cc"}]: dispatch 2024-09-12T03:24:04.525 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:04 smithi064 ceph-mon[30952]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad7c4151-cd98-4113-879b-ca2b7c83c7cc"}]': finished 2024-09-12T03:24:04.525 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:04 smithi064 ceph-mon[30952]: osdmap e45: 8 total, 7 up, 8 in 2024-09-12T03:24:04.525 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:04 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:24:04.525 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:04 smithi064 ceph-mon[30952]: from='client.? 172.21.15.115:0/1678423578' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:24:04.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:04 smithi195 ceph-mon[34786]: osdmap e44: 7 total, 7 up, 7 in 2024-09-12T03:24:04.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:04 smithi195 ceph-mon[34786]: from='client.? 172.21.15.115:0/3537392417' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad7c4151-cd98-4113-879b-ca2b7c83c7cc"}]: dispatch 2024-09-12T03:24:04.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:04 smithi195 ceph-mon[34786]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad7c4151-cd98-4113-879b-ca2b7c83c7cc"}]: dispatch 2024-09-12T03:24:04.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:04 smithi195 ceph-mon[34786]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad7c4151-cd98-4113-879b-ca2b7c83c7cc"}]': finished 2024-09-12T03:24:04.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:04 smithi195 ceph-mon[34786]: osdmap e45: 8 total, 7 up, 8 in 2024-09-12T03:24:04.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:04 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:24:04.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:04 smithi195 ceph-mon[34786]: from='client.? 172.21.15.115:0/1678423578' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:24:05.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:05 smithi064 ceph-mon[30952]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:24:05.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:05 smithi115 ceph-mon[34801]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:24:05.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:05 smithi195 ceph-mon[34786]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:24:07.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:07 smithi064 ceph-mon[30952]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:24:07.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:07 smithi115 ceph-mon[34801]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:24:07.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:07 smithi195 ceph-mon[34786]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:24:09.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:09 smithi115 ceph-mon[34801]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:24:09.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:09 smithi064 ceph-mon[30952]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:24:09.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:09 smithi195 ceph-mon[34786]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:24:11.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:11 smithi064 ceph-mon[30952]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:24:11.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:11 smithi115 ceph-mon[34801]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:24:11.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:11 smithi195 ceph-mon[34786]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:24:12.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:12 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T03:24:12.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:12 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:12.563 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:12 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T03:24:12.563 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:12 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:12.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:12 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T03:24:12.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:12 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:13.310 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:13 smithi115 ceph-mon[34801]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:24:13.310 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:13 smithi115 ceph-mon[34801]: Deploying daemon osd.7 on smithi115 2024-09-12T03:24:13.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:13 smithi064 ceph-mon[30952]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:24:13.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:13 smithi064 ceph-mon[30952]: Deploying daemon osd.7 on smithi115 2024-09-12T03:24:13.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:13 smithi195 ceph-mon[34786]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:24:13.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:13 smithi195 ceph-mon[34786]: Deploying daemon osd.7 on smithi115 2024-09-12T03:24:15.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:15 smithi064 ceph-mon[30952]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:24:15.555 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:15 smithi115 ceph-mon[34801]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:24:15.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:15 smithi195 ceph-mon[34786]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:24:17.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:17 smithi115 ceph-mon[34801]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:24:17.508 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:17 smithi195 ceph-mon[34786]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:24:17.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:17 smithi064 ceph-mon[30952]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:24:18.517 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:18 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:24:18.517 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:18 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:18.517 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:18 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:18.521 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:18 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:24:18.521 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:18 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:18.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:18 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:18.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:18 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:24:18.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:18 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:18.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:18 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:19.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:19 smithi064 ceph-mon[30952]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:24:19.548 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:19 smithi115 ceph-mon[34801]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:24:19.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:19 smithi195 ceph-mon[34786]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:24:21.375 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:21 smithi115 ceph-mon[34801]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:24:21.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:21.413+0000 7fd9337fe640 1 -- 172.21.15.115:0/3648525695 <== mgr.14150 v2:172.21.15.64:6800/759931848 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd94406a730 con 0x7fd92c0a2bb0 2024-09-12T03:24:21.414 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 7 on host 'smithi115' 2024-09-12T03:24:21.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:21.415+0000 7fd948f5f640 1 -- 172.21.15.115:0/3648525695 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fd92c0a2bb0 msgr2=0x7fd92c0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:21.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:21.415+0000 7fd948f5f640 1 --2- 172.21.15.115:0/3648525695 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fd92c0a2bb0 0x7fd92c0a5070 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd944199360 tx=0x7fd93800a250 comp rx=0 tx=0).stop 2024-09-12T03:24:21.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:21.415+0000 7fd948f5f640 1 -- 172.21.15.115:0/3648525695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd944100ad0 msgr2=0x7fd944197e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:21.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:21.415+0000 7fd948f5f640 1 --2- 172.21.15.115:0/3648525695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd944100ad0 0x7fd944197e40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd9340314e0 tx=0x7fd934031a30 comp rx=0 tx=0).stop 2024-09-12T03:24:21.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:21.415+0000 7fd948f5f640 1 -- 172.21.15.115:0/3648525695 shutdown_connections 2024-09-12T03:24:21.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:21.415+0000 7fd948f5f640 1 --2- 172.21.15.115:0/3648525695 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fd92c0a2bb0 0x7fd92c0a5070 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:21.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:21.415+0000 7fd948f5f640 1 --2- 172.21.15.115:0/3648525695 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd944102690 0x7fd94419f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:21.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:21.415+0000 7fd948f5f640 1 --2- 172.21.15.115:0/3648525695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd944101cd0 0x7fd944198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:21.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:21.415+0000 7fd948f5f640 1 --2- 172.21.15.115:0/3648525695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd944100ad0 0x7fd944197e40 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:21.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:21.415+0000 7fd948f5f640 1 -- 172.21.15.115:0/3648525695 >> 172.21.15.115:0/3648525695 conn(0x7fd9440fc280 msgr2=0x7fd94406a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:21.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:21.415+0000 7fd948f5f640 1 -- 172.21.15.115:0/3648525695 shutdown_connections 2024-09-12T03:24:21.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:24:21.415+0000 7fd948f5f640 1 -- 172.21.15.115:0/3648525695 wait complete. 2024-09-12T03:24:21.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:21 smithi064 ceph-mon[30952]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:24:21.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:21 smithi195 ceph-mon[34786]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:24:22.125 DEBUG:teuthology.orchestra.run.smithi115:osd.7> sudo journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@osd.7.service 2024-09-12T03:24:22.127 INFO:tasks.cephadm:Deploying osd.8 on smithi195 with /dev/vg_nvme/lv_2... 2024-09-12T03:24:22.127 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T03:24:22.383 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:24:22.664 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:22 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:22.664 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:22 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:22.664 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:22 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:22.664 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:22 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:22.684 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:22 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:22.684 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:22 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:22.684 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:22 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:22.685 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:22 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:22.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:22 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:22.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:22 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:22.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:22 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:22.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:22 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:23.690 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:23 smithi195 ceph-mon[34786]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:24:23.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:23 smithi064 ceph-mon[30952]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:24:23.820 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:23 smithi115 ceph-mon[34801]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:24:23.820 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:24:23 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:24:23.646+0000 7f15a83ec640 -1 osd.7 0 log_to_monitors true 2024-09-12T03:24:24.507 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:24 smithi195 ceph-mon[34786]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:24:24.507 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:24 smithi195 ceph-mon[34786]: from='osd.7 [v2:172.21.15.115:6816/3075734174,v1:172.21.15.115:6817/3075734174]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T03:24:24.507 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:24 smithi195 ceph-mon[34786]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T03:24:24.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:24 smithi064 ceph-mon[30952]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:24:24.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:24 smithi064 ceph-mon[30952]: from='osd.7 [v2:172.21.15.115:6816/3075734174,v1:172.21.15.115:6817/3075734174]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T03:24:24.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:24 smithi064 ceph-mon[30952]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T03:24:24.808 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-12T03:24:24.832 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:24 smithi115 ceph-mon[34801]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:24:24.832 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:24 smithi115 ceph-mon[34801]: from='osd.7 [v2:172.21.15.115:6816/3075734174,v1:172.21.15.115:6817/3075734174]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T03:24:24.832 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:24 smithi115 ceph-mon[34801]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T03:24:24.834 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi195:vg_nvme/lv_2 2024-09-12T03:24:25.083 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:24:25.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:25 smithi195 ceph-mon[34786]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T03:24:25.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:25 smithi195 ceph-mon[34786]: from='osd.7 [v2:172.21.15.115:6816/3075734174,v1:172.21.15.115:6817/3075734174]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-12T03:24:25.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:25 smithi195 ceph-mon[34786]: osdmap e46: 8 total, 7 up, 8 in 2024-09-12T03:24:25.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:25 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:24:25.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:25 smithi195 ceph-mon[34786]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-12T03:24:25.542 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:25.529+0000 7fe788f15640 1 -- 172.21.15.195:0/3978025317 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7841026c0 msgr2=0x7fe784108f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:25.542 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:25.529+0000 7fe788f15640 1 --2- 172.21.15.195:0/3978025317 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7841026c0 0x7fe784108f50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe774009900 tx=0x7fe77402a410 comp rx=0 tx=0).stop 2024-09-12T03:24:25.542 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:25.529+0000 7fe788f15640 1 -- 172.21.15.195:0/3978025317 shutdown_connections 2024-09-12T03:24:25.542 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:25.529+0000 7fe788f15640 1 --2- 172.21.15.195:0/3978025317 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7841026c0 0x7fe784108f50 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:25.542 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:25.529+0000 7fe788f15640 1 --2- 172.21.15.195:0/3978025317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe784101d00 0x7fe784102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:25.542 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:25.529+0000 7fe788f15640 1 --2- 172.21.15.195:0/3978025317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe784100b00 0x7fe784100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:25.542 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:25.529+0000 7fe788f15640 1 -- 172.21.15.195:0/3978025317 >> 172.21.15.195:0/3978025317 conn(0x7fe7840fc290 msgr2=0x7fe7840fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:25.542 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:25.529+0000 7fe788f15640 1 -- 172.21.15.195:0/3978025317 shutdown_connections 2024-09-12T03:24:25.543 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:25.529+0000 7fe788f15640 1 -- 172.21.15.195:0/3978025317 wait complete. 2024-09-12T03:24:25.543 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:25.530+0000 7fe788f15640 1 Processor -- start 2024-09-12T03:24:25.543 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:25.530+0000 7fe788f15640 1 -- start start 2024-09-12T03:24:25.543 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:25.530+0000 7fe788f15640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe784100b00 0x7fe784104a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:25.543 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:25.530+0000 7fe788f15640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe784101d00 0x7fe7841030e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:25.543 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:25.530+0000 7fe788f15640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7841026c0 0x7fe784103620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:25.543 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:25.530+0000 7fe788f15640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe78410c690 con 0x7fe784101d00 2024-09-12T03:24:25.543 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:25.530+0000 7fe788f15640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe78410c510 con 0x7fe7841026c0 2024-09-12T03:24:25.543 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:25.530+0000 7fe788f15640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe78410c810 con 0x7fe784100b00 2024-09-12T03:24:25.543 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:25.530+0000 7fe78259c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe784100b00 0x7fe784104a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:25.543 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:25.530+0000 7fe78259c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe784100b00 0x7fe784104a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:47282/0 (socket says 172.21.15.195:47282) 2024-09-12T03:24:25.543 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:25.530+0000 7fe78259c640 1 -- 172.21.15.195:0/1805117951 learned_addr learned my addr 172.21.15.195:0/1805117951 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-09-12T03:24:25.544 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:25.530+0000 7fe782d9d640 1 --2- 172.21.15.195:0/1805117951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7841026c0 0x7fe784103620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:25.544 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:25.530+0000 7fe781d9b640 1 --2- 172.21.15.195:0/1805117951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe784101d00 0x7fe7841030e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:25.544 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:25.531+0000 7fe78259c640 1 -- 172.21.15.195:0/1805117951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7841026c0 msgr2=0x7fe784103620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:25.544 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:25.531+0000 7fe78259c640 1 --2- 172.21.15.195:0/1805117951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7841026c0 0x7fe784103620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:25.544 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:25.531+0000 7fe78259c640 1 -- 172.21.15.195:0/1805117951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe784101d00 msgr2=0x7fe7841030e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:25.544 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:25.531+0000 7fe78259c640 1 --2- 172.21.15.195:0/1805117951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe784101d00 0x7fe7841030e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:25.544 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:25.531+0000 7fe78259c640 1 -- 172.21.15.195:0/1805117951 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe784103ee0 con 0x7fe784100b00 2024-09-12T03:24:25.544 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:25.531+0000 7fe78259c640 1 --2- 172.21.15.195:0/1805117951 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe784100b00 0x7fe784104a90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe77c009fd0 tx=0x7fe77c00ef00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:25.544 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:25.533+0000 7fe77b7fe640 1 -- 172.21.15.195:0/1805117951 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77c00cde0 con 0x7fe784100b00 2024-09-12T03:24:25.544 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:25.533+0000 7fe77b7fe640 1 -- 172.21.15.195:0/1805117951 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe77c011920 con 0x7fe784100b00 2024-09-12T03:24:25.544 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:25.533+0000 7fe788f15640 1 -- 172.21.15.195:0/1805117951 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7841a1ff0 con 0x7fe784100b00 2024-09-12T03:24:25.544 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:25.533+0000 7fe77b7fe640 1 -- 172.21.15.195:0/1805117951 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77c01aa00 con 0x7fe784100b00 2024-09-12T03:24:25.544 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:25.533+0000 7fe788f15640 1 -- 172.21.15.195:0/1805117951 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7841a23e0 con 0x7fe784100b00 2024-09-12T03:24:25.545 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:25.534+0000 7fe788f15640 1 -- 172.21.15.195:0/1805117951 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7841a26a0 con 0x7fe784100b00 2024-09-12T03:24:25.545 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:25.534+0000 7fe77b7fe640 1 -- 172.21.15.195:0/1805117951 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe77c011ad0 con 0x7fe784100b00 2024-09-12T03:24:25.545 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:25.535+0000 7fe77b7fe640 1 --2- 172.21.15.195:0/1805117951 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe7700a2b80 0x7fe7700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:25.545 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:25.535+0000 7fe77b7fe640 1 -- 172.21.15.195:0/1805117951 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5117+0+0 (secure 0 0 0) 0x7fe77c0ce900 con 0x7fe784100b00 2024-09-12T03:24:25.545 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:25.535+0000 7fe781d9b640 1 --2- 172.21.15.195:0/1805117951 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe7700a2b80 0x7fe7700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:25.545 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:25.536+0000 7fe781d9b640 1 --2- 172.21.15.195:0/1805117951 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe7700a2b80 0x7fe7700a5040 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe784104810 tx=0x7fe76c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:25.545 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:25.539+0000 7fe77b7fe640 1 -- 172.21.15.195:0/1805117951 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe77c016030 con 0x7fe784100b00 2024-09-12T03:24:25.718 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:25.717+0000 7fe788f15640 1 -- 172.21.15.195:0/1805117951 --> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fe78406af10 con 0x7fe7700a2b80 2024-09-12T03:24:25.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:25 smithi064 ceph-mon[30952]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T03:24:25.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:25 smithi064 ceph-mon[30952]: from='osd.7 [v2:172.21.15.115:6816/3075734174,v1:172.21.15.115:6817/3075734174]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-12T03:24:25.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:25 smithi064 ceph-mon[30952]: osdmap e46: 8 total, 7 up, 8 in 2024-09-12T03:24:25.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:25 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:24:25.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:25 smithi064 ceph-mon[30952]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-12T03:24:25.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:25 smithi115 ceph-mon[34801]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T03:24:25.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:25 smithi115 ceph-mon[34801]: from='osd.7 [v2:172.21.15.115:6816/3075734174,v1:172.21.15.115:6817/3075734174]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-12T03:24:25.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:25 smithi115 ceph-mon[34801]: osdmap e46: 8 total, 7 up, 8 in 2024-09-12T03:24:25.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:25 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:24:25.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:25 smithi115 ceph-mon[34801]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-12T03:24:25.893 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:24:25 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:24:25.430+0000 7f15a436f640 -1 osd.7 0 waiting for initial osdmap 2024-09-12T03:24:25.893 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:24:25 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:24:25.442+0000 7f15a0197640 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:24:26.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:26 smithi195 ceph-mon[34786]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-12T03:24:26.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:26 smithi195 ceph-mon[34786]: osdmap e47: 8 total, 7 up, 8 in 2024-09-12T03:24:26.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:26 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:24:26.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:26 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:24:26.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:26 smithi195 ceph-mon[34786]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:24:26.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:26 smithi195 ceph-mon[34786]: Detected new or changed devices on smithi115 2024-09-12T03:24:26.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:26 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:26.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:26 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:26.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:26 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:24:26.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:26 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:24:26.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:26 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:24:26.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:26 smithi195 ceph-mon[34786]: Adjusting osd_memory_target on smithi115 to 5696M 2024-09-12T03:24:26.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:26 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:26.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:26 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:26.708 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:26 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:24:26.708 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:26 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:26.708 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:26 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:24:26.708 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:26 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:26.708 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:26 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:24:26.708 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:26 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:26.708 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:26 smithi195 ceph-mon[34786]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:24:26.708 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:26 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:24:26.708 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:26 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:24:26.708 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:26 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:26.708 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:26 smithi195 ceph-mon[34786]: osd.7 [v2:172.21.15.115:6816/3075734174,v1:172.21.15.115:6817/3075734174] boot 2024-09-12T03:24:26.708 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:26 smithi195 ceph-mon[34786]: osdmap e48: 8 total, 8 up, 8 in 2024-09-12T03:24:26.708 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:26 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:24:26.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:26 smithi064 ceph-mon[30952]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-12T03:24:26.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:26 smithi064 ceph-mon[30952]: osdmap e47: 8 total, 7 up, 8 in 2024-09-12T03:24:26.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:26 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:24:26.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:26 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:24:26.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:26 smithi064 ceph-mon[30952]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:24:26.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:26 smithi064 ceph-mon[30952]: Detected new or changed devices on smithi115 2024-09-12T03:24:26.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:26 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:26.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:26 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:26.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:26 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:24:26.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:26 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:24:26.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:26 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:24:26.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:26 smithi064 ceph-mon[30952]: Adjusting osd_memory_target on smithi115 to 5696M 2024-09-12T03:24:26.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:26 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:26.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:26 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:26.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:26 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:24:26.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:26 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:26.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:26 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:24:26.774 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:26 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:26.774 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:26 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:24:26.774 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:26 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:26.774 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:26 smithi064 ceph-mon[30952]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:24:26.774 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:26 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:24:26.774 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:26 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:24:26.774 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:26 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:26.774 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:26 smithi064 ceph-mon[30952]: osd.7 [v2:172.21.15.115:6816/3075734174,v1:172.21.15.115:6817/3075734174] boot 2024-09-12T03:24:26.774 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:26 smithi064 ceph-mon[30952]: osdmap e48: 8 total, 8 up, 8 in 2024-09-12T03:24:26.774 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:26 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:24:26.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:26 smithi115 ceph-mon[34801]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-12T03:24:26.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:26 smithi115 ceph-mon[34801]: osdmap e47: 8 total, 7 up, 8 in 2024-09-12T03:24:26.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:26 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:24:26.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:26 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:24:26.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:26 smithi115 ceph-mon[34801]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:24:26.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:26 smithi115 ceph-mon[34801]: Detected new or changed devices on smithi115 2024-09-12T03:24:26.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:26 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:26.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:26 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:26.894 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:26 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:24:26.894 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:26 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:24:26.894 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:26 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:24:26.894 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:26 smithi115 ceph-mon[34801]: Adjusting osd_memory_target on smithi115 to 5696M 2024-09-12T03:24:26.894 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:26 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:26.894 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:26 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:26.894 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:26 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:24:26.894 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:26 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:26.894 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:26 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:24:26.894 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:26 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:26.894 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:26 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:24:26.894 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:26 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:26.894 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:26 smithi115 ceph-mon[34801]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:24:26.895 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:26 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:24:26.895 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:26 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:24:26.895 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:26 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:26.895 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:26 smithi115 ceph-mon[34801]: osd.7 [v2:172.21.15.115:6816/3075734174,v1:172.21.15.115:6817/3075734174] boot 2024-09-12T03:24:26.895 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:26 smithi115 ceph-mon[34801]: osdmap e48: 8 total, 8 up, 8 in 2024-09-12T03:24:26.895 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:26 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:24:27.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:27 smithi195 ceph-mon[34786]: purged_snaps scrub starts 2024-09-12T03:24:27.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:27 smithi195 ceph-mon[34786]: purged_snaps scrub ok 2024-09-12T03:24:27.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:27 smithi195 ceph-mon[34786]: osdmap e49: 8 total, 8 up, 8 in 2024-09-12T03:24:27.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:27 smithi064 ceph-mon[30952]: purged_snaps scrub starts 2024-09-12T03:24:27.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:27 smithi064 ceph-mon[30952]: purged_snaps scrub ok 2024-09-12T03:24:27.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:27 smithi064 ceph-mon[30952]: osdmap e49: 8 total, 8 up, 8 in 2024-09-12T03:24:27.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:27 smithi115 ceph-mon[34801]: purged_snaps scrub starts 2024-09-12T03:24:27.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:27 smithi115 ceph-mon[34801]: purged_snaps scrub ok 2024-09-12T03:24:27.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:27 smithi115 ceph-mon[34801]: osdmap e49: 8 total, 8 up, 8 in 2024-09-12T03:24:29.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:28 smithi115 ceph-mon[34801]: pgmap v169: 1 pgs: 1 remapped; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:24:29.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:28 smithi115 ceph-mon[34801]: osdmap e50: 8 total, 8 up, 8 in 2024-09-12T03:24:29.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:28 smithi115 ceph-mon[34801]: from='client.? 172.21.15.195:0/18012287' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6de50049-c28f-4f21-b123-05e55bf59127"}]: dispatch 2024-09-12T03:24:29.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:28 smithi115 ceph-mon[34801]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6de50049-c28f-4f21-b123-05e55bf59127"}]: dispatch 2024-09-12T03:24:29.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:28 smithi115 ceph-mon[34801]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6de50049-c28f-4f21-b123-05e55bf59127"}]': finished 2024-09-12T03:24:29.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:28 smithi115 ceph-mon[34801]: osdmap e51: 9 total, 8 up, 9 in 2024-09-12T03:24:29.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:28 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:24:29.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:28 smithi115 ceph-mon[34801]: from='client.? 172.21.15.195:0/2342705829' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:24:29.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:28 smithi195 ceph-mon[34786]: pgmap v169: 1 pgs: 1 remapped; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:24:29.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:28 smithi195 ceph-mon[34786]: osdmap e50: 8 total, 8 up, 8 in 2024-09-12T03:24:29.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:28 smithi195 ceph-mon[34786]: from='client.? 172.21.15.195:0/18012287' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6de50049-c28f-4f21-b123-05e55bf59127"}]: dispatch 2024-09-12T03:24:29.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:28 smithi195 ceph-mon[34786]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6de50049-c28f-4f21-b123-05e55bf59127"}]: dispatch 2024-09-12T03:24:29.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:28 smithi195 ceph-mon[34786]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6de50049-c28f-4f21-b123-05e55bf59127"}]': finished 2024-09-12T03:24:29.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:28 smithi195 ceph-mon[34786]: osdmap e51: 9 total, 8 up, 9 in 2024-09-12T03:24:29.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:28 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:24:29.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:28 smithi195 ceph-mon[34786]: from='client.? 172.21.15.195:0/2342705829' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:24:29.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:28 smithi064 ceph-mon[30952]: pgmap v169: 1 pgs: 1 remapped; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:24:29.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:28 smithi064 ceph-mon[30952]: osdmap e50: 8 total, 8 up, 8 in 2024-09-12T03:24:29.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:28 smithi064 ceph-mon[30952]: from='client.? 172.21.15.195:0/18012287' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6de50049-c28f-4f21-b123-05e55bf59127"}]: dispatch 2024-09-12T03:24:29.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:28 smithi064 ceph-mon[30952]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6de50049-c28f-4f21-b123-05e55bf59127"}]: dispatch 2024-09-12T03:24:29.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:28 smithi064 ceph-mon[30952]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6de50049-c28f-4f21-b123-05e55bf59127"}]': finished 2024-09-12T03:24:29.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:28 smithi064 ceph-mon[30952]: osdmap e51: 9 total, 8 up, 9 in 2024-09-12T03:24:29.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:28 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:24:29.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:28 smithi064 ceph-mon[30952]: from='client.? 172.21.15.195:0/2342705829' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:24:30.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:30 smithi115 ceph-mon[34801]: osdmap e52: 9 total, 8 up, 9 in 2024-09-12T03:24:30.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:30 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:24:30.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:30 smithi195 ceph-mon[34786]: osdmap e52: 9 total, 8 up, 9 in 2024-09-12T03:24:30.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:30 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:24:30.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:30 smithi064 ceph-mon[30952]: osdmap e52: 9 total, 8 up, 9 in 2024-09-12T03:24:30.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:30 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:24:31.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:31 smithi115 ceph-mon[34801]: pgmap v173: 1 pgs: 1 remapped; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:24:31.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:31 smithi195 ceph-mon[34786]: pgmap v173: 1 pgs: 1 remapped; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:24:31.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:31 smithi064 ceph-mon[30952]: pgmap v173: 1 pgs: 1 remapped; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:24:33.324 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:33 smithi195 ceph-mon[34786]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:24:33.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:33 smithi115 ceph-mon[34801]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:24:33.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:33 smithi064 ceph-mon[30952]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:24:35.355 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:35 smithi195 ceph-mon[34786]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:24:35.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:35 smithi115 ceph-mon[34801]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:24:35.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:35 smithi064 ceph-mon[30952]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:24:37.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:37 smithi115 ceph-mon[34801]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:24:37.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:37 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T03:24:37.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:37 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:37.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:37 smithi115 ceph-mon[34801]: Deploying daemon osd.8 on smithi195 2024-09-12T03:24:37.446 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:37 smithi195 ceph-mon[34786]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:24:37.446 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:37 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T03:24:37.446 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:37 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:37.446 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:37 smithi195 ceph-mon[34786]: Deploying daemon osd.8 on smithi195 2024-09-12T03:24:37.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:37 smithi064 ceph-mon[30952]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:24:37.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:37 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T03:24:37.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:37 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:37.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:37 smithi064 ceph-mon[30952]: Deploying daemon osd.8 on smithi195 2024-09-12T03:24:39.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:39 smithi115 ceph-mon[34801]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:24:39.398 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:39 smithi195 ceph-mon[34786]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:24:39.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:39 smithi064 ceph-mon[30952]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:24:41.191 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:41 smithi195 ceph-mon[34786]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:24:41.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:41 smithi115 ceph-mon[34801]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:24:41.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:41 smithi064 ceph-mon[30952]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:24:43.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:43 smithi195 ceph-mon[34786]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:24:43.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:43 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:24:43.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:43 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:43.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:43 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:43.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:43 smithi115 ceph-mon[34801]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:24:43.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:43 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:24:43.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:43 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:43.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:43 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:43.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:43 smithi064 ceph-mon[30952]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:24:43.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:43 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:24:43.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:43 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:43.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:43 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:45.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:45 smithi115 ceph-mon[34801]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:24:45.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:45 smithi195 ceph-mon[34786]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:24:45.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:45 smithi064 ceph-mon[30952]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:24:46.227 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:46.227+0000 7fe77b7fe640 1 -- 172.21.15.195:0/1805117951 <== mgr.14150 v2:172.21.15.64:6800/759931848 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe78406af10 con 0x7fe7700a2b80 2024-09-12T03:24:46.227 INFO:teuthology.orchestra.run.smithi195.stdout:Created osd(s) 8 on host 'smithi195' 2024-09-12T03:24:46.229 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:46.229+0000 7fe788f15640 1 -- 172.21.15.195:0/1805117951 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe7700a2b80 msgr2=0x7fe7700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:46.229 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:46.229+0000 7fe788f15640 1 --2- 172.21.15.195:0/1805117951 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe7700a2b80 0x7fe7700a5040 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe784104810 tx=0x7fe76c009110 comp rx=0 tx=0).stop 2024-09-12T03:24:46.229 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:46.229+0000 7fe788f15640 1 -- 172.21.15.195:0/1805117951 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe784100b00 msgr2=0x7fe784104a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:46.230 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:46.229+0000 7fe788f15640 1 --2- 172.21.15.195:0/1805117951 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe784100b00 0x7fe784104a90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe77c009fd0 tx=0x7fe77c00ef00 comp rx=0 tx=0).stop 2024-09-12T03:24:46.230 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:46.229+0000 7fe788f15640 1 -- 172.21.15.195:0/1805117951 shutdown_connections 2024-09-12T03:24:46.230 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:46.229+0000 7fe788f15640 1 --2- 172.21.15.195:0/1805117951 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe7700a2b80 0x7fe7700a5040 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:46.230 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:46.229+0000 7fe788f15640 1 --2- 172.21.15.195:0/1805117951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7841026c0 0x7fe784103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:46.230 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:46.229+0000 7fe788f15640 1 --2- 172.21.15.195:0/1805117951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe784101d00 0x7fe7841030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:46.230 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:46.229+0000 7fe788f15640 1 --2- 172.21.15.195:0/1805117951 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe784100b00 0x7fe784104a90 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:46.230 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:46.229+0000 7fe788f15640 1 -- 172.21.15.195:0/1805117951 >> 172.21.15.195:0/1805117951 conn(0x7fe7840fc290 msgr2=0x7fe78406a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:46.230 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:46.229+0000 7fe788f15640 1 -- 172.21.15.195:0/1805117951 shutdown_connections 2024-09-12T03:24:46.230 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:24:46.229+0000 7fe788f15640 1 -- 172.21.15.195:0/1805117951 wait complete. 2024-09-12T03:24:46.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:46 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:46.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:46 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:46.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:46 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:46.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:46 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:24:46.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:46 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:46.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:46 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:24:46.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:46 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:46.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:46 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:46.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:46 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:46.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:46 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:46.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:46 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:46.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:46 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:24:46.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:46 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:46.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:46 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:24:46.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:46 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:46.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:46 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:46.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:46 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:46.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:46 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:46.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:46 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:46.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:46 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:24:46.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:46 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:46.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:46 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:24:46.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:46 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:46.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:46 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:46.885 DEBUG:teuthology.orchestra.run.smithi195:osd.8> sudo journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@osd.8.service 2024-09-12T03:24:46.888 INFO:tasks.cephadm:Deploying osd.9 on smithi064 with /dev/vg_nvme/lv_1... 2024-09-12T03:24:46.888 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T03:24:47.154 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:47.534 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:47 smithi064 ceph-mon[30952]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:24:47.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:47 smithi115 ceph-mon[34801]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:24:47.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:47 smithi195 ceph-mon[34786]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:24:48.607 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:24:48 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:24:48.527+0000 7fa76ebd3640 -1 osd.8 0 log_to_monitors true 2024-09-12T03:24:49.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:49 smithi064 ceph-mon[30952]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:24:49.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:49 smithi064 ceph-mon[30952]: from='osd.8 [v2:172.21.15.195:6816/2184742494,v1:172.21.15.195:6817/2184742494]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T03:24:49.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:49 smithi064 ceph-mon[30952]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T03:24:49.642 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:49 smithi195 ceph-mon[34786]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:24:49.642 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:49 smithi195 ceph-mon[34786]: from='osd.8 [v2:172.21.15.195:6816/2184742494,v1:172.21.15.195:6817/2184742494]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T03:24:49.642 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:49 smithi195 ceph-mon[34786]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T03:24:49.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:49 smithi115 ceph-mon[34801]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:24:49.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:49 smithi115 ceph-mon[34801]: from='osd.8 [v2:172.21.15.195:6816/2184742494,v1:172.21.15.195:6817/2184742494]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T03:24:49.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:49 smithi115 ceph-mon[34801]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T03:24:49.953 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:24:49.989 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi064:vg_nvme/lv_1 2024-09-12T03:24:50.251 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:50.633 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:50 smithi064 ceph-mon[30952]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T03:24:50.633 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:50 smithi064 ceph-mon[30952]: osdmap e53: 9 total, 8 up, 9 in 2024-09-12T03:24:50.633 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:50 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:24:50.633 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:50 smithi064 ceph-mon[30952]: from='osd.8 [v2:172.21.15.195:6816/2184742494,v1:172.21.15.195:6817/2184742494]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-09-12T03:24:50.633 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:50 smithi064 ceph-mon[30952]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-09-12T03:24:50.633 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:50 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:50.633 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:50 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:50.633 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:50 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:24:50.633 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:50 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:24:50.634 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:50 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:24:50.634 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:50 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:50.634 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:50 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:50.634 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:50 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:24:50.634 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:50 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:50.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:50 smithi115 ceph-mon[34801]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T03:24:50.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:50 smithi115 ceph-mon[34801]: osdmap e53: 9 total, 8 up, 9 in 2024-09-12T03:24:50.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:50 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:24:50.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:50 smithi115 ceph-mon[34801]: from='osd.8 [v2:172.21.15.195:6816/2184742494,v1:172.21.15.195:6817/2184742494]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-09-12T03:24:50.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:50 smithi115 ceph-mon[34801]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-09-12T03:24:50.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:50 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:50.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:50 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:50.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:50 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:24:50.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:50 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:24:50.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:50 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:24:50.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:50 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:50.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:50 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:50.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:50 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:24:50.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:50 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:50.706 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:24:50 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:24:50.331+0000 7fa76ab56640 -1 osd.8 0 waiting for initial osdmap 2024-09-12T03:24:50.706 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:24:50 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:24:50.341+0000 7fa76697e640 -1 osd.8 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:24:50.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:50 smithi195 ceph-mon[34786]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T03:24:50.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:50 smithi195 ceph-mon[34786]: osdmap e53: 9 total, 8 up, 9 in 2024-09-12T03:24:50.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:50 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:24:50.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:50 smithi195 ceph-mon[34786]: from='osd.8 [v2:172.21.15.195:6816/2184742494,v1:172.21.15.195:6817/2184742494]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-09-12T03:24:50.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:50 smithi195 ceph-mon[34786]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-09-12T03:24:50.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:50 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:50.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:50 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:50.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:50 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:24:50.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:50 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:24:50.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:50 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:24:50.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:50 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:50.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:50 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:50.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:50 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:24:50.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:50 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:24:50.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.747+0000 7fd81d156640 1 -- 172.21.15.64:0/415316271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd818101cf0 msgr2=0x7fd818102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:50.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.747+0000 7fd81d156640 1 --2- 172.21.15.64:0/415316271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd818101cf0 0x7fd818102170 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd808009900 tx=0x7fd80802a410 comp rx=0 tx=0).stop 2024-09-12T03:24:50.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.747+0000 7fd81d156640 1 -- 172.21.15.64:0/415316271 shutdown_connections 2024-09-12T03:24:50.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.747+0000 7fd81d156640 1 --2- 172.21.15.64:0/415316271 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8181026b0 0x7fd818108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:50.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.747+0000 7fd81d156640 1 --2- 172.21.15.64:0/415316271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd818101cf0 0x7fd818102170 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:50.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.747+0000 7fd81d156640 1 --2- 172.21.15.64:0/415316271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd818100af0 0x7fd818100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:50.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.747+0000 7fd81d156640 1 -- 172.21.15.64:0/415316271 >> 172.21.15.64:0/415316271 conn(0x7fd8180fc2a0 msgr2=0x7fd8180fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:50.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.747+0000 7fd81d156640 1 -- 172.21.15.64:0/415316271 shutdown_connections 2024-09-12T03:24:50.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.747+0000 7fd81d156640 1 -- 172.21.15.64:0/415316271 wait complete. 2024-09-12T03:24:50.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.748+0000 7fd81d156640 1 Processor -- start 2024-09-12T03:24:50.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.748+0000 7fd81d156640 1 -- start start 2024-09-12T03:24:50.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.748+0000 7fd81d156640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd818100af0 0x7fd818197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:50.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.748+0000 7fd81d156640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd818101cf0 0x7fd818198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:50.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.748+0000 7fd81d156640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8181026b0 0x7fd81819f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:50.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.748+0000 7fd81d156640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd81810d7b0 con 0x7fd818101cf0 2024-09-12T03:24:50.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.748+0000 7fd81d156640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd81810d630 con 0x7fd818100af0 2024-09-12T03:24:50.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.748+0000 7fd81d156640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd81810d930 con 0x7fd8181026b0 2024-09-12T03:24:50.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.748+0000 7fd81659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd818101cf0 0x7fd818198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:50.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.748+0000 7fd81659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd818101cf0 0x7fd818198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53194/0 (socket says 172.21.15.64:53194) 2024-09-12T03:24:50.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.748+0000 7fd81659c640 1 -- 172.21.15.64:0/4211960262 learned_addr learned my addr 172.21.15.64:0/4211960262 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:24:50.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.748+0000 7fd81759e640 1 --2- 172.21.15.64:0/4211960262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8181026b0 0x7fd81819f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:50.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.748+0000 7fd816d9d640 1 --2- 172.21.15.64:0/4211960262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd818100af0 0x7fd818197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:50.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.749+0000 7fd81659c640 1 -- 172.21.15.64:0/4211960262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8181026b0 msgr2=0x7fd81819f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:50.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.749+0000 7fd81659c640 1 --2- 172.21.15.64:0/4211960262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8181026b0 0x7fd81819f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:50.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.749+0000 7fd81659c640 1 -- 172.21.15.64:0/4211960262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd818100af0 msgr2=0x7fd818197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:50.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.749+0000 7fd81659c640 1 --2- 172.21.15.64:0/4211960262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd818100af0 0x7fd818197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:50.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.749+0000 7fd81659c640 1 -- 172.21.15.64:0/4211960262 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd81819fa20 con 0x7fd818101cf0 2024-09-12T03:24:50.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.749+0000 7fd816d9d640 1 --2- 172.21.15.64:0/4211960262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd818100af0 0x7fd818197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:24:50.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.749+0000 7fd81659c640 1 --2- 172.21.15.64:0/4211960262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd818101cf0 0x7fd818198330 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd8080314e0 tx=0x7fd808003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:50.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.749+0000 7fd80ffff640 1 -- 172.21.15.64:0/4211960262 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80803f070 con 0x7fd818101cf0 2024-09-12T03:24:50.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.749+0000 7fd81d156640 1 -- 172.21.15.64:0/4211960262 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd81819fcb0 con 0x7fd818101cf0 2024-09-12T03:24:50.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.749+0000 7fd80ffff640 1 -- 172.21.15.64:0/4211960262 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd808031c40 con 0x7fd818101cf0 2024-09-12T03:24:50.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.749+0000 7fd80ffff640 1 -- 172.21.15.64:0/4211960262 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd808035030 con 0x7fd818101cf0 2024-09-12T03:24:50.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.749+0000 7fd81d156640 1 -- 172.21.15.64:0/4211960262 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8181a0190 con 0x7fd818101cf0 2024-09-12T03:24:50.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.750+0000 7fd81d156640 1 -- 172.21.15.64:0/4211960262 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd818107f90 con 0x7fd818101cf0 2024-09-12T03:24:50.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.751+0000 7fd80ffff640 1 -- 172.21.15.64:0/4211960262 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd808042070 con 0x7fd818101cf0 2024-09-12T03:24:50.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.752+0000 7fd80ffff640 1 --2- 172.21.15.64:0/4211960262 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fd8040a2bb0 0x7fd8040a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:50.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.752+0000 7fd816d9d640 1 --2- 172.21.15.64:0/4211960262 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fd8040a2bb0 0x7fd8040a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:50.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.752+0000 7fd80ffff640 1 -- 172.21.15.64:0/4211960262 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5549+0+0 (secure 0 0 0) 0x7fd8080ef400 con 0x7fd818101cf0 2024-09-12T03:24:50.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.753+0000 7fd816d9d640 1 --2- 172.21.15.64:0/4211960262 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fd8040a2bb0 0x7fd8040a5070 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd818101b50 tx=0x7fd800005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:50.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.755+0000 7fd80ffff640 1 -- 172.21.15.64:0/4211960262 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd8080bce30 con 0x7fd818101cf0 2024-09-12T03:24:50.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:24:50.939+0000 7fd81d156640 1 -- 172.21.15.64:0/4211960262 --> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fd818062630 con 0x7fd8040a2bb0 2024-09-12T03:24:51.582 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:51 smithi064 ceph-mon[30952]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:24:51.582 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:51 smithi064 ceph-mon[30952]: Detected new or changed devices on smithi195 2024-09-12T03:24:51.582 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:51 smithi064 ceph-mon[30952]: Adjusting osd_memory_target on smithi195 to 5696M 2024-09-12T03:24:51.582 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:51 smithi064 ceph-mon[30952]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-09-12T03:24:51.582 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:51 smithi064 ceph-mon[30952]: osdmap e54: 9 total, 8 up, 9 in 2024-09-12T03:24:51.582 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:51 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:24:51.583 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:51 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:24:51.583 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:51 smithi064 ceph-mon[30952]: osd.8 [v2:172.21.15.195:6816/2184742494,v1:172.21.15.195:6817/2184742494] boot 2024-09-12T03:24:51.583 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:51 smithi064 ceph-mon[30952]: osdmap e55: 9 total, 9 up, 9 in 2024-09-12T03:24:51.583 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:51 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:24:51.583 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:51 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:24:51.583 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:51 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:24:51.583 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:51 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:51.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:51 smithi115 ceph-mon[34801]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:24:51.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:51 smithi115 ceph-mon[34801]: Detected new or changed devices on smithi195 2024-09-12T03:24:51.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:51 smithi115 ceph-mon[34801]: Adjusting osd_memory_target on smithi195 to 5696M 2024-09-12T03:24:51.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:51 smithi115 ceph-mon[34801]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-09-12T03:24:51.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:51 smithi115 ceph-mon[34801]: osdmap e54: 9 total, 8 up, 9 in 2024-09-12T03:24:51.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:51 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:24:51.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:51 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:24:51.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:51 smithi115 ceph-mon[34801]: osd.8 [v2:172.21.15.195:6816/2184742494,v1:172.21.15.195:6817/2184742494] boot 2024-09-12T03:24:51.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:51 smithi115 ceph-mon[34801]: osdmap e55: 9 total, 9 up, 9 in 2024-09-12T03:24:51.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:51 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:24:51.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:51 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:24:51.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:51 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:24:51.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:51 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:51.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:51 smithi195 ceph-mon[34786]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:24:51.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:51 smithi195 ceph-mon[34786]: Detected new or changed devices on smithi195 2024-09-12T03:24:51.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:51 smithi195 ceph-mon[34786]: Adjusting osd_memory_target on smithi195 to 5696M 2024-09-12T03:24:51.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:51 smithi195 ceph-mon[34786]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-09-12T03:24:51.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:51 smithi195 ceph-mon[34786]: osdmap e54: 9 total, 8 up, 9 in 2024-09-12T03:24:51.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:51 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:24:51.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:51 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:24:51.708 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:51 smithi195 ceph-mon[34786]: osd.8 [v2:172.21.15.195:6816/2184742494,v1:172.21.15.195:6817/2184742494] boot 2024-09-12T03:24:51.708 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:51 smithi195 ceph-mon[34786]: osdmap e55: 9 total, 9 up, 9 in 2024-09-12T03:24:51.708 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:51 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:24:51.708 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:51 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:24:51.708 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:51 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:24:51.708 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:51 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:52.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:52 smithi064 ceph-mon[30952]: purged_snaps scrub starts 2024-09-12T03:24:52.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:52 smithi064 ceph-mon[30952]: purged_snaps scrub ok 2024-09-12T03:24:52.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:52 smithi064 ceph-mon[30952]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:24:52.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:52 smithi064 ceph-mon[30952]: osdmap e56: 9 total, 9 up, 9 in 2024-09-12T03:24:52.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:52 smithi115 ceph-mon[34801]: purged_snaps scrub starts 2024-09-12T03:24:52.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:52 smithi115 ceph-mon[34801]: purged_snaps scrub ok 2024-09-12T03:24:52.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:52 smithi115 ceph-mon[34801]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:24:52.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:52 smithi115 ceph-mon[34801]: osdmap e56: 9 total, 9 up, 9 in 2024-09-12T03:24:52.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:52 smithi195 ceph-mon[34786]: purged_snaps scrub starts 2024-09-12T03:24:52.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:52 smithi195 ceph-mon[34786]: purged_snaps scrub ok 2024-09-12T03:24:52.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:52 smithi195 ceph-mon[34786]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:24:52.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:52 smithi195 ceph-mon[34786]: osdmap e56: 9 total, 9 up, 9 in 2024-09-12T03:24:53.634 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:53 smithi064 ceph-mon[30952]: pgmap v187: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:24:53.634 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:53 smithi064 ceph-mon[30952]: osdmap e57: 9 total, 9 up, 9 in 2024-09-12T03:24:53.634 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:53 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/3909799021' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36ab792c-58f2-4beb-9024-7f3d1e3785dd"}]: dispatch 2024-09-12T03:24:53.634 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:53 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/3909799021' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "36ab792c-58f2-4beb-9024-7f3d1e3785dd"}]': finished 2024-09-12T03:24:53.634 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:53 smithi064 ceph-mon[30952]: osdmap e58: 10 total, 9 up, 10 in 2024-09-12T03:24:53.634 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:53 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:24:53.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:53 smithi115 ceph-mon[34801]: pgmap v187: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:24:53.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:53 smithi115 ceph-mon[34801]: osdmap e57: 9 total, 9 up, 9 in 2024-09-12T03:24:53.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:53 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/3909799021' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36ab792c-58f2-4beb-9024-7f3d1e3785dd"}]: dispatch 2024-09-12T03:24:53.645 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:53 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/3909799021' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "36ab792c-58f2-4beb-9024-7f3d1e3785dd"}]': finished 2024-09-12T03:24:53.645 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:53 smithi115 ceph-mon[34801]: osdmap e58: 10 total, 9 up, 10 in 2024-09-12T03:24:53.645 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:53 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:24:53.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:53 smithi195 ceph-mon[34786]: pgmap v187: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:24:53.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:53 smithi195 ceph-mon[34786]: osdmap e57: 9 total, 9 up, 9 in 2024-09-12T03:24:53.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:53 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/3909799021' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36ab792c-58f2-4beb-9024-7f3d1e3785dd"}]: dispatch 2024-09-12T03:24:53.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:53 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/3909799021' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "36ab792c-58f2-4beb-9024-7f3d1e3785dd"}]': finished 2024-09-12T03:24:53.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:53 smithi195 ceph-mon[34786]: osdmap e58: 10 total, 9 up, 10 in 2024-09-12T03:24:53.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:53 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:24:54.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:54 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/2200276169' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:24:54.709 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:54 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/2200276169' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:24:54.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:54 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/2200276169' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:24:55.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:55 smithi115 ceph-mon[34801]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:24:55.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:55 smithi115 ceph-mon[34801]: osdmap e59: 10 total, 9 up, 10 in 2024-09-12T03:24:55.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:55 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:24:55.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:55 smithi195 ceph-mon[34786]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:24:55.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:55 smithi195 ceph-mon[34786]: osdmap e59: 10 total, 9 up, 10 in 2024-09-12T03:24:55.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:55 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:24:55.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:55 smithi064 ceph-mon[30952]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:24:55.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:55 smithi064 ceph-mon[30952]: osdmap e59: 10 total, 9 up, 10 in 2024-09-12T03:24:55.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:55 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:24:56.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:56 smithi115 ceph-mon[34801]: osdmap e60: 10 total, 9 up, 10 in 2024-09-12T03:24:56.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:56 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:24:56.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:56 smithi195 ceph-mon[34786]: osdmap e60: 10 total, 9 up, 10 in 2024-09-12T03:24:56.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:56 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:24:56.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:56 smithi064 ceph-mon[30952]: osdmap e60: 10 total, 9 up, 10 in 2024-09-12T03:24:56.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:56 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:24:57.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:57 smithi115 ceph-mon[34801]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:24:57.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:57 smithi195 ceph-mon[34786]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:24:57.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:57 smithi064 ceph-mon[30952]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:24:59.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:24:59 smithi064 ceph-mon[30952]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-09-12T03:24:59.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:24:59 smithi115 ceph-mon[34801]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-09-12T03:24:59.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:24:59 smithi195 ceph-mon[34786]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-09-12T03:25:01.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:01 smithi115 ceph-mon[34801]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-09-12T03:25:01.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:01 smithi195 ceph-mon[34786]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-09-12T03:25:01.729 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:01 smithi064 ceph-mon[30952]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-09-12T03:25:02.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:02 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T03:25:02.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:02 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:25:02.706 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:02 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T03:25:02.706 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:02 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:25:02.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:02 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T03:25:02.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:02 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:25:03.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:03 smithi064 ceph-mon[30952]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T03:25:03.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:03 smithi064 ceph-mon[30952]: Deploying daemon osd.9 on smithi064 2024-09-12T03:25:03.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:03 smithi115 ceph-mon[34801]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T03:25:03.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:03 smithi115 ceph-mon[34801]: Deploying daemon osd.9 on smithi064 2024-09-12T03:25:03.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:03 smithi195 ceph-mon[34786]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T03:25:03.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:03 smithi195 ceph-mon[34786]: Deploying daemon osd.9 on smithi064 2024-09-12T03:25:05.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:05 smithi064 ceph-mon[30952]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-12T03:25:05.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:05 smithi115 ceph-mon[34801]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-12T03:25:05.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:05 smithi195 ceph-mon[34786]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-12T03:25:07.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:07 smithi115 ceph-mon[34801]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-12T03:25:07.689 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:07 smithi064 ceph-mon[30952]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-12T03:25:07.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:07 smithi195 ceph-mon[34786]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-12T03:25:08.471 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:08 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:25:08.471 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:08 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:08.472 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:08 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:08.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:08 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:25:08.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:08 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:08.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:08 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:08.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:08 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:25:08.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:08 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:08.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:08 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:09.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:09 smithi115 ceph-mon[34801]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T03:25:09.699 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:09 smithi064 ceph-mon[30952]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T03:25:09.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:09 smithi195 ceph-mon[34786]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T03:25:11.338 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 9 on host 'smithi064' 2024-09-12T03:25:11.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:25:11.337+0000 7fd80ffff640 1 -- 172.21.15.64:0/4211960262 <== mgr.14150 v2:172.21.15.64:6800/759931848 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd818062630 con 0x7fd8040a2bb0 2024-09-12T03:25:11.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:25:11.340+0000 7fd81d156640 1 -- 172.21.15.64:0/4211960262 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fd8040a2bb0 msgr2=0x7fd8040a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:11.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:25:11.340+0000 7fd81d156640 1 --2- 172.21.15.64:0/4211960262 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fd8040a2bb0 0x7fd8040a5070 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd818101b50 tx=0x7fd800005f90 comp rx=0 tx=0).stop 2024-09-12T03:25:11.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:25:11.340+0000 7fd81d156640 1 -- 172.21.15.64:0/4211960262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd818101cf0 msgr2=0x7fd818198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:11.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:25:11.340+0000 7fd81d156640 1 --2- 172.21.15.64:0/4211960262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd818101cf0 0x7fd818198330 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd8080314e0 tx=0x7fd808003c10 comp rx=0 tx=0).stop 2024-09-12T03:25:11.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:25:11.340+0000 7fd81d156640 1 -- 172.21.15.64:0/4211960262 shutdown_connections 2024-09-12T03:25:11.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:25:11.340+0000 7fd81d156640 1 --2- 172.21.15.64:0/4211960262 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fd8040a2bb0 0x7fd8040a5070 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:11.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:25:11.340+0000 7fd81d156640 1 --2- 172.21.15.64:0/4211960262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8181026b0 0x7fd81819f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:11.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:25:11.340+0000 7fd81d156640 1 --2- 172.21.15.64:0/4211960262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd818101cf0 0x7fd818198330 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:11.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:25:11.340+0000 7fd81d156640 1 --2- 172.21.15.64:0/4211960262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd818100af0 0x7fd818197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:11.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:25:11.340+0000 7fd81d156640 1 -- 172.21.15.64:0/4211960262 >> 172.21.15.64:0/4211960262 conn(0x7fd8180fc2a0 msgr2=0x7fd818105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:11.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:25:11.340+0000 7fd81d156640 1 -- 172.21.15.64:0/4211960262 shutdown_connections 2024-09-12T03:25:11.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:25:11.340+0000 7fd81d156640 1 -- 172.21.15.64:0/4211960262 wait complete. 2024-09-12T03:25:11.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:11 smithi115 ceph-mon[34801]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:25:11.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:11 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:11.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:11 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:11.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:11 smithi064 ceph-mon[30952]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:25:11.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:11 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:11.655 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:11 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:11.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:11 smithi195 ceph-mon[34786]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:25:11.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:11 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:11.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:11 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:12.014 DEBUG:teuthology.orchestra.run.smithi064:osd.9> sudo journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@osd.9.service 2024-09-12T03:25:12.017 INFO:tasks.cephadm:Deploying osd.10 on smithi115 with /dev/vg_nvme/lv_1... 2024-09-12T03:25:12.017 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T03:25:12.275 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:25:13.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:13 smithi195 ceph-mon[34786]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:25:13.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:13 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:13.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:13 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:13.481 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:13 smithi115 ceph-mon[34801]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:25:13.481 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:13 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:13.481 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:13 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:13.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:13 smithi064 ceph-mon[30952]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:25:13.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:13 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:13.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:13 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:14.185 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:25:14 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:25:14.034+0000 7fe5b259d640 -1 osd.9 0 log_to_monitors true 2024-09-12T03:25:14.414 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:14 smithi115 ceph-mon[34801]: from='osd.9 [v2:172.21.15.64:6826/561728112,v1:172.21.15.64:6827/561728112]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T03:25:14.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:14 smithi195 ceph-mon[34786]: from='osd.9 [v2:172.21.15.64:6826/561728112,v1:172.21.15.64:6827/561728112]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T03:25:14.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:14 smithi064 ceph-mon[30952]: from='osd.9 [v2:172.21.15.64:6826/561728112,v1:172.21.15.64:6827/561728112]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T03:25:14.951 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T03:25:14.976 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi115:vg_nvme/lv_1 2024-09-12T03:25:15.225 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:25:15.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:25:15 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:25:15.197+0000 7fe5ae520640 -1 osd.9 0 waiting for initial osdmap 2024-09-12T03:25:15.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:25:15 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:25:15.207+0000 7fe5aa348640 -1 osd.9 62 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:25:15.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:15 smithi064 ceph-mon[30952]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:25:15.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:15 smithi064 ceph-mon[30952]: from='osd.9 [v2:172.21.15.64:6826/561728112,v1:172.21.15.64:6827/561728112]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T03:25:15.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:15 smithi064 ceph-mon[30952]: osdmap e61: 10 total, 9 up, 10 in 2024-09-12T03:25:15.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:15 smithi064 ceph-mon[30952]: from='osd.9 [v2:172.21.15.64:6826/561728112,v1:172.21.15.64:6827/561728112]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-12T03:25:15.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:15 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:25:15.354 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:15 smithi115 ceph-mon[34801]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:25:15.354 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:15 smithi115 ceph-mon[34801]: from='osd.9 [v2:172.21.15.64:6826/561728112,v1:172.21.15.64:6827/561728112]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T03:25:15.354 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:15 smithi115 ceph-mon[34801]: osdmap e61: 10 total, 9 up, 10 in 2024-09-12T03:25:15.354 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:15 smithi115 ceph-mon[34801]: from='osd.9 [v2:172.21.15.64:6826/561728112,v1:172.21.15.64:6827/561728112]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-12T03:25:15.354 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:15 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:25:15.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:15 smithi195 ceph-mon[34786]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:25:15.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:15 smithi195 ceph-mon[34786]: from='osd.9 [v2:172.21.15.64:6826/561728112,v1:172.21.15.64:6827/561728112]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T03:25:15.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:15 smithi195 ceph-mon[34786]: osdmap e61: 10 total, 9 up, 10 in 2024-09-12T03:25:15.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:15 smithi195 ceph-mon[34786]: from='osd.9 [v2:172.21.15.64:6826/561728112,v1:172.21.15.64:6827/561728112]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-09-12T03:25:15.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:15 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:25:15.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.755+0000 7f1d44f19640 1 -- 172.21.15.115:0/42275037 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d40100ad0 msgr2=0x7f1d40100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:15.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.755+0000 7f1d44f19640 1 --2- 172.21.15.115:0/42275037 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d40100ad0 0x7f1d40100ed0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1d28009900 tx=0x7f1d2802a410 comp rx=0 tx=0).stop 2024-09-12T03:25:15.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.755+0000 7f1d44f19640 1 -- 172.21.15.115:0/42275037 shutdown_connections 2024-09-12T03:25:15.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.755+0000 7f1d44f19640 1 --2- 172.21.15.115:0/42275037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d40102690 0x7f1d40108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:15.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.755+0000 7f1d44f19640 1 --2- 172.21.15.115:0/42275037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d40101cd0 0x7f1d40102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:15.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.755+0000 7f1d44f19640 1 --2- 172.21.15.115:0/42275037 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d40100ad0 0x7f1d40100ed0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:15.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.755+0000 7f1d44f19640 1 -- 172.21.15.115:0/42275037 >> 172.21.15.115:0/42275037 conn(0x7f1d400fc280 msgr2=0x7f1d400fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:15.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.755+0000 7f1d44f19640 1 -- 172.21.15.115:0/42275037 shutdown_connections 2024-09-12T03:25:15.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.755+0000 7f1d44f19640 1 -- 172.21.15.115:0/42275037 wait complete. 2024-09-12T03:25:15.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.756+0000 7f1d44f19640 1 Processor -- start 2024-09-12T03:25:15.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.756+0000 7f1d44f19640 1 -- start start 2024-09-12T03:25:15.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.756+0000 7f1d44f19640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d40100ad0 0x7f1d40197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:15.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.756+0000 7f1d44f19640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d40101cd0 0x7f1d40198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:15.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.756+0000 7f1d44f19640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d40102690 0x7f1d4019f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:15.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.756+0000 7f1d44f19640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4010c600 con 0x7f1d40101cd0 2024-09-12T03:25:15.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.756+0000 7f1d44f19640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4010c480 con 0x7f1d40102690 2024-09-12T03:25:15.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.756+0000 7f1d44f19640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4010c780 con 0x7f1d40100ad0 2024-09-12T03:25:15.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.756+0000 7f1d3ed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d40102690 0x7f1d4019f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:15.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.756+0000 7f1d3ed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d40102690 0x7f1d4019f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55892/0 (socket says 172.21.15.115:55892) 2024-09-12T03:25:15.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.756+0000 7f1d3ed9d640 1 -- 172.21.15.115:0/21799217 learned_addr learned my addr 172.21.15.115:0/21799217 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-09-12T03:25:15.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.756+0000 7f1d3e59c640 1 --2- 172.21.15.115:0/21799217 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d40100ad0 0x7f1d40197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:15.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.756+0000 7f1d3dd9b640 1 --2- 172.21.15.115:0/21799217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d40101cd0 0x7f1d40198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:15.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.756+0000 7f1d3ed9d640 1 -- 172.21.15.115:0/21799217 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d40100ad0 msgr2=0x7f1d40197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:15.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.756+0000 7f1d3ed9d640 1 --2- 172.21.15.115:0/21799217 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d40100ad0 0x7f1d40197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:15.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.756+0000 7f1d3ed9d640 1 -- 172.21.15.115:0/21799217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d40101cd0 msgr2=0x7f1d40198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:15.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.756+0000 7f1d3ed9d640 1 --2- 172.21.15.115:0/21799217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d40101cd0 0x7f1d40198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:15.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.756+0000 7f1d3ed9d640 1 -- 172.21.15.115:0/21799217 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d4019fa40 con 0x7f1d40102690 2024-09-12T03:25:15.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.756+0000 7f1d3e59c640 1 --2- 172.21.15.115:0/21799217 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d40100ad0 0x7f1d40197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:25:15.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.756+0000 7f1d3ed9d640 1 --2- 172.21.15.115:0/21799217 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d40102690 0x7f1d4019f3d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1d3800cce0 tx=0x7f1d3800a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:15.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.757+0000 7f1d377fe640 1 -- 172.21.15.115:0/21799217 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3800f450 con 0x7f1d40102690 2024-09-12T03:25:15.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.757+0000 7f1d377fe640 1 -- 172.21.15.115:0/21799217 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1d3800faa0 con 0x7f1d40102690 2024-09-12T03:25:15.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.757+0000 7f1d44f19640 1 -- 172.21.15.115:0/21799217 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d4019fd30 con 0x7f1d40102690 2024-09-12T03:25:15.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.757+0000 7f1d377fe640 1 -- 172.21.15.115:0/21799217 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d38018b70 con 0x7f1d40102690 2024-09-12T03:25:15.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.757+0000 7f1d44f19640 1 -- 172.21.15.115:0/21799217 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d401a01c0 con 0x7f1d40102690 2024-09-12T03:25:15.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.758+0000 7f1d44f19640 1 -- 172.21.15.115:0/21799217 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d401a0450 con 0x7f1d40102690 2024-09-12T03:25:15.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.760+0000 7f1d377fe640 1 -- 172.21.15.115:0/21799217 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1d38018d10 con 0x7f1d40102690 2024-09-12T03:25:15.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.760+0000 7f1d377fe640 1 --2- 172.21.15.115:0/21799217 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f1d2c0a2bb0 0x7f1d2c0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:15.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.761+0000 7f1d377fe640 1 -- 172.21.15.115:0/21799217 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5981+0+0 (secure 0 0 0) 0x7f1d380cecc0 con 0x7f1d40102690 2024-09-12T03:25:15.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.761+0000 7f1d3e59c640 1 --2- 172.21.15.115:0/21799217 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f1d2c0a2bb0 0x7f1d2c0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:15.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.761+0000 7f1d3e59c640 1 --2- 172.21.15.115:0/21799217 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f1d2c0a2bb0 0x7f1d2c0a5070 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1d28000c00 tx=0x7f1d28032d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:15.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.764+0000 7f1d377fe640 1 -- 172.21.15.115:0/21799217 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1d38014030 con 0x7f1d40102690 2024-09-12T03:25:15.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:15.939+0000 7f1d44f19640 1 -- 172.21.15.115:0/21799217 --> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f1d401a0620 con 0x7f1d2c0a2bb0 2024-09-12T03:25:16.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:16 smithi195 ceph-mon[34786]: from='osd.9 [v2:172.21.15.64:6826/561728112,v1:172.21.15.64:6827/561728112]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-12T03:25:16.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:16 smithi195 ceph-mon[34786]: osdmap e62: 10 total, 9 up, 10 in 2024-09-12T03:25:16.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:16 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:25:16.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:16 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:25:16.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:16 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:16.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:16 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:16.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:16 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:25:16.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:16 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:25:16.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:16 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:25:16.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:16 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:25:16.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:16 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:16.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:16 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:25:16.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:16 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:25:16.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:16 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:16.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:16 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:25:16.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:16 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:25:16.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:16 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:25:16.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:16 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:16.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:16 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:25:16.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:16 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:25:16.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:16 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:25:16.498 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:16 smithi115 ceph-mon[34801]: from='osd.9 [v2:172.21.15.64:6826/561728112,v1:172.21.15.64:6827/561728112]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-12T03:25:16.498 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:16 smithi115 ceph-mon[34801]: osdmap e62: 10 total, 9 up, 10 in 2024-09-12T03:25:16.498 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:16 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:25:16.498 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:16 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:25:16.498 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:16 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:16.498 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:16 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:16.498 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:16 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:25:16.498 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:16 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:25:16.499 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:16 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:25:16.499 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:16 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:25:16.499 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:16 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:16.499 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:16 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:25:16.499 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:16 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:25:16.499 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:16 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:16.499 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:16 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:25:16.499 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:16 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:25:16.499 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:16 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:25:16.500 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:16 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:16.500 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:16 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:25:16.500 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:16 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:25:16.500 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:16 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:25:16.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:16 smithi064 ceph-mon[30952]: from='osd.9 [v2:172.21.15.64:6826/561728112,v1:172.21.15.64:6827/561728112]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-12T03:25:16.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:16 smithi064 ceph-mon[30952]: osdmap e62: 10 total, 9 up, 10 in 2024-09-12T03:25:16.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:16 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:25:16.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:16 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:25:16.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:16 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:16.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:16 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:16.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:16 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:25:16.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:16 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:25:16.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:16 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:25:16.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:16 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:25:16.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:16 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:16.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:16 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:25:16.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:16 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:25:16.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:16 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:16.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:16 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:25:16.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:16 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:25:16.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:16 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:25:16.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:16 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:16.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:16 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:25:16.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:16 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:25:16.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:16 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:25:17.359 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:17 smithi115 ceph-mon[34801]: purged_snaps scrub starts 2024-09-12T03:25:17.359 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:17 smithi115 ceph-mon[34801]: purged_snaps scrub ok 2024-09-12T03:25:17.359 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:17 smithi115 ceph-mon[34801]: Detected new or changed devices on smithi064 2024-09-12T03:25:17.360 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:17 smithi115 ceph-mon[34801]: Adjusting osd_memory_target on smithi064 to 4272M 2024-09-12T03:25:17.360 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:17 smithi115 ceph-mon[34801]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:25:17.360 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:17 smithi115 ceph-mon[34801]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:25:17.360 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:17 smithi115 ceph-mon[34801]: osd.9 [v2:172.21.15.64:6826/561728112,v1:172.21.15.64:6827/561728112] boot 2024-09-12T03:25:17.360 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:17 smithi115 ceph-mon[34801]: osdmap e63: 10 total, 10 up, 10 in 2024-09-12T03:25:17.360 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:17 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:25:17.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:17 smithi064 ceph-mon[30952]: purged_snaps scrub starts 2024-09-12T03:25:17.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:17 smithi064 ceph-mon[30952]: purged_snaps scrub ok 2024-09-12T03:25:17.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:17 smithi064 ceph-mon[30952]: Detected new or changed devices on smithi064 2024-09-12T03:25:17.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:17 smithi064 ceph-mon[30952]: Adjusting osd_memory_target on smithi064 to 4272M 2024-09-12T03:25:17.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:17 smithi064 ceph-mon[30952]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:25:17.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:17 smithi064 ceph-mon[30952]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:25:17.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:17 smithi064 ceph-mon[30952]: osd.9 [v2:172.21.15.64:6826/561728112,v1:172.21.15.64:6827/561728112] boot 2024-09-12T03:25:17.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:17 smithi064 ceph-mon[30952]: osdmap e63: 10 total, 10 up, 10 in 2024-09-12T03:25:17.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:17 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:25:17.628 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:17 smithi195 ceph-mon[34786]: purged_snaps scrub starts 2024-09-12T03:25:17.628 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:17 smithi195 ceph-mon[34786]: purged_snaps scrub ok 2024-09-12T03:25:17.628 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:17 smithi195 ceph-mon[34786]: Detected new or changed devices on smithi064 2024-09-12T03:25:17.628 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:17 smithi195 ceph-mon[34786]: Adjusting osd_memory_target on smithi064 to 4272M 2024-09-12T03:25:17.628 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:17 smithi195 ceph-mon[34786]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:25:17.628 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:17 smithi195 ceph-mon[34786]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:25:17.629 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:17 smithi195 ceph-mon[34786]: osd.9 [v2:172.21.15.64:6826/561728112,v1:172.21.15.64:6827/561728112] boot 2024-09-12T03:25:17.629 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:17 smithi195 ceph-mon[34786]: osdmap e63: 10 total, 10 up, 10 in 2024-09-12T03:25:17.629 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:17 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:25:18.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:18 smithi064 ceph-mon[30952]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:25:18.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:18 smithi064 ceph-mon[30952]: osdmap e64: 10 total, 10 up, 10 in 2024-09-12T03:25:18.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:18 smithi064 ceph-mon[30952]: from='client.? 172.21.15.115:0/3356359559' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1f7881c-ec3f-466e-9cea-1d5cb441d970"}]: dispatch 2024-09-12T03:25:18.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:18 smithi064 ceph-mon[30952]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1f7881c-ec3f-466e-9cea-1d5cb441d970"}]: dispatch 2024-09-12T03:25:18.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:18 smithi064 ceph-mon[30952]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d1f7881c-ec3f-466e-9cea-1d5cb441d970"}]': finished 2024-09-12T03:25:18.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:18 smithi064 ceph-mon[30952]: osdmap e65: 11 total, 10 up, 11 in 2024-09-12T03:25:18.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:18 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:25:18.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:18 smithi115 ceph-mon[34801]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:25:18.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:18 smithi115 ceph-mon[34801]: osdmap e64: 10 total, 10 up, 10 in 2024-09-12T03:25:18.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:18 smithi115 ceph-mon[34801]: from='client.? 172.21.15.115:0/3356359559' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1f7881c-ec3f-466e-9cea-1d5cb441d970"}]: dispatch 2024-09-12T03:25:18.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:18 smithi115 ceph-mon[34801]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1f7881c-ec3f-466e-9cea-1d5cb441d970"}]: dispatch 2024-09-12T03:25:18.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:18 smithi115 ceph-mon[34801]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d1f7881c-ec3f-466e-9cea-1d5cb441d970"}]': finished 2024-09-12T03:25:18.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:18 smithi115 ceph-mon[34801]: osdmap e65: 11 total, 10 up, 11 in 2024-09-12T03:25:18.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:18 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:25:18.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:18 smithi195 ceph-mon[34786]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:25:18.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:18 smithi195 ceph-mon[34786]: osdmap e64: 10 total, 10 up, 10 in 2024-09-12T03:25:18.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:18 smithi195 ceph-mon[34786]: from='client.? 172.21.15.115:0/3356359559' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1f7881c-ec3f-466e-9cea-1d5cb441d970"}]: dispatch 2024-09-12T03:25:18.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:18 smithi195 ceph-mon[34786]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1f7881c-ec3f-466e-9cea-1d5cb441d970"}]: dispatch 2024-09-12T03:25:18.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:18 smithi195 ceph-mon[34786]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d1f7881c-ec3f-466e-9cea-1d5cb441d970"}]': finished 2024-09-12T03:25:18.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:18 smithi195 ceph-mon[34786]: osdmap e65: 11 total, 10 up, 11 in 2024-09-12T03:25:18.958 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:18 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:25:19.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:19 smithi064 ceph-mon[30952]: from='client.? 172.21.15.115:0/1568810661' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:25:19.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:19 smithi115 ceph-mon[34801]: from='client.? 172.21.15.115:0/1568810661' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:25:19.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:19 smithi195 ceph-mon[34786]: from='client.? 172.21.15.115:0/1568810661' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:25:20.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:20 smithi064 ceph-mon[30952]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:25:20.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:20 smithi115 ceph-mon[34801]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:25:20.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:20 smithi195 ceph-mon[34786]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:25:22.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:22 smithi064 ceph-mon[30952]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:25:22.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:22 smithi115 ceph-mon[34801]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:25:22.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:22 smithi195 ceph-mon[34786]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:25:24.631 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:24 smithi115 ceph-mon[34801]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:25:24.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:24 smithi064 ceph-mon[30952]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:25:24.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:24 smithi195 ceph-mon[34786]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:25:26.746 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:26 smithi115 ceph-mon[34801]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:25:26.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:26 smithi064 ceph-mon[30952]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:25:26.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:26 smithi195 ceph-mon[34786]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:25:27.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:27 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T03:25:27.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:27 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:25:27.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:27 smithi064 ceph-mon[30952]: Deploying daemon osd.10 on smithi115 2024-09-12T03:25:27.853 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:27 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T03:25:27.853 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:27 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:25:27.853 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:27 smithi115 ceph-mon[34801]: Deploying daemon osd.10 on smithi115 2024-09-12T03:25:27.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:27 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T03:25:27.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:27 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:25:27.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:27 smithi195 ceph-mon[34786]: Deploying daemon osd.10 on smithi115 2024-09-12T03:25:28.763 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:28 smithi115 ceph-mon[34801]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:25:28.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:28 smithi064 ceph-mon[30952]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:25:28.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:28 smithi195 ceph-mon[34786]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:25:30.622 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:30 smithi115 ceph-mon[34801]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:25:30.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:30 smithi064 ceph-mon[30952]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:25:30.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:30 smithi195 ceph-mon[34786]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:25:32.767 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:32 smithi115 ceph-mon[34801]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:25:32.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:32 smithi064 ceph-mon[30952]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:25:32.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:32 smithi195 ceph-mon[34786]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:25:33.555 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:33 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:25:33.555 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:33 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:33.555 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:33 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:33.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:33 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:25:33.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:33 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:33.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:33 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:33.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:33 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:25:33.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:33 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:33.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:33 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:34.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:34 smithi115 ceph-mon[34801]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:25:35.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:34 smithi195 ceph-mon[34786]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:25:35.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:34 smithi064 ceph-mon[30952]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:25:36.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:36.178+0000 7f1d377fe640 1 -- 172.21.15.115:0/21799217 <== mgr.14150 v2:172.21.15.64:6800/759931848 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f1d401a0620 con 0x7f1d2c0a2bb0 2024-09-12T03:25:36.179 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 10 on host 'smithi115' 2024-09-12T03:25:36.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:36.180+0000 7f1d44f19640 1 -- 172.21.15.115:0/21799217 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f1d2c0a2bb0 msgr2=0x7f1d2c0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:36.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:36.180+0000 7f1d44f19640 1 --2- 172.21.15.115:0/21799217 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f1d2c0a2bb0 0x7f1d2c0a5070 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1d28000c00 tx=0x7f1d28032d20 comp rx=0 tx=0).stop 2024-09-12T03:25:36.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:36.180+0000 7f1d44f19640 1 -- 172.21.15.115:0/21799217 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d40102690 msgr2=0x7f1d4019f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:36.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:36.180+0000 7f1d44f19640 1 --2- 172.21.15.115:0/21799217 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d40102690 0x7f1d4019f3d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1d3800cce0 tx=0x7f1d3800a620 comp rx=0 tx=0).stop 2024-09-12T03:25:36.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:36.180+0000 7f1d44f19640 1 -- 172.21.15.115:0/21799217 shutdown_connections 2024-09-12T03:25:36.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:36.180+0000 7f1d44f19640 1 --2- 172.21.15.115:0/21799217 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f1d2c0a2bb0 0x7f1d2c0a5070 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:36.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:36.180+0000 7f1d44f19640 1 --2- 172.21.15.115:0/21799217 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d40102690 0x7f1d4019f3d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:36.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:36.180+0000 7f1d44f19640 1 --2- 172.21.15.115:0/21799217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d40101cd0 0x7f1d40198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:36.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:36.180+0000 7f1d44f19640 1 --2- 172.21.15.115:0/21799217 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d40100ad0 0x7f1d40197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:36.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:36.180+0000 7f1d44f19640 1 -- 172.21.15.115:0/21799217 >> 172.21.15.115:0/21799217 conn(0x7f1d400fc280 msgr2=0x7f1d4006a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:36.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:36.181+0000 7f1d44f19640 1 -- 172.21.15.115:0/21799217 shutdown_connections 2024-09-12T03:25:36.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T03:25:36.181+0000 7f1d44f19640 1 -- 172.21.15.115:0/21799217 wait complete. 2024-09-12T03:25:36.778 DEBUG:teuthology.orchestra.run.smithi115:osd.10> sudo journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@osd.10.service 2024-09-12T03:25:36.781 INFO:tasks.cephadm:Deploying osd.11 on smithi195 with /dev/vg_nvme/lv_1... 2024-09-12T03:25:36.781 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T03:25:37.037 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:25:37.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:37 smithi195 ceph-mon[34786]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:25:37.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:37 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:37.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:37 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:37.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:37 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:37.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:37 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:37.513 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:37 smithi115 ceph-mon[34801]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:25:37.513 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:37 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:37.513 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:37 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:37.513 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:37 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:37.513 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:37 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:37.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:37 smithi064 ceph-mon[30952]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:25:37.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:37 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:37.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:37 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:37.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:37 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:37.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:37 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:39.142 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:25:38 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:25:38.860+0000 7f6f19f96640 -1 osd.10 0 log_to_monitors true 2024-09-12T03:25:39.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:39 smithi195 ceph-mon[34786]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:25:39.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:39 smithi195 ceph-mon[34786]: from='osd.10 [v2:172.21.15.115:6824/2977887627,v1:172.21.15.115:6825/2977887627]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T03:25:39.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:39 smithi195 ceph-mon[34786]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T03:25:39.517 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:39 smithi115 ceph-mon[34801]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:25:39.517 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:39 smithi115 ceph-mon[34801]: from='osd.10 [v2:172.21.15.115:6824/2977887627,v1:172.21.15.115:6825/2977887627]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T03:25:39.517 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:39 smithi115 ceph-mon[34801]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T03:25:39.521 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:39 smithi064 ceph-mon[30952]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:25:39.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:39 smithi064 ceph-mon[30952]: from='osd.10 [v2:172.21.15.115:6824/2977887627,v1:172.21.15.115:6825/2977887627]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T03:25:39.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:39 smithi064 ceph-mon[30952]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T03:25:39.760 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-12T03:25:39.786 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi195:vg_nvme/lv_1 2024-09-12T03:25:40.039 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:25:40.435 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:40 smithi195 ceph-mon[34786]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T03:25:40.436 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:40 smithi195 ceph-mon[34786]: osdmap e66: 11 total, 10 up, 11 in 2024-09-12T03:25:40.436 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:40 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:25:40.436 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:40 smithi195 ceph-mon[34786]: from='osd.10 [v2:172.21.15.115:6824/2977887627,v1:172.21.15.115:6825/2977887627]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-12T03:25:40.437 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:40 smithi195 ceph-mon[34786]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-12T03:25:40.437 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:40 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:40.437 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:40 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:40.437 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:40 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:25:40.437 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:40 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:25:40.437 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:40 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:25:40.438 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:40 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:25:40.438 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:40 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:40.438 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:40 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:25:40.438 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:40 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:25:40.438 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:40 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:40.438 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:40 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:25:40.439 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:40 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:25:40.439 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:40 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:25:40.439 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:40 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:40.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:40 smithi064 ceph-mon[30952]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T03:25:40.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:40 smithi064 ceph-mon[30952]: osdmap e66: 11 total, 10 up, 11 in 2024-09-12T03:25:40.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:40 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:25:40.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:40 smithi064 ceph-mon[30952]: from='osd.10 [v2:172.21.15.115:6824/2977887627,v1:172.21.15.115:6825/2977887627]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-12T03:25:40.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:40 smithi064 ceph-mon[30952]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-12T03:25:40.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:40 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:40.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:40 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:40.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:40 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:25:40.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:40 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:25:40.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:40 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:25:40.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:40 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:25:40.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:40 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:40.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:40 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:25:40.525 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:40 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:25:40.525 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:40 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:40.525 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:40 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:25:40.525 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:40 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:25:40.525 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:40 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:25:40.525 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:40 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:40.535 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.518+0000 7f5b35074640 1 -- 172.21.15.195:0/2868033765 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b30101cd0 msgr2=0x7f5b30102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:40.536 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.518+0000 7f5b35074640 1 --2- 172.21.15.195:0/2868033765 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b30101cd0 0x7f5b30102150 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5b20009900 tx=0x7f5b2002a410 comp rx=0 tx=0).stop 2024-09-12T03:25:40.536 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.519+0000 7f5b35074640 1 -- 172.21.15.195:0/2868033765 shutdown_connections 2024-09-12T03:25:40.536 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.519+0000 7f5b35074640 1 --2- 172.21.15.195:0/2868033765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b30102690 0x7f5b30108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.536 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.519+0000 7f5b35074640 1 --2- 172.21.15.195:0/2868033765 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b30101cd0 0x7f5b30102150 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.536 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.519+0000 7f5b35074640 1 --2- 172.21.15.195:0/2868033765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b30100ad0 0x7f5b30100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.536 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.519+0000 7f5b35074640 1 -- 172.21.15.195:0/2868033765 >> 172.21.15.195:0/2868033765 conn(0x7f5b300fc280 msgr2=0x7f5b300fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:40.536 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.519+0000 7f5b35074640 1 -- 172.21.15.195:0/2868033765 shutdown_connections 2024-09-12T03:25:40.536 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.519+0000 7f5b35074640 1 -- 172.21.15.195:0/2868033765 wait complete. 2024-09-12T03:25:40.536 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.519+0000 7f5b35074640 1 Processor -- start 2024-09-12T03:25:40.536 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.520+0000 7f5b35074640 1 -- start start 2024-09-12T03:25:40.536 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.520+0000 7f5b35074640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b30100ad0 0x7f5b30104a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:40.537 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.520+0000 7f5b35074640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b30101cd0 0x7f5b301030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:40.537 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.520+0000 7f5b35074640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b30102690 0x7f5b301035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:40.537 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.520+0000 7f5b35074640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3010c600 con 0x7f5b30100ad0 2024-09-12T03:25:40.537 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.520+0000 7f5b35074640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3010c480 con 0x7f5b30101cd0 2024-09-12T03:25:40.537 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.520+0000 7f5b35074640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3010c780 con 0x7f5b30102690 2024-09-12T03:25:40.537 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.520+0000 7f5b2f59e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b30102690 0x7f5b301035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:40.537 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.520+0000 7f5b2e59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b30101cd0 0x7f5b301030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:40.537 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.520+0000 7f5b2f59e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b30102690 0x7f5b301035f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:49260/0 (socket says 172.21.15.195:49260) 2024-09-12T03:25:40.537 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.520+0000 7f5b2f59e640 1 -- 172.21.15.195:0/3773989204 learned_addr learned my addr 172.21.15.195:0/3773989204 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-09-12T03:25:40.537 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.520+0000 7f5b2ed9d640 1 --2- 172.21.15.195:0/3773989204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b30100ad0 0x7f5b30104a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:40.537 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.520+0000 7f5b2f59e640 1 -- 172.21.15.195:0/3773989204 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b30101cd0 msgr2=0x7f5b301030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:40.538 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.520+0000 7f5b2f59e640 1 --2- 172.21.15.195:0/3773989204 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b30101cd0 0x7f5b301030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.538 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.520+0000 7f5b2f59e640 1 -- 172.21.15.195:0/3773989204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b30100ad0 msgr2=0x7f5b30104a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:40.538 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.520+0000 7f5b2f59e640 1 --2- 172.21.15.195:0/3773989204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b30100ad0 0x7f5b30104a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:40.538 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.520+0000 7f5b2f59e640 1 -- 172.21.15.195:0/3773989204 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b30103eb0 con 0x7f5b30102690 2024-09-12T03:25:40.538 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.520+0000 7f5b2e59c640 1 --2- 172.21.15.195:0/3773989204 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b30101cd0 0x7f5b301030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:25:40.538 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.521+0000 7f5b2f59e640 1 --2- 172.21.15.195:0/3773989204 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b30102690 0x7f5b301035f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5b2800cce0 tx=0x7f5b2800a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:40.538 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.521+0000 7f5b27fff640 1 -- 172.21.15.195:0/3773989204 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2800f450 con 0x7f5b30102690 2024-09-12T03:25:40.538 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.521+0000 7f5b27fff640 1 -- 172.21.15.195:0/3773989204 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b2800faa0 con 0x7f5b30102690 2024-09-12T03:25:40.538 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.521+0000 7f5b35074640 1 -- 172.21.15.195:0/3773989204 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b301a1f80 con 0x7f5b30102690 2024-09-12T03:25:40.538 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.521+0000 7f5b27fff640 1 -- 172.21.15.195:0/3773989204 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b28018b70 con 0x7f5b30102690 2024-09-12T03:25:40.538 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.521+0000 7f5b35074640 1 -- 172.21.15.195:0/3773989204 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b301a2430 con 0x7f5b30102690 2024-09-12T03:25:40.538 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.522+0000 7f5b35074640 1 -- 172.21.15.195:0/3773989204 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b301a26c0 con 0x7f5b30102690 2024-09-12T03:25:40.538 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.523+0000 7f5b27fff640 1 -- 172.21.15.195:0/3773989204 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5b28018d10 con 0x7f5b30102690 2024-09-12T03:25:40.539 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.524+0000 7f5b27fff640 1 --2- 172.21.15.195:0/3773989204 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f5b1c0a2c00 0x7f5b1c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:40.539 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.524+0000 7f5b27fff640 1 -- 172.21.15.195:0/3773989204 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6414+0+0 (secure 0 0 0) 0x7f5b280cec10 con 0x7f5b30102690 2024-09-12T03:25:40.539 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.524+0000 7f5b2ed9d640 1 --2- 172.21.15.195:0/3773989204 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f5b1c0a2c00 0x7f5b1c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:40.539 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.525+0000 7f5b2ed9d640 1 --2- 172.21.15.195:0/3773989204 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f5b1c0a2c00 0x7f5b1c0a50c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5b18005fd0 tx=0x7f5b1800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:40.539 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.528+0000 7f5b27fff640 1 -- 172.21.15.195:0/3773989204 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5b28014030 con 0x7f5b30102690 2024-09-12T03:25:40.568 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:40 smithi115 ceph-mon[34801]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T03:25:40.568 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:40 smithi115 ceph-mon[34801]: osdmap e66: 11 total, 10 up, 11 in 2024-09-12T03:25:40.568 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:40 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:25:40.568 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:40 smithi115 ceph-mon[34801]: from='osd.10 [v2:172.21.15.115:6824/2977887627,v1:172.21.15.115:6825/2977887627]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-12T03:25:40.569 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:40 smithi115 ceph-mon[34801]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-12T03:25:40.569 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:40 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:40.569 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:40 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:40.569 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:40 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:25:40.569 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:40 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:25:40.569 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:40 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:25:40.569 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:40 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:25:40.569 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:40 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:40.569 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:40 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:25:40.569 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:40 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:25:40.569 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:40 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:40.569 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:40 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:25:40.569 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:40 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:25:40.570 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:40 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:25:40.570 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:40 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:40.707 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:25:40.707+0000 7f5b35074640 1 -- 172.21.15.195:0/3773989204 --> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f5b3006a6d0 con 0x7f5b1c0a2c00 2024-09-12T03:25:40.893 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:25:40 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:25:40.567+0000 7f6f1672d640 -1 osd.10 0 waiting for initial osdmap 2024-09-12T03:25:40.893 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:25:40 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:25:40.577+0000 7f6f0dd39640 -1 osd.10 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:25:41.441 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:41 smithi195 ceph-mon[34786]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:25:41.441 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:41 smithi195 ceph-mon[34786]: Detected new or changed devices on smithi115 2024-09-12T03:25:41.441 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:41 smithi195 ceph-mon[34786]: Adjusting osd_memory_target on smithi115 to 4272M 2024-09-12T03:25:41.441 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:41 smithi195 ceph-mon[34786]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-12T03:25:41.441 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:41 smithi195 ceph-mon[34786]: osdmap e67: 11 total, 10 up, 11 in 2024-09-12T03:25:41.441 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:41 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:25:41.441 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:41 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:25:41.441 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:41 smithi195 ceph-mon[34786]: from='osd.10 ' entity='osd.10' 2024-09-12T03:25:41.441 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:41 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:25:41.442 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:41 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:25:41.442 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:41 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:25:41.442 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:41 smithi195 ceph-mon[34786]: osd.10 [v2:172.21.15.115:6824/2977887627,v1:172.21.15.115:6825/2977887627] boot 2024-09-12T03:25:41.442 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:41 smithi195 ceph-mon[34786]: osdmap e68: 11 total, 11 up, 11 in 2024-09-12T03:25:41.442 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:41 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:25:41.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:41 smithi064 ceph-mon[30952]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:25:41.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:41 smithi064 ceph-mon[30952]: Detected new or changed devices on smithi115 2024-09-12T03:25:41.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:41 smithi064 ceph-mon[30952]: Adjusting osd_memory_target on smithi115 to 4272M 2024-09-12T03:25:41.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:41 smithi064 ceph-mon[30952]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-12T03:25:41.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:41 smithi064 ceph-mon[30952]: osdmap e67: 11 total, 10 up, 11 in 2024-09-12T03:25:41.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:41 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:25:41.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:41 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:25:41.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:41 smithi064 ceph-mon[30952]: from='osd.10 ' entity='osd.10' 2024-09-12T03:25:41.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:41 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:25:41.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:41 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:25:41.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:41 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:25:41.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:41 smithi064 ceph-mon[30952]: osd.10 [v2:172.21.15.115:6824/2977887627,v1:172.21.15.115:6825/2977887627] boot 2024-09-12T03:25:41.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:41 smithi064 ceph-mon[30952]: osdmap e68: 11 total, 11 up, 11 in 2024-09-12T03:25:41.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:41 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:25:41.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:41 smithi115 ceph-mon[34801]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:25:41.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:41 smithi115 ceph-mon[34801]: Detected new or changed devices on smithi115 2024-09-12T03:25:41.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:41 smithi115 ceph-mon[34801]: Adjusting osd_memory_target on smithi115 to 4272M 2024-09-12T03:25:41.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:41 smithi115 ceph-mon[34801]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-12T03:25:41.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:41 smithi115 ceph-mon[34801]: osdmap e67: 11 total, 10 up, 11 in 2024-09-12T03:25:41.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:41 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:25:41.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:41 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:25:41.645 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:41 smithi115 ceph-mon[34801]: from='osd.10 ' entity='osd.10' 2024-09-12T03:25:41.645 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:41 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:25:41.645 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:41 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:25:41.645 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:41 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:25:41.646 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:41 smithi115 ceph-mon[34801]: osd.10 [v2:172.21.15.115:6824/2977887627,v1:172.21.15.115:6825/2977887627] boot 2024-09-12T03:25:41.646 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:41 smithi115 ceph-mon[34801]: osdmap e68: 11 total, 11 up, 11 in 2024-09-12T03:25:41.646 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:41 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:25:42.425 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:42 smithi195 ceph-mon[34786]: purged_snaps scrub starts 2024-09-12T03:25:42.425 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:42 smithi195 ceph-mon[34786]: purged_snaps scrub ok 2024-09-12T03:25:42.425 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:42 smithi195 ceph-mon[34786]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:25:42.425 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:42 smithi195 ceph-mon[34786]: osdmap e69: 11 total, 11 up, 11 in 2024-09-12T03:25:42.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:42 smithi064 ceph-mon[30952]: purged_snaps scrub starts 2024-09-12T03:25:42.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:42 smithi064 ceph-mon[30952]: purged_snaps scrub ok 2024-09-12T03:25:42.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:42 smithi064 ceph-mon[30952]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:25:42.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:42 smithi064 ceph-mon[30952]: osdmap e69: 11 total, 11 up, 11 in 2024-09-12T03:25:42.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:42 smithi115 ceph-mon[34801]: purged_snaps scrub starts 2024-09-12T03:25:42.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:42 smithi115 ceph-mon[34801]: purged_snaps scrub ok 2024-09-12T03:25:42.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:42 smithi115 ceph-mon[34801]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:25:42.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:42 smithi115 ceph-mon[34801]: osdmap e69: 11 total, 11 up, 11 in 2024-09-12T03:25:43.454 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:43 smithi195 ceph-mon[34786]: pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:25:43.454 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:43 smithi195 ceph-mon[34786]: osdmap e70: 11 total, 11 up, 11 in 2024-09-12T03:25:43.454 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:43 smithi195 ceph-mon[34786]: from='client.? 172.21.15.195:0/1352680680' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c46c028-22c7-450f-9927-2144a7a09e9e"}]: dispatch 2024-09-12T03:25:43.454 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:43 smithi195 ceph-mon[34786]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c46c028-22c7-450f-9927-2144a7a09e9e"}]: dispatch 2024-09-12T03:25:43.454 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:43 smithi195 ceph-mon[34786]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7c46c028-22c7-450f-9927-2144a7a09e9e"}]': finished 2024-09-12T03:25:43.454 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:43 smithi195 ceph-mon[34786]: osdmap e71: 12 total, 11 up, 12 in 2024-09-12T03:25:43.454 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:43 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:25:43.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:43 smithi064 ceph-mon[30952]: pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:25:43.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:43 smithi064 ceph-mon[30952]: osdmap e70: 11 total, 11 up, 11 in 2024-09-12T03:25:43.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:43 smithi064 ceph-mon[30952]: from='client.? 172.21.15.195:0/1352680680' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c46c028-22c7-450f-9927-2144a7a09e9e"}]: dispatch 2024-09-12T03:25:43.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:43 smithi064 ceph-mon[30952]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c46c028-22c7-450f-9927-2144a7a09e9e"}]: dispatch 2024-09-12T03:25:43.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:43 smithi064 ceph-mon[30952]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7c46c028-22c7-450f-9927-2144a7a09e9e"}]': finished 2024-09-12T03:25:43.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:43 smithi064 ceph-mon[30952]: osdmap e71: 12 total, 11 up, 12 in 2024-09-12T03:25:43.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:43 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:25:43.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:43 smithi115 ceph-mon[34801]: pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:25:43.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:43 smithi115 ceph-mon[34801]: osdmap e70: 11 total, 11 up, 11 in 2024-09-12T03:25:43.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:43 smithi115 ceph-mon[34801]: from='client.? 172.21.15.195:0/1352680680' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c46c028-22c7-450f-9927-2144a7a09e9e"}]: dispatch 2024-09-12T03:25:43.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:43 smithi115 ceph-mon[34801]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c46c028-22c7-450f-9927-2144a7a09e9e"}]: dispatch 2024-09-12T03:25:43.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:43 smithi115 ceph-mon[34801]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7c46c028-22c7-450f-9927-2144a7a09e9e"}]': finished 2024-09-12T03:25:43.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:43 smithi115 ceph-mon[34801]: osdmap e71: 12 total, 11 up, 12 in 2024-09-12T03:25:43.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:43 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:25:44.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:44 smithi195 ceph-mon[34786]: from='client.? 172.21.15.195:0/465687977' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:25:44.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:44 smithi064 ceph-mon[30952]: from='client.? 172.21.15.195:0/465687977' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:25:44.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:44 smithi115 ceph-mon[34801]: from='client.? 172.21.15.195:0/465687977' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:25:45.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:45 smithi064 ceph-mon[30952]: pgmap v229: 1 pgs: 1 remapped+peering; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:25:45.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:45 smithi064 ceph-mon[30952]: osdmap e72: 12 total, 11 up, 12 in 2024-09-12T03:25:45.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:45 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:25:45.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:45 smithi115 ceph-mon[34801]: pgmap v229: 1 pgs: 1 remapped+peering; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:25:45.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:45 smithi115 ceph-mon[34801]: osdmap e72: 12 total, 11 up, 12 in 2024-09-12T03:25:45.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:45 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:25:45.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:45 smithi195 ceph-mon[34786]: pgmap v229: 1 pgs: 1 remapped+peering; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:25:45.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:45 smithi195 ceph-mon[34786]: osdmap e72: 12 total, 11 up, 12 in 2024-09-12T03:25:45.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:45 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:25:46.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:46 smithi064 ceph-mon[30952]: osdmap e73: 12 total, 11 up, 12 in 2024-09-12T03:25:46.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:46 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:25:46.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:46 smithi115 ceph-mon[34801]: osdmap e73: 12 total, 11 up, 12 in 2024-09-12T03:25:46.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:46 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:25:46.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:46 smithi195 ceph-mon[34786]: osdmap e73: 12 total, 11 up, 12 in 2024-09-12T03:25:46.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:46 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:25:47.498 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:47 smithi195 ceph-mon[34786]: pgmap v232: 1 pgs: 1 remapped+peering; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:25:47.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:47 smithi064 ceph-mon[30952]: pgmap v232: 1 pgs: 1 remapped+peering; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:25:47.544 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:47 smithi115 ceph-mon[34801]: pgmap v232: 1 pgs: 1 remapped+peering; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:25:49.383 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:49 smithi195 ceph-mon[34786]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-09-12T03:25:49.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:49 smithi064 ceph-mon[30952]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-09-12T03:25:49.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:49 smithi115 ceph-mon[34801]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-09-12T03:25:51.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:51 smithi195 ceph-mon[34786]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-09-12T03:25:51.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:51 smithi064 ceph-mon[30952]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-09-12T03:25:51.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:51 smithi115 ceph-mon[34801]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-09-12T03:25:52.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:52 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T03:25:52.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:52 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:25:52.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:52 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T03:25:52.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:52 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:25:52.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:52 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T03:25:52.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:52 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:25:53.483 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:53 smithi195 ceph-mon[34786]: Deploying daemon osd.11 on smithi195 2024-09-12T03:25:53.483 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:53 smithi195 ceph-mon[34786]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T03:25:53.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:53 smithi064 ceph-mon[30952]: Deploying daemon osd.11 on smithi195 2024-09-12T03:25:53.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:53 smithi064 ceph-mon[30952]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T03:25:53.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:53 smithi115 ceph-mon[34801]: Deploying daemon osd.11 on smithi195 2024-09-12T03:25:53.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:53 smithi115 ceph-mon[34801]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T03:25:55.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:55 smithi195 ceph-mon[34786]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-12T03:25:55.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:55 smithi064 ceph-mon[30952]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-12T03:25:55.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:55 smithi115 ceph-mon[34801]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-12T03:25:57.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:57 smithi195 ceph-mon[34786]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-12T03:25:57.521 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:57 smithi064 ceph-mon[30952]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-12T03:25:57.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:57 smithi115 ceph-mon[34801]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-12T03:25:58.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:58 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:25:58.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:58 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:58.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:58 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:58.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:58 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:25:58.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:58 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:58.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:58 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:58.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:58 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:25:58.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:58 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:58.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:58 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:25:59.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:25:59 smithi195 ceph-mon[34786]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T03:25:59.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:25:59 smithi115 ceph-mon[34801]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T03:25:59.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:25:59 smithi064 ceph-mon[30952]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T03:26:00.896 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:00.896+0000 7f5b27fff640 1 -- 172.21.15.195:0/3773989204 <== mgr.14150 v2:172.21.15.64:6800/759931848 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f5b3006a6d0 con 0x7f5b1c0a2c00 2024-09-12T03:26:00.896 INFO:teuthology.orchestra.run.smithi195.stdout:Created osd(s) 11 on host 'smithi195' 2024-09-12T03:26:00.900 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:00.898+0000 7f5b35074640 1 -- 172.21.15.195:0/3773989204 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f5b1c0a2c00 msgr2=0x7f5b1c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:00.900 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:00.898+0000 7f5b35074640 1 --2- 172.21.15.195:0/3773989204 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f5b1c0a2c00 0x7f5b1c0a50c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5b18005fd0 tx=0x7f5b1800b390 comp rx=0 tx=0).stop 2024-09-12T03:26:00.900 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:00.898+0000 7f5b35074640 1 -- 172.21.15.195:0/3773989204 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b30102690 msgr2=0x7f5b301035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:00.900 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:00.898+0000 7f5b35074640 1 --2- 172.21.15.195:0/3773989204 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b30102690 0x7f5b301035f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5b2800cce0 tx=0x7f5b2800a620 comp rx=0 tx=0).stop 2024-09-12T03:26:00.901 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:00.898+0000 7f5b35074640 1 -- 172.21.15.195:0/3773989204 shutdown_connections 2024-09-12T03:26:00.901 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:00.898+0000 7f5b35074640 1 --2- 172.21.15.195:0/3773989204 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f5b1c0a2c00 0x7f5b1c0a50c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:00.901 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:00.898+0000 7f5b35074640 1 --2- 172.21.15.195:0/3773989204 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b30102690 0x7f5b301035f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:00.901 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:00.898+0000 7f5b35074640 1 --2- 172.21.15.195:0/3773989204 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b30101cd0 0x7f5b301030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:00.901 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:00.898+0000 7f5b35074640 1 --2- 172.21.15.195:0/3773989204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b30100ad0 0x7f5b30104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:00.901 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:00.898+0000 7f5b35074640 1 -- 172.21.15.195:0/3773989204 >> 172.21.15.195:0/3773989204 conn(0x7f5b300fc280 msgr2=0x7f5b30069fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:00.901 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:00.898+0000 7f5b35074640 1 -- 172.21.15.195:0/3773989204 shutdown_connections 2024-09-12T03:26:00.901 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:00.898+0000 7f5b35074640 1 -- 172.21.15.195:0/3773989204 wait complete. 2024-09-12T03:26:01.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:01 smithi195 ceph-mon[34786]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:26:01.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:01 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:01.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:01 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:01.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:01 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:01.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:01 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:26:01.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:01 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:01.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:01 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:26:01.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:01 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:01.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:01 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:01.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:01 smithi115 ceph-mon[34801]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:26:01.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:01 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:01.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:01 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:01.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:01 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:01.394 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:01 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:26:01.394 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:01 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:01.394 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:01 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:26:01.394 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:01 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:01.394 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:01 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:01.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:01 smithi064 ceph-mon[30952]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:26:01.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:01 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:01.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:01 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:01.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:01 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:01.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:01 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:26:01.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:01 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:01.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:01 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:26:01.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:01 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:01.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:01 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:01.681 DEBUG:teuthology.orchestra.run.smithi195:osd.11> sudo journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@osd.11.service 2024-09-12T03:26:01.683 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-12T03:26:01.684 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-12T03:26:01.952 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:02.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.428+0000 7f1853ca0640 1 -- 172.21.15.64:0/3597562117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f184c102690 msgr2=0x7f184c108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:02.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.428+0000 7f1853ca0640 1 --2- 172.21.15.64:0/3597562117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f184c102690 0x7f184c108f20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1844009ca0 tx=0x7f184402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.428+0000 7f1853ca0640 1 -- 172.21.15.64:0/3597562117 shutdown_connections 2024-09-12T03:26:02.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.428+0000 7f1853ca0640 1 --2- 172.21.15.64:0/3597562117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f184c102690 0x7f184c108f20 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.428+0000 7f1853ca0640 1 --2- 172.21.15.64:0/3597562117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f184c101cd0 0x7f184c102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.428+0000 7f1853ca0640 1 --2- 172.21.15.64:0/3597562117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f184c100ad0 0x7f184c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.428+0000 7f1853ca0640 1 -- 172.21.15.64:0/3597562117 >> 172.21.15.64:0/3597562117 conn(0x7f184c0fc280 msgr2=0x7f184c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:02.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.428+0000 7f1853ca0640 1 -- 172.21.15.64:0/3597562117 shutdown_connections 2024-09-12T03:26:02.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.428+0000 7f1853ca0640 1 -- 172.21.15.64:0/3597562117 wait complete. 2024-09-12T03:26:02.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.429+0000 7f1853ca0640 1 Processor -- start 2024-09-12T03:26:02.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.429+0000 7f1853ca0640 1 -- start start 2024-09-12T03:26:02.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.429+0000 7f1853ca0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f184c100ad0 0x7f184c06aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:02.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.429+0000 7f1853ca0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f184c101cd0 0x7f184c19caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:02.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.429+0000 7f1853ca0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f184c102690 0x7f184c19f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:02.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.429+0000 7f1853ca0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f184c10c5e0 con 0x7f184c101cd0 2024-09-12T03:26:02.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.429+0000 7f1853ca0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f184c10c460 con 0x7f184c102690 2024-09-12T03:26:02.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.429+0000 7f1853ca0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f184c10c760 con 0x7f184c100ad0 2024-09-12T03:26:02.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.430+0000 7f185123b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f184c101cd0 0x7f184c19caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:02.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.430+0000 7f1851a3c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f184c100ad0 0x7f184c06aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:02.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.430+0000 7f185123b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f184c101cd0 0x7f184c19caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41928/0 (socket says 172.21.15.64:41928) 2024-09-12T03:26:02.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.430+0000 7f185123b640 1 -- 172.21.15.64:0/1920845513 learned_addr learned my addr 172.21.15.64:0/1920845513 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:26:02.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.430+0000 7f185223d640 1 --2- 172.21.15.64:0/1920845513 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f184c102690 0x7f184c19f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:02.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.430+0000 7f185123b640 1 -- 172.21.15.64:0/1920845513 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f184c100ad0 msgr2=0x7f184c06aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:02.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.430+0000 7f185123b640 1 --2- 172.21.15.64:0/1920845513 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f184c100ad0 0x7f184c06aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.430+0000 7f185123b640 1 -- 172.21.15.64:0/1920845513 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f184c102690 msgr2=0x7f184c19f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:02.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.430+0000 7f185123b640 1 --2- 172.21.15.64:0/1920845513 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f184c102690 0x7f184c19f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.430+0000 7f185123b640 1 -- 172.21.15.64:0/1920845513 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f184c19fa20 con 0x7f184c101cd0 2024-09-12T03:26:02.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.430+0000 7f1851a3c640 1 --2- 172.21.15.64:0/1920845513 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f184c100ad0 0x7f184c06aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:26:02.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.430+0000 7f185123b640 1 --2- 172.21.15.64:0/1920845513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f184c101cd0 0x7f184c19caf0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f183c00d950 tx=0x7f183c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:02.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.430+0000 7f1842ffd640 1 -- 172.21.15.64:0/1920845513 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c017030 con 0x7f184c101cd0 2024-09-12T03:26:02.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.430+0000 7f1853ca0640 1 -- 172.21.15.64:0/1920845513 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f184c19fcb0 con 0x7f184c101cd0 2024-09-12T03:26:02.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.430+0000 7f1842ffd640 1 -- 172.21.15.64:0/1920845513 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f183c004020 con 0x7f184c101cd0 2024-09-12T03:26:02.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.430+0000 7f1842ffd640 1 -- 172.21.15.64:0/1920845513 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c0047a0 con 0x7f184c101cd0 2024-09-12T03:26:02.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.430+0000 7f1853ca0640 1 -- 172.21.15.64:0/1920845513 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f184c1a01a0 con 0x7f184c101cd0 2024-09-12T03:26:02.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.432+0000 7f1853ca0640 1 -- 172.21.15.64:0/1920845513 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f184c107f20 con 0x7f184c101cd0 2024-09-12T03:26:02.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.433+0000 7f1842ffd640 1 -- 172.21.15.64:0/1920845513 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f183c0041d0 con 0x7f184c101cd0 2024-09-12T03:26:02.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.434+0000 7f1842ffd640 1 --2- 172.21.15.64:0/1920845513 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f18380a2b80 0x7f18380a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:02.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.434+0000 7f1851a3c640 1 --2- 172.21.15.64:0/1920845513 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f18380a2b80 0x7f18380a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:02.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.434+0000 7f1842ffd640 1 -- 172.21.15.64:0/1920845513 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6809+0+0 (secure 0 0 0) 0x7f183c0ce470 con 0x7f184c101cd0 2024-09-12T03:26:02.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.434+0000 7f1851a3c640 1 --2- 172.21.15.64:0/1920845513 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f18380a2b80 0x7f18380a5040 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1848005fd0 tx=0x7f184800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:02.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.438+0000 7f1842ffd640 1 -- 172.21.15.64:0/1920845513 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f183c09b910 con 0x7f184c101cd0 2024-09-12T03:26:02.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.614+0000 7f1853ca0640 1 -- 172.21.15.64:0/1920845513 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f184c062e90 con 0x7f184c101cd0 2024-09-12T03:26:02.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.614+0000 7f1842ffd640 1 -- 172.21.15.64:0/1920845513 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) ==== 74+0+133 (secure 0 0 0) 0x7f183c09b210 con 0x7f184c101cd0 2024-09-12T03:26:02.614 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:26:02.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.616+0000 7f1853ca0640 1 -- 172.21.15.64:0/1920845513 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f18380a2b80 msgr2=0x7f18380a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:02.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.616+0000 7f1853ca0640 1 --2- 172.21.15.64:0/1920845513 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f18380a2b80 0x7f18380a5040 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1848005fd0 tx=0x7f184800b410 comp rx=0 tx=0).stop 2024-09-12T03:26:02.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.616+0000 7f1853ca0640 1 -- 172.21.15.64:0/1920845513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f184c101cd0 msgr2=0x7f184c19caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:02.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.616+0000 7f1853ca0640 1 --2- 172.21.15.64:0/1920845513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f184c101cd0 0x7f184c19caf0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f183c00d950 tx=0x7f183c00de10 comp rx=0 tx=0).stop 2024-09-12T03:26:02.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.617+0000 7f1853ca0640 1 -- 172.21.15.64:0/1920845513 shutdown_connections 2024-09-12T03:26:02.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.617+0000 7f1853ca0640 1 --2- 172.21.15.64:0/1920845513 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f18380a2b80 0x7f18380a5040 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.617+0000 7f1853ca0640 1 --2- 172.21.15.64:0/1920845513 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f184c102690 0x7f184c19f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.617+0000 7f1853ca0640 1 --2- 172.21.15.64:0/1920845513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f184c101cd0 0x7f184c19caf0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.617+0000 7f1853ca0640 1 --2- 172.21.15.64:0/1920845513 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f184c100ad0 0x7f184c06aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.617+0000 7f1853ca0640 1 -- 172.21.15.64:0/1920845513 >> 172.21.15.64:0/1920845513 conn(0x7f184c0fc280 msgr2=0x7f184c1050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:02.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.617+0000 7f1853ca0640 1 -- 172.21.15.64:0/1920845513 shutdown_connections 2024-09-12T03:26:02.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:02.617+0000 7f1853ca0640 1 -- 172.21.15.64:0/1920845513 wait complete. 2024-09-12T03:26:03.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:03 smithi195 ceph-mon[34786]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:26:03.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:03 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/1920845513' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:26:03.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:03 smithi115 ceph-mon[34801]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:26:03.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:03 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/1920845513' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:26:03.473 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:03 smithi064 ceph-mon[30952]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:26:03.473 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:03 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/1920845513' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:26:03.501 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":73,"num_osds":12,"num_up_osds":11,"osd_up_since":1726111540,"num_in_osds":12,"osd_in_since":1726111543,"num_remapped_pgs":0} 2024-09-12T03:26:03.707 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:26:03 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:26:03.398+0000 7f97d3b6f640 -1 osd.11 0 log_to_monitors true 2024-09-12T03:26:04.336 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:04 smithi195 ceph-mon[34786]: from='osd.11 [v2:172.21.15.195:6824/591599352,v1:172.21.15.195:6825/591599352]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T03:26:04.336 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:04 smithi195 ceph-mon[34786]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T03:26:04.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:04 smithi115 ceph-mon[34801]: from='osd.11 [v2:172.21.15.195:6824/591599352,v1:172.21.15.195:6825/591599352]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T03:26:04.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:04 smithi115 ceph-mon[34801]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T03:26:04.503 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-12T03:26:04.521 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:04 smithi064 ceph-mon[30952]: from='osd.11 [v2:172.21.15.195:6824/591599352,v1:172.21.15.195:6825/591599352]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T03:26:04.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:04 smithi064 ceph-mon[30952]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T03:26:04.782 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:05.186 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:05 smithi064 ceph-mon[30952]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:26:05.187 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:05 smithi064 ceph-mon[30952]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T03:26:05.187 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:05 smithi064 ceph-mon[30952]: osdmap e74: 12 total, 11 up, 12 in 2024-09-12T03:26:05.187 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:05 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:26:05.187 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:05 smithi064 ceph-mon[30952]: from='osd.11 [v2:172.21.15.195:6824/591599352,v1:172.21.15.195:6825/591599352]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-09-12T03:26:05.187 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:05 smithi064 ceph-mon[30952]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-09-12T03:26:05.187 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:05 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:05.187 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:05 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:05.187 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:05 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:26:05.187 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:05 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:26:05.187 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:05 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:26:05.187 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:05 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:26:05.187 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:05 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:05.187 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:05 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:05.187 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:05 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:26:05.188 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:05 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:05.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:05 smithi115 ceph-mon[34801]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:26:05.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:05 smithi115 ceph-mon[34801]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T03:26:05.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:05 smithi115 ceph-mon[34801]: osdmap e74: 12 total, 11 up, 12 in 2024-09-12T03:26:05.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:05 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:26:05.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:05 smithi115 ceph-mon[34801]: from='osd.11 [v2:172.21.15.195:6824/591599352,v1:172.21.15.195:6825/591599352]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-09-12T03:26:05.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:05 smithi115 ceph-mon[34801]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-09-12T03:26:05.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:05 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:05.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:05 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:05.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:05 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:26:05.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:05 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:26:05.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:05 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:26:05.394 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:05 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:26:05.394 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:05 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:05.394 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:05 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:05.394 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:05 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:26:05.394 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:05 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:05.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.395+0000 7ff757b28640 1 -- 172.21.15.64:0/2079341674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff750100ad0 msgr2=0x7ff750100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:05.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.395+0000 7ff757b28640 1 --2- 172.21.15.64:0/2079341674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff750100ad0 0x7ff750100ed0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff74c009ca0 tx=0x7ff74c02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:26:05.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.395+0000 7ff757b28640 1 -- 172.21.15.64:0/2079341674 shutdown_connections 2024-09-12T03:26:05.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.395+0000 7ff757b28640 1 --2- 172.21.15.64:0/2079341674 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff750102690 0x7ff750108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:05.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.395+0000 7ff757b28640 1 --2- 172.21.15.64:0/2079341674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff750101cd0 0x7ff750102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:05.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.395+0000 7ff757b28640 1 --2- 172.21.15.64:0/2079341674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff750100ad0 0x7ff750100ed0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:05.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.395+0000 7ff757b28640 1 -- 172.21.15.64:0/2079341674 >> 172.21.15.64:0/2079341674 conn(0x7ff7500fc280 msgr2=0x7ff7500fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:05.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.395+0000 7ff757b28640 1 -- 172.21.15.64:0/2079341674 shutdown_connections 2024-09-12T03:26:05.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.395+0000 7ff757b28640 1 -- 172.21.15.64:0/2079341674 wait complete. 2024-09-12T03:26:05.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.396+0000 7ff757b28640 1 Processor -- start 2024-09-12T03:26:05.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.396+0000 7ff757b28640 1 -- start start 2024-09-12T03:26:05.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.396+0000 7ff757b28640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff750100ad0 0x7ff75006aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:05.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.396+0000 7ff757b28640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff750101cd0 0x7ff75019cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:05.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.396+0000 7ff757b28640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff750102690 0x7ff75019f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:05.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.396+0000 7ff757b28640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff75010c610 con 0x7ff750100ad0 2024-09-12T03:26:05.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.396+0000 7ff757b28640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff75010c490 con 0x7ff750102690 2024-09-12T03:26:05.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.396+0000 7ff757b28640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff75010c790 con 0x7ff750101cd0 2024-09-12T03:26:05.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.396+0000 7ff7558c4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff750100ad0 0x7ff75006aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:05.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.396+0000 7ff7558c4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff750100ad0 0x7ff75006aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41936/0 (socket says 172.21.15.64:41936) 2024-09-12T03:26:05.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.396+0000 7ff7558c4640 1 -- 172.21.15.64:0/2056978463 learned_addr learned my addr 172.21.15.64:0/2056978463 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:26:05.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.396+0000 7ff7550c3640 1 --2- 172.21.15.64:0/2056978463 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff750101cd0 0x7ff75019cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:05.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.396+0000 7ff7560c5640 1 --2- 172.21.15.64:0/2056978463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff750102690 0x7ff75019f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:05.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.396+0000 7ff7558c4640 1 -- 172.21.15.64:0/2056978463 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff750101cd0 msgr2=0x7ff75019cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:05.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.396+0000 7ff7558c4640 1 --2- 172.21.15.64:0/2056978463 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff750101cd0 0x7ff75019cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:05.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.397+0000 7ff7558c4640 1 -- 172.21.15.64:0/2056978463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff750102690 msgr2=0x7ff75019f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:05.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.397+0000 7ff7558c4640 1 --2- 172.21.15.64:0/2056978463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff750102690 0x7ff75019f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:05.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.397+0000 7ff7558c4640 1 -- 172.21.15.64:0/2056978463 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff75019fa50 con 0x7ff750100ad0 2024-09-12T03:26:05.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.397+0000 7ff7550c3640 1 --2- 172.21.15.64:0/2056978463 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff750101cd0 0x7ff75019cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:26:05.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.397+0000 7ff7558c4640 1 --2- 172.21.15.64:0/2056978463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff750100ad0 0x7ff75006aad0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff74c003ba0 tx=0x7ff74c003bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:05.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.397+0000 7ff742ffd640 1 -- 172.21.15.64:0/2056978463 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c004040 con 0x7ff750100ad0 2024-09-12T03:26:05.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.397+0000 7ff742ffd640 1 -- 172.21.15.64:0/2056978463 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff74c004690 con 0x7ff750100ad0 2024-09-12T03:26:05.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.397+0000 7ff757b28640 1 -- 172.21.15.64:0/2056978463 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff75019fc80 con 0x7ff750100ad0 2024-09-12T03:26:05.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.397+0000 7ff742ffd640 1 -- 172.21.15.64:0/2056978463 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c030cd0 con 0x7ff750100ad0 2024-09-12T03:26:05.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.397+0000 7ff757b28640 1 -- 172.21.15.64:0/2056978463 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7501a01c0 con 0x7ff750100ad0 2024-09-12T03:26:05.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.398+0000 7ff757b28640 1 -- 172.21.15.64:0/2056978463 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7501a03f0 con 0x7ff750100ad0 2024-09-12T03:26:05.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.399+0000 7ff742ffd640 1 -- 172.21.15.64:0/2056978463 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff74c0041e0 con 0x7ff750100ad0 2024-09-12T03:26:05.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.400+0000 7ff742ffd640 1 --2- 172.21.15.64:0/2056978463 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff73c0a2c00 0x7ff73c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:05.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.400+0000 7ff7550c3640 1 --2- 172.21.15.64:0/2056978463 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff73c0a2c00 0x7ff73c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:05.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.400+0000 7ff742ffd640 1 -- 172.21.15.64:0/2056978463 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6847+0+0 (secure 0 0 0) 0x7ff74c0f01e0 con 0x7ff750100ad0 2024-09-12T03:26:05.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.400+0000 7ff7550c3640 1 --2- 172.21.15.64:0/2056978463 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff73c0a2c00 0x7ff73c0a50c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff75006b050 tx=0x7ff744005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:05.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.403+0000 7ff742ffd640 1 -- 172.21.15.64:0/2056978463 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff74c0bd790 con 0x7ff750100ad0 2024-09-12T03:26:05.456 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:26:05 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:26:05.120+0000 7f97cfaf2640 -1 osd.11 0 waiting for initial osdmap 2024-09-12T03:26:05.456 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:26:05 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:26:05.133+0000 7f97cb91a640 -1 osd.11 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:26:05.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:05 smithi195 ceph-mon[34786]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:26:05.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:05 smithi195 ceph-mon[34786]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T03:26:05.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:05 smithi195 ceph-mon[34786]: osdmap e74: 12 total, 11 up, 12 in 2024-09-12T03:26:05.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:05 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:26:05.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:05 smithi195 ceph-mon[34786]: from='osd.11 [v2:172.21.15.195:6824/591599352,v1:172.21.15.195:6825/591599352]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-09-12T03:26:05.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:05 smithi195 ceph-mon[34786]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-09-12T03:26:05.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:05 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:05.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:05 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:05.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:05 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:26:05.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:05 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:26:05.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:05 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:26:05.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:05 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:26:05.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:05 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:05.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:05 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:05.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:05 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:26:05.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:05 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:05.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.580+0000 7ff757b28640 1 -- 172.21.15.64:0/2056978463 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7ff750107960 con 0x7ff750100ad0 2024-09-12T03:26:05.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.581+0000 7ff742ffd640 1 -- 172.21.15.64:0/2056978463 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) ==== 74+0+133 (secure 0 0 0) 0x7ff74c0bd090 con 0x7ff750100ad0 2024-09-12T03:26:05.581 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:26:05.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.583+0000 7ff757b28640 1 -- 172.21.15.64:0/2056978463 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff73c0a2c00 msgr2=0x7ff73c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:05.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.583+0000 7ff757b28640 1 --2- 172.21.15.64:0/2056978463 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff73c0a2c00 0x7ff73c0a50c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff75006b050 tx=0x7ff744005f90 comp rx=0 tx=0).stop 2024-09-12T03:26:05.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.583+0000 7ff757b28640 1 -- 172.21.15.64:0/2056978463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff750100ad0 msgr2=0x7ff75006aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:05.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.583+0000 7ff757b28640 1 --2- 172.21.15.64:0/2056978463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff750100ad0 0x7ff75006aad0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff74c003ba0 tx=0x7ff74c003bd0 comp rx=0 tx=0).stop 2024-09-12T03:26:05.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.583+0000 7ff757b28640 1 -- 172.21.15.64:0/2056978463 shutdown_connections 2024-09-12T03:26:05.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.583+0000 7ff757b28640 1 --2- 172.21.15.64:0/2056978463 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff73c0a2c00 0x7ff73c0a50c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:05.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.583+0000 7ff757b28640 1 --2- 172.21.15.64:0/2056978463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff750102690 0x7ff75019f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:05.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.583+0000 7ff757b28640 1 --2- 172.21.15.64:0/2056978463 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff750101cd0 0x7ff75019cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:05.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.583+0000 7ff757b28640 1 --2- 172.21.15.64:0/2056978463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff750100ad0 0x7ff75006aad0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:05.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.583+0000 7ff757b28640 1 -- 172.21.15.64:0/2056978463 >> 172.21.15.64:0/2056978463 conn(0x7ff7500fc280 msgr2=0x7ff7500fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:05.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.583+0000 7ff757b28640 1 -- 172.21.15.64:0/2056978463 shutdown_connections 2024-09-12T03:26:05.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:05.583+0000 7ff757b28640 1 -- 172.21.15.64:0/2056978463 wait complete. 2024-09-12T03:26:06.196 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":75,"num_osds":12,"num_up_osds":11,"osd_up_since":1726111540,"num_in_osds":12,"osd_in_since":1726111543,"num_remapped_pgs":0} 2024-09-12T03:26:06.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:06 smithi115 ceph-mon[34801]: Detected new or changed devices on smithi195 2024-09-12T03:26:06.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:06 smithi115 ceph-mon[34801]: Adjusting osd_memory_target on smithi195 to 4272M 2024-09-12T03:26:06.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:06 smithi115 ceph-mon[34801]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-09-12T03:26:06.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:06 smithi115 ceph-mon[34801]: osdmap e75: 12 total, 11 up, 12 in 2024-09-12T03:26:06.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:06 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:26:06.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:06 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:26:06.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:06 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/2056978463' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:26:06.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:06 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:26:06.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:06 smithi195 ceph-mon[34786]: Detected new or changed devices on smithi195 2024-09-12T03:26:06.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:06 smithi195 ceph-mon[34786]: Adjusting osd_memory_target on smithi195 to 4272M 2024-09-12T03:26:06.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:06 smithi195 ceph-mon[34786]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-09-12T03:26:06.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:06 smithi195 ceph-mon[34786]: osdmap e75: 12 total, 11 up, 12 in 2024-09-12T03:26:06.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:06 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:26:06.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:06 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:26:06.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:06 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/2056978463' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:26:06.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:06 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:26:06.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:06 smithi064 ceph-mon[30952]: Detected new or changed devices on smithi195 2024-09-12T03:26:06.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:06 smithi064 ceph-mon[30952]: Adjusting osd_memory_target on smithi195 to 4272M 2024-09-12T03:26:06.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:06 smithi064 ceph-mon[30952]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-09-12T03:26:06.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:06 smithi064 ceph-mon[30952]: osdmap e75: 12 total, 11 up, 12 in 2024-09-12T03:26:06.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:06 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:26:06.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:06 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:26:06.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:06 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/2056978463' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:26:06.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:06 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:26:07.197 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-12T03:26:07.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:07 smithi115 ceph-mon[34801]: purged_snaps scrub starts 2024-09-12T03:26:07.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:07 smithi115 ceph-mon[34801]: purged_snaps scrub ok 2024-09-12T03:26:07.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:07 smithi115 ceph-mon[34801]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:26:07.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:07 smithi115 ceph-mon[34801]: osd.11 [v2:172.21.15.195:6824/591599352,v1:172.21.15.195:6825/591599352] boot 2024-09-12T03:26:07.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:07 smithi115 ceph-mon[34801]: osdmap e76: 12 total, 12 up, 12 in 2024-09-12T03:26:07.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:07 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:26:07.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:07 smithi195 ceph-mon[34786]: purged_snaps scrub starts 2024-09-12T03:26:07.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:07 smithi195 ceph-mon[34786]: purged_snaps scrub ok 2024-09-12T03:26:07.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:07 smithi195 ceph-mon[34786]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:26:07.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:07 smithi195 ceph-mon[34786]: osd.11 [v2:172.21.15.195:6824/591599352,v1:172.21.15.195:6825/591599352] boot 2024-09-12T03:26:07.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:07 smithi195 ceph-mon[34786]: osdmap e76: 12 total, 12 up, 12 in 2024-09-12T03:26:07.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:07 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:26:07.467 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:07.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:07 smithi064 ceph-mon[30952]: purged_snaps scrub starts 2024-09-12T03:26:07.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:07 smithi064 ceph-mon[30952]: purged_snaps scrub ok 2024-09-12T03:26:07.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:07 smithi064 ceph-mon[30952]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:26:07.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:07 smithi064 ceph-mon[30952]: osd.11 [v2:172.21.15.195:6824/591599352,v1:172.21.15.195:6825/591599352] boot 2024-09-12T03:26:07.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:07 smithi064 ceph-mon[30952]: osdmap e76: 12 total, 12 up, 12 in 2024-09-12T03:26:07.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:07 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:26:07.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:07.944+0000 7f56dda0f640 1 -- 172.21.15.64:0/480040268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56d80ffed0 msgr2=0x7f56d81002d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:07.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:07.944+0000 7f56dda0f640 1 --2- 172.21.15.64:0/480040268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56d80ffed0 0x7f56d81002d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f56c8009930 tx=0x7f56c802a420 comp rx=0 tx=0).stop 2024-09-12T03:26:07.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:07.944+0000 7f56dda0f640 1 -- 172.21.15.64:0/480040268 shutdown_connections 2024-09-12T03:26:07.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:07.944+0000 7f56dda0f640 1 --2- 172.21.15.64:0/480040268 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56d8103870 0x7f56d810a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:07.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:07.944+0000 7f56dda0f640 1 --2- 172.21.15.64:0/480040268 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56d8102eb0 0x7f56d8103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:07.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:07.944+0000 7f56dda0f640 1 --2- 172.21.15.64:0/480040268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56d80ffed0 0x7f56d81002d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:07.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:07.944+0000 7f56dda0f640 1 -- 172.21.15.64:0/480040268 >> 172.21.15.64:0/480040268 conn(0x7f56d80fba40 msgr2=0x7f56d80fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:07.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:07.944+0000 7f56dda0f640 1 -- 172.21.15.64:0/480040268 shutdown_connections 2024-09-12T03:26:07.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:07.944+0000 7f56dda0f640 1 -- 172.21.15.64:0/480040268 wait complete. 2024-09-12T03:26:07.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:07.945+0000 7f56dda0f640 1 Processor -- start 2024-09-12T03:26:07.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:07.945+0000 7f56dda0f640 1 -- start start 2024-09-12T03:26:07.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:07.945+0000 7f56dda0f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56d80ffed0 0x7f56d8197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:07.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:07.945+0000 7f56dda0f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56d8102eb0 0x7f56d81983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:07.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:07.945+0000 7f56dda0f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56d8103870 0x7f56d819f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:07.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:07.945+0000 7f56dda0f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f56d810d740 con 0x7f56d8102eb0 2024-09-12T03:26:07.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:07.945+0000 7f56dda0f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f56d810d5c0 con 0x7f56d80ffed0 2024-09-12T03:26:07.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:07.945+0000 7f56dda0f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f56d810d8c0 con 0x7f56d8103870 2024-09-12T03:26:07.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:07.945+0000 7f56d67fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56d8102eb0 0x7f56d81983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:07.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:07.945+0000 7f56d67fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56d8102eb0 0x7f56d81983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53912/0 (socket says 172.21.15.64:53912) 2024-09-12T03:26:07.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:07.945+0000 7f56d67fc640 1 -- 172.21.15.64:0/1697060959 learned_addr learned my addr 172.21.15.64:0/1697060959 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:26:07.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:07.945+0000 7f56d6ffd640 1 --2- 172.21.15.64:0/1697060959 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56d80ffed0 0x7f56d8197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:07.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:07.945+0000 7f56d77fe640 1 --2- 172.21.15.64:0/1697060959 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56d8103870 0x7f56d819f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:07.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:07.945+0000 7f56d67fc640 1 -- 172.21.15.64:0/1697060959 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56d8103870 msgr2=0x7f56d819f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:07.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:07.945+0000 7f56d67fc640 1 --2- 172.21.15.64:0/1697060959 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56d8103870 0x7f56d819f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:07.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:07.945+0000 7f56d67fc640 1 -- 172.21.15.64:0/1697060959 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56d80ffed0 msgr2=0x7f56d8197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:07.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:07.945+0000 7f56d67fc640 1 --2- 172.21.15.64:0/1697060959 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56d80ffed0 0x7f56d8197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:07.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:07.945+0000 7f56d67fc640 1 -- 172.21.15.64:0/1697060959 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d819fab0 con 0x7f56d8102eb0 2024-09-12T03:26:07.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:07.945+0000 7f56d6ffd640 1 --2- 172.21.15.64:0/1697060959 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56d80ffed0 0x7f56d8197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:26:07.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:07.946+0000 7f56d67fc640 1 --2- 172.21.15.64:0/1697060959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56d8102eb0 0x7f56d81983c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f56cc00d950 tx=0x7f56cc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:07.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:07.946+0000 7f56dca0d640 1 -- 172.21.15.64:0/1697060959 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56cc017070 con 0x7f56d8102eb0 2024-09-12T03:26:07.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:07.946+0000 7f56dca0d640 1 -- 172.21.15.64:0/1697060959 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56cc009c30 con 0x7f56d8102eb0 2024-09-12T03:26:07.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:07.946+0000 7f56dda0f640 1 -- 172.21.15.64:0/1697060959 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56d819fda0 con 0x7f56d8102eb0 2024-09-12T03:26:07.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:07.946+0000 7f56dca0d640 1 -- 172.21.15.64:0/1697060959 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56cc00f620 con 0x7f56d8102eb0 2024-09-12T03:26:07.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:07.946+0000 7f56dda0f640 1 -- 172.21.15.64:0/1697060959 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56d81a0260 con 0x7f56d8102eb0 2024-09-12T03:26:07.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:07.947+0000 7f56dda0f640 1 -- 172.21.15.64:0/1697060959 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56d81091f0 con 0x7f56d8102eb0 2024-09-12T03:26:07.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:07.948+0000 7f56dca0d640 1 -- 172.21.15.64:0/1697060959 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f56cc0109a0 con 0x7f56d8102eb0 2024-09-12T03:26:07.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:07.949+0000 7f56dca0d640 1 --2- 172.21.15.64:0/1697060959 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f56c00a2c30 0x7f56c00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:07.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:07.949+0000 7f56d6ffd640 1 --2- 172.21.15.64:0/1697060959 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f56c00a2c30 0x7f56c00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:07.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:07.949+0000 7f56dca0d640 1 -- 172.21.15.64:0/1697060959 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f56cc0ce7a0 con 0x7f56d8102eb0 2024-09-12T03:26:07.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:07.949+0000 7f56d6ffd640 1 --2- 172.21.15.64:0/1697060959 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f56c00a2c30 0x7f56c00a50f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f56c8009900 tx=0x7f56c8005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:07.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:07.953+0000 7f56dca0d640 1 -- 172.21.15.64:0/1697060959 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f56cc09bba0 con 0x7f56d8102eb0 2024-09-12T03:26:08.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:08.128+0000 7f56dda0f640 1 -- 172.21.15.64:0/1697060959 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f56d8062e90 con 0x7f56d8102eb0 2024-09-12T03:26:08.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:08.128+0000 7f56dca0d640 1 -- 172.21.15.64:0/1697060959 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v77) ==== 74+0+133 (secure 0 0 0) 0x7f56cc09b4a0 con 0x7f56d8102eb0 2024-09-12T03:26:08.128 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:26:08.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:08.131+0000 7f56dda0f640 1 -- 172.21.15.64:0/1697060959 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f56c00a2c30 msgr2=0x7f56c00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:08.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:08.131+0000 7f56dda0f640 1 --2- 172.21.15.64:0/1697060959 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f56c00a2c30 0x7f56c00a50f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f56c8009900 tx=0x7f56c8005fd0 comp rx=0 tx=0).stop 2024-09-12T03:26:08.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:08.131+0000 7f56dda0f640 1 -- 172.21.15.64:0/1697060959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56d8102eb0 msgr2=0x7f56d81983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:08.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:08.131+0000 7f56dda0f640 1 --2- 172.21.15.64:0/1697060959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56d8102eb0 0x7f56d81983c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f56cc00d950 tx=0x7f56cc00de10 comp rx=0 tx=0).stop 2024-09-12T03:26:08.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:08.131+0000 7f56dda0f640 1 -- 172.21.15.64:0/1697060959 shutdown_connections 2024-09-12T03:26:08.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:08.131+0000 7f56dda0f640 1 --2- 172.21.15.64:0/1697060959 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f56c00a2c30 0x7f56c00a50f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:08.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:08.131+0000 7f56dda0f640 1 --2- 172.21.15.64:0/1697060959 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56d8103870 0x7f56d819f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:08.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:08.131+0000 7f56dda0f640 1 --2- 172.21.15.64:0/1697060959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56d8102eb0 0x7f56d81983c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:08.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:08.131+0000 7f56dda0f640 1 --2- 172.21.15.64:0/1697060959 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56d80ffed0 0x7f56d8197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:08.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:08.131+0000 7f56dda0f640 1 -- 172.21.15.64:0/1697060959 >> 172.21.15.64:0/1697060959 conn(0x7f56d80fba40 msgr2=0x7f56d81063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:08.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:08.131+0000 7f56dda0f640 1 -- 172.21.15.64:0/1697060959 shutdown_connections 2024-09-12T03:26:08.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:08.131+0000 7f56dda0f640 1 -- 172.21.15.64:0/1697060959 wait complete. 2024-09-12T03:26:08.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:08 smithi195 ceph-mon[34786]: osdmap e77: 12 total, 12 up, 12 in 2024-09-12T03:26:08.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:08 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/1697060959' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:26:08.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:08 smithi064 ceph-mon[30952]: osdmap e77: 12 total, 12 up, 12 in 2024-09-12T03:26:08.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:08 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/1697060959' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:26:08.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:08 smithi115 ceph-mon[34801]: osdmap e77: 12 total, 12 up, 12 in 2024-09-12T03:26:08.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:08 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/1697060959' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:26:08.898 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":77,"num_osds":12,"num_up_osds":12,"osd_up_since":1726111566,"num_in_osds":12,"osd_in_since":1726111543,"num_remapped_pgs":0} 2024-09-12T03:26:08.899 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:26:09.158 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:09.411 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:09 smithi064 ceph-mon[30952]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:09.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:09 smithi195 ceph-mon[34786]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:09.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.593+0000 7f4955730640 1 -- 172.21.15.64:0/2035983418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4950102d30 msgr2=0x7f4950103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:09.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.593+0000 7f4955730640 1 --2- 172.21.15.64:0/2035983418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4950102d30 0x7f4950103110 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4940009cd0 tx=0x7f494002a8e0 comp rx=0 tx=0).stop 2024-09-12T03:26:09.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.593+0000 7f4955730640 1 -- 172.21.15.64:0/2035983418 shutdown_connections 2024-09-12T03:26:09.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.593+0000 7f4955730640 1 --2- 172.21.15.64:0/2035983418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49501082a0 0x7f495010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:09.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.593+0000 7f4955730640 1 --2- 172.21.15.64:0/2035983418 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49501036e0 0x7f4950107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:09.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.593+0000 7f4955730640 1 --2- 172.21.15.64:0/2035983418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4950102d30 0x7f4950103110 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:09.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.593+0000 7f4955730640 1 -- 172.21.15.64:0/2035983418 >> 172.21.15.64:0/2035983418 conn(0x7f49500fc7d0 msgr2=0x7f49500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:09.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.593+0000 7f4955730640 1 -- 172.21.15.64:0/2035983418 shutdown_connections 2024-09-12T03:26:09.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.593+0000 7f4955730640 1 -- 172.21.15.64:0/2035983418 wait complete. 2024-09-12T03:26:09.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.594+0000 7f4955730640 1 Processor -- start 2024-09-12T03:26:09.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.594+0000 7f4955730640 1 -- start start 2024-09-12T03:26:09.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.594+0000 7f4955730640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4950102d30 0x7f4950198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:09.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.594+0000 7f4955730640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49501036e0 0x7f4950198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:09.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.594+0000 7f4955730640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49501082a0 0x7f495019cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:09.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.594+0000 7f4955730640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4950111f50 con 0x7f49501082a0 2024-09-12T03:26:09.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.594+0000 7f4955730640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4950111dd0 con 0x7f4950102d30 2024-09-12T03:26:09.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.594+0000 7f4955730640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f49501120d0 con 0x7f49501036e0 2024-09-12T03:26:09.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.594+0000 7f494f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49501082a0 0x7f495019cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:09.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.595+0000 7f494f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49501082a0 0x7f495019cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53942/0 (socket says 172.21.15.64:53942) 2024-09-12T03:26:09.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.595+0000 7f494f7fe640 1 -- 172.21.15.64:0/4264004436 learned_addr learned my addr 172.21.15.64:0/4264004436 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:26:09.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.595+0000 7f494effd640 1 --2- 172.21.15.64:0/4264004436 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4950102d30 0x7f4950198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:09.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.595+0000 7f494e7fc640 1 --2- 172.21.15.64:0/4264004436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49501036e0 0x7f4950198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:09.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.595+0000 7f494f7fe640 1 -- 172.21.15.64:0/4264004436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49501036e0 msgr2=0x7f4950198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:09.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.595+0000 7f494f7fe640 1 --2- 172.21.15.64:0/4264004436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49501036e0 0x7f4950198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:09.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.595+0000 7f494f7fe640 1 -- 172.21.15.64:0/4264004436 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4950102d30 msgr2=0x7f4950198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:09.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.595+0000 7f494f7fe640 1 --2- 172.21.15.64:0/4264004436 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4950102d30 0x7f4950198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:09.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.595+0000 7f494f7fe640 1 -- 172.21.15.64:0/4264004436 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495019d4c0 con 0x7f49501082a0 2024-09-12T03:26:09.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.595+0000 7f494effd640 1 --2- 172.21.15.64:0/4264004436 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4950102d30 0x7f4950198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:26:09.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.595+0000 7f494f7fe640 1 --2- 172.21.15.64:0/4264004436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49501082a0 0x7f495019cde0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f494800d750 tx=0x7f494800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:09.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.595+0000 7f4947fff640 1 -- 172.21.15.64:0/4264004436 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4948017070 con 0x7f49501082a0 2024-09-12T03:26:09.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.595+0000 7f4947fff640 1 -- 172.21.15.64:0/4264004436 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49480054d0 con 0x7f49501082a0 2024-09-12T03:26:09.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.595+0000 7f4955730640 1 -- 172.21.15.64:0/4264004436 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f495019d7b0 con 0x7f49501082a0 2024-09-12T03:26:09.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.595+0000 7f4947fff640 1 -- 172.21.15.64:0/4264004436 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4948004c60 con 0x7f49501082a0 2024-09-12T03:26:09.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.595+0000 7f4955730640 1 -- 172.21.15.64:0/4264004436 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49501a5090 con 0x7f49501082a0 2024-09-12T03:26:09.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.596+0000 7f4955730640 1 -- 172.21.15.64:0/4264004436 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f495006a850 con 0x7f49501082a0 2024-09-12T03:26:09.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.597+0000 7f4947fff640 1 -- 172.21.15.64:0/4264004436 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4948004e00 con 0x7f49501082a0 2024-09-12T03:26:09.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.598+0000 7f4947fff640 1 --2- 172.21.15.64:0/4264004436 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f493c0a2b80 0x7f493c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:09.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.598+0000 7f494effd640 1 --2- 172.21.15.64:0/4264004436 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f493c0a2b80 0x7f493c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:09.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.598+0000 7f4947fff640 1 -- 172.21.15.64:0/4264004436 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f49480ce590 con 0x7f49501082a0 2024-09-12T03:26:09.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.598+0000 7f494effd640 1 --2- 172.21.15.64:0/4264004436 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f493c0a2b80 0x7f493c0a5040 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4940006010 tx=0x7f49400093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:09.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.602+0000 7f4947fff640 1 -- 172.21.15.64:0/4264004436 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f494809b990 con 0x7f49501082a0 2024-09-12T03:26:09.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:09 smithi115 ceph-mon[34801]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:09.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.779+0000 7f4955730640 1 -- 172.21.15.64:0/4264004436 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f495010a2b0 con 0x7f49501082a0 2024-09-12T03:26:09.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.780+0000 7f4947fff640 1 -- 172.21.15.64:0/4264004436 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+18465 (secure 0 0 0) 0x7f494809b290 con 0x7f49501082a0 2024-09-12T03:26:09.781 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:26:09.781 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":77,"fsid":"b0812460-70b5-11ef-bcea-c7b262605968","created":"2024-09-12T03:19:15.900322+0000","modified":"2024-09-12T03:26:07.133658+0000","last_up_change":"2024-09-12T03:26:06.122971+0000","last_in_change":"2024-09-12T03:25:43.130689+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:22:21.915870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"326271c9-1f28-4ce6-b3be-fcc57479d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6803","nonce":1307832590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6805","nonce":1307832590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6809","nonce":1307832590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6807","nonce":1307832590}]},"public_addr":"172.21.15.64:6803/1307832590","cluster_addr":"172.21.15.64:6805/1307832590","heartbeat_back_addr":"172.21.15.64:6809/1307832590","heartbeat_front_addr":"172.21.15.64:6807/1307832590","state":["exists","up"]},{"osd":1,"uuid":"7dca0c28-c234-423c-936e-2351d63f918e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6801","nonce":3123728622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6803","nonce":3123728622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6807","nonce":3123728622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6805","nonce":3123728622}]},"public_addr":"172.21.15.115:6801/3123728622","cluster_addr":"172.21.15.115:6803/3123728622","heartbeat_back_addr":"172.21.15.115:6807/3123728622","heartbeat_front_addr":"172.21.15.115:6805/3123728622","state":["exists","up"]},{"osd":2,"uuid":"4b3a3c7a-aec6-455a-9a9a-1ac9cc900f24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6801","nonce":1664125451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6803","nonce":1664125451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6807","nonce":1664125451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6805","nonce":1664125451}]},"public_addr":"172.21.15.195:6801/1664125451","cluster_addr":"172.21.15.195:6803/1664125451","heartbeat_back_addr":"172.21.15.195:6807/1664125451","heartbeat_front_addr":"172.21.15.195:6805/1664125451","state":["exists","up"]},{"osd":3,"uuid":"8768c339-2c1e-4294-8b78-3c2adab6750f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6811","nonce":3935517185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6813","nonce":3935517185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6817","nonce":3935517185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6815","nonce":3935517185}]},"public_addr":"172.21.15.64:6811/3935517185","cluster_addr":"172.21.15.64:6813/3935517185","heartbeat_back_addr":"172.21.15.64:6817/3935517185","heartbeat_front_addr":"172.21.15.64:6815/3935517185","state":["exists","up"]},{"osd":4,"uuid":"123aac47-a8ae-47c7-93a7-0dd293356322","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6809","nonce":446665227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6811","nonce":446665227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6815","nonce":446665227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6813","nonce":446665227}]},"public_addr":"172.21.15.115:6809/446665227","cluster_addr":"172.21.15.115:6811/446665227","heartbeat_back_addr":"172.21.15.115:6815/446665227","heartbeat_front_addr":"172.21.15.115:6813/446665227","state":["exists","up"]},{"osd":5,"uuid":"c90bcaed-e7a8-4852-8677-b01dc73ea44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6809","nonce":1187293668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6811","nonce":1187293668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6815","nonce":1187293668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6813","nonce":1187293668}]},"public_addr":"172.21.15.195:6809/1187293668","cluster_addr":"172.21.15.195:6811/1187293668","heartbeat_back_addr":"172.21.15.195:6815/1187293668","heartbeat_front_addr":"172.21.15.195:6813/1187293668","state":["exists","up"]},{"osd":6,"uuid":"fd292547-6695-4889-8867-e74318bbb00f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6819","nonce":560343274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6821","nonce":560343274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6825","nonce":560343274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6823","nonce":560343274}]},"public_addr":"172.21.15.64:6819/560343274","cluster_addr":"172.21.15.64:6821/560343274","heartbeat_back_addr":"172.21.15.64:6825/560343274","heartbeat_front_addr":"172.21.15.64:6823/560343274","state":["exists","up"]},{"osd":7,"uuid":"ad7c4151-cd98-4113-879b-ca2b7c83c7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6817","nonce":3075734174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6819","nonce":3075734174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6823","nonce":3075734174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6821","nonce":3075734174}]},"public_addr":"172.21.15.115:6817/3075734174","cluster_addr":"172.21.15.115:6819/3075734174","heartbeat_back_addr":"172.21.15.115:6823/3075734174","heartbeat_front_addr":"172.21.15.115:6821/3075734174","state":["exists","up"]},{"osd":8,"uuid":"6de50049-c28f-4f21-b123-05e55bf59127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6817","nonce":2184742494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6819","nonce":2184742494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6823","nonce":2184742494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6821","nonce":2184742494}]},"public_addr":"172.21.15.195:6817/2184742494","cluster_addr":"172.21.15.195:6819/2184742494","heartbeat_back_addr":"172.21.15.195:6823/2184742494","heartbeat_front_addr":"172.21.15.195:6821/2184742494","state":["exists","up"]},{"osd":9,"uuid":"36ab792c-58f2-4beb-9024-7f3d1e3785dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6827","nonce":561728112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6829","nonce":561728112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6833","nonce":561728112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6831","nonce":561728112}]},"public_addr":"172.21.15.64:6827/561728112","cluster_addr":"172.21.15.64:6829/561728112","heartbeat_back_addr":"172.21.15.64:6833/561728112","heartbeat_front_addr":"172.21.15.64:6831/561728112","state":["exists","up"]},{"osd":10,"uuid":"d1f7881c-ec3f-466e-9cea-1d5cb441d970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6825","nonce":2977887627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6827","nonce":2977887627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6831","nonce":2977887627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6829","nonce":2977887627}]},"public_addr":"172.21.15.115:6825/2977887627","cluster_addr":"172.21.15.115:6827/2977887627","heartbeat_back_addr":"172.21.15.115:6831/2977887627","heartbeat_front_addr":"172.21.15.115:6829/2977887627","state":["exists","up"]},{"osd":11,"uuid":"7c46c028-22c7-450f-9927-2144a7a09e9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6825","nonce":591599352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6827","nonce":591599352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6831","nonce":591599352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6829","nonce":591599352}]},"public_addr":"172.21.15.195:6825/591599352","cluster_addr":"172.21.15.195:6827/591599352","heartbeat_back_addr":"172.21.15.195:6831/591599352","heartbeat_front_addr":"172.21.15.195:6829/591599352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:29.151454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:54.308909+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:18.680995+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:44.687548+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:10.166131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:35.026132+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:24:00.242966+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:24:24.600848+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:24:49.563060+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:25:15.025730+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:25:39.830995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:26:04.394276+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6801/68603379":"2024-09-13T03:20:07.411102+0000","172.21.15.64:0/3827202979":"2024-09-13T03:20:07.411102+0000","172.21.15.64:6800/68603379":"2024-09-13T03:20:07.411102+0000","172.21.15.64:0/1217136221":"2024-09-13T03:20:07.411102+0000","172.21.15.64:0/3079679964":"2024-09-13T03:19:36.839431+0000","172.21.15.64:6801/3186640984":"2024-09-13T03:19:36.839431+0000","172.21.15.64:0/3334629354":"2024-09-13T03:20:07.411102+0000","172.21.15.64:6800/3186640984":"2024-09-13T03:19:36.839431+0000","172.21.15.64:0/3564418115":"2024-09-13T03:19:36.839431+0000","172.21.15.64:0/686137382":"2024-09-13T03:19:36.839431+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:26:09.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.783+0000 7f4955730640 1 -- 172.21.15.64:0/4264004436 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f493c0a2b80 msgr2=0x7f493c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:09.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.783+0000 7f4955730640 1 --2- 172.21.15.64:0/4264004436 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f493c0a2b80 0x7f493c0a5040 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4940006010 tx=0x7f49400093f0 comp rx=0 tx=0).stop 2024-09-12T03:26:09.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.783+0000 7f4955730640 1 -- 172.21.15.64:0/4264004436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49501082a0 msgr2=0x7f495019cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:09.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.783+0000 7f4955730640 1 --2- 172.21.15.64:0/4264004436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49501082a0 0x7f495019cde0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f494800d750 tx=0x7f494800dc10 comp rx=0 tx=0).stop 2024-09-12T03:26:09.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.783+0000 7f4955730640 1 -- 172.21.15.64:0/4264004436 shutdown_connections 2024-09-12T03:26:09.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.783+0000 7f4955730640 1 --2- 172.21.15.64:0/4264004436 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f493c0a2b80 0x7f493c0a5040 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:09.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.783+0000 7f4955730640 1 --2- 172.21.15.64:0/4264004436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49501082a0 0x7f495019cde0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:09.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.783+0000 7f4955730640 1 --2- 172.21.15.64:0/4264004436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49501036e0 0x7f4950198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:09.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.783+0000 7f4955730640 1 --2- 172.21.15.64:0/4264004436 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4950102d30 0x7f4950198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:09.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.783+0000 7f4955730640 1 -- 172.21.15.64:0/4264004436 >> 172.21.15.64:0/4264004436 conn(0x7f49500fc7d0 msgr2=0x7f4950105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:09.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.783+0000 7f4955730640 1 -- 172.21.15.64:0/4264004436 shutdown_connections 2024-09-12T03:26:09.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:09.783+0000 7f4955730640 1 -- 172.21.15.64:0/4264004436 wait complete. 2024-09-12T03:26:10.375 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:10 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/4264004436' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:26:10.407 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T03:22:21.915870+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T03:26:10.407 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-12T03:26:10.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:10 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/4264004436' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:26:10.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:10 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/4264004436' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:26:10.660 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:11.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.104+0000 7f9b0c7d2640 1 -- 172.21.15.64:0/2423084107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b04102570 msgr2=0x7f9b041029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:11.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.104+0000 7f9b0c7d2640 1 --2- 172.21.15.64:0/2423084107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b04102570 0x7f9b041029b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9af8009900 tx=0x7f9af802a480 comp rx=0 tx=0).stop 2024-09-12T03:26:11.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.104+0000 7f9b0c7d2640 1 -- 172.21.15.64:0/2423084107 shutdown_connections 2024-09-12T03:26:11.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.104+0000 7f9b0c7d2640 1 --2- 172.21.15.64:0/2423084107 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b04102ef0 0x7f9b0410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:11.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.104+0000 7f9b0c7d2640 1 --2- 172.21.15.64:0/2423084107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b04102570 0x7f9b041029b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:11.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.104+0000 7f9b0c7d2640 1 --2- 172.21.15.64:0/2423084107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b04100e20 0x7f9b04101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:11.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.104+0000 7f9b0c7d2640 1 -- 172.21.15.64:0/2423084107 >> 172.21.15.64:0/2423084107 conn(0x7f9b040fc7d0 msgr2=0x7f9b040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:11.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.104+0000 7f9b0c7d2640 1 -- 172.21.15.64:0/2423084107 shutdown_connections 2024-09-12T03:26:11.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.104+0000 7f9b0c7d2640 1 -- 172.21.15.64:0/2423084107 wait complete. 2024-09-12T03:26:11.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.105+0000 7f9b0c7d2640 1 Processor -- start 2024-09-12T03:26:11.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.105+0000 7f9b0c7d2640 1 -- start start 2024-09-12T03:26:11.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.105+0000 7f9b0c7d2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b04100e20 0x7f9b0410e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:11.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.105+0000 7f9b0c7d2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b04102570 0x7f9b0410ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:11.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.105+0000 7f9b0c7d2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b04102ef0 0x7f9b0410ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:11.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.105+0000 7f9b0c7d2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9b04111fa0 con 0x7f9b04102ef0 2024-09-12T03:26:11.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.105+0000 7f9b0c7d2640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9b04111e20 con 0x7f9b04102570 2024-09-12T03:26:11.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.105+0000 7f9b0c7d2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9b04112120 con 0x7f9b04100e20 2024-09-12T03:26:11.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.105+0000 7f9b0ad6f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b04102ef0 0x7f9b0410ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:11.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.106+0000 7f9b0ad6f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b04102ef0 0x7f9b0410ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53950/0 (socket says 172.21.15.64:53950) 2024-09-12T03:26:11.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.106+0000 7f9b0ad6f640 1 -- 172.21.15.64:0/3858866751 learned_addr learned my addr 172.21.15.64:0/3858866751 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:26:11.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.106+0000 7f9b09d6d640 1 --2- 172.21.15.64:0/3858866751 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b04102570 0x7f9b0410ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:11.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.106+0000 7f9b0a56e640 1 --2- 172.21.15.64:0/3858866751 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b04100e20 0x7f9b0410e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:11.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.106+0000 7f9b0ad6f640 1 -- 172.21.15.64:0/3858866751 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b04100e20 msgr2=0x7f9b0410e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:11.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.106+0000 7f9b0ad6f640 1 --2- 172.21.15.64:0/3858866751 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b04100e20 0x7f9b0410e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:11.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.106+0000 7f9b0ad6f640 1 -- 172.21.15.64:0/3858866751 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b04102570 msgr2=0x7f9b0410ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:11.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.106+0000 7f9b0ad6f640 1 --2- 172.21.15.64:0/3858866751 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b04102570 0x7f9b0410ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:11.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.106+0000 7f9b0ad6f640 1 -- 172.21.15.64:0/3858866751 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b041a68e0 con 0x7f9b04102ef0 2024-09-12T03:26:11.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.106+0000 7f9b09d6d640 1 --2- 172.21.15.64:0/3858866751 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b04102570 0x7f9b0410ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:26:11.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.106+0000 7f9b0ad6f640 1 --2- 172.21.15.64:0/3858866751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b04102ef0 0x7f9b0410ef50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9af400b970 tx=0x7f9af400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:11.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.106+0000 7f9b037fe640 1 -- 172.21.15.64:0/3858866751 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af4019070 con 0x7f9b04102ef0 2024-09-12T03:26:11.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.106+0000 7f9b037fe640 1 -- 172.21.15.64:0/3858866751 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9af4014440 con 0x7f9b04102ef0 2024-09-12T03:26:11.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.106+0000 7f9b0c7d2640 1 -- 172.21.15.64:0/3858866751 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b041a6bd0 con 0x7f9b04102ef0 2024-09-12T03:26:11.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.106+0000 7f9b0c7d2640 1 -- 172.21.15.64:0/3858866751 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b041a70e0 con 0x7f9b04102ef0 2024-09-12T03:26:11.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.106+0000 7f9b037fe640 1 -- 172.21.15.64:0/3858866751 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af40075d0 con 0x7f9b04102ef0 2024-09-12T03:26:11.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.108+0000 7f9b0c7d2640 1 -- 172.21.15.64:0/3858866751 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9af0005290 con 0x7f9b04102ef0 2024-09-12T03:26:11.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.109+0000 7f9b037fe640 1 -- 172.21.15.64:0/3858866751 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9af401c070 con 0x7f9b04102ef0 2024-09-12T03:26:11.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.109+0000 7f9b037fe640 1 --2- 172.21.15.64:0/3858866751 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f9aec0a2cd0 0x7f9aec0a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:11.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.110+0000 7f9b0a56e640 1 --2- 172.21.15.64:0/3858866751 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f9aec0a2cd0 0x7f9aec0a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:11.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.110+0000 7f9b037fe640 1 -- 172.21.15.64:0/3858866751 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f9af40ce4d0 con 0x7f9b04102ef0 2024-09-12T03:26:11.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.110+0000 7f9b0a56e640 1 --2- 172.21.15.64:0/3858866751 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f9aec0a2cd0 0x7f9aec0a5190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9afc005fd0 tx=0x7f9afc005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:11.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.113+0000 7f9b037fe640 1 -- 172.21.15.64:0/3858866751 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9af409b950 con 0x7f9b04102ef0 2024-09-12T03:26:11.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.287+0000 7f9b0c7d2640 1 -- 172.21.15.64:0/3858866751 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9af0005850 con 0x7f9b04102ef0 2024-09-12T03:26:11.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.288+0000 7f9b037fe640 1 -- 172.21.15.64:0/3858866751 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7f9af409b250 con 0x7f9b04102ef0 2024-09-12T03:26:11.288 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-09-12T03:26:11.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.290+0000 7f9b0c7d2640 1 -- 172.21.15.64:0/3858866751 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f9aec0a2cd0 msgr2=0x7f9aec0a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:11.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.290+0000 7f9b0c7d2640 1 --2- 172.21.15.64:0/3858866751 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f9aec0a2cd0 0x7f9aec0a5190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9afc005fd0 tx=0x7f9afc005850 comp rx=0 tx=0).stop 2024-09-12T03:26:11.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.290+0000 7f9b0c7d2640 1 -- 172.21.15.64:0/3858866751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b04102ef0 msgr2=0x7f9b0410ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:11.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.290+0000 7f9b0c7d2640 1 --2- 172.21.15.64:0/3858866751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b04102ef0 0x7f9b0410ef50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9af400b970 tx=0x7f9af400be30 comp rx=0 tx=0).stop 2024-09-12T03:26:11.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.290+0000 7f9b0c7d2640 1 -- 172.21.15.64:0/3858866751 shutdown_connections 2024-09-12T03:26:11.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.290+0000 7f9b0c7d2640 1 --2- 172.21.15.64:0/3858866751 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f9aec0a2cd0 0x7f9aec0a5190 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:11.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.290+0000 7f9b0c7d2640 1 --2- 172.21.15.64:0/3858866751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b04102ef0 0x7f9b0410ef50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:11.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.290+0000 7f9b0c7d2640 1 --2- 172.21.15.64:0/3858866751 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b04102570 0x7f9b0410ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:11.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.290+0000 7f9b0c7d2640 1 --2- 172.21.15.64:0/3858866751 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b04100e20 0x7f9b0410e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:11.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.290+0000 7f9b0c7d2640 1 -- 172.21.15.64:0/3858866751 >> 172.21.15.64:0/3858866751 conn(0x7f9b040fc7d0 msgr2=0x7f9b040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:11.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.290+0000 7f9b0c7d2640 1 -- 172.21.15.64:0/3858866751 shutdown_connections 2024-09-12T03:26:11.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:11.290+0000 7f9b0c7d2640 1 -- 172.21.15.64:0/3858866751 wait complete. 2024-09-12T03:26:11.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:11 smithi195 ceph-mon[34786]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:11.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:11 smithi064 ceph-mon[30952]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:11.645 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:11 smithi115 ceph-mon[34801]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:11.953 INFO:tasks.cephadm:Adding mds.a on smithi064 2024-09-12T03:26:11.954 INFO:tasks.cephadm:Adding mds.d on smithi064 2024-09-12T03:26:11.954 INFO:tasks.cephadm:Adding mds.g on smithi064 2024-09-12T03:26:11.954 INFO:tasks.cephadm:Adding mds.j on smithi064 2024-09-12T03:26:11.954 INFO:tasks.cephadm:Adding mds.b on smithi115 2024-09-12T03:26:11.954 INFO:tasks.cephadm:Adding mds.e on smithi115 2024-09-12T03:26:11.954 INFO:tasks.cephadm:Adding mds.h on smithi115 2024-09-12T03:26:11.954 INFO:tasks.cephadm:Adding mds.k on smithi115 2024-09-12T03:26:11.954 INFO:tasks.cephadm:Adding mds.c on smithi195 2024-09-12T03:26:11.954 INFO:tasks.cephadm:Adding mds.f on smithi195 2024-09-12T03:26:11.954 INFO:tasks.cephadm:Adding mds.i on smithi195 2024-09-12T03:26:11.954 INFO:tasks.cephadm:Adding mds.l on smithi195 2024-09-12T03:26:11.955 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph orch apply mds all '12;smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi115=b;smithi115=e;smithi115=h;smithi115=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l' 2024-09-12T03:26:12.215 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:26:12.335 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:12 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/3858866751' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:26:12.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:12 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/3858866751' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:26:12.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:12 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/3858866751' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:26:12.709 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.693+0000 7f44e4168640 1 -- 172.21.15.195:0/2993415398 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44dc1026a0 msgr2=0x7f44dc104a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:12.709 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.693+0000 7f44e4168640 1 --2- 172.21.15.195:0/2993415398 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44dc1026a0 0x7f44dc104a90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f44d8009900 tx=0x7f44d802a3f0 comp rx=0 tx=0).stop 2024-09-12T03:26:12.709 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.693+0000 7f44e4168640 1 -- 172.21.15.195:0/2993415398 shutdown_connections 2024-09-12T03:26:12.709 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.693+0000 7f44e4168640 1 --2- 172.21.15.195:0/2993415398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44dc107900 0x7f44dc109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:12.709 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.693+0000 7f44e4168640 1 --2- 172.21.15.195:0/2993415398 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44dc104fd0 0x7f44dc1073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:12.709 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.693+0000 7f44e4168640 1 --2- 172.21.15.195:0/2993415398 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44dc1026a0 0x7f44dc104a90 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:12.709 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.693+0000 7f44e4168640 1 -- 172.21.15.195:0/2993415398 >> 172.21.15.195:0/2993415398 conn(0x7f44dc0fc280 msgr2=0x7f44dc0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:12.709 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.693+0000 7f44e4168640 1 -- 172.21.15.195:0/2993415398 shutdown_connections 2024-09-12T03:26:12.709 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.693+0000 7f44e4168640 1 -- 172.21.15.195:0/2993415398 wait complete. 2024-09-12T03:26:12.709 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.694+0000 7f44e4168640 1 Processor -- start 2024-09-12T03:26:12.710 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.694+0000 7f44e4168640 1 -- start start 2024-09-12T03:26:12.710 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.694+0000 7f44e4168640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44dc1026a0 0x7f44dc197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:12.710 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.694+0000 7f44e4168640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44dc104fd0 0x7f44dc198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:12.710 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.694+0000 7f44e4168640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44dc107900 0x7f44dc19f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:12.710 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.694+0000 7f44e4168640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f44dc10d760 con 0x7f44dc1026a0 2024-09-12T03:26:12.710 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.694+0000 7f44e4168640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f44dc10d5e0 con 0x7f44dc104fd0 2024-09-12T03:26:12.710 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.694+0000 7f44e4168640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f44dc10d8e0 con 0x7f44dc107900 2024-09-12T03:26:12.710 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.694+0000 7f44e2705640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44dc107900 0x7f44dc19f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:12.710 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.694+0000 7f44e2705640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44dc107900 0x7f44dc19f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:56768/0 (socket says 172.21.15.195:56768) 2024-09-12T03:26:12.710 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.694+0000 7f44e2705640 1 -- 172.21.15.195:0/2360078259 learned_addr learned my addr 172.21.15.195:0/2360078259 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-09-12T03:26:12.710 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.695+0000 7f44e1f04640 1 --2- 172.21.15.195:0/2360078259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44dc1026a0 0x7f44dc197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:12.710 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.695+0000 7f44e1703640 1 --2- 172.21.15.195:0/2360078259 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44dc104fd0 0x7f44dc198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:12.710 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.695+0000 7f44e2705640 1 -- 172.21.15.195:0/2360078259 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44dc104fd0 msgr2=0x7f44dc198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:12.710 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.695+0000 7f44e2705640 1 --2- 172.21.15.195:0/2360078259 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44dc104fd0 0x7f44dc198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:12.711 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.695+0000 7f44e2705640 1 -- 172.21.15.195:0/2360078259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44dc1026a0 msgr2=0x7f44dc197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:12.711 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.695+0000 7f44e2705640 1 --2- 172.21.15.195:0/2360078259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44dc1026a0 0x7f44dc197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:12.711 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.695+0000 7f44e2705640 1 -- 172.21.15.195:0/2360078259 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44dc19f960 con 0x7f44dc107900 2024-09-12T03:26:12.711 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.695+0000 7f44e2705640 1 --2- 172.21.15.195:0/2360078259 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44dc107900 0x7f44dc19f2f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f44d400cce0 tx=0x7f44d400a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:12.711 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.695+0000 7f44d2ffd640 1 -- 172.21.15.195:0/2360078259 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d400f450 con 0x7f44dc107900 2024-09-12T03:26:12.711 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.695+0000 7f44e4168640 1 -- 172.21.15.195:0/2360078259 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44dc19fc50 con 0x7f44dc107900 2024-09-12T03:26:12.711 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.695+0000 7f44d2ffd640 1 -- 172.21.15.195:0/2360078259 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44d400faa0 con 0x7f44dc107900 2024-09-12T03:26:12.711 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.695+0000 7f44d2ffd640 1 -- 172.21.15.195:0/2360078259 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d4018b70 con 0x7f44dc107900 2024-09-12T03:26:12.711 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.695+0000 7f44e4168640 1 -- 172.21.15.195:0/2360078259 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44dc1a0190 con 0x7f44dc107900 2024-09-12T03:26:12.711 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.696+0000 7f44e4168640 1 -- 172.21.15.195:0/2360078259 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44dc0fdd70 con 0x7f44dc107900 2024-09-12T03:26:12.711 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.698+0000 7f44d2ffd640 1 -- 172.21.15.195:0/2360078259 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f44d4018d10 con 0x7f44dc107900 2024-09-12T03:26:12.711 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.699+0000 7f44d2ffd640 1 --2- 172.21.15.195:0/2360078259 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f44c80a2c30 0x7f44c80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:12.711 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.699+0000 7f44d2ffd640 1 -- 172.21.15.195:0/2360078259 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f44d40cf0c0 con 0x7f44dc107900 2024-09-12T03:26:12.712 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.699+0000 7f44e1f04640 1 --2- 172.21.15.195:0/2360078259 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f44c80a2c30 0x7f44c80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:12.712 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.700+0000 7f44e1f04640 1 --2- 172.21.15.195:0/2360078259 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f44c80a2c30 0x7f44c80a50f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f44d8000c00 tx=0x7f44d80314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:12.712 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.701+0000 7f44d2ffd640 1 -- 172.21.15.195:0/2360078259 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f44d4014030 con 0x7f44dc107900 2024-09-12T03:26:12.881 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.879+0000 7f44e4168640 1 -- 172.21.15.195:0/2360078259 --> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi115=b;smithi115=e;smithi115=h;smithi115=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l", "target": ["mon-mgr", ""]}) -- 0x7f44dc062730 con 0x7f44c80a2c30 2024-09-12T03:26:12.889 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.888+0000 7f44d2ffd640 1 -- 172.21.15.195:0/2360078259 <== mgr.14150 v2:172.21.15.64:6800/759931848 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f44dc062730 con 0x7f44c80a2c30 2024-09-12T03:26:12.889 INFO:teuthology.orchestra.run.smithi195.stdout:Scheduled mds.all update... 2024-09-12T03:26:12.891 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.890+0000 7f44e4168640 1 -- 172.21.15.195:0/2360078259 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f44c80a2c30 msgr2=0x7f44c80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:12.891 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.890+0000 7f44e4168640 1 --2- 172.21.15.195:0/2360078259 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f44c80a2c30 0x7f44c80a50f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f44d8000c00 tx=0x7f44d80314a0 comp rx=0 tx=0).stop 2024-09-12T03:26:12.891 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.890+0000 7f44e4168640 1 -- 172.21.15.195:0/2360078259 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44dc107900 msgr2=0x7f44dc19f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:12.891 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.890+0000 7f44e4168640 1 --2- 172.21.15.195:0/2360078259 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44dc107900 0x7f44dc19f2f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f44d400cce0 tx=0x7f44d400a620 comp rx=0 tx=0).stop 2024-09-12T03:26:12.891 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.890+0000 7f44e4168640 1 -- 172.21.15.195:0/2360078259 shutdown_connections 2024-09-12T03:26:12.891 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.890+0000 7f44e4168640 1 --2- 172.21.15.195:0/2360078259 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f44c80a2c30 0x7f44c80a50f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:12.892 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.890+0000 7f44e4168640 1 --2- 172.21.15.195:0/2360078259 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44dc107900 0x7f44dc19f2f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:12.892 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.890+0000 7f44e4168640 1 --2- 172.21.15.195:0/2360078259 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44dc104fd0 0x7f44dc198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:12.892 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.890+0000 7f44e4168640 1 --2- 172.21.15.195:0/2360078259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44dc1026a0 0x7f44dc197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:12.892 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.890+0000 7f44e4168640 1 -- 172.21.15.195:0/2360078259 >> 172.21.15.195:0/2360078259 conn(0x7f44dc0fc280 msgr2=0x7f44dc102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:12.892 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.890+0000 7f44e4168640 1 -- 172.21.15.195:0/2360078259 shutdown_connections 2024-09-12T03:26:12.892 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-12T03:26:12.890+0000 7f44e4168640 1 -- 172.21.15.195:0/2360078259 wait complete. 2024-09-12T03:26:13.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:13 smithi195 ceph-mon[34786]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:13.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:13 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:13.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:13 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:26:13.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:13 smithi064 ceph-mon[30952]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:13.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:13 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:13.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:13 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:26:13.537 DEBUG:teuthology.orchestra.run.smithi064:mds.a> sudo journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@mds.a.service 2024-09-12T03:26:13.540 DEBUG:teuthology.orchestra.run.smithi064:mds.d> sudo journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@mds.d.service 2024-09-12T03:26:13.584 DEBUG:teuthology.orchestra.run.smithi064:mds.g> sudo journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@mds.g.service 2024-09-12T03:26:13.627 DEBUG:teuthology.orchestra.run.smithi064:mds.j> sudo journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@mds.j.service 2024-09-12T03:26:13.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:13 smithi115 ceph-mon[34801]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:13.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:13 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:13.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:13 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:26:13.670 DEBUG:teuthology.orchestra.run.smithi115:mds.b> sudo journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@mds.b.service 2024-09-12T03:26:13.674 DEBUG:teuthology.orchestra.run.smithi115:mds.e> sudo journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@mds.e.service 2024-09-12T03:26:13.718 DEBUG:teuthology.orchestra.run.smithi115:mds.h> sudo journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@mds.h.service 2024-09-12T03:26:13.761 DEBUG:teuthology.orchestra.run.smithi115:mds.k> sudo journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@mds.k.service 2024-09-12T03:26:13.805 DEBUG:teuthology.orchestra.run.smithi195:mds.c> sudo journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@mds.c.service 2024-09-12T03:26:13.808 DEBUG:teuthology.orchestra.run.smithi195:mds.f> sudo journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@mds.f.service 2024-09-12T03:26:13.851 DEBUG:teuthology.orchestra.run.smithi195:mds.i> sudo journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@mds.i.service 2024-09-12T03:26:13.895 DEBUG:teuthology.orchestra.run.smithi195:mds.l> sudo journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@mds.l.service 2024-09-12T03:26:13.938 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-12T03:26:13.938 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:26:14.203 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:14.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:14 smithi064 ceph-mon[30952]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi115=b;smithi115=e;smithi115=h;smithi115=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:26:14.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:14 smithi064 ceph-mon[30952]: Saving service mds.all spec with placement smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi115=b;smithi115=e;smithi115=h;smithi115=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l;count:12 2024-09-12T03:26:14.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:14 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:14.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:14 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:26:14.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:14 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:14.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:14 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:26:14.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:14 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:26:14.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:14 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:14.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:14 smithi064 ceph-mon[30952]: Deploying daemon mds.l on smithi195 2024-09-12T03:26:14.422 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:14 smithi195 ceph-mon[34786]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi115=b;smithi115=e;smithi115=h;smithi115=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:26:14.422 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:14 smithi195 ceph-mon[34786]: Saving service mds.all spec with placement smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi115=b;smithi115=e;smithi115=h;smithi115=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l;count:12 2024-09-12T03:26:14.422 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:14 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:14.423 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:14 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:26:14.423 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:14 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:14.423 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:14 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:26:14.423 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:14 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:26:14.423 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:14 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:14.423 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:14 smithi195 ceph-mon[34786]: Deploying daemon mds.l on smithi195 2024-09-12T03:26:14.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.611+0000 7fcb29bc2640 1 -- 172.21.15.64:0/907510177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb24106f10 msgr2=0x7fcb2410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:14.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.611+0000 7fcb29bc2640 1 --2- 172.21.15.64:0/907510177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb24106f10 0x7fcb2410ea90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c009ca0 tx=0x7fcb1c02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:26:14.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.611+0000 7fcb29bc2640 1 -- 172.21.15.64:0/907510177 shutdown_connections 2024-09-12T03:26:14.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.611+0000 7fcb29bc2640 1 --2- 172.21.15.64:0/907510177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb24106f10 0x7fcb2410ea90 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:14.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.611+0000 7fcb29bc2640 1 --2- 172.21.15.64:0/907510177 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb24102790 0x7fcb241067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:14.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.611+0000 7fcb29bc2640 1 --2- 172.21.15.64:0/907510177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb24101de0 0x7fcb241021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:14.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.611+0000 7fcb29bc2640 1 -- 172.21.15.64:0/907510177 >> 172.21.15.64:0/907510177 conn(0x7fcb240fd8c0 msgr2=0x7fcb240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:14.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.611+0000 7fcb29bc2640 1 -- 172.21.15.64:0/907510177 shutdown_connections 2024-09-12T03:26:14.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.611+0000 7fcb29bc2640 1 -- 172.21.15.64:0/907510177 wait complete. 2024-09-12T03:26:14.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.612+0000 7fcb29bc2640 1 Processor -- start 2024-09-12T03:26:14.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.612+0000 7fcb29bc2640 1 -- start start 2024-09-12T03:26:14.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.612+0000 7fcb29bc2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb24101de0 0x7fcb241982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:14.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.612+0000 7fcb29bc2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb24102790 0x7fcb241987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:14.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.612+0000 7fcb29bc2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb24106f10 0x7fcb2419cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:14.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.612+0000 7fcb29bc2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcb24111e20 con 0x7fcb24106f10 2024-09-12T03:26:14.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.612+0000 7fcb29bc2640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcb24111ca0 con 0x7fcb24102790 2024-09-12T03:26:14.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.612+0000 7fcb29bc2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcb24111fa0 con 0x7fcb24101de0 2024-09-12T03:26:14.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.613+0000 7fcb23fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb24106f10 0x7fcb2419cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:14.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.613+0000 7fcb22ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb24102790 0x7fcb241987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:14.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.613+0000 7fcb23fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb24106f10 0x7fcb2419cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53964/0 (socket says 172.21.15.64:53964) 2024-09-12T03:26:14.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.613+0000 7fcb23fff640 1 -- 172.21.15.64:0/2477604931 learned_addr learned my addr 172.21.15.64:0/2477604931 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:26:14.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.613+0000 7fcb237fe640 1 --2- 172.21.15.64:0/2477604931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb24101de0 0x7fcb241982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:14.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.613+0000 7fcb22ffd640 1 -- 172.21.15.64:0/2477604931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb24101de0 msgr2=0x7fcb241982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:14.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.613+0000 7fcb22ffd640 1 --2- 172.21.15.64:0/2477604931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb24101de0 0x7fcb241982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:14.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.613+0000 7fcb22ffd640 1 -- 172.21.15.64:0/2477604931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb24106f10 msgr2=0x7fcb2419cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:14.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.613+0000 7fcb22ffd640 1 --2- 172.21.15.64:0/2477604931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb24106f10 0x7fcb2419cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:14.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.613+0000 7fcb22ffd640 1 -- 172.21.15.64:0/2477604931 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb2419d250 con 0x7fcb24102790 2024-09-12T03:26:14.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.613+0000 7fcb23fff640 1 --2- 172.21.15.64:0/2477604931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb24106f10 0x7fcb2419cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:26:14.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.613+0000 7fcb22ffd640 1 --2- 172.21.15.64:0/2477604931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb24102790 0x7fcb241987e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c00e9f0 tx=0x7fcb0c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:14.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.613+0000 7fcb20ff9640 1 -- 172.21.15.64:0/2477604931 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c00ccc0 con 0x7fcb24102790 2024-09-12T03:26:14.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.613+0000 7fcb29bc2640 1 -- 172.21.15.64:0/2477604931 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb2419d540 con 0x7fcb24102790 2024-09-12T03:26:14.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.613+0000 7fcb20ff9640 1 -- 172.21.15.64:0/2477604931 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb0c005020 con 0x7fcb24102790 2024-09-12T03:26:14.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.613+0000 7fcb20ff9640 1 -- 172.21.15.64:0/2477604931 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c00ce60 con 0x7fcb24102790 2024-09-12T03:26:14.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.613+0000 7fcb29bc2640 1 -- 172.21.15.64:0/2477604931 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb241a4e20 con 0x7fcb24102790 2024-09-12T03:26:14.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.615+0000 7fcb29bc2640 1 -- 172.21.15.64:0/2477604931 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb241021c0 con 0x7fcb24102790 2024-09-12T03:26:14.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.616+0000 7fcb20ff9640 1 -- 172.21.15.64:0/2477604931 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fcb0c0051d0 con 0x7fcb24102790 2024-09-12T03:26:14.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.616+0000 7fcb20ff9640 1 --2- 172.21.15.64:0/2477604931 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fcb100a2b80 0x7fcb100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:14.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.616+0000 7fcb237fe640 1 --2- 172.21.15.64:0/2477604931 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fcb100a2b80 0x7fcb100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:14.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.617+0000 7fcb20ff9640 1 -- 172.21.15.64:0/2477604931 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fcb0c0cea60 con 0x7fcb24102790 2024-09-12T03:26:14.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.617+0000 7fcb237fe640 1 --2- 172.21.15.64:0/2477604931 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fcb100a2b80 0x7fcb100a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400abe0 tx=0x7fcb1400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:14.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.620+0000 7fcb20ff9640 1 -- 172.21.15.64:0/2477604931 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcb0c09be60 con 0x7fcb24102790 2024-09-12T03:26:14.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:14 smithi115 ceph-mon[34801]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi115=b;smithi115=e;smithi115=h;smithi115=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:26:14.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:14 smithi115 ceph-mon[34801]: Saving service mds.all spec with placement smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi115=b;smithi115=e;smithi115=h;smithi115=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l;count:12 2024-09-12T03:26:14.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:14 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:14.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:14 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:26:14.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:14 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:14.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:14 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:26:14.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:14 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:26:14.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:14 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:14.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:14 smithi115 ceph-mon[34801]: Deploying daemon mds.l on smithi195 2024-09-12T03:26:14.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.830+0000 7fcb29bc2640 1 -- 172.21.15.64:0/2477604931 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb2402d7c0 con 0x7fcb24102790 2024-09-12T03:26:14.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.830+0000 7fcb20ff9640 1 -- 172.21.15.64:0/2477604931 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7fcb0c09b760 con 0x7fcb24102790 2024-09-12T03:26:14.830 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:26:14.831 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":1,"btime":"2024-09-12T03:19:15:900094+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-12T03:26:14.831 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 1 2024-09-12T03:26:14.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.833+0000 7fcb29bc2640 1 -- 172.21.15.64:0/2477604931 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fcb100a2b80 msgr2=0x7fcb100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:14.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.833+0000 7fcb29bc2640 1 --2- 172.21.15.64:0/2477604931 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fcb100a2b80 0x7fcb100a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400abe0 tx=0x7fcb1400a2d0 comp rx=0 tx=0).stop 2024-09-12T03:26:14.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.833+0000 7fcb29bc2640 1 -- 172.21.15.64:0/2477604931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb24102790 msgr2=0x7fcb241987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:14.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.833+0000 7fcb29bc2640 1 --2- 172.21.15.64:0/2477604931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb24102790 0x7fcb241987e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c00e9f0 tx=0x7fcb0c00edf0 comp rx=0 tx=0).stop 2024-09-12T03:26:14.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.833+0000 7fcb29bc2640 1 -- 172.21.15.64:0/2477604931 shutdown_connections 2024-09-12T03:26:14.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.833+0000 7fcb29bc2640 1 --2- 172.21.15.64:0/2477604931 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fcb100a2b80 0x7fcb100a5040 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:14.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.833+0000 7fcb29bc2640 1 --2- 172.21.15.64:0/2477604931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb24106f10 0x7fcb2419cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:14.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.833+0000 7fcb29bc2640 1 --2- 172.21.15.64:0/2477604931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb24102790 0x7fcb241987e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:14.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.833+0000 7fcb29bc2640 1 --2- 172.21.15.64:0/2477604931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb24101de0 0x7fcb241982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:14.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.833+0000 7fcb29bc2640 1 -- 172.21.15.64:0/2477604931 >> 172.21.15.64:0/2477604931 conn(0x7fcb240fd8c0 msgr2=0x7fcb240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:14.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.833+0000 7fcb29bc2640 1 -- 172.21.15.64:0/2477604931 shutdown_connections 2024-09-12T03:26:14.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:14.833+0000 7fcb29bc2640 1 -- 172.21.15.64:0/2477604931 wait complete. 2024-09-12T03:26:15.273 INFO:journalctl@ceph.mds.l.smithi195.stdout:Sep 12 03:26:15 smithi195 systemd[1]: Starting Ceph mds.l for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:26:15.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:15 smithi064 ceph-mon[30952]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:15.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:15 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/2477604931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:15.636 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:15 smithi195 ceph-mon[34786]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:15.636 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:15 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/2477604931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:15.637 INFO:journalctl@ceph.mds.l.smithi195.stdout:Sep 12 03:26:15 smithi195 podman[64213]: 2024-09-12 03:26:15.26549732 +0000 UTC m=+0.015893884 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:26:15.637 INFO:journalctl@ceph.mds.l.smithi195.stdout:Sep 12 03:26:15 smithi195 podman[64213]: 2024-09-12 03:26:15.367434036 +0000 UTC m=+0.117830587 container create bdd5cfba81e90cc0dfcbe2fed1c1cd4bc6dbb1ec97aa24225af47e28b5e4ecac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-l, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:26:15.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:15 smithi115 ceph-mon[34801]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:15.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:15 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/2477604931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:15.956 INFO:journalctl@ceph.mds.l.smithi195.stdout:Sep 12 03:26:15 smithi195 podman[64213]: 2024-09-12 03:26:15.637125185 +0000 UTC m=+0.387521735 container init bdd5cfba81e90cc0dfcbe2fed1c1cd4bc6dbb1ec97aa24225af47e28b5e4ecac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-l, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-12T03:26:15.956 INFO:journalctl@ceph.mds.l.smithi195.stdout:Sep 12 03:26:15 smithi195 podman[64213]: 2024-09-12 03:26:15.639997245 +0000 UTC m=+0.390393797 container start bdd5cfba81e90cc0dfcbe2fed1c1cd4bc6dbb1ec97aa24225af47e28b5e4ecac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-l, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-12T03:26:15.956 INFO:journalctl@ceph.mds.l.smithi195.stdout:Sep 12 03:26:15 smithi195 ceph-mds[64232]: -- 172.21.15.195:0/439094111 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563ffe359860 con 0x563ffe337000 2024-09-12T03:26:15.957 INFO:journalctl@ceph.mds.l.smithi195.stdout:Sep 12 03:26:15 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-l[64228]: starting mds.l at 2024-09-12T03:26:15.957 INFO:journalctl@ceph.mds.l.smithi195.stdout:Sep 12 03:26:15 smithi195 bash[64213]: bdd5cfba81e90cc0dfcbe2fed1c1cd4bc6dbb1ec97aa24225af47e28b5e4ecac 2024-09-12T03:26:15.957 INFO:journalctl@ceph.mds.l.smithi195.stdout:Sep 12 03:26:15 smithi195 systemd[1]: Started Ceph mds.l for b0812460-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:26:16.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:16 smithi115 ceph-mon[34801]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:16.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:16 smithi115 ceph-mon[34801]: mds.? [v2:172.21.15.195:6832/2991933266,v1:172.21.15.195:6833/2991933266] up:boot 2024-09-12T03:26:16.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:16 smithi115 ceph-mon[34801]: fsmap 1 up:standby 2024-09-12T03:26:16.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:16 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T03:26:16.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:16 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:16.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:16 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:16.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:16 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:16.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:16 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:26:16.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:16 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:26:16.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:16 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:16.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:16 smithi115 ceph-mon[34801]: Deploying daemon mds.k on smithi115 2024-09-12T03:26:16.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:16 smithi195 ceph-mon[34786]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:16.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:16 smithi195 ceph-mon[34786]: mds.? [v2:172.21.15.195:6832/2991933266,v1:172.21.15.195:6833/2991933266] up:boot 2024-09-12T03:26:16.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:16 smithi195 ceph-mon[34786]: fsmap 1 up:standby 2024-09-12T03:26:16.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:16 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T03:26:16.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:16 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:16.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:16 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:16.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:16 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:16.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:16 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:26:16.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:16 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:26:16.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:16 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:16.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:16 smithi195 ceph-mon[34786]: Deploying daemon mds.k on smithi115 2024-09-12T03:26:17.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:16 smithi064 ceph-mon[30952]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:17.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:16 smithi064 ceph-mon[30952]: mds.? [v2:172.21.15.195:6832/2991933266,v1:172.21.15.195:6833/2991933266] up:boot 2024-09-12T03:26:17.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:16 smithi064 ceph-mon[30952]: fsmap 1 up:standby 2024-09-12T03:26:17.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:16 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T03:26:17.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:16 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:17.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:16 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:17.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:16 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:17.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:16 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:26:17.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:16 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:26:17.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:16 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:17.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:16 smithi064 ceph-mon[30952]: Deploying daemon mds.k on smithi115 2024-09-12T03:26:17.564 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:26:17.831 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:17.893 INFO:journalctl@ceph.mds.k.smithi115.stdout:Sep 12 03:26:17 smithi115 systemd[1]: Starting Ceph mds.k for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:26:18.228 INFO:journalctl@ceph.mds.k.smithi115.stdout:Sep 12 03:26:17 smithi115 podman[63181]: 2024-09-12 03:26:17.868148712 +0000 UTC m=+0.015814868 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:26:18.228 INFO:journalctl@ceph.mds.k.smithi115.stdout:Sep 12 03:26:17 smithi115 podman[63181]: 2024-09-12 03:26:17.97589997 +0000 UTC m=+0.123566125 container create cf7a19fc47bbeb338ffeeeade29b9fdc3ea8a3dafb33b591bce89c85fc1e2199 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-12T03:26:18.228 INFO:journalctl@ceph.mds.k.smithi115.stdout:Sep 12 03:26:18 smithi115 podman[63181]: 2024-09-12 03:26:18.228649389 +0000 UTC m=+0.376315539 container init cf7a19fc47bbeb338ffeeeade29b9fdc3ea8a3dafb33b591bce89c85fc1e2199 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-k, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-12T03:26:18.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.237+0000 7f0abb6ff640 1 -- 172.21.15.64:0/3414700978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ab41082a0 msgr2=0x7f0ab410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:18.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.237+0000 7f0abb6ff640 1 --2- 172.21.15.64:0/3414700978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ab41082a0 0x7f0ab410ec40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4009cd0 tx=0x7f0aa402a8e0 comp rx=0 tx=0).stop 2024-09-12T03:26:18.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.237+0000 7f0abb6ff640 1 -- 172.21.15.64:0/3414700978 shutdown_connections 2024-09-12T03:26:18.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.237+0000 7f0abb6ff640 1 --2- 172.21.15.64:0/3414700978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ab41082a0 0x7f0ab410ec40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:18.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.237+0000 7f0abb6ff640 1 --2- 172.21.15.64:0/3414700978 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ab41036e0 0x7f0ab4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:18.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.237+0000 7f0abb6ff640 1 --2- 172.21.15.64:0/3414700978 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ab4102d30 0x7f0ab4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:18.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.237+0000 7f0abb6ff640 1 -- 172.21.15.64:0/3414700978 >> 172.21.15.64:0/3414700978 conn(0x7f0ab40fc7d0 msgr2=0x7f0ab40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:18.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.237+0000 7f0abb6ff640 1 -- 172.21.15.64:0/3414700978 shutdown_connections 2024-09-12T03:26:18.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.237+0000 7f0abb6ff640 1 -- 172.21.15.64:0/3414700978 wait complete. 2024-09-12T03:26:18.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.237+0000 7f0abb6ff640 1 Processor -- start 2024-09-12T03:26:18.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.237+0000 7f0abb6ff640 1 -- start start 2024-09-12T03:26:18.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.238+0000 7f0abb6ff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ab4102d30 0x7f0ab4100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:18.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.238+0000 7f0abb6ff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ab41036e0 0x7f0ab4100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:18.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.238+0000 7f0abb6ff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ab41082a0 0x7f0ab4101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:18.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.238+0000 7f0abb6ff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab4111f50 con 0x7f0ab4102d30 2024-09-12T03:26:18.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.238+0000 7f0abb6ff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab4111dd0 con 0x7f0ab41036e0 2024-09-12T03:26:18.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.238+0000 7f0abb6ff640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab41120d0 con 0x7f0ab41082a0 2024-09-12T03:26:18.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.238+0000 7f0ab949b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ab4102d30 0x7f0ab4100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:18.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.238+0000 7f0ab949b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ab4102d30 0x7f0ab4100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58530/0 (socket says 172.21.15.64:58530) 2024-09-12T03:26:18.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.238+0000 7f0ab949b640 1 -- 172.21.15.64:0/3037764023 learned_addr learned my addr 172.21.15.64:0/3037764023 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:26:18.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.238+0000 7f0ab8c9a640 1 --2- 172.21.15.64:0/3037764023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ab41036e0 0x7f0ab4100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:18.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.238+0000 7f0ab9c9c640 1 --2- 172.21.15.64:0/3037764023 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ab41082a0 0x7f0ab4101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:18.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.238+0000 7f0ab949b640 1 -- 172.21.15.64:0/3037764023 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ab41082a0 msgr2=0x7f0ab4101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:18.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.238+0000 7f0ab949b640 1 --2- 172.21.15.64:0/3037764023 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ab41082a0 0x7f0ab4101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:18.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.238+0000 7f0ab949b640 1 -- 172.21.15.64:0/3037764023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ab41036e0 msgr2=0x7f0ab4100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:18.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.238+0000 7f0ab949b640 1 --2- 172.21.15.64:0/3037764023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ab41036e0 0x7f0ab4100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:18.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.238+0000 7f0ab949b640 1 -- 172.21.15.64:0/3037764023 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ab41a68b0 con 0x7f0ab4102d30 2024-09-12T03:26:18.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.238+0000 7f0ab949b640 1 --2- 172.21.15.64:0/3037764023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ab4102d30 0x7f0ab4100990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0aac00e9f0 tx=0x7f0aac00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:18.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.238+0000 7f0ab27fc640 1 -- 172.21.15.64:0/3037764023 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aac00ccc0 con 0x7f0ab4102d30 2024-09-12T03:26:18.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.238+0000 7f0abb6ff640 1 -- 172.21.15.64:0/3037764023 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ab41a6ba0 con 0x7f0ab4102d30 2024-09-12T03:26:18.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.238+0000 7f0ab27fc640 1 -- 172.21.15.64:0/3037764023 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0aac0108e0 con 0x7f0ab4102d30 2024-09-12T03:26:18.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.238+0000 7f0ab27fc640 1 -- 172.21.15.64:0/3037764023 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aac0187c0 con 0x7f0ab4102d30 2024-09-12T03:26:18.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.239+0000 7f0abb6ff640 1 -- 172.21.15.64:0/3037764023 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ab41a7130 con 0x7f0ab4102d30 2024-09-12T03:26:18.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.240+0000 7f0abb6ff640 1 -- 172.21.15.64:0/3037764023 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ab4103110 con 0x7f0ab4102d30 2024-09-12T03:26:18.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.241+0000 7f0ab27fc640 1 -- 172.21.15.64:0/3037764023 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0aac010a90 con 0x7f0ab4102d30 2024-09-12T03:26:18.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.241+0000 7f0ab27fc640 1 --2- 172.21.15.64:0/3037764023 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f0a9c0a2b80 0x7f0a9c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:18.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.242+0000 7f0ab8c9a640 1 --2- 172.21.15.64:0/3037764023 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f0a9c0a2b80 0x7f0a9c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:18.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.242+0000 7f0ab27fc640 1 -- 172.21.15.64:0/3037764023 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f0aac0cf4b0 con 0x7f0ab4102d30 2024-09-12T03:26:18.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.242+0000 7f0ab8c9a640 1 --2- 172.21.15.64:0/3037764023 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f0a9c0a2b80 0x7f0a9c0a5040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4102420 tx=0x7f0aa800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:18.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.245+0000 7f0ab27fc640 1 -- 172.21.15.64:0/3037764023 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0aac09c8b0 con 0x7f0ab4102d30 2024-09-12T03:26:18.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.457+0000 7f0abb6ff640 1 -- 172.21.15.64:0/3037764023 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ab41a7300 con 0x7f0ab4102d30 2024-09-12T03:26:18.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.458+0000 7f0ab27fc640 1 -- 172.21.15.64:0/3037764023 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7f0aac09c1b0 con 0x7f0ab4102d30 2024-09-12T03:26:18.458 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:26:18.458 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":2,"btime":"2024-09-12T03:26:15:667905+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2991933266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2991933266},{"type":"v1","addr":"172.21.15.195:6833","nonce":2991933266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-12T03:26:18.459 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 2 2024-09-12T03:26:18.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.460+0000 7f0abb6ff640 1 -- 172.21.15.64:0/3037764023 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f0a9c0a2b80 msgr2=0x7f0a9c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:18.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.460+0000 7f0abb6ff640 1 --2- 172.21.15.64:0/3037764023 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f0a9c0a2b80 0x7f0a9c0a5040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4102420 tx=0x7f0aa800a2d0 comp rx=0 tx=0).stop 2024-09-12T03:26:18.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.460+0000 7f0abb6ff640 1 -- 172.21.15.64:0/3037764023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ab4102d30 msgr2=0x7f0ab4100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:18.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.460+0000 7f0abb6ff640 1 --2- 172.21.15.64:0/3037764023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ab4102d30 0x7f0ab4100990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0aac00e9f0 tx=0x7f0aac00edf0 comp rx=0 tx=0).stop 2024-09-12T03:26:18.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.460+0000 7f0abb6ff640 1 -- 172.21.15.64:0/3037764023 shutdown_connections 2024-09-12T03:26:18.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.460+0000 7f0abb6ff640 1 --2- 172.21.15.64:0/3037764023 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f0a9c0a2b80 0x7f0a9c0a5040 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:18.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.460+0000 7f0abb6ff640 1 --2- 172.21.15.64:0/3037764023 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ab41082a0 0x7f0ab4101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:18.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.460+0000 7f0abb6ff640 1 --2- 172.21.15.64:0/3037764023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ab41036e0 0x7f0ab4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:18.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.460+0000 7f0abb6ff640 1 --2- 172.21.15.64:0/3037764023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ab4102d30 0x7f0ab4100990 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:18.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.460+0000 7f0abb6ff640 1 -- 172.21.15.64:0/3037764023 >> 172.21.15.64:0/3037764023 conn(0x7f0ab40fc7d0 msgr2=0x7f0ab4105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:18.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.460+0000 7f0abb6ff640 1 -- 172.21.15.64:0/3037764023 shutdown_connections 2024-09-12T03:26:18.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:18.460+0000 7f0abb6ff640 1 -- 172.21.15.64:0/3037764023 wait complete. 2024-09-12T03:26:18.643 INFO:journalctl@ceph.mds.k.smithi115.stdout:Sep 12 03:26:18 smithi115 podman[63181]: 2024-09-12 03:26:18.231590394 +0000 UTC m=+0.379256554 container start cf7a19fc47bbeb338ffeeeade29b9fdc3ea8a3dafb33b591bce89c85fc1e2199 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-k, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-12T03:26:18.643 INFO:journalctl@ceph.mds.k.smithi115.stdout:Sep 12 03:26:18 smithi115 ceph-mds[63200]: -- 172.21.15.115:0/270463151 <== mon.2 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563c699ebc20 con 0x563c699c9400 2024-09-12T03:26:18.643 INFO:journalctl@ceph.mds.k.smithi115.stdout:Sep 12 03:26:18 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-k[63196]: starting mds.k at 2024-09-12T03:26:18.643 INFO:journalctl@ceph.mds.k.smithi115.stdout:Sep 12 03:26:18 smithi115 bash[63181]: cf7a19fc47bbeb338ffeeeade29b9fdc3ea8a3dafb33b591bce89c85fc1e2199 2024-09-12T03:26:18.643 INFO:journalctl@ceph.mds.k.smithi115.stdout:Sep 12 03:26:18 smithi115 systemd[1]: Started Ceph mds.k for b0812460-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:26:19.514 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:19 smithi064 ceph-mon[30952]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:19.514 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:19 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:19.514 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:19 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:19.514 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:19 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:19.514 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:19 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:26:19.515 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:19 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:26:19.515 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:19 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:19.515 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:19 smithi064 ceph-mon[30952]: Deploying daemon mds.j on smithi064 2024-09-12T03:26:19.515 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:19 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/3037764023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:19.515 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:19 smithi064 ceph-mon[30952]: mds.? [v2:172.21.15.115:6832/1138960346,v1:172.21.15.115:6833/1138960346] up:boot 2024-09-12T03:26:19.515 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:19 smithi064 ceph-mon[30952]: fsmap 2 up:standby 2024-09-12T03:26:19.515 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:19 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T03:26:19.637 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:19 smithi115 ceph-mon[34801]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:19.637 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:19 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:19.637 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:19 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:19.637 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:19 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:19.637 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:19 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:26:19.637 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:19 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:26:19.638 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:19 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:19.638 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:19 smithi115 ceph-mon[34801]: Deploying daemon mds.j on smithi064 2024-09-12T03:26:19.638 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:19 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/3037764023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:19.638 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:19 smithi115 ceph-mon[34801]: mds.? [v2:172.21.15.115:6832/1138960346,v1:172.21.15.115:6833/1138960346] up:boot 2024-09-12T03:26:19.638 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:19 smithi115 ceph-mon[34801]: fsmap 2 up:standby 2024-09-12T03:26:19.638 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:19 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T03:26:19.660 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:19 smithi195 ceph-mon[34786]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:19.660 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:19 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:19.660 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:19 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:19.660 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:19 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:19.661 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:19 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:26:19.661 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:19 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:26:19.661 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:19 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:19.661 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:19 smithi195 ceph-mon[34786]: Deploying daemon mds.j on smithi064 2024-09-12T03:26:19.661 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:19 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/3037764023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:19.661 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:19 smithi195 ceph-mon[34786]: mds.? [v2:172.21.15.115:6832/1138960346,v1:172.21.15.115:6833/1138960346] up:boot 2024-09-12T03:26:19.662 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:19 smithi195 ceph-mon[34786]: fsmap 2 up:standby 2024-09-12T03:26:19.662 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:19 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T03:26:21.022 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 12 03:26:20 smithi064 systemd[1]: Starting Ceph mds.j for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:26:21.279 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:26:21.343 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 12 03:26:21 smithi064 podman[70428]: 2024-09-12 03:26:21.071898744 +0000 UTC m=+0.110040669 container create 38ce48cd536e5a8a60c76775c13186460a659a13751e4b5136abfa9566a3669e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-j, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:26:21.343 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 12 03:26:21 smithi064 podman[70428]: 2024-09-12 03:26:20.978178994 +0000 UTC m=+0.016320916 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:26:21.344 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 12 03:26:21 smithi064 podman[70428]: 2024-09-12 03:26:21.316151865 +0000 UTC m=+0.354293788 container init 38ce48cd536e5a8a60c76775c13186460a659a13751e4b5136abfa9566a3669e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-12T03:26:21.344 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 12 03:26:21 smithi064 podman[70428]: 2024-09-12 03:26:21.319368473 +0000 UTC m=+0.357510405 container start 38ce48cd536e5a8a60c76775c13186460a659a13751e4b5136abfa9566a3669e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-j, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:26:21.548 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:21.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:21 smithi115 ceph-mon[34801]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:21.677 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:21 smithi064 ceph-mon[30952]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:21.678 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 12 03:26:21 smithi064 ceph-mds[70465]: -- 172.21.15.64:0/75220596 <== mon.0 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555f4bd7d860 con 0x555f4bd5a800 2024-09-12T03:26:21.678 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 12 03:26:21 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-j[70443]: starting mds.j at 2024-09-12T03:26:21.678 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 12 03:26:21 smithi064 bash[70428]: 38ce48cd536e5a8a60c76775c13186460a659a13751e4b5136abfa9566a3669e 2024-09-12T03:26:21.678 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 12 03:26:21 smithi064 systemd[1]: Started Ceph mds.j for b0812460-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:26:21.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:21 smithi195 ceph-mon[34786]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:22.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.030+0000 7f39ed175640 1 -- 172.21.15.64:0/1523054965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39e8100100 msgr2=0x7f39e8100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:22.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.030+0000 7f39ed175640 1 --2- 172.21.15.64:0/1523054965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39e8100100 0x7f39e8100560 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f39dc009900 tx=0x7f39dc02a410 comp rx=0 tx=0).stop 2024-09-12T03:26:22.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.030+0000 7f39ed175640 1 -- 172.21.15.64:0/1523054965 shutdown_connections 2024-09-12T03:26:22.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.030+0000 7f39ed175640 1 --2- 172.21.15.64:0/1523054965 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39e8100aa0 0x7f39e810ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.030+0000 7f39ed175640 1 --2- 172.21.15.64:0/1523054965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39e8100100 0x7f39e8100560 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.030+0000 7f39ed175640 1 --2- 172.21.15.64:0/1523054965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39e8106b80 0x7f39e8106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.030+0000 7f39ed175640 1 -- 172.21.15.64:0/1523054965 >> 172.21.15.64:0/1523054965 conn(0x7f39e8077230 msgr2=0x7f39e80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:22.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.030+0000 7f39ed175640 1 -- 172.21.15.64:0/1523054965 shutdown_connections 2024-09-12T03:26:22.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.030+0000 7f39ed175640 1 -- 172.21.15.64:0/1523054965 wait complete. 2024-09-12T03:26:22.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.031+0000 7f39ed175640 1 Processor -- start 2024-09-12T03:26:22.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.031+0000 7f39ed175640 1 -- start start 2024-09-12T03:26:22.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.031+0000 7f39ed175640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39e8100100 0x7f39e81984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:22.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.031+0000 7f39ed175640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39e8100aa0 0x7f39e8198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:22.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.031+0000 7f39ed175640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39e8106b80 0x7f39e819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:22.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.031+0000 7f39ed175640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f39e8111f40 con 0x7f39e8106b80 2024-09-12T03:26:22.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.031+0000 7f39ed175640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f39e8111dc0 con 0x7f39e8100aa0 2024-09-12T03:26:22.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.031+0000 7f39ed175640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f39e81120c0 con 0x7f39e8100100 2024-09-12T03:26:22.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.032+0000 7f39e659c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39e8100aa0 0x7f39e8198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:22.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.032+0000 7f39e759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39e8106b80 0x7f39e819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:22.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.032+0000 7f39e659c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39e8100aa0 0x7f39e8198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.64:36122/0 (socket says 172.21.15.64:36122) 2024-09-12T03:26:22.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.032+0000 7f39e659c640 1 -- 172.21.15.64:0/2181710509 learned_addr learned my addr 172.21.15.64:0/2181710509 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:26:22.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.032+0000 7f39e6d9d640 1 --2- 172.21.15.64:0/2181710509 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39e8100100 0x7f39e81984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:22.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.032+0000 7f39e659c640 1 -- 172.21.15.64:0/2181710509 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39e8100100 msgr2=0x7f39e81984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:22.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.032+0000 7f39e659c640 1 --2- 172.21.15.64:0/2181710509 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39e8100100 0x7f39e81984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.032+0000 7f39e659c640 1 -- 172.21.15.64:0/2181710509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39e8106b80 msgr2=0x7f39e819cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:22.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.032+0000 7f39e659c640 1 --2- 172.21.15.64:0/2181710509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39e8106b80 0x7f39e819cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.032+0000 7f39e659c640 1 -- 172.21.15.64:0/2181710509 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e819d470 con 0x7f39e8100aa0 2024-09-12T03:26:22.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.032+0000 7f39e759e640 1 --2- 172.21.15.64:0/2181710509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39e8106b80 0x7f39e819cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:26:22.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.032+0000 7f39e659c640 1 --2- 172.21.15.64:0/2181710509 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39e8100aa0 0x7f39e8198a00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f39dc005940 tx=0x7f39dc004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:22.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.033+0000 7f39cffff640 1 -- 172.21.15.64:0/2181710509 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39dc03f030 con 0x7f39e8100aa0 2024-09-12T03:26:22.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.033+0000 7f39cffff640 1 -- 172.21.15.64:0/2181710509 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39dc003aa0 con 0x7f39e8100aa0 2024-09-12T03:26:22.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.033+0000 7f39ed175640 1 -- 172.21.15.64:0/2181710509 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39e819d700 con 0x7f39e8100aa0 2024-09-12T03:26:22.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.033+0000 7f39cffff640 1 -- 172.21.15.64:0/2181710509 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39dc035070 con 0x7f39e8100aa0 2024-09-12T03:26:22.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.033+0000 7f39ed175640 1 -- 172.21.15.64:0/2181710509 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39e81a5040 con 0x7f39e8100aa0 2024-09-12T03:26:22.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.034+0000 7f39ed175640 1 -- 172.21.15.64:0/2181710509 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39e8100560 con 0x7f39e8100aa0 2024-09-12T03:26:22.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.035+0000 7f39cffff640 1 -- 172.21.15.64:0/2181710509 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f39dc042030 con 0x7f39e8100aa0 2024-09-12T03:26:22.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.036+0000 7f39cffff640 1 --2- 172.21.15.64:0/2181710509 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f39d00a2b80 0x7f39d00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:22.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.036+0000 7f39e6d9d640 1 --2- 172.21.15.64:0/2181710509 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f39d00a2b80 0x7f39d00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:22.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.036+0000 7f39cffff640 1 -- 172.21.15.64:0/2181710509 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f39dc0ef7b0 con 0x7f39e8100aa0 2024-09-12T03:26:22.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.037+0000 7f39e6d9d640 1 --2- 172.21.15.64:0/2181710509 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f39d00a2b80 0x7f39d00a5040 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f39d800bbe0 tx=0x7f39d800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:22.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.039+0000 7f39cffff640 1 -- 172.21.15.64:0/2181710509 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f39dc049030 con 0x7f39e8100aa0 2024-09-12T03:26:22.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.254+0000 7f39ed175640 1 -- 172.21.15.64:0/2181710509 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39e810a0e0 con 0x7f39e8100aa0 2024-09-12T03:26:22.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.255+0000 7f39cffff640 1 -- 172.21.15.64:0/2181710509 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3091 (secure 0 0 0) 0x7f39dc0bc540 con 0x7f39e8100aa0 2024-09-12T03:26:22.255 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:26:22.255 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":4,"btime":"2024-09-12T03:26:21:383644+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2580078738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2580078738},{"type":"v1","addr":"172.21.15.64:6835","nonce":2580078738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2991933266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2991933266},{"type":"v1","addr":"172.21.15.195:6833","nonce":2991933266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1138960346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1138960346},{"type":"v1","addr":"172.21.15.115:6833","nonce":1138960346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-12T03:26:22.256 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 4 2024-09-12T03:26:22.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.257+0000 7f39ed175640 1 -- 172.21.15.64:0/2181710509 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f39d00a2b80 msgr2=0x7f39d00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:22.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.257+0000 7f39ed175640 1 --2- 172.21.15.64:0/2181710509 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f39d00a2b80 0x7f39d00a5040 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f39d800bbe0 tx=0x7f39d800b390 comp rx=0 tx=0).stop 2024-09-12T03:26:22.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.257+0000 7f39ed175640 1 -- 172.21.15.64:0/2181710509 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39e8100aa0 msgr2=0x7f39e8198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:22.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.257+0000 7f39ed175640 1 --2- 172.21.15.64:0/2181710509 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39e8100aa0 0x7f39e8198a00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f39dc005940 tx=0x7f39dc004270 comp rx=0 tx=0).stop 2024-09-12T03:26:22.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.257+0000 7f39ed175640 1 -- 172.21.15.64:0/2181710509 shutdown_connections 2024-09-12T03:26:22.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.257+0000 7f39ed175640 1 --2- 172.21.15.64:0/2181710509 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f39d00a2b80 0x7f39d00a5040 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.257+0000 7f39ed175640 1 --2- 172.21.15.64:0/2181710509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39e8106b80 0x7f39e819cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.257+0000 7f39ed175640 1 --2- 172.21.15.64:0/2181710509 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39e8100aa0 0x7f39e8198a00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.257+0000 7f39ed175640 1 --2- 172.21.15.64:0/2181710509 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39e8100100 0x7f39e81984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:22.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.257+0000 7f39ed175640 1 -- 172.21.15.64:0/2181710509 >> 172.21.15.64:0/2181710509 conn(0x7f39e8077230 msgr2=0x7f39e8104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:22.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.258+0000 7f39ed175640 1 -- 172.21.15.64:0/2181710509 shutdown_connections 2024-09-12T03:26:22.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:22.258+0000 7f39ed175640 1 -- 172.21.15.64:0/2181710509 wait complete. 2024-09-12T03:26:22.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:22 smithi064 ceph-mon[30952]: mds.? [v2:172.21.15.64:6834/2580078738,v1:172.21.15.64:6835/2580078738] up:boot 2024-09-12T03:26:22.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:22 smithi064 ceph-mon[30952]: fsmap 3 up:standby 2024-09-12T03:26:22.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:22 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T03:26:22.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:22 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:22.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:22 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:22.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:22 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:22.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:22 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:26:22.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:22 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:26:22.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:22 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:22.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:22 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/2181710509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:22.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:22 smithi115 ceph-mon[34801]: mds.? [v2:172.21.15.64:6834/2580078738,v1:172.21.15.64:6835/2580078738] up:boot 2024-09-12T03:26:22.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:22 smithi115 ceph-mon[34801]: fsmap 3 up:standby 2024-09-12T03:26:22.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:22 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T03:26:22.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:22 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:22.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:22 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:22.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:22 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:22.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:22 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:26:22.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:22 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:26:22.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:22 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:22.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:22 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/2181710509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:22.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:22 smithi195 ceph-mon[34786]: mds.? [v2:172.21.15.64:6834/2580078738,v1:172.21.15.64:6835/2580078738] up:boot 2024-09-12T03:26:22.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:22 smithi195 ceph-mon[34786]: fsmap 3 up:standby 2024-09-12T03:26:22.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:22 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T03:26:22.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:22 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:22.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:22 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:22.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:22 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:22.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:22 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:26:22.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:22 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:26:22.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:22 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:22.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:22 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/2181710509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:23.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:23 smithi064 ceph-mon[30952]: Deploying daemon mds.d on smithi064 2024-09-12T03:26:23.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:23 smithi064 ceph-mon[30952]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:23.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:23 smithi195 ceph-mon[34786]: Deploying daemon mds.d on smithi064 2024-09-12T03:26:23.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:23 smithi195 ceph-mon[34786]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:23.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:23 smithi115 ceph-mon[34801]: Deploying daemon mds.d on smithi064 2024-09-12T03:26:23.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:23 smithi115 ceph-mon[34801]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:24.522 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 12 03:26:24 smithi064 systemd[1]: Starting Ceph mds.d for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:26:24.974 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 12 03:26:24 smithi064 podman[70847]: 2024-09-12 03:26:24.571772881 +0000 UTC m=+0.016339334 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:26:24.974 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 12 03:26:24 smithi064 podman[70847]: 2024-09-12 03:26:24.671498692 +0000 UTC m=+0.116065176 container create 379a334559f53db66547b4c713e13426a992120fdfb8aa9605c19d65ecb01d3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-d, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:26:24.974 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 12 03:26:24 smithi064 podman[70847]: 2024-09-12 03:26:24.882451428 +0000 UTC m=+0.327017868 container init 379a334559f53db66547b4c713e13426a992120fdfb8aa9605c19d65ecb01d3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-d, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , ceph=True) 2024-09-12T03:26:24.974 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 12 03:26:24 smithi064 podman[70847]: 2024-09-12 03:26:24.88537487 +0000 UTC m=+0.329941320 container start 379a334559f53db66547b4c713e13426a992120fdfb8aa9605c19d65ecb01d3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-d, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:26:24.974 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 12 03:26:24 smithi064 ceph-mds[70866]: -- 172.21.15.64:0/3303129116 <== mon.0 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559e672ff860 con 0x559e672dc800 2024-09-12T03:26:24.975 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 12 03:26:24 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-d[70862]: starting mds.d at 2024-09-12T03:26:24.975 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 12 03:26:24 smithi064 bash[70847]: 379a334559f53db66547b4c713e13426a992120fdfb8aa9605c19d65ecb01d3e 2024-09-12T03:26:25.001 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:26:25.263 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:25.271 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 12 03:26:24 smithi064 systemd[1]: Started Ceph mds.d for b0812460-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:26:25.655 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:25 smithi064 ceph-mon[30952]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:25.655 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:25 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:25.655 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:25 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:25.655 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:25 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:25.655 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:25 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:26:25.655 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:25 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:26:25.655 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:25 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:25.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:25 smithi195 ceph-mon[34786]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:25.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:25 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:25.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:25 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:25.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:25 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:25.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:25 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:26:25.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:25 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:26:25.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:25 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:25.724 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:25 smithi115 ceph-mon[34801]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:25.725 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:25 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:25.725 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:25 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:25.725 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:25 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:25.725 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:25 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:26:25.725 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:25 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:26:25.725 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:25 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:25.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.738+0000 7f95eaa5a640 1 -- 172.21.15.64:0/808522357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95e4106f20 msgr2=0x7f95e410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:25.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.738+0000 7f95eaa5a640 1 --2- 172.21.15.64:0/808522357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95e4106f20 0x7f95e410eaa0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f95dc009ca0 tx=0x7f95dc02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:26:25.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.738+0000 7f95eaa5a640 1 -- 172.21.15.64:0/808522357 shutdown_connections 2024-09-12T03:26:25.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.738+0000 7f95eaa5a640 1 --2- 172.21.15.64:0/808522357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95e4106f20 0x7f95e410eaa0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:25.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.738+0000 7f95eaa5a640 1 --2- 172.21.15.64:0/808522357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95e41027a0 0x7f95e41067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:25.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.738+0000 7f95eaa5a640 1 --2- 172.21.15.64:0/808522357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95e4101df0 0x7f95e41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:25.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.738+0000 7f95eaa5a640 1 -- 172.21.15.64:0/808522357 >> 172.21.15.64:0/808522357 conn(0x7f95e40fd8b0 msgr2=0x7f95e40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:25.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.738+0000 7f95eaa5a640 1 -- 172.21.15.64:0/808522357 shutdown_connections 2024-09-12T03:26:25.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.738+0000 7f95eaa5a640 1 -- 172.21.15.64:0/808522357 wait complete. 2024-09-12T03:26:25.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.739+0000 7f95eaa5a640 1 Processor -- start 2024-09-12T03:26:25.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.739+0000 7f95eaa5a640 1 -- start start 2024-09-12T03:26:25.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.739+0000 7f95eaa5a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95e4101df0 0x7f95e419a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:25.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.739+0000 7f95eaa5a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95e41027a0 0x7f95e419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:25.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.739+0000 7f95eaa5a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95e4106f20 0x7f95e419ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:25.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.739+0000 7f95eaa5a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f95e4111e30 con 0x7f95e41027a0 2024-09-12T03:26:25.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.739+0000 7f95eaa5a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f95e4111cb0 con 0x7f95e4106f20 2024-09-12T03:26:25.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.739+0000 7f95eaa5a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f95e4111fb0 con 0x7f95e4101df0 2024-09-12T03:26:25.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.739+0000 7f95dbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95e41027a0 0x7f95e419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:25.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.739+0000 7f95dbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95e41027a0 0x7f95e419aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58592/0 (socket says 172.21.15.64:58592) 2024-09-12T03:26:25.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.739+0000 7f95dbfff640 1 -- 172.21.15.64:0/3187720793 learned_addr learned my addr 172.21.15.64:0/3187720793 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:26:25.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.739+0000 7f95e8ff7640 1 --2- 172.21.15.64:0/3187720793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95e4106f20 0x7f95e419ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:25.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.739+0000 7f95e3fff640 1 --2- 172.21.15.64:0/3187720793 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95e4101df0 0x7f95e419a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:25.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.739+0000 7f95dbfff640 1 -- 172.21.15.64:0/3187720793 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95e4101df0 msgr2=0x7f95e419a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:25.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.739+0000 7f95dbfff640 1 --2- 172.21.15.64:0/3187720793 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95e4101df0 0x7f95e419a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:25.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.739+0000 7f95dbfff640 1 -- 172.21.15.64:0/3187720793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95e4106f20 msgr2=0x7f95e419ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:25.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.739+0000 7f95dbfff640 1 --2- 172.21.15.64:0/3187720793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95e4106f20 0x7f95e419ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:25.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.739+0000 7f95dbfff640 1 -- 172.21.15.64:0/3187720793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e419f4e0 con 0x7f95e41027a0 2024-09-12T03:26:25.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.739+0000 7f95e8ff7640 1 --2- 172.21.15.64:0/3187720793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95e4106f20 0x7f95e419ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:26:25.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.740+0000 7f95dbfff640 1 --2- 172.21.15.64:0/3187720793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95e41027a0 0x7f95e419aa70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f95cc00e9f0 tx=0x7f95cc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:25.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.740+0000 7f95e1ffb640 1 -- 172.21.15.64:0/3187720793 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc00ccc0 con 0x7f95e41027a0 2024-09-12T03:26:25.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.740+0000 7f95e1ffb640 1 -- 172.21.15.64:0/3187720793 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95cc0108e0 con 0x7f95e41027a0 2024-09-12T03:26:25.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.740+0000 7f95eaa5a640 1 -- 172.21.15.64:0/3187720793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95e419f7d0 con 0x7f95e41027a0 2024-09-12T03:26:25.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.740+0000 7f95e1ffb640 1 -- 172.21.15.64:0/3187720793 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc0187c0 con 0x7f95e41027a0 2024-09-12T03:26:25.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.740+0000 7f95eaa5a640 1 -- 172.21.15.64:0/3187720793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95e41a70b0 con 0x7f95e41027a0 2024-09-12T03:26:25.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.741+0000 7f95eaa5a640 1 -- 172.21.15.64:0/3187720793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95e41021d0 con 0x7f95e41027a0 2024-09-12T03:26:25.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.742+0000 7f95e1ffb640 1 -- 172.21.15.64:0/3187720793 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f95cc010a90 con 0x7f95e41027a0 2024-09-12T03:26:25.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.743+0000 7f95e1ffb640 1 --2- 172.21.15.64:0/3187720793 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f95d00a2b80 0x7f95d00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:25.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.743+0000 7f95e3fff640 1 --2- 172.21.15.64:0/3187720793 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f95d00a2b80 0x7f95d00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:25.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.743+0000 7f95e1ffb640 1 -- 172.21.15.64:0/3187720793 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f95cc0ceeb0 con 0x7f95e41027a0 2024-09-12T03:26:25.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.743+0000 7f95e3fff640 1 --2- 172.21.15.64:0/3187720793 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f95d00a2b80 0x7f95d00a5040 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f95d400abe0 tx=0x7f95d400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:25.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.747+0000 7f95e1ffb640 1 -- 172.21.15.64:0/3187720793 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f95cc09c2b0 con 0x7f95e41027a0 2024-09-12T03:26:25.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.958+0000 7f95eaa5a640 1 -- 172.21.15.64:0/3187720793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f95e41a7310 con 0x7f95e41027a0 2024-09-12T03:26:25.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.958+0000 7f95e1ffb640 1 -- 172.21.15.64:0/3187720793 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3916 (secure 0 0 0) 0x7f95cc09bbb0 con 0x7f95e41027a0 2024-09-12T03:26:25.958 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:26:25.959 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":5,"btime":"2024-09-12T03:26:25:396933+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2580078738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2580078738},{"type":"v1","addr":"172.21.15.64:6835","nonce":2580078738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2333415004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2333415004},{"type":"v1","addr":"172.21.15.64:6837","nonce":2333415004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2991933266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2991933266},{"type":"v1","addr":"172.21.15.195:6833","nonce":2991933266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1138960346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1138960346},{"type":"v1","addr":"172.21.15.115:6833","nonce":1138960346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-12T03:26:25.959 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 5 2024-09-12T03:26:25.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.960+0000 7f95eaa5a640 1 -- 172.21.15.64:0/3187720793 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f95d00a2b80 msgr2=0x7f95d00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:25.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.960+0000 7f95eaa5a640 1 --2- 172.21.15.64:0/3187720793 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f95d00a2b80 0x7f95d00a5040 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f95d400abe0 tx=0x7f95d400a2d0 comp rx=0 tx=0).stop 2024-09-12T03:26:25.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.960+0000 7f95eaa5a640 1 -- 172.21.15.64:0/3187720793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95e41027a0 msgr2=0x7f95e419aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:25.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.960+0000 7f95eaa5a640 1 --2- 172.21.15.64:0/3187720793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95e41027a0 0x7f95e419aa70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f95cc00e9f0 tx=0x7f95cc00edf0 comp rx=0 tx=0).stop 2024-09-12T03:26:25.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.960+0000 7f95eaa5a640 1 -- 172.21.15.64:0/3187720793 shutdown_connections 2024-09-12T03:26:25.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.960+0000 7f95eaa5a640 1 --2- 172.21.15.64:0/3187720793 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f95d00a2b80 0x7f95d00a5040 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:25.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.960+0000 7f95eaa5a640 1 --2- 172.21.15.64:0/3187720793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95e4106f20 0x7f95e419ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:25.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.960+0000 7f95eaa5a640 1 --2- 172.21.15.64:0/3187720793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95e41027a0 0x7f95e419aa70 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:25.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.960+0000 7f95eaa5a640 1 --2- 172.21.15.64:0/3187720793 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95e4101df0 0x7f95e419a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:25.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.960+0000 7f95eaa5a640 1 -- 172.21.15.64:0/3187720793 >> 172.21.15.64:0/3187720793 conn(0x7f95e40fd8b0 msgr2=0x7f95e40febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:25.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.960+0000 7f95eaa5a640 1 -- 172.21.15.64:0/3187720793 shutdown_connections 2024-09-12T03:26:25.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:25.961+0000 7f95eaa5a640 1 -- 172.21.15.64:0/3187720793 wait complete. 2024-09-12T03:26:26.636 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:26 smithi064 ceph-mon[30952]: Deploying daemon mds.e on smithi115 2024-09-12T03:26:26.636 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:26 smithi064 ceph-mon[30952]: mds.? [v2:172.21.15.64:6836/2333415004,v1:172.21.15.64:6837/2333415004] up:boot 2024-09-12T03:26:26.636 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:26 smithi064 ceph-mon[30952]: fsmap 4 up:standby 2024-09-12T03:26:26.636 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:26 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T03:26:26.637 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:26 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/3187720793' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:26.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:26 smithi195 ceph-mon[34786]: Deploying daemon mds.e on smithi115 2024-09-12T03:26:26.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:26 smithi195 ceph-mon[34786]: mds.? [v2:172.21.15.64:6836/2333415004,v1:172.21.15.64:6837/2333415004] up:boot 2024-09-12T03:26:26.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:26 smithi195 ceph-mon[34786]: fsmap 4 up:standby 2024-09-12T03:26:26.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:26 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T03:26:26.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:26 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/3187720793' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:26.742 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:26 smithi115 ceph-mon[34801]: Deploying daemon mds.e on smithi115 2024-09-12T03:26:26.742 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:26 smithi115 ceph-mon[34801]: mds.? [v2:172.21.15.64:6836/2333415004,v1:172.21.15.64:6837/2333415004] up:boot 2024-09-12T03:26:26.742 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:26 smithi115 ceph-mon[34801]: fsmap 4 up:standby 2024-09-12T03:26:26.743 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:26 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T03:26:26.743 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:26 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/3187720793' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:27.137 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 12 03:26:26 smithi115 systemd[1]: Starting Ceph mds.e for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:26:27.392 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 12 03:26:27 smithi115 podman[63494]: 2024-09-12 03:26:27.040427928 +0000 UTC m=+0.015922772 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:26:27.393 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 12 03:26:27 smithi115 podman[63494]: 2024-09-12 03:26:27.164338185 +0000 UTC m=+0.139833029 container create e75ad4f3457546d974c8bdcfba8dcbbfac057c087cf2e3ca5c4d7fecf7a1a378 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-12T03:26:27.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:27 smithi195 ceph-mon[34786]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:27.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:27 smithi064 ceph-mon[30952]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:27.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:27 smithi115 ceph-mon[34801]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:27.893 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 12 03:26:27 smithi115 podman[63494]: 2024-09-12 03:26:27.408855656 +0000 UTC m=+0.384350500 container init e75ad4f3457546d974c8bdcfba8dcbbfac057c087cf2e3ca5c4d7fecf7a1a378 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0) 2024-09-12T03:26:27.893 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 12 03:26:27 smithi115 podman[63494]: 2024-09-12 03:26:27.411776446 +0000 UTC m=+0.387271300 container start e75ad4f3457546d974c8bdcfba8dcbbfac057c087cf2e3ca5c4d7fecf7a1a378 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:26:27.893 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 12 03:26:27 smithi115 ceph-mds[63513]: -- 172.21.15.115:0/2825759362 <== mon.2 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556134891c20 con 0x55613486e800 2024-09-12T03:26:27.893 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 12 03:26:27 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-e[63509]: starting mds.e at 2024-09-12T03:26:27.893 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 12 03:26:27 smithi115 bash[63494]: e75ad4f3457546d974c8bdcfba8dcbbfac057c087cf2e3ca5c4d7fecf7a1a378 2024-09-12T03:26:27.893 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 12 03:26:27 smithi115 systemd[1]: Started Ceph mds.e for b0812460-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:26:28.667 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:26:28.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:28 smithi115 ceph-mon[34801]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:28.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:28 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:28.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:28 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:28.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:28 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:28.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:28 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:26:28.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:28 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:26:28.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:28 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:28.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:28 smithi115 ceph-mon[34801]: Deploying daemon mds.f on smithi195 2024-09-12T03:26:28.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:28 smithi115 ceph-mon[34801]: mds.? [v2:172.21.15.115:6834/332737916,v1:172.21.15.115:6835/332737916] up:boot 2024-09-12T03:26:28.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:28 smithi115 ceph-mon[34801]: fsmap 5 up:standby 2024-09-12T03:26:28.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:28 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:26:28.940 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:28.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:28 smithi195 ceph-mon[34786]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:28.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:28 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:28.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:28 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:28.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:28 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:28.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:28 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:26:28.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:28 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:26:28.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:28 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:28.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:28 smithi195 ceph-mon[34786]: Deploying daemon mds.f on smithi195 2024-09-12T03:26:28.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:28 smithi195 ceph-mon[34786]: mds.? [v2:172.21.15.115:6834/332737916,v1:172.21.15.115:6835/332737916] up:boot 2024-09-12T03:26:28.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:28 smithi195 ceph-mon[34786]: fsmap 5 up:standby 2024-09-12T03:26:28.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:28 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:26:29.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:28 smithi064 ceph-mon[30952]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:29.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:28 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:29.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:28 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:29.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:28 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:29.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:28 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:26:29.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:28 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:26:29.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:28 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:29.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:28 smithi064 ceph-mon[30952]: Deploying daemon mds.f on smithi195 2024-09-12T03:26:29.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:28 smithi064 ceph-mon[30952]: mds.? [v2:172.21.15.115:6834/332737916,v1:172.21.15.115:6835/332737916] up:boot 2024-09-12T03:26:29.024 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:28 smithi064 ceph-mon[30952]: fsmap 5 up:standby 2024-09-12T03:26:29.024 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:28 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:26:29.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.422+0000 7ffb959c5640 1 -- 172.21.15.64:0/1191735796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb90102d40 msgr2=0x7ffb90103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:29.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.422+0000 7ffb959c5640 1 --2- 172.21.15.64:0/1191735796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb90102d40 0x7ffb90103120 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ffb80009900 tx=0x7ffb8002a480 comp rx=0 tx=0).stop 2024-09-12T03:26:29.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.422+0000 7ffb959c5640 1 -- 172.21.15.64:0/1191735796 shutdown_connections 2024-09-12T03:26:29.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.422+0000 7ffb959c5640 1 --2- 172.21.15.64:0/1191735796 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb9010c170 0x7ffb9010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:29.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.422+0000 7ffb959c5640 1 --2- 172.21.15.64:0/1191735796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb90103660 0x7ffb9010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:29.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.422+0000 7ffb959c5640 1 --2- 172.21.15.64:0/1191735796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb90102d40 0x7ffb90103120 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:29.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.422+0000 7ffb959c5640 1 -- 172.21.15.64:0/1191735796 >> 172.21.15.64:0/1191735796 conn(0x7ffb900fc7a0 msgr2=0x7ffb900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:29.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.422+0000 7ffb959c5640 1 -- 172.21.15.64:0/1191735796 shutdown_connections 2024-09-12T03:26:29.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.422+0000 7ffb959c5640 1 -- 172.21.15.64:0/1191735796 wait complete. 2024-09-12T03:26:29.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.423+0000 7ffb959c5640 1 Processor -- start 2024-09-12T03:26:29.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.423+0000 7ffb959c5640 1 -- start start 2024-09-12T03:26:29.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.423+0000 7ffb959c5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb90102d40 0x7ffb90198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:29.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.423+0000 7ffb959c5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb90103660 0x7ffb901988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:29.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.423+0000 7ffb959c5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb9010c170 0x7ffb9019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:29.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.423+0000 7ffb959c5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffb90111e70 con 0x7ffb9010c170 2024-09-12T03:26:29.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.423+0000 7ffb959c5640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ffb90111cf0 con 0x7ffb90103660 2024-09-12T03:26:29.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.423+0000 7ffb959c5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ffb90111ff0 con 0x7ffb90102d40 2024-09-12T03:26:29.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.424+0000 7ffb8f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb9010c170 0x7ffb9019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:29.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.424+0000 7ffb8f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb9010c170 0x7ffb9019cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54230/0 (socket says 172.21.15.64:54230) 2024-09-12T03:26:29.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.424+0000 7ffb8f7fe640 1 -- 172.21.15.64:0/3428938137 learned_addr learned my addr 172.21.15.64:0/3428938137 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:26:29.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.424+0000 7ffb8effd640 1 --2- 172.21.15.64:0/3428938137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb90102d40 0x7ffb90198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:29.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.424+0000 7ffb8e7fc640 1 --2- 172.21.15.64:0/3428938137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb90103660 0x7ffb901988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:29.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.424+0000 7ffb8f7fe640 1 -- 172.21.15.64:0/3428938137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb90102d40 msgr2=0x7ffb90198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:29.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.424+0000 7ffb8f7fe640 1 --2- 172.21.15.64:0/3428938137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb90102d40 0x7ffb90198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:29.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.424+0000 7ffb8f7fe640 1 -- 172.21.15.64:0/3428938137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb90103660 msgr2=0x7ffb901988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:29.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.424+0000 7ffb8f7fe640 1 --2- 172.21.15.64:0/3428938137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb90103660 0x7ffb901988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:29.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.424+0000 7ffb8f7fe640 1 -- 172.21.15.64:0/3428938137 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb9019d340 con 0x7ffb9010c170 2024-09-12T03:26:29.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.424+0000 7ffb8e7fc640 1 --2- 172.21.15.64:0/3428938137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb90103660 0x7ffb901988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:26:29.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.424+0000 7ffb8f7fe640 1 --2- 172.21.15.64:0/3428938137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb9010c170 0x7ffb9019cc60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ffb8800cce0 tx=0x7ffb8800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:29.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.424+0000 7ffb949c3640 1 -- 172.21.15.64:0/3428938137 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8800f450 con 0x7ffb9010c170 2024-09-12T03:26:29.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.424+0000 7ffb949c3640 1 -- 172.21.15.64:0/3428938137 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb8800faa0 con 0x7ffb9010c170 2024-09-12T03:26:29.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.424+0000 7ffb959c5640 1 -- 172.21.15.64:0/3428938137 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb9019d630 con 0x7ffb9010c170 2024-09-12T03:26:29.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.424+0000 7ffb949c3640 1 -- 172.21.15.64:0/3428938137 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb88018b70 con 0x7ffb9010c170 2024-09-12T03:26:29.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.424+0000 7ffb959c5640 1 -- 172.21.15.64:0/3428938137 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb901a4f10 con 0x7ffb9010c170 2024-09-12T03:26:29.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.425+0000 7ffb959c5640 1 -- 172.21.15.64:0/3428938137 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb90103120 con 0x7ffb9010c170 2024-09-12T03:26:29.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.426+0000 7ffb949c3640 1 -- 172.21.15.64:0/3428938137 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ffb88018d10 con 0x7ffb9010c170 2024-09-12T03:26:29.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.427+0000 7ffb949c3640 1 --2- 172.21.15.64:0/3428938137 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ffb780a2c00 0x7ffb780a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:29.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.427+0000 7ffb8effd640 1 --2- 172.21.15.64:0/3428938137 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ffb780a2c00 0x7ffb780a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:29.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.428+0000 7ffb949c3640 1 -- 172.21.15.64:0/3428938137 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7ffb880ceef0 con 0x7ffb9010c170 2024-09-12T03:26:29.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.428+0000 7ffb8effd640 1 --2- 172.21.15.64:0/3428938137 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ffb780a2c00 0x7ffb780a50c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ffb80005ab0 tx=0x7ffb80005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:29.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.432+0000 7ffb949c3640 1 -- 172.21.15.64:0/3428938137 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffb88014030 con 0x7ffb9010c170 2024-09-12T03:26:29.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.647+0000 7ffb959c5640 1 -- 172.21.15.64:0/3428938137 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb9002d7c0 con 0x7ffb9010c170 2024-09-12T03:26:29.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.647+0000 7ffb949c3640 1 -- 172.21.15.64:0/3428938137 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4741 (secure 0 0 0) 0x7ffb8809c2f0 con 0x7ffb9010c170 2024-09-12T03:26:29.648 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:26:29.648 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":6,"btime":"2024-09-12T03:26:28:412382+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2580078738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2580078738},{"type":"v1","addr":"172.21.15.64:6835","nonce":2580078738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2333415004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2333415004},{"type":"v1","addr":"172.21.15.64:6837","nonce":2333415004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2991933266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2991933266},{"type":"v1","addr":"172.21.15.195:6833","nonce":2991933266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1138960346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1138960346},{"type":"v1","addr":"172.21.15.115:6833","nonce":1138960346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/332737916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":332737916},{"type":"v1","addr":"172.21.15.115:6835","nonce":332737916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-12T03:26:29.648 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 6 2024-09-12T03:26:29.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.650+0000 7ffb959c5640 1 -- 172.21.15.64:0/3428938137 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ffb780a2c00 msgr2=0x7ffb780a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:29.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.650+0000 7ffb959c5640 1 --2- 172.21.15.64:0/3428938137 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ffb780a2c00 0x7ffb780a50c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ffb80005ab0 tx=0x7ffb80005a20 comp rx=0 tx=0).stop 2024-09-12T03:26:29.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.650+0000 7ffb959c5640 1 -- 172.21.15.64:0/3428938137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb9010c170 msgr2=0x7ffb9019cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:29.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.650+0000 7ffb959c5640 1 --2- 172.21.15.64:0/3428938137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb9010c170 0x7ffb9019cc60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ffb8800cce0 tx=0x7ffb8800a620 comp rx=0 tx=0).stop 2024-09-12T03:26:29.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.650+0000 7ffb959c5640 1 -- 172.21.15.64:0/3428938137 shutdown_connections 2024-09-12T03:26:29.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.650+0000 7ffb959c5640 1 --2- 172.21.15.64:0/3428938137 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ffb780a2c00 0x7ffb780a50c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:29.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.650+0000 7ffb959c5640 1 --2- 172.21.15.64:0/3428938137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb9010c170 0x7ffb9019cc60 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:29.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.650+0000 7ffb959c5640 1 --2- 172.21.15.64:0/3428938137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb90103660 0x7ffb901988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:29.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.650+0000 7ffb959c5640 1 --2- 172.21.15.64:0/3428938137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb90102d40 0x7ffb90198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:29.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.650+0000 7ffb959c5640 1 -- 172.21.15.64:0/3428938137 >> 172.21.15.64:0/3428938137 conn(0x7ffb900fc7a0 msgr2=0x7ffb900fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:29.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.650+0000 7ffb959c5640 1 -- 172.21.15.64:0/3428938137 shutdown_connections 2024-09-12T03:26:29.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:29.650+0000 7ffb959c5640 1 -- 172.21.15.64:0/3428938137 wait complete. 2024-09-12T03:26:29.677 INFO:journalctl@ceph.mds.f.smithi195.stdout:Sep 12 03:26:29 smithi195 systemd[1]: Starting Ceph mds.f for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:26:29.956 INFO:journalctl@ceph.mds.f.smithi195.stdout:Sep 12 03:26:29 smithi195 podman[64513]: 2024-09-12 03:26:29.579766418 +0000 UTC m=+0.015738181 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:26:29.956 INFO:journalctl@ceph.mds.f.smithi195.stdout:Sep 12 03:26:29 smithi195 podman[64513]: 2024-09-12 03:26:29.71440002 +0000 UTC m=+0.150371773 container create ccaea7d954ed2cec35084927ab1806dc44b6fcdbe0aebf1253005ebad4f86cef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-f, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:26:30.456 INFO:journalctl@ceph.mds.f.smithi195.stdout:Sep 12 03:26:30 smithi195 podman[64513]: 2024-09-12 03:26:30.009521887 +0000 UTC m=+0.445493648 container init ccaea7d954ed2cec35084927ab1806dc44b6fcdbe0aebf1253005ebad4f86cef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-f, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-12T03:26:30.457 INFO:journalctl@ceph.mds.f.smithi195.stdout:Sep 12 03:26:30 smithi195 podman[64513]: 2024-09-12 03:26:30.012328974 +0000 UTC m=+0.448300728 container start ccaea7d954ed2cec35084927ab1806dc44b6fcdbe0aebf1253005ebad4f86cef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-f, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3) 2024-09-12T03:26:30.457 INFO:journalctl@ceph.mds.f.smithi195.stdout:Sep 12 03:26:30 smithi195 ceph-mds[64532]: -- 172.21.15.195:0/2164644202 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55854467f860 con 0x55854465c800 2024-09-12T03:26:30.457 INFO:journalctl@ceph.mds.f.smithi195.stdout:Sep 12 03:26:30 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-f[64528]: starting mds.f at 2024-09-12T03:26:30.457 INFO:journalctl@ceph.mds.f.smithi195.stdout:Sep 12 03:26:30 smithi195 bash[64513]: ccaea7d954ed2cec35084927ab1806dc44b6fcdbe0aebf1253005ebad4f86cef 2024-09-12T03:26:30.457 INFO:journalctl@ceph.mds.f.smithi195.stdout:Sep 12 03:26:30 smithi195 systemd[1]: Started Ceph mds.f for b0812460-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:26:30.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:30 smithi064 ceph-mon[30952]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:30.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:30 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/3428938137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:30.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:30 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:30.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:30 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:30.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:30 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:30.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:30 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:26:30.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:30 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:26:30.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:30 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:30.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:30 smithi064 ceph-mon[30952]: Deploying daemon mds.c on smithi195 2024-09-12T03:26:30.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:30 smithi115 ceph-mon[34801]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:30.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:30 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/3428938137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:30.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:30 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:30.894 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:30 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:30.894 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:30 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:30.894 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:30 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:26:30.894 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:30 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:26:30.894 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:30 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:30.894 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:30 smithi115 ceph-mon[34801]: Deploying daemon mds.c on smithi195 2024-09-12T03:26:30.913 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:30 smithi195 ceph-mon[34786]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:30.913 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:30 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/3428938137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:30.913 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:30 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:30.914 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:30 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:30.914 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:30 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:30.914 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:30 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:26:30.914 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:30 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:26:30.914 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:30 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:30.914 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:30 smithi195 ceph-mon[34786]: Deploying daemon mds.c on smithi195 2024-09-12T03:26:31.870 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:31 smithi195 ceph-mon[34786]: mds.? [v2:172.21.15.195:6834/2531045583,v1:172.21.15.195:6835/2531045583] up:boot 2024-09-12T03:26:31.870 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:31 smithi195 ceph-mon[34786]: fsmap 6 up:standby 2024-09-12T03:26:31.870 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:31 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:26:31.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:31 smithi115 ceph-mon[34801]: mds.? [v2:172.21.15.195:6834/2531045583,v1:172.21.15.195:6835/2531045583] up:boot 2024-09-12T03:26:31.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:31 smithi115 ceph-mon[34801]: fsmap 6 up:standby 2024-09-12T03:26:31.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:31 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:26:32.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:31 smithi064 ceph-mon[30952]: mds.? [v2:172.21.15.195:6834/2531045583,v1:172.21.15.195:6835/2531045583] up:boot 2024-09-12T03:26:32.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:31 smithi064 ceph-mon[30952]: fsmap 6 up:standby 2024-09-12T03:26:32.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:31 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:26:32.207 INFO:journalctl@ceph.mds.c.smithi195.stdout:Sep 12 03:26:32 smithi195 systemd[1]: Starting Ceph mds.c for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:26:32.350 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:26:32.579 INFO:journalctl@ceph.mds.c.smithi195.stdout:Sep 12 03:26:32 smithi195 podman[64812]: 2024-09-12 03:26:32.165175764 +0000 UTC m=+0.016006344 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:26:32.579 INFO:journalctl@ceph.mds.c.smithi195.stdout:Sep 12 03:26:32 smithi195 podman[64812]: 2024-09-12 03:26:32.295073784 +0000 UTC m=+0.145904397 container create 1c5adae5c26de23ebc8ca5c3fbcb3410b5e82f95d73cc28190cd84db98806983 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-c, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, ceph=True, GIT_BRANCH=HEAD) 2024-09-12T03:26:32.619 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:32.740 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:32 smithi064 ceph-mon[30952]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:32.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:32 smithi115 ceph-mon[34801]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:32.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:32 smithi195 ceph-mon[34786]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:32.957 INFO:journalctl@ceph.mds.c.smithi195.stdout:Sep 12 03:26:32 smithi195 podman[64812]: 2024-09-12 03:26:32.581008194 +0000 UTC m=+0.431838771 container init 1c5adae5c26de23ebc8ca5c3fbcb3410b5e82f95d73cc28190cd84db98806983 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-12T03:26:32.957 INFO:journalctl@ceph.mds.c.smithi195.stdout:Sep 12 03:26:32 smithi195 podman[64812]: 2024-09-12 03:26:32.583975177 +0000 UTC m=+0.434805746 container start 1c5adae5c26de23ebc8ca5c3fbcb3410b5e82f95d73cc28190cd84db98806983 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-c, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-12T03:26:32.957 INFO:journalctl@ceph.mds.c.smithi195.stdout:Sep 12 03:26:32 smithi195 ceph-mds[64832]: -- 172.21.15.195:0/2662069111 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dd08acd860 con 0x55dd08aab000 2024-09-12T03:26:32.957 INFO:journalctl@ceph.mds.c.smithi195.stdout:Sep 12 03:26:32 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-c[64828]: starting mds.c at 2024-09-12T03:26:32.957 INFO:journalctl@ceph.mds.c.smithi195.stdout:Sep 12 03:26:32 smithi195 bash[64812]: 1c5adae5c26de23ebc8ca5c3fbcb3410b5e82f95d73cc28190cd84db98806983 2024-09-12T03:26:32.957 INFO:journalctl@ceph.mds.c.smithi195.stdout:Sep 12 03:26:32 smithi195 systemd[1]: Started Ceph mds.c for b0812460-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:26:33.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.114+0000 7efeb9b56640 1 -- 172.21.15.64:0/3038877563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efeb4100e20 msgr2=0x7efeb4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:33.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.114+0000 7efeb9b56640 1 --2- 172.21.15.64:0/3038877563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efeb4100e20 0x7efeb4101200 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efea4009900 tx=0x7efea402a480 comp rx=0 tx=0).stop 2024-09-12T03:26:33.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.114+0000 7efeb9b56640 1 -- 172.21.15.64:0/3038877563 shutdown_connections 2024-09-12T03:26:33.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.114+0000 7efeb9b56640 1 --2- 172.21.15.64:0/3038877563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efeb4102ef0 0x7efeb410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:33.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.114+0000 7efeb9b56640 1 --2- 172.21.15.64:0/3038877563 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efeb4102570 0x7efeb41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:33.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.114+0000 7efeb9b56640 1 --2- 172.21.15.64:0/3038877563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efeb4100e20 0x7efeb4101200 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:33.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.114+0000 7efeb9b56640 1 -- 172.21.15.64:0/3038877563 >> 172.21.15.64:0/3038877563 conn(0x7efeb40fc7d0 msgr2=0x7efeb40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:33.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.114+0000 7efeb9b56640 1 -- 172.21.15.64:0/3038877563 shutdown_connections 2024-09-12T03:26:33.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.114+0000 7efeb9b56640 1 -- 172.21.15.64:0/3038877563 wait complete. 2024-09-12T03:26:33.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.115+0000 7efeb9b56640 1 Processor -- start 2024-09-12T03:26:33.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.115+0000 7efeb9b56640 1 -- start start 2024-09-12T03:26:33.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.115+0000 7efeb9b56640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efeb4100e20 0x7efeb4103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:33.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.115+0000 7efeb9b56640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efeb4102570 0x7efeb4103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:33.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.115+0000 7efeb9b56640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efeb4102ef0 0x7efeb41041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:33.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.115+0000 7efeb9b56640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efeb4110e30 con 0x7efeb4102570 2024-09-12T03:26:33.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.115+0000 7efeb9b56640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efeb4110cb0 con 0x7efeb4102ef0 2024-09-12T03:26:33.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.115+0000 7efeb9b56640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efeb4110fb0 con 0x7efeb4100e20 2024-09-12T03:26:33.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.116+0000 7efeb2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efeb4102570 0x7efeb4103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:33.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.116+0000 7efeb37fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efeb4100e20 0x7efeb4103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:33.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.116+0000 7efeb2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efeb4102570 0x7efeb4103c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54246/0 (socket says 172.21.15.64:54246) 2024-09-12T03:26:33.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.116+0000 7efeb37fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efeb4100e20 0x7efeb4103750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.64:55232/0 (socket says 172.21.15.64:55232) 2024-09-12T03:26:33.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.116+0000 7efeb2ffd640 1 -- 172.21.15.64:0/1696971302 learned_addr learned my addr 172.21.15.64:0/1696971302 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:26:33.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.116+0000 7efeb3fff640 1 --2- 172.21.15.64:0/1696971302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efeb4102ef0 0x7efeb41041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:33.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.116+0000 7efeb2ffd640 1 -- 172.21.15.64:0/1696971302 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efeb4100e20 msgr2=0x7efeb4103750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:33.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.116+0000 7efeb2ffd640 1 --2- 172.21.15.64:0/1696971302 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efeb4100e20 0x7efeb4103750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:33.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.116+0000 7efeb2ffd640 1 -- 172.21.15.64:0/1696971302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efeb4102ef0 msgr2=0x7efeb41041d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:33.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.116+0000 7efeb2ffd640 1 --2- 172.21.15.64:0/1696971302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efeb4102ef0 0x7efeb41041d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:33.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.116+0000 7efeb2ffd640 1 -- 172.21.15.64:0/1696971302 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb41a6900 con 0x7efeb4102570 2024-09-12T03:26:33.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.116+0000 7efeb2ffd640 1 --2- 172.21.15.64:0/1696971302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efeb4102570 0x7efeb4103c90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7efea800d950 tx=0x7efea800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:33.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.116+0000 7efeb0ff9640 1 -- 172.21.15.64:0/1696971302 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea8017070 con 0x7efeb4102570 2024-09-12T03:26:33.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.116+0000 7efeb0ff9640 1 -- 172.21.15.64:0/1696971302 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efea8009c30 con 0x7efeb4102570 2024-09-12T03:26:33.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.116+0000 7efeb9b56640 1 -- 172.21.15.64:0/1696971302 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeb41a6bf0 con 0x7efeb4102570 2024-09-12T03:26:33.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.116+0000 7efeb0ff9640 1 -- 172.21.15.64:0/1696971302 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea800f620 con 0x7efeb4102570 2024-09-12T03:26:33.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.116+0000 7efeb9b56640 1 -- 172.21.15.64:0/1696971302 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeb41a7180 con 0x7efeb4102570 2024-09-12T03:26:33.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.117+0000 7efeb9b56640 1 -- 172.21.15.64:0/1696971302 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeb4101200 con 0x7efeb4102570 2024-09-12T03:26:33.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.119+0000 7efeb0ff9640 1 -- 172.21.15.64:0/1696971302 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7efea80109a0 con 0x7efeb4102570 2024-09-12T03:26:33.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.120+0000 7efeb0ff9640 1 --2- 172.21.15.64:0/1696971302 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7efe9c0a2c00 0x7efe9c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:33.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.120+0000 7efeb37fe640 1 --2- 172.21.15.64:0/1696971302 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7efe9c0a2c00 0x7efe9c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:33.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.120+0000 7efeb0ff9640 1 -- 172.21.15.64:0/1696971302 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7efea80ce700 con 0x7efeb4102570 2024-09-12T03:26:33.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.120+0000 7efeb37fe640 1 --2- 172.21.15.64:0/1696971302 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7efe9c0a2c00 0x7efe9c0a50c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efea4009230 tx=0x7efea40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:33.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.123+0000 7efeb0ff9640 1 -- 172.21.15.64:0/1696971302 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efea8009de0 con 0x7efeb4102570 2024-09-12T03:26:33.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.339+0000 7efeb9b56640 1 -- 172.21.15.64:0/1696971302 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efeb4104c80 con 0x7efeb4102570 2024-09-12T03:26:33.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.340+0000 7efeb0ff9640 1 -- 172.21.15.64:0/1696971302 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5569 (secure 0 0 0) 0x7efeb4104c80 con 0x7efeb4102570 2024-09-12T03:26:33.341 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:26:33.341 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":7,"btime":"2024-09-12T03:26:30:559899+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2580078738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2580078738},{"type":"v1","addr":"172.21.15.64:6835","nonce":2580078738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2333415004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2333415004},{"type":"v1","addr":"172.21.15.64:6837","nonce":2333415004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2991933266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2991933266},{"type":"v1","addr":"172.21.15.195:6833","nonce":2991933266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1138960346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1138960346},{"type":"v1","addr":"172.21.15.115:6833","nonce":1138960346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/332737916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":332737916},{"type":"v1","addr":"172.21.15.115:6835","nonce":332737916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2531045583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2531045583},{"type":"v1","addr":"172.21.15.195:6835","nonce":2531045583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-09-12T03:26:33.342 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 7 2024-09-12T03:26:33.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.342+0000 7efeb9b56640 1 -- 172.21.15.64:0/1696971302 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7efe9c0a2c00 msgr2=0x7efe9c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:33.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.342+0000 7efeb9b56640 1 --2- 172.21.15.64:0/1696971302 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7efe9c0a2c00 0x7efe9c0a50c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efea4009230 tx=0x7efea40058d0 comp rx=0 tx=0).stop 2024-09-12T03:26:33.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.342+0000 7efeb9b56640 1 -- 172.21.15.64:0/1696971302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efeb4102570 msgr2=0x7efeb4103c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:33.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.342+0000 7efeb9b56640 1 --2- 172.21.15.64:0/1696971302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efeb4102570 0x7efeb4103c90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7efea800d950 tx=0x7efea800de10 comp rx=0 tx=0).stop 2024-09-12T03:26:33.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.342+0000 7efeb9b56640 1 -- 172.21.15.64:0/1696971302 shutdown_connections 2024-09-12T03:26:33.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.343+0000 7efeb9b56640 1 --2- 172.21.15.64:0/1696971302 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7efe9c0a2c00 0x7efe9c0a50c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:33.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.343+0000 7efeb9b56640 1 --2- 172.21.15.64:0/1696971302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efeb4102ef0 0x7efeb41041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:33.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.343+0000 7efeb9b56640 1 --2- 172.21.15.64:0/1696971302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efeb4102570 0x7efeb4103c90 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:33.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.343+0000 7efeb9b56640 1 --2- 172.21.15.64:0/1696971302 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efeb4100e20 0x7efeb4103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:33.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.343+0000 7efeb9b56640 1 -- 172.21.15.64:0/1696971302 >> 172.21.15.64:0/1696971302 conn(0x7efeb40fc7d0 msgr2=0x7efeb406a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:33.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.343+0000 7efeb9b56640 1 -- 172.21.15.64:0/1696971302 shutdown_connections 2024-09-12T03:26:33.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:33.343+0000 7efeb9b56640 1 -- 172.21.15.64:0/1696971302 wait complete. 2024-09-12T03:26:34.005 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:33 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:34.005 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:33 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:34.005 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:33 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:34.005 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:33 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:26:34.005 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:33 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:26:34.005 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:33 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:34.006 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:33 smithi064 ceph-mon[30952]: Deploying daemon mds.h on smithi115 2024-09-12T03:26:34.006 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:33 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/1696971302' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:34.006 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:33 smithi064 ceph-mon[30952]: mds.? [v2:172.21.15.195:6836/71647401,v1:172.21.15.195:6837/71647401] up:boot 2024-09-12T03:26:34.006 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:33 smithi064 ceph-mon[30952]: fsmap 7 up:standby 2024-09-12T03:26:34.006 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:33 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:26:34.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:33 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:34.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:33 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:34.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:33 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:34.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:33 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:26:34.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:33 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:26:34.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:33 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:34.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:33 smithi115 ceph-mon[34801]: Deploying daemon mds.h on smithi115 2024-09-12T03:26:34.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:33 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/1696971302' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:34.144 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:33 smithi115 ceph-mon[34801]: mds.? [v2:172.21.15.195:6836/71647401,v1:172.21.15.195:6837/71647401] up:boot 2024-09-12T03:26:34.144 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:33 smithi115 ceph-mon[34801]: fsmap 7 up:standby 2024-09-12T03:26:34.144 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:33 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:26:34.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:33 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:34.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:33 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:34.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:33 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:34.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:33 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:26:34.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:33 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:26:34.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:33 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:34.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:33 smithi195 ceph-mon[34786]: Deploying daemon mds.h on smithi115 2024-09-12T03:26:34.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:33 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/1696971302' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:34.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:33 smithi195 ceph-mon[34786]: mds.? [v2:172.21.15.195:6836/71647401,v1:172.21.15.195:6837/71647401] up:boot 2024-09-12T03:26:34.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:33 smithi195 ceph-mon[34786]: fsmap 7 up:standby 2024-09-12T03:26:34.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:33 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:26:34.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:34 smithi115 ceph-mon[34801]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:34.894 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 12 03:26:34 smithi115 systemd[1]: Starting Ceph mds.h for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:26:35.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:34 smithi064 ceph-mon[30952]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:35.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:34 smithi195 ceph-mon[34786]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:35.256 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 12 03:26:34 smithi115 podman[63795]: 2024-09-12 03:26:34.882461288 +0000 UTC m=+0.016079929 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:26:35.256 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 12 03:26:35 smithi115 podman[63795]: 2024-09-12 03:26:35.004206736 +0000 UTC m=+0.137825380 container create d0ac6da922dcbe63728c1d2bdbc27d07befd2d84980bc17af774f35fbfcff5d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-h, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-12T03:26:35.257 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 12 03:26:35 smithi115 podman[63795]: 2024-09-12 03:26:35.257072972 +0000 UTC m=+0.390691614 container init d0ac6da922dcbe63728c1d2bdbc27d07befd2d84980bc17af774f35fbfcff5d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-h, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:26:35.643 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 12 03:26:35 smithi115 podman[63795]: 2024-09-12 03:26:35.259883012 +0000 UTC m=+0.393501653 container start d0ac6da922dcbe63728c1d2bdbc27d07befd2d84980bc17af774f35fbfcff5d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-h, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:26:35.643 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 12 03:26:35 smithi115 ceph-mds[63814]: -- 172.21.15.115:0/4203847833 <== mon.2 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5598c5819860 con 0x5598c57f7000 2024-09-12T03:26:35.643 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 12 03:26:35 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-h[63810]: starting mds.h at 2024-09-12T03:26:35.643 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 12 03:26:35 smithi115 bash[63795]: d0ac6da922dcbe63728c1d2bdbc27d07befd2d84980bc17af774f35fbfcff5d5 2024-09-12T03:26:35.643 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 12 03:26:35 smithi115 systemd[1]: Started Ceph mds.h for b0812460-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:26:36.034 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:26:36.306 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:36.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:36 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:36.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:36 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:36.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:36 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:36.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:36 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:26:36.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:36 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:26:36.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:36 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:36.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:36 smithi195 ceph-mon[34786]: mds.? [v2:172.21.15.115:6836/2915155705,v1:172.21.15.115:6837/2915155705] up:boot 2024-09-12T03:26:36.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:36 smithi195 ceph-mon[34786]: fsmap 8 up:standby 2024-09-12T03:26:36.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:36 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:26:36.700 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:36 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:36.700 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:36 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:36.701 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:36 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:36.701 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:36 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:26:36.701 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:36 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:26:36.701 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:36 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:36.701 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:36 smithi064 ceph-mon[30952]: mds.? [v2:172.21.15.115:6836/2915155705,v1:172.21.15.115:6837/2915155705] up:boot 2024-09-12T03:26:36.701 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:36 smithi064 ceph-mon[30952]: fsmap 8 up:standby 2024-09-12T03:26:36.701 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:36 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:26:36.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:36.782+0000 7f802facc640 1 -- 172.21.15.64:0/917411113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f802810e480 msgr2=0x7f8028110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:36.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:36.782+0000 7f802facc640 1 --2- 172.21.15.64:0/917411113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f802810e480 0x7f8028110870 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8020009900 tx=0x7f802002a410 comp rx=0 tx=0).stop 2024-09-12T03:26:36.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:36.782+0000 7f802facc640 1 -- 172.21.15.64:0/917411113 shutdown_connections 2024-09-12T03:26:36.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:36.782+0000 7f802facc640 1 --2- 172.21.15.64:0/917411113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f802810e480 0x7f8028110870 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:36.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:36.782+0000 7f802facc640 1 --2- 172.21.15.64:0/917411113 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8028101280 0x7f802810df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:36.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:36.782+0000 7f802facc640 1 --2- 172.21.15.64:0/917411113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80281008d0 0x7f8028100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:36.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:36.782+0000 7f802facc640 1 -- 172.21.15.64:0/917411113 >> 172.21.15.64:0/917411113 conn(0x7f80280fc7a0 msgr2=0x7f80280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:36.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:36.782+0000 7f802facc640 1 -- 172.21.15.64:0/917411113 shutdown_connections 2024-09-12T03:26:36.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:36.782+0000 7f802facc640 1 -- 172.21.15.64:0/917411113 wait complete. 2024-09-12T03:26:36.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:36.782+0000 7f802facc640 1 Processor -- start 2024-09-12T03:26:36.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:36.783+0000 7f802facc640 1 -- start start 2024-09-12T03:26:36.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:36.783+0000 7f802facc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80281008d0 0x7f802819a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:36.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:36.783+0000 7f802facc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8028101280 0x7f802819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:36.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:36.783+0000 7f802facc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f802810e480 0x7f802819ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:36.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:36.783+0000 7f802facc640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8028114010 con 0x7f80281008d0 2024-09-12T03:26:36.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:36.783+0000 7f802facc640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8028113e90 con 0x7f8028101280 2024-09-12T03:26:36.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:36.783+0000 7f802facc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8028114190 con 0x7f802810e480 2024-09-12T03:26:36.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:36.783+0000 7f802d868640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80281008d0 0x7f802819a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:36.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:36.783+0000 7f802d868640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80281008d0 0x7f802819a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54264/0 (socket says 172.21.15.64:54264) 2024-09-12T03:26:36.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:36.783+0000 7f802d868640 1 -- 172.21.15.64:0/97054514 learned_addr learned my addr 172.21.15.64:0/97054514 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:26:36.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:36.783+0000 7f802d067640 1 --2- 172.21.15.64:0/97054514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8028101280 0x7f802819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:36.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:36.783+0000 7f802e069640 1 --2- 172.21.15.64:0/97054514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f802810e480 0x7f802819ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:36.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:36.783+0000 7f802d868640 1 -- 172.21.15.64:0/97054514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f802810e480 msgr2=0x7f802819ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:36.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:36.783+0000 7f802d868640 1 --2- 172.21.15.64:0/97054514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f802810e480 0x7f802819ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:36.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:36.783+0000 7f802d868640 1 -- 172.21.15.64:0/97054514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8028101280 msgr2=0x7f802819aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:36.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:36.783+0000 7f802d868640 1 --2- 172.21.15.64:0/97054514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8028101280 0x7f802819aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:36.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:36.783+0000 7f802d868640 1 -- 172.21.15.64:0/97054514 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f802819f520 con 0x7f80281008d0 2024-09-12T03:26:36.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:36.783+0000 7f802e069640 1 --2- 172.21.15.64:0/97054514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f802810e480 0x7f802819ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:26:36.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:36.783+0000 7f802d868640 1 --2- 172.21.15.64:0/97054514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80281008d0 0x7f802819a570 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f802400db80 tx=0x7f802400b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:36.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:36.784+0000 7f801effd640 1 -- 172.21.15.64:0/97054514 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8024017070 con 0x7f80281008d0 2024-09-12T03:26:36.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:36.784+0000 7f801effd640 1 -- 172.21.15.64:0/97054514 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8024003b70 con 0x7f80281008d0 2024-09-12T03:26:36.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:36.784+0000 7f802facc640 1 -- 172.21.15.64:0/97054514 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f802819f810 con 0x7f80281008d0 2024-09-12T03:26:36.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:36.784+0000 7f801effd640 1 -- 172.21.15.64:0/97054514 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80240104c0 con 0x7f80281008d0 2024-09-12T03:26:36.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:36.784+0000 7f802facc640 1 -- 172.21.15.64:0/97054514 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80281a70f0 con 0x7f80281008d0 2024-09-12T03:26:36.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:36.785+0000 7f802facc640 1 -- 172.21.15.64:0/97054514 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8028100cb0 con 0x7f80281008d0 2024-09-12T03:26:36.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:36.786+0000 7f801effd640 1 -- 172.21.15.64:0/97054514 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8024004020 con 0x7f80281008d0 2024-09-12T03:26:36.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:36.786+0000 7f801effd640 1 --2- 172.21.15.64:0/97054514 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f80140a2b80 0x7f80140a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:36.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:36.786+0000 7f802d067640 1 --2- 172.21.15.64:0/97054514 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f80140a2b80 0x7f80140a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:36.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:36.787+0000 7f801effd640 1 -- 172.21.15.64:0/97054514 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f80240cec80 con 0x7f80281008d0 2024-09-12T03:26:36.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:36.787+0000 7f802d067640 1 --2- 172.21.15.64:0/97054514 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f80140a2b80 0x7f80140a5040 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f802810e8a0 tx=0x7f801800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:36.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:36.791+0000 7f801effd640 1 -- 172.21.15.64:0/97054514 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f802409c080 con 0x7f80281008d0 2024-09-12T03:26:36.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:36 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:36.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:36 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:36.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:36 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:36.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:36 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:26:36.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:36 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:26:36.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:36 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:36.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:36 smithi115 ceph-mon[34801]: mds.? [v2:172.21.15.115:6836/2915155705,v1:172.21.15.115:6837/2915155705] up:boot 2024-09-12T03:26:36.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:36 smithi115 ceph-mon[34801]: fsmap 8 up:standby 2024-09-12T03:26:36.894 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:36 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:26:37.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:37.002+0000 7f802facc640 1 -- 172.21.15.64:0/97054514 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f802819b8d0 con 0x7f80281008d0 2024-09-12T03:26:37.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:37.003+0000 7f801effd640 1 -- 172.21.15.64:0/97054514 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7219 (secure 0 0 0) 0x7f802409b980 con 0x7f80281008d0 2024-09-12T03:26:37.004 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:26:37.005 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":9,"btime":"2024-09-12T03:26:35:757760+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2580078738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2580078738},{"type":"v1","addr":"172.21.15.64:6835","nonce":2580078738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2333415004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2333415004},{"type":"v1","addr":"172.21.15.64:6837","nonce":2333415004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2991933266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2991933266},{"type":"v1","addr":"172.21.15.195:6833","nonce":2991933266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1138960346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1138960346},{"type":"v1","addr":"172.21.15.115:6833","nonce":1138960346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/332737916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":332737916},{"type":"v1","addr":"172.21.15.115:6835","nonce":332737916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2531045583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2531045583},{"type":"v1","addr":"172.21.15.195:6835","nonce":2531045583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/71647401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":71647401},{"type":"v1","addr":"172.21.15.195:6837","nonce":71647401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2915155705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2915155705},{"type":"v1","addr":"172.21.15.115:6837","nonce":2915155705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-12T03:26:37.005 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 9 2024-09-12T03:26:37.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:37.005+0000 7f802facc640 1 -- 172.21.15.64:0/97054514 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f80140a2b80 msgr2=0x7f80140a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:37.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:37.005+0000 7f802facc640 1 --2- 172.21.15.64:0/97054514 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f80140a2b80 0x7f80140a5040 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f802810e8a0 tx=0x7f801800b390 comp rx=0 tx=0).stop 2024-09-12T03:26:37.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:37.005+0000 7f802facc640 1 -- 172.21.15.64:0/97054514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80281008d0 msgr2=0x7f802819a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:37.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:37.005+0000 7f802facc640 1 --2- 172.21.15.64:0/97054514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80281008d0 0x7f802819a570 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f802400db80 tx=0x7f802400b630 comp rx=0 tx=0).stop 2024-09-12T03:26:37.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:37.005+0000 7f802facc640 1 -- 172.21.15.64:0/97054514 shutdown_connections 2024-09-12T03:26:37.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:37.005+0000 7f802facc640 1 --2- 172.21.15.64:0/97054514 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f80140a2b80 0x7f80140a5040 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:37.005+0000 7f802facc640 1 --2- 172.21.15.64:0/97054514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f802810e480 0x7f802819ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:37.005+0000 7f802facc640 1 --2- 172.21.15.64:0/97054514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8028101280 0x7f802819aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:37.005+0000 7f802facc640 1 --2- 172.21.15.64:0/97054514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80281008d0 0x7f802819a570 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:37.005+0000 7f802facc640 1 -- 172.21.15.64:0/97054514 >> 172.21.15.64:0/97054514 conn(0x7f80280fc7a0 msgr2=0x7f802810f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:37.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:37.006+0000 7f802facc640 1 -- 172.21.15.64:0/97054514 shutdown_connections 2024-09-12T03:26:37.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:37.006+0000 7f802facc640 1 -- 172.21.15.64:0/97054514 wait complete. 2024-09-12T03:26:37.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:37 smithi195 ceph-mon[34786]: Deploying daemon mds.i on smithi195 2024-09-12T03:26:37.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:37 smithi195 ceph-mon[34786]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:37.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:37 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/97054514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:37.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:37 smithi064 ceph-mon[30952]: Deploying daemon mds.i on smithi195 2024-09-12T03:26:37.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:37 smithi064 ceph-mon[30952]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:37.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:37 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/97054514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:37.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:37 smithi115 ceph-mon[34801]: Deploying daemon mds.i on smithi195 2024-09-12T03:26:37.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:37 smithi115 ceph-mon[34801]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:37.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:37 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/97054514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:38.206 INFO:journalctl@ceph.mds.i.smithi195.stdout:Sep 12 03:26:37 smithi195 systemd[1]: Started Ceph mds.i for b0812460-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:26:39.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:38 smithi064 ceph-mon[30952]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:39.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:38 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:39.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:38 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:39.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:38 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:39.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:38 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:26:39.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:38 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:26:39.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:38 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:39.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:38 smithi064 ceph-mon[30952]: Deploying daemon mds.g on smithi064 2024-09-12T03:26:39.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:38 smithi064 ceph-mon[30952]: mds.? [v2:172.21.15.195:6838/2574695134,v1:172.21.15.195:6839/2574695134] up:boot 2024-09-12T03:26:39.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:38 smithi064 ceph-mon[30952]: fsmap 9 up:standby 2024-09-12T03:26:39.274 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:38 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:26:39.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:38 smithi115 ceph-mon[34801]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:39.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:38 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:39.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:38 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:39.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:38 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:39.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:38 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:26:39.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:38 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:26:39.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:38 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:39.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:38 smithi115 ceph-mon[34801]: Deploying daemon mds.g on smithi064 2024-09-12T03:26:39.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:38 smithi115 ceph-mon[34801]: mds.? [v2:172.21.15.195:6838/2574695134,v1:172.21.15.195:6839/2574695134] up:boot 2024-09-12T03:26:39.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:38 smithi115 ceph-mon[34801]: fsmap 9 up:standby 2024-09-12T03:26:39.394 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:38 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:26:39.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:38 smithi195 ceph-mon[34786]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:39.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:38 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:39.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:38 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:39.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:38 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:39.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:38 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:26:39.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:38 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:26:39.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:38 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:39.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:38 smithi195 ceph-mon[34786]: Deploying daemon mds.g on smithi064 2024-09-12T03:26:39.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:38 smithi195 ceph-mon[34786]: mds.? [v2:172.21.15.195:6838/2574695134,v1:172.21.15.195:6839/2574695134] up:boot 2024-09-12T03:26:39.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:38 smithi195 ceph-mon[34786]: fsmap 9 up:standby 2024-09-12T03:26:39.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:38 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:26:39.671 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:26:40.104 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:40.111 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 12 03:26:40 smithi064 systemd[1]: Starting Ceph mds.g for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:26:40.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.561+0000 7f784941f640 1 -- 172.21.15.64:0/4272115509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7844102ef0 msgr2=0x7f784410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:40.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.561+0000 7f784941f640 1 --2- 172.21.15.64:0/4272115509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7844102ef0 0x7f784410da40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7834009ca0 tx=0x7f783402a950 comp rx=0 tx=0).stop 2024-09-12T03:26:40.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.561+0000 7f784941f640 1 -- 172.21.15.64:0/4272115509 shutdown_connections 2024-09-12T03:26:40.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.561+0000 7f784941f640 1 --2- 172.21.15.64:0/4272115509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7844102ef0 0x7f784410da40 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.561+0000 7f784941f640 1 --2- 172.21.15.64:0/4272115509 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7844102570 0x7f78441029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.561+0000 7f784941f640 1 --2- 172.21.15.64:0/4272115509 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7844100e20 0x7f7844101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.561+0000 7f784941f640 1 -- 172.21.15.64:0/4272115509 >> 172.21.15.64:0/4272115509 conn(0x7f78440fc7d0 msgr2=0x7f78440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:40.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.561+0000 7f784941f640 1 -- 172.21.15.64:0/4272115509 shutdown_connections 2024-09-12T03:26:40.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.561+0000 7f784941f640 1 -- 172.21.15.64:0/4272115509 wait complete. 2024-09-12T03:26:40.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.561+0000 7f784941f640 1 Processor -- start 2024-09-12T03:26:40.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.562+0000 7f784941f640 1 -- start start 2024-09-12T03:26:40.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.562+0000 7f784941f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7844100e20 0x7f78440fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:40.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.562+0000 7f784941f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7844102570 0x7f78440fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:40.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.562+0000 7f784941f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7844102ef0 0x7f78440fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:40.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.562+0000 7f784941f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7844110e50 con 0x7f7844102ef0 2024-09-12T03:26:40.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.562+0000 7f784941f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7844110cd0 con 0x7f7844102570 2024-09-12T03:26:40.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.562+0000 7f784941f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7844110fd0 con 0x7f7844100e20 2024-09-12T03:26:40.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.562+0000 7f78437fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7844102ef0 0x7f78440fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:40.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.562+0000 7f78437fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7844102ef0 0x7f78440fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60692/0 (socket says 172.21.15.64:60692) 2024-09-12T03:26:40.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.562+0000 7f78437fe640 1 -- 172.21.15.64:0/3409132008 learned_addr learned my addr 172.21.15.64:0/3409132008 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:26:40.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.562+0000 7f7842ffd640 1 --2- 172.21.15.64:0/3409132008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7844100e20 0x7f78440fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:40.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.562+0000 7f78427fc640 1 --2- 172.21.15.64:0/3409132008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7844102570 0x7f78440fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:40.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.562+0000 7f78437fe640 1 -- 172.21.15.64:0/3409132008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7844100e20 msgr2=0x7f78440fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:40.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.562+0000 7f78437fe640 1 --2- 172.21.15.64:0/3409132008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7844100e20 0x7f78440fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.562+0000 7f78437fe640 1 -- 172.21.15.64:0/3409132008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7844102570 msgr2=0x7f78440fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:40.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.562+0000 7f78437fe640 1 --2- 172.21.15.64:0/3409132008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7844102570 0x7f78440fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.562+0000 7f78437fe640 1 -- 172.21.15.64:0/3409132008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78441a68b0 con 0x7f7844102ef0 2024-09-12T03:26:40.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.562+0000 7f78427fc640 1 --2- 172.21.15.64:0/3409132008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7844102570 0x7f78440fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:26:40.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.563+0000 7f78437fe640 1 --2- 172.21.15.64:0/3409132008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7844102ef0 0x7f78440fd700 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7834006010 tx=0x7f783402f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:40.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.563+0000 7f783bfff640 1 -- 172.21.15.64:0/3409132008 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7834034040 con 0x7f7844102ef0 2024-09-12T03:26:40.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.563+0000 7f784941f640 1 -- 172.21.15.64:0/3409132008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78441a6b40 con 0x7f7844102ef0 2024-09-12T03:26:40.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.563+0000 7f783bfff640 1 -- 172.21.15.64:0/3409132008 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7834003e00 con 0x7f7844102ef0 2024-09-12T03:26:40.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.563+0000 7f783bfff640 1 -- 172.21.15.64:0/3409132008 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7834045480 con 0x7f7844102ef0 2024-09-12T03:26:40.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.563+0000 7f784941f640 1 -- 172.21.15.64:0/3409132008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78441a7070 con 0x7f7844102ef0 2024-09-12T03:26:40.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.564+0000 7f784941f640 1 -- 172.21.15.64:0/3409132008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7844101200 con 0x7f7844102ef0 2024-09-12T03:26:40.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.565+0000 7f783bfff640 1 -- 172.21.15.64:0/3409132008 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7834037020 con 0x7f7844102ef0 2024-09-12T03:26:40.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.566+0000 7f783bfff640 1 --2- 172.21.15.64:0/3409132008 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f78300a2b80 0x7f78300a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:40.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.566+0000 7f7842ffd640 1 --2- 172.21.15.64:0/3409132008 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f78300a2b80 0x7f78300a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:40.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.566+0000 7f783bfff640 1 -- 172.21.15.64:0/3409132008 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f78340efec0 con 0x7f7844102ef0 2024-09-12T03:26:40.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.566+0000 7f7842ffd640 1 --2- 172.21.15.64:0/3409132008 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f78300a2b80 0x7f78300a5040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f783c009990 tx=0x7f783c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:40.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.569+0000 7f783bfff640 1 -- 172.21.15.64:0/3409132008 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f78340811b0 con 0x7f7844102ef0 2024-09-12T03:26:40.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.797+0000 7f784941f640 1 -- 172.21.15.64:0/3409132008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f784410a0a0 con 0x7f7844102ef0 2024-09-12T03:26:40.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.798+0000 7f783bfff640 1 -- 172.21.15.64:0/3409132008 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8049 (secure 0 0 0) 0x7f78340bd2c0 con 0x7f7844102ef0 2024-09-12T03:26:40.799 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:26:40.800 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":10,"btime":"2024-09-12T03:26:38:428743+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2580078738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2580078738},{"type":"v1","addr":"172.21.15.64:6835","nonce":2580078738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2333415004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2333415004},{"type":"v1","addr":"172.21.15.64:6837","nonce":2333415004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2991933266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2991933266},{"type":"v1","addr":"172.21.15.195:6833","nonce":2991933266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1138960346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1138960346},{"type":"v1","addr":"172.21.15.115:6833","nonce":1138960346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/332737916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":332737916},{"type":"v1","addr":"172.21.15.115:6835","nonce":332737916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2531045583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2531045583},{"type":"v1","addr":"172.21.15.195:6835","nonce":2531045583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/71647401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":71647401},{"type":"v1","addr":"172.21.15.195:6837","nonce":71647401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2915155705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2915155705},{"type":"v1","addr":"172.21.15.115:6837","nonce":2915155705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6839/2574695134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":2574695134},{"type":"v1","addr":"172.21.15.195:6839","nonce":2574695134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-12T03:26:40.800 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 10 2024-09-12T03:26:40.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.800+0000 7f784941f640 1 -- 172.21.15.64:0/3409132008 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f78300a2b80 msgr2=0x7f78300a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:40.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.800+0000 7f784941f640 1 --2- 172.21.15.64:0/3409132008 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f78300a2b80 0x7f78300a5040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f783c009990 tx=0x7f783c009110 comp rx=0 tx=0).stop 2024-09-12T03:26:40.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.800+0000 7f784941f640 1 -- 172.21.15.64:0/3409132008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7844102ef0 msgr2=0x7f78440fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:40.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.800+0000 7f784941f640 1 --2- 172.21.15.64:0/3409132008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7844102ef0 0x7f78440fd700 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7834006010 tx=0x7f783402f710 comp rx=0 tx=0).stop 2024-09-12T03:26:40.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.800+0000 7f784941f640 1 -- 172.21.15.64:0/3409132008 shutdown_connections 2024-09-12T03:26:40.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.800+0000 7f784941f640 1 --2- 172.21.15.64:0/3409132008 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f78300a2b80 0x7f78300a5040 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.800+0000 7f784941f640 1 --2- 172.21.15.64:0/3409132008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7844102ef0 0x7f78440fd700 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.800+0000 7f784941f640 1 --2- 172.21.15.64:0/3409132008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7844102570 0x7f78440fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.800+0000 7f784941f640 1 --2- 172.21.15.64:0/3409132008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7844100e20 0x7f78440fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.800+0000 7f784941f640 1 -- 172.21.15.64:0/3409132008 >> 172.21.15.64:0/3409132008 conn(0x7f78440fc7d0 msgr2=0x7f784406a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:40.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.800+0000 7f784941f640 1 -- 172.21.15.64:0/3409132008 shutdown_connections 2024-09-12T03:26:40.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:40.800+0000 7f784941f640 1 -- 172.21.15.64:0/3409132008 wait complete. 2024-09-12T03:26:41.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:40 smithi064 ceph-mon[30952]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:41.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:40 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/3409132008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:41.273 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 12 03:26:40 smithi064 podman[71712]: 2024-09-12 03:26:40.704482893 +0000 UTC m=+0.016877766 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:26:41.273 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 12 03:26:40 smithi064 podman[71712]: 2024-09-12 03:26:40.847586179 +0000 UTC m=+0.159981043 container create 30355fb6af3a02f81de8046bc7af0f96c5f5b3958722b1560692fd41224b39ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-g, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-12T03:26:41.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:40 smithi115 ceph-mon[34801]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:41.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:40 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/3409132008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:41.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:40 smithi195 ceph-mon[34786]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:41.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:40 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/3409132008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:41.946 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 12 03:26:41 smithi064 podman[71712]: 2024-09-12 03:26:41.847169368 +0000 UTC m=+1.159564232 container init 30355fb6af3a02f81de8046bc7af0f96c5f5b3958722b1560692fd41224b39ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-g, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-12T03:26:41.947 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 12 03:26:41 smithi064 podman[71712]: 2024-09-12 03:26:41.850616295 +0000 UTC m=+1.163011162 container start 30355fb6af3a02f81de8046bc7af0f96c5f5b3958722b1560692fd41224b39ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-g, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-12T03:26:41.947 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 12 03:26:41 smithi064 ceph-mds[71748]: -- 172.21.15.64:0/1390910792 <== mon.0 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55df77399860 con 0x55df77377400 2024-09-12T03:26:41.947 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 12 03:26:41 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-g[71741]: starting mds.g at 2024-09-12T03:26:42.271 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 12 03:26:41 smithi064 bash[71712]: 30355fb6af3a02f81de8046bc7af0f96c5f5b3958722b1560692fd41224b39ef 2024-09-12T03:26:42.272 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 12 03:26:41 smithi064 systemd[1]: Started Ceph mds.g for b0812460-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:26:43.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:43 smithi064 ceph-mon[30952]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:43.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:43 smithi064 ceph-mon[30952]: mds.? [v2:172.21.15.64:6838/3030618395,v1:172.21.15.64:6839/3030618395] up:boot 2024-09-12T03:26:43.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:43 smithi064 ceph-mon[30952]: fsmap 10 up:standby 2024-09-12T03:26:43.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:43 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T03:26:43.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:43 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:43.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:43 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:43.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:43 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:43.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:43 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:26:43.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:43 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:26:43.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:43 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:43.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:43 smithi064 ceph-mon[30952]: Deploying daemon mds.a on smithi064 2024-09-12T03:26:43.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:43 smithi115 ceph-mon[34801]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:43.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:43 smithi115 ceph-mon[34801]: mds.? [v2:172.21.15.64:6838/3030618395,v1:172.21.15.64:6839/3030618395] up:boot 2024-09-12T03:26:43.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:43 smithi115 ceph-mon[34801]: fsmap 10 up:standby 2024-09-12T03:26:43.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:43 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T03:26:43.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:43 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:43.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:43 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:43.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:43 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:43.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:43 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:26:43.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:43 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:26:43.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:43 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:43.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:43 smithi115 ceph-mon[34801]: Deploying daemon mds.a on smithi064 2024-09-12T03:26:43.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:43 smithi195 ceph-mon[34786]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:43.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:43 smithi195 ceph-mon[34786]: mds.? [v2:172.21.15.64:6838/3030618395,v1:172.21.15.64:6839/3030618395] up:boot 2024-09-12T03:26:43.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:43 smithi195 ceph-mon[34786]: fsmap 10 up:standby 2024-09-12T03:26:43.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:43 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T03:26:43.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:43 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:43.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:43 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:43.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:43 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:43.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:43 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:26:43.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:43 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:26:43.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:43 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:43.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:43 smithi195 ceph-mon[34786]: Deploying daemon mds.a on smithi064 2024-09-12T03:26:43.704 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:26:43.989 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:44.022 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 12 03:26:43 smithi064 systemd[1]: Starting Ceph mds.a for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:26:44.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.446+0000 7f0fbf128640 1 -- 172.21.15.64:0/2314673220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fb8103660 msgr2=0x7f0fb810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:44.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.446+0000 7f0fbf128640 1 --2- 172.21.15.64:0/2314673220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fb8103660 0x7f0fb810bb50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4009900 tx=0x7f0fa402a480 comp rx=0 tx=0).stop 2024-09-12T03:26:44.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.446+0000 7f0fbf128640 1 -- 172.21.15.64:0/2314673220 shutdown_connections 2024-09-12T03:26:44.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.446+0000 7f0fbf128640 1 --2- 172.21.15.64:0/2314673220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fb810c170 0x7f0fb810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:44.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.446+0000 7f0fbf128640 1 --2- 172.21.15.64:0/2314673220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fb8103660 0x7f0fb810bb50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:44.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.446+0000 7f0fbf128640 1 --2- 172.21.15.64:0/2314673220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0fb8102d40 0x7f0fb8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:44.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.446+0000 7f0fbf128640 1 -- 172.21.15.64:0/2314673220 >> 172.21.15.64:0/2314673220 conn(0x7f0fb80fc7a0 msgr2=0x7f0fb80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:44.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.447+0000 7f0fbf128640 1 -- 172.21.15.64:0/2314673220 shutdown_connections 2024-09-12T03:26:44.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.447+0000 7f0fbf128640 1 -- 172.21.15.64:0/2314673220 wait complete. 2024-09-12T03:26:44.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.447+0000 7f0fbf128640 1 Processor -- start 2024-09-12T03:26:44.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.447+0000 7f0fbf128640 1 -- start start 2024-09-12T03:26:44.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.447+0000 7f0fbf128640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fb8102d40 0x7f0fb8198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:44.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.447+0000 7f0fbf128640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fb8103660 0x7f0fb8198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:44.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.448+0000 7f0fbf128640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0fb810c170 0x7f0fb819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:44.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.448+0000 7f0fbf128640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb8111e20 con 0x7f0fb8103660 2024-09-12T03:26:44.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.448+0000 7f0fbf128640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb8111ca0 con 0x7f0fb8102d40 2024-09-12T03:26:44.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.448+0000 7f0fbf128640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb8111fa0 con 0x7f0fb810c170 2024-09-12T03:26:44.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.448+0000 7f0fb7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fb8103660 0x7f0fb8198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:44.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.448+0000 7f0fb7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fb8103660 0x7f0fb8198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60732/0 (socket says 172.21.15.64:60732) 2024-09-12T03:26:44.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.448+0000 7f0fb7fff640 1 -- 172.21.15.64:0/2178796358 learned_addr learned my addr 172.21.15.64:0/2178796358 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:26:44.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.448+0000 7f0fbcec4640 1 --2- 172.21.15.64:0/2178796358 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fb8102d40 0x7f0fb8198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:44.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.448+0000 7f0fbd6c5640 1 --2- 172.21.15.64:0/2178796358 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0fb810c170 0x7f0fb819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:44.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.448+0000 7f0fb7fff640 1 -- 172.21.15.64:0/2178796358 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0fb810c170 msgr2=0x7f0fb819cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:44.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.448+0000 7f0fb7fff640 1 --2- 172.21.15.64:0/2178796358 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0fb810c170 0x7f0fb819cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:44.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.448+0000 7f0fb7fff640 1 -- 172.21.15.64:0/2178796358 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fb8102d40 msgr2=0x7f0fb8198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:44.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.448+0000 7f0fb7fff640 1 --2- 172.21.15.64:0/2178796358 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fb8102d40 0x7f0fb8198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:44.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.448+0000 7f0fb7fff640 1 -- 172.21.15.64:0/2178796358 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fb819d2b0 con 0x7f0fb8103660 2024-09-12T03:26:44.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.448+0000 7f0fbd6c5640 1 --2- 172.21.15.64:0/2178796358 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0fb810c170 0x7f0fb819cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:26:44.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.448+0000 7f0fb7fff640 1 --2- 172.21.15.64:0/2178796358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fb8103660 0x7f0fb8198840 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4005940 tx=0x7f0fa40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:44.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.448+0000 7f0fb5ffb640 1 -- 172.21.15.64:0/2178796358 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa403f030 con 0x7f0fb8103660 2024-09-12T03:26:44.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.448+0000 7f0fb5ffb640 1 -- 172.21.15.64:0/2178796358 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0fa4003cb0 con 0x7f0fb8103660 2024-09-12T03:26:44.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.448+0000 7f0fbf128640 1 -- 172.21.15.64:0/2178796358 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fb819d540 con 0x7f0fb8103660 2024-09-12T03:26:44.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.448+0000 7f0fb5ffb640 1 -- 172.21.15.64:0/2178796358 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa4035070 con 0x7f0fb8103660 2024-09-12T03:26:44.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.449+0000 7f0fbf128640 1 -- 172.21.15.64:0/2178796358 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fb81a4e80 con 0x7f0fb8103660 2024-09-12T03:26:44.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.450+0000 7f0fbf128640 1 -- 172.21.15.64:0/2178796358 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fb8103120 con 0x7f0fb8103660 2024-09-12T03:26:44.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.451+0000 7f0fb5ffb640 1 -- 172.21.15.64:0/2178796358 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0fa4042030 con 0x7f0fb8103660 2024-09-12T03:26:44.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.451+0000 7f0fb5ffb640 1 --2- 172.21.15.64:0/2178796358 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f0fa80a2b80 0x7f0fa80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:44.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.451+0000 7f0fb5ffb640 1 -- 172.21.15.64:0/2178796358 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f0fa40ef450 con 0x7f0fb8103660 2024-09-12T03:26:44.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.451+0000 7f0fbcec4640 1 --2- 172.21.15.64:0/2178796358 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f0fa80a2b80 0x7f0fa80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:44.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.452+0000 7f0fbcec4640 1 --2- 172.21.15.64:0/2178796358 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f0fa80a2b80 0x7f0fa80a5040 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0fac00abe0 tx=0x7f0fac00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:44.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.456+0000 7f0fb5ffb640 1 -- 172.21.15.64:0/2178796358 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0fa40bc7b0 con 0x7f0fb8103660 2024-09-12T03:26:44.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.675+0000 7f0fbf128640 1 -- 172.21.15.64:0/2178796358 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0fb81071e0 con 0x7f0fb8103660 2024-09-12T03:26:44.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.676+0000 7f0fb5ffb640 1 -- 172.21.15.64:0/2178796358 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8875 (secure 0 0 0) 0x7f0fa40bc0b0 con 0x7f0fb8103660 2024-09-12T03:26:44.677 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:26:44.677 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":11,"btime":"2024-09-12T03:26:41:999183+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2580078738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2580078738},{"type":"v1","addr":"172.21.15.64:6835","nonce":2580078738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2333415004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2333415004},{"type":"v1","addr":"172.21.15.64:6837","nonce":2333415004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/3030618395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":3030618395},{"type":"v1","addr":"172.21.15.64:6839","nonce":3030618395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2991933266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2991933266},{"type":"v1","addr":"172.21.15.195:6833","nonce":2991933266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1138960346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1138960346},{"type":"v1","addr":"172.21.15.115:6833","nonce":1138960346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/332737916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":332737916},{"type":"v1","addr":"172.21.15.115:6835","nonce":332737916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2531045583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2531045583},{"type":"v1","addr":"172.21.15.195:6835","nonce":2531045583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/71647401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":71647401},{"type":"v1","addr":"172.21.15.195:6837","nonce":71647401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2915155705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2915155705},{"type":"v1","addr":"172.21.15.115:6837","nonce":2915155705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6839/2574695134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":2574695134},{"type":"v1","addr":"172.21.15.195:6839","nonce":2574695134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-12T03:26:44.678 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 11 2024-09-12T03:26:44.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.678+0000 7f0fbf128640 1 -- 172.21.15.64:0/2178796358 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f0fa80a2b80 msgr2=0x7f0fa80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:44.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.678+0000 7f0fbf128640 1 --2- 172.21.15.64:0/2178796358 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f0fa80a2b80 0x7f0fa80a5040 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0fac00abe0 tx=0x7f0fac00a2d0 comp rx=0 tx=0).stop 2024-09-12T03:26:44.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.678+0000 7f0fbf128640 1 -- 172.21.15.64:0/2178796358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fb8103660 msgr2=0x7f0fb8198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:44.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.678+0000 7f0fbf128640 1 --2- 172.21.15.64:0/2178796358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fb8103660 0x7f0fb8198840 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4005940 tx=0x7f0fa40042f0 comp rx=0 tx=0).stop 2024-09-12T03:26:44.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.678+0000 7f0fbf128640 1 -- 172.21.15.64:0/2178796358 shutdown_connections 2024-09-12T03:26:44.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.678+0000 7f0fbf128640 1 --2- 172.21.15.64:0/2178796358 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f0fa80a2b80 0x7f0fa80a5040 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:44.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.678+0000 7f0fbf128640 1 --2- 172.21.15.64:0/2178796358 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0fb810c170 0x7f0fb819cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:44.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.678+0000 7f0fbf128640 1 --2- 172.21.15.64:0/2178796358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fb8103660 0x7f0fb8198840 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:44.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.678+0000 7f0fbf128640 1 --2- 172.21.15.64:0/2178796358 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fb8102d40 0x7f0fb8198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:44.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.678+0000 7f0fbf128640 1 -- 172.21.15.64:0/2178796358 >> 172.21.15.64:0/2178796358 conn(0x7f0fb80fc7a0 msgr2=0x7f0fb80fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:44.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.678+0000 7f0fbf128640 1 -- 172.21.15.64:0/2178796358 shutdown_connections 2024-09-12T03:26:44.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:44.678+0000 7f0fbf128640 1 -- 172.21.15.64:0/2178796358 wait complete. 2024-09-12T03:26:45.007 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 12 03:26:44 smithi064 podman[72128]: 2024-09-12 03:26:44.534034687 +0000 UTC m=+0.017296088 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:26:45.008 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 12 03:26:44 smithi064 podman[72128]: 2024-09-12 03:26:44.663259923 +0000 UTC m=+0.146521317 container create 74a579f1e1e3f7b42dd170ff33c1d2f8c4dd5c8fb7b33939a5b2a19ea66da36a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-a, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-12T03:26:45.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:45 smithi064 ceph-mon[30952]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:45.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:45 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/2178796358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:45.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:45 smithi115 ceph-mon[34801]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:45.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:45 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/2178796358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:45.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:45 smithi195 ceph-mon[34786]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:45.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:45 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/2178796358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:45.772 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 12 03:26:45 smithi064 podman[72128]: 2024-09-12 03:26:45.42922892 +0000 UTC m=+0.912490312 container init 74a579f1e1e3f7b42dd170ff33c1d2f8c4dd5c8fb7b33939a5b2a19ea66da36a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-a, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:26:45.772 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 12 03:26:45 smithi064 podman[72128]: 2024-09-12 03:26:45.432346881 +0000 UTC m=+0.915608273 container start 74a579f1e1e3f7b42dd170ff33c1d2f8c4dd5c8fb7b33939a5b2a19ea66da36a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-a, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-12T03:26:45.772 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 12 03:26:45 smithi064 ceph-mds[72165]: -- 172.21.15.64:0/3221235908 <== mon.0 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a5e4c05860 con 0x55a5e4be3000 2024-09-12T03:26:45.772 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 12 03:26:45 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-a[72157]: starting mds.a at 2024-09-12T03:26:45.772 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 12 03:26:45 smithi064 bash[72128]: 74a579f1e1e3f7b42dd170ff33c1d2f8c4dd5c8fb7b33939a5b2a19ea66da36a 2024-09-12T03:26:45.772 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 12 03:26:45 smithi064 systemd[1]: Started Ceph mds.a for b0812460-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:26:46.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:46 smithi115 ceph-mon[34801]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:46.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:46 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:46.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:46 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:46.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:46 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:46.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:46 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:26:46.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:46 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:26:46.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:46 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:46.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:46 smithi115 ceph-mon[34801]: Deploying daemon mds.b on smithi115 2024-09-12T03:26:46.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:46 smithi115 ceph-mon[34801]: mds.? [v2:172.21.15.64:6840/3333057625,v1:172.21.15.64:6841/3333057625] up:boot 2024-09-12T03:26:46.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:46 smithi115 ceph-mon[34801]: fsmap 11 up:standby 2024-09-12T03:26:46.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:46 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T03:26:46.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:46 smithi195 ceph-mon[34786]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:46.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:46 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:46.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:46 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:46.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:46 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:46.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:46 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:26:46.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:46 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:26:46.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:46 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:46.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:46 smithi195 ceph-mon[34786]: Deploying daemon mds.b on smithi115 2024-09-12T03:26:46.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:46 smithi195 ceph-mon[34786]: mds.? [v2:172.21.15.64:6840/3333057625,v1:172.21.15.64:6841/3333057625] up:boot 2024-09-12T03:26:46.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:46 smithi195 ceph-mon[34786]: fsmap 11 up:standby 2024-09-12T03:26:46.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:46 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T03:26:47.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:46 smithi064 ceph-mon[30952]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:47.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:46 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:47.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:46 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:47.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:46 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:47.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:46 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:26:47.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:46 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:26:47.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:46 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:47.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:46 smithi064 ceph-mon[30952]: Deploying daemon mds.b on smithi115 2024-09-12T03:26:47.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:46 smithi064 ceph-mon[30952]: mds.? [v2:172.21.15.64:6840/3333057625,v1:172.21.15.64:6841/3333057625] up:boot 2024-09-12T03:26:47.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:46 smithi064 ceph-mon[30952]: fsmap 11 up:standby 2024-09-12T03:26:47.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:46 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T03:26:47.354 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:26:47.629 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:47.643 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 12 03:26:47 smithi115 systemd[1]: Starting Ceph mds.b for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:26:48.053 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 12 03:26:47 smithi115 podman[64096]: 2024-09-12 03:26:47.671115213 +0000 UTC m=+0.016249871 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:26:48.054 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 12 03:26:47 smithi115 podman[64096]: 2024-09-12 03:26:47.784515177 +0000 UTC m=+0.129649827 container create 66e746a883690558d4d72d2fb1dcc9cc5545824a5c263c8363f530a150594b4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-12T03:26:48.054 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 12 03:26:48 smithi115 podman[64096]: 2024-09-12 03:26:48.054089564 +0000 UTC m=+0.399224210 container init 66e746a883690558d4d72d2fb1dcc9cc5545824a5c263c8363f530a150594b4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:26:48.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.124+0000 7f41c6cee640 1 -- 172.21.15.64:0/723871562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41c0101420 msgr2=0x7f41c010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:48.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.124+0000 7f41c6cee640 1 --2- 172.21.15.64:0/723871562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41c0101420 0x7f41c010d950 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f41b8009cd0 tx=0x7f41b802a8e0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.124+0000 7f41c6cee640 1 -- 172.21.15.64:0/723871562 shutdown_connections 2024-09-12T03:26:48.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.124+0000 7f41c6cee640 1 --2- 172.21.15.64:0/723871562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41c0101420 0x7f41c010d950 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.124+0000 7f41c6cee640 1 --2- 172.21.15.64:0/723871562 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41c0100a80 0x7f41c0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.124+0000 7f41c6cee640 1 --2- 172.21.15.64:0/723871562 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41c0106a80 0x7f41c0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.124+0000 7f41c6cee640 1 -- 172.21.15.64:0/723871562 >> 172.21.15.64:0/723871562 conn(0x7f41c00fc7a0 msgr2=0x7f41c00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:48.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.124+0000 7f41c6cee640 1 -- 172.21.15.64:0/723871562 shutdown_connections 2024-09-12T03:26:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.124+0000 7f41c6cee640 1 -- 172.21.15.64:0/723871562 wait complete. 2024-09-12T03:26:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.125+0000 7f41c6cee640 1 Processor -- start 2024-09-12T03:26:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.125+0000 7f41c6cee640 1 -- start start 2024-09-12T03:26:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.125+0000 7f41c6cee640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41c0100a80 0x7f41c0198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.125+0000 7f41c6cee640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41c0101420 0x7f41c01988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.125+0000 7f41c6cee640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41c0106a80 0x7f41c019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.125+0000 7f41c6cee640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f41c0111e40 con 0x7f41c0106a80 2024-09-12T03:26:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.125+0000 7f41c6cee640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f41c0111cc0 con 0x7f41c0101420 2024-09-12T03:26:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.125+0000 7f41c6cee640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f41c0111fc0 con 0x7f41c0100a80 2024-09-12T03:26:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.126+0000 7f41c528b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41c0106a80 0x7f41c019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.126+0000 7f41c528b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41c0106a80 0x7f41c019cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43152/0 (socket says 172.21.15.64:43152) 2024-09-12T03:26:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.126+0000 7f41c528b640 1 -- 172.21.15.64:0/1119211421 learned_addr learned my addr 172.21.15.64:0/1119211421 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:26:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.126+0000 7f41bffff640 1 --2- 172.21.15.64:0/1119211421 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41c0101420 0x7f41c01988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:48.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.126+0000 7f41c4a8a640 1 --2- 172.21.15.64:0/1119211421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41c0100a80 0x7f41c0198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:48.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.126+0000 7f41c528b640 1 -- 172.21.15.64:0/1119211421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41c0100a80 msgr2=0x7f41c0198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:48.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.126+0000 7f41c528b640 1 --2- 172.21.15.64:0/1119211421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41c0100a80 0x7f41c0198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.126+0000 7f41c528b640 1 -- 172.21.15.64:0/1119211421 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41c0101420 msgr2=0x7f41c01988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:48.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.126+0000 7f41c528b640 1 --2- 172.21.15.64:0/1119211421 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41c0101420 0x7f41c01988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.126+0000 7f41c528b640 1 -- 172.21.15.64:0/1119211421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c019d310 con 0x7f41c0106a80 2024-09-12T03:26:48.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.126+0000 7f41c528b640 1 --2- 172.21.15.64:0/1119211421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41c0106a80 0x7f41c019cc30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f41b8005cc0 tx=0x7f41b8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:48.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.126+0000 7f41bdffb640 1 -- 172.21.15.64:0/1119211421 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b8004450 con 0x7f41c0106a80 2024-09-12T03:26:48.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.126+0000 7f41bdffb640 1 -- 172.21.15.64:0/1119211421 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41b8031c90 con 0x7f41c0106a80 2024-09-12T03:26:48.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.126+0000 7f41c6cee640 1 -- 172.21.15.64:0/1119211421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41c019d5a0 con 0x7f41c0106a80 2024-09-12T03:26:48.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.126+0000 7f41bdffb640 1 -- 172.21.15.64:0/1119211421 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b803b400 con 0x7f41c0106a80 2024-09-12T03:26:48.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.126+0000 7f41c6cee640 1 -- 172.21.15.64:0/1119211421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41c01a4ee0 con 0x7f41c0106a80 2024-09-12T03:26:48.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.127+0000 7f41c6cee640 1 -- 172.21.15.64:0/1119211421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41c0100ee0 con 0x7f41c0106a80 2024-09-12T03:26:48.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.129+0000 7f41bdffb640 1 -- 172.21.15.64:0/1119211421 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f41b8004680 con 0x7f41c0106a80 2024-09-12T03:26:48.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.129+0000 7f41bdffb640 1 --2- 172.21.15.64:0/1119211421 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f41b00a2c00 0x7f41b00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:48.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.129+0000 7f41c4a8a640 1 --2- 172.21.15.64:0/1119211421 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f41b00a2c00 0x7f41b00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:48.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.129+0000 7f41bdffb640 1 -- 172.21.15.64:0/1119211421 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f41b802e030 con 0x7f41c0106a80 2024-09-12T03:26:48.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.130+0000 7f41c4a8a640 1 --2- 172.21.15.64:0/1119211421 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f41b00a2c00 0x7f41b00a50c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f41b400bb60 tx=0x7f41b400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:48.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.134+0000 7f41bdffb640 1 -- 172.21.15.64:0/1119211421 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f41b80bd070 con 0x7f41c0106a80 2024-09-12T03:26:48.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.351+0000 7f41c6cee640 1 -- 172.21.15.64:0/1119211421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41c01996c0 con 0x7f41c0106a80 2024-09-12T03:26:48.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.352+0000 7f41bdffb640 1 -- 172.21.15.64:0/1119211421 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9701 (secure 0 0 0) 0x7f41b80098b0 con 0x7f41c0106a80 2024-09-12T03:26:48.353 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:26:48.353 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":12,"btime":"2024-09-12T03:26:46:008592+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2580078738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2580078738},{"type":"v1","addr":"172.21.15.64:6835","nonce":2580078738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2333415004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2333415004},{"type":"v1","addr":"172.21.15.64:6837","nonce":2333415004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/3030618395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":3030618395},{"type":"v1","addr":"172.21.15.64:6839","nonce":3030618395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3333057625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3333057625},{"type":"v1","addr":"172.21.15.64:6841","nonce":3333057625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2991933266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2991933266},{"type":"v1","addr":"172.21.15.195:6833","nonce":2991933266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1138960346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1138960346},{"type":"v1","addr":"172.21.15.115:6833","nonce":1138960346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/332737916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":332737916},{"type":"v1","addr":"172.21.15.115:6835","nonce":332737916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2531045583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2531045583},{"type":"v1","addr":"172.21.15.195:6835","nonce":2531045583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/71647401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":71647401},{"type":"v1","addr":"172.21.15.195:6837","nonce":71647401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2915155705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2915155705},{"type":"v1","addr":"172.21.15.115:6837","nonce":2915155705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6839/2574695134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":2574695134},{"type":"v1","addr":"172.21.15.195:6839","nonce":2574695134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-12T03:26:48.354 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 12 2024-09-12T03:26:48.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.356+0000 7f41c6cee640 1 -- 172.21.15.64:0/1119211421 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f41b00a2c00 msgr2=0x7f41b00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:48.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.356+0000 7f41c6cee640 1 --2- 172.21.15.64:0/1119211421 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f41b00a2c00 0x7f41b00a50c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f41b400bb60 tx=0x7f41b400b390 comp rx=0 tx=0).stop 2024-09-12T03:26:48.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.356+0000 7f41c6cee640 1 -- 172.21.15.64:0/1119211421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41c0106a80 msgr2=0x7f41c019cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:48.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.356+0000 7f41c6cee640 1 --2- 172.21.15.64:0/1119211421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41c0106a80 0x7f41c019cc30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f41b8005cc0 tx=0x7f41b8003c20 comp rx=0 tx=0).stop 2024-09-12T03:26:48.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.356+0000 7f41c6cee640 1 -- 172.21.15.64:0/1119211421 shutdown_connections 2024-09-12T03:26:48.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.356+0000 7f41c6cee640 1 --2- 172.21.15.64:0/1119211421 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f41b00a2c00 0x7f41b00a50c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.356+0000 7f41c6cee640 1 --2- 172.21.15.64:0/1119211421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41c0106a80 0x7f41c019cc30 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.356+0000 7f41c6cee640 1 --2- 172.21.15.64:0/1119211421 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41c0101420 0x7f41c01988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.356+0000 7f41c6cee640 1 --2- 172.21.15.64:0/1119211421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41c0100a80 0x7f41c0198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.356+0000 7f41c6cee640 1 -- 172.21.15.64:0/1119211421 >> 172.21.15.64:0/1119211421 conn(0x7f41c00fc7a0 msgr2=0x7f41c00fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:48.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.357+0000 7f41c6cee640 1 -- 172.21.15.64:0/1119211421 shutdown_connections 2024-09-12T03:26:48.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:48.357+0000 7f41c6cee640 1 -- 172.21.15.64:0/1119211421 wait complete. 2024-09-12T03:26:48.392 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 12 03:26:48 smithi115 podman[64096]: 2024-09-12 03:26:48.05761971 +0000 UTC m=+0.402754370 container start 66e746a883690558d4d72d2fb1dcc9cc5545824a5c263c8363f530a150594b4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_CLEAN=True) 2024-09-12T03:26:48.393 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 12 03:26:48 smithi115 ceph-mds[64115]: -- 172.21.15.115:0/726775088 <== mon.2 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a885757860 con 0x55a885735400 2024-09-12T03:26:48.393 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 12 03:26:48 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-b[64111]: starting mds.b at 2024-09-12T03:26:48.393 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 12 03:26:48 smithi115 bash[64096]: 66e746a883690558d4d72d2fb1dcc9cc5545824a5c263c8363f530a150594b4e 2024-09-12T03:26:48.393 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 12 03:26:48 smithi115 systemd[1]: Started Ceph mds.b for b0812460-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:26:49.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:49 smithi064 ceph-mon[30952]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:49.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:49 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:49.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:49 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:49.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:49 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:49.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:49 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:49.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:49 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:49.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:49 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:26:49.401 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:49 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/1119211421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:49.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:49 smithi064 ceph-mon[30952]: mds.? [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] up:boot 2024-09-12T03:26:49.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:49 smithi064 ceph-mon[30952]: fsmap 12 up:standby 2024-09-12T03:26:49.402 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:49 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:26:49.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:49 smithi195 ceph-mon[34786]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:49.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:49 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:49.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:49 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:49.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:49 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:49.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:49 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:49.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:49 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:49.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:49 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:26:49.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:49 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/1119211421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:49.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:49 smithi195 ceph-mon[34786]: mds.? [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] up:boot 2024-09-12T03:26:49.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:49 smithi195 ceph-mon[34786]: fsmap 12 up:standby 2024-09-12T03:26:49.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:49 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:26:49.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:49 smithi115 ceph-mon[34801]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:49.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:49 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:49.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:49 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:49.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:49 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:49.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:49 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:49.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:49 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:49.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:49 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:26:49.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:49 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/1119211421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:49.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:49 smithi115 ceph-mon[34801]: mds.? [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] up:boot 2024-09-12T03:26:49.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:49 smithi115 ceph-mon[34801]: fsmap 12 up:standby 2024-09-12T03:26:49.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:49 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:26:50.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:50 smithi115 ceph-mon[34801]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:50.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:50 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:50.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:50 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:50.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:50 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:50.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:50 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:50.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:50 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:50.989 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:50 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:50.989 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:50 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:50.989 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:50 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:26:50.989 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:50 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:50.989 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:50 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:26:51.070 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:26:51.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:50 smithi195 ceph-mon[34786]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:51.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:50 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:51.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:50 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:51.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:50 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:51.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:50 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:51.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:50 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:51.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:50 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:51.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:50 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:51.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:50 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:26:51.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:50 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:51.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:50 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:26:51.189 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:50 smithi064 ceph-mon[30952]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:51.189 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:50 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:51.189 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:50 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:51.189 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:50 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:51.189 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:50 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:51.189 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:50 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:51.190 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:50 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:51.190 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:50 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:51.190 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:50 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:26:51.190 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:50 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:51.190 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:50 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:26:51.467 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:51.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:51.957+0000 7f88467a2640 1 -- 172.21.15.64:0/4023421493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f884010e2d0 msgr2=0x7f88401106c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:51.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:51.957+0000 7f88467a2640 1 --2- 172.21.15.64:0/4023421493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f884010e2d0 0x7f88401106c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8838009ca0 tx=0x7f883802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:26:51.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:51.957+0000 7f88467a2640 1 -- 172.21.15.64:0/4023421493 shutdown_connections 2024-09-12T03:26:51.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:51.957+0000 7f88467a2640 1 --2- 172.21.15.64:0/4023421493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f884010e2d0 0x7f88401106c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:51.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:51.957+0000 7f88467a2640 1 --2- 172.21.15.64:0/4023421493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88401010d0 0x7f884010dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:51.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:51.957+0000 7f88467a2640 1 --2- 172.21.15.64:0/4023421493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8840100720 0x7f8840100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:51.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:51.957+0000 7f88467a2640 1 -- 172.21.15.64:0/4023421493 >> 172.21.15.64:0/4023421493 conn(0x7f88400fc630 msgr2=0x7f88400fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:51.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:51.957+0000 7f88467a2640 1 -- 172.21.15.64:0/4023421493 shutdown_connections 2024-09-12T03:26:51.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:51.957+0000 7f88467a2640 1 -- 172.21.15.64:0/4023421493 wait complete. 2024-09-12T03:26:51.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:51.958+0000 7f88467a2640 1 Processor -- start 2024-09-12T03:26:51.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:51.958+0000 7f88467a2640 1 -- start start 2024-09-12T03:26:51.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:51.958+0000 7f88467a2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8840100720 0x7f884019a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:51.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:51.958+0000 7f88467a2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88401010d0 0x7f884019a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:51.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:51.958+0000 7f88467a2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f884010e2d0 0x7f884019ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:51.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:51.958+0000 7f88467a2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8840113e80 con 0x7f88401010d0 2024-09-12T03:26:51.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:51.958+0000 7f88467a2640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8840113d00 con 0x7f8840100720 2024-09-12T03:26:51.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:51.958+0000 7f88467a2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8840114000 con 0x7f884010e2d0 2024-09-12T03:26:51.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:51.958+0000 7f8844f9f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88401010d0 0x7f884019a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:51.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:51.958+0000 7f8844f9f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88401010d0 0x7f884019a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43176/0 (socket says 172.21.15.64:43176) 2024-09-12T03:26:51.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:51.958+0000 7f8844f9f640 1 -- 172.21.15.64:0/500423996 learned_addr learned my addr 172.21.15.64:0/500423996 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:26:51.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:51.958+0000 7f88457a0640 1 --2- 172.21.15.64:0/500423996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8840100720 0x7f884019a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:51.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:51.959+0000 7f8845fa1640 1 --2- 172.21.15.64:0/500423996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f884010e2d0 0x7f884019ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:51.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:51.959+0000 7f8844f9f640 1 -- 172.21.15.64:0/500423996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f884010e2d0 msgr2=0x7f884019ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:51.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:51.959+0000 7f8844f9f640 1 --2- 172.21.15.64:0/500423996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f884010e2d0 0x7f884019ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:51.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:51.959+0000 7f8844f9f640 1 -- 172.21.15.64:0/500423996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8840100720 msgr2=0x7f884019a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:51.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:51.959+0000 7f8844f9f640 1 --2- 172.21.15.64:0/500423996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8840100720 0x7f884019a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:51.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:51.959+0000 7f8844f9f640 1 -- 172.21.15.64:0/500423996 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f884019f350 con 0x7f88401010d0 2024-09-12T03:26:51.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:51.959+0000 7f88457a0640 1 --2- 172.21.15.64:0/500423996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8840100720 0x7f884019a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:26:51.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:51.959+0000 7f8844f9f640 1 --2- 172.21.15.64:0/500423996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88401010d0 0x7f884019a8e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f883000e9f0 tx=0x7f883000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:51.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:51.959+0000 7f88367fc640 1 -- 172.21.15.64:0/500423996 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883000cd80 con 0x7f88401010d0 2024-09-12T03:26:51.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:51.959+0000 7f88367fc640 1 -- 172.21.15.64:0/500423996 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88300108e0 con 0x7f88401010d0 2024-09-12T03:26:51.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:51.959+0000 7f88367fc640 1 -- 172.21.15.64:0/500423996 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8830018920 con 0x7f88401010d0 2024-09-12T03:26:51.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:51.959+0000 7f88467a2640 1 -- 172.21.15.64:0/500423996 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f884019f640 con 0x7f88401010d0 2024-09-12T03:26:51.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:51.959+0000 7f88467a2640 1 -- 172.21.15.64:0/500423996 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88401a6f20 con 0x7f88401010d0 2024-09-12T03:26:51.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:51.960+0000 7f88467a2640 1 -- 172.21.15.64:0/500423996 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8840100b00 con 0x7f88401010d0 2024-09-12T03:26:51.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:51.961+0000 7f88367fc640 1 -- 172.21.15.64:0/500423996 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8830010a90 con 0x7f88401010d0 2024-09-12T03:26:51.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:51.962+0000 7f88367fc640 1 --2- 172.21.15.64:0/500423996 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f882c0a2c00 0x7f882c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:51.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:51.962+0000 7f88457a0640 1 --2- 172.21.15.64:0/500423996 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f882c0a2c00 0x7f882c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:51.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:51.962+0000 7f88367fc640 1 -- 172.21.15.64:0/500423996 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f88300cf2d0 con 0x7f88401010d0 2024-09-12T03:26:51.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:51.962+0000 7f88457a0640 1 --2- 172.21.15.64:0/500423996 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f882c0a2c00 0x7f882c0a50c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f883c00bfd0 tx=0x7f883c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:51.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:51.965+0000 7f88367fc640 1 -- 172.21.15.64:0/500423996 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f883009c6d0 con 0x7f88401010d0 2024-09-12T03:26:52.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:52.207+0000 7f88467a2640 1 -- 172.21.15.64:0/500423996 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f884002d7c0 con 0x7f88401010d0 2024-09-12T03:26:52.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:52.208+0000 7f88367fc640 1 -- 172.21.15.64:0/500423996 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10530 (secure 0 0 0) 0x7f883009bfd0 con 0x7f88401010d0 2024-09-12T03:26:52.208 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:26:52.209 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":13,"btime":"2024-09-12T03:26:48:558154+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2580078738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2580078738},{"type":"v1","addr":"172.21.15.64:6835","nonce":2580078738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2333415004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2333415004},{"type":"v1","addr":"172.21.15.64:6837","nonce":2333415004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/3030618395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":3030618395},{"type":"v1","addr":"172.21.15.64:6839","nonce":3030618395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3333057625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3333057625},{"type":"v1","addr":"172.21.15.64:6841","nonce":3333057625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2991933266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2991933266},{"type":"v1","addr":"172.21.15.195:6833","nonce":2991933266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1138960346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1138960346},{"type":"v1","addr":"172.21.15.115:6833","nonce":1138960346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/332737916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":332737916},{"type":"v1","addr":"172.21.15.115:6835","nonce":332737916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2531045583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2531045583},{"type":"v1","addr":"172.21.15.195:6835","nonce":2531045583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/71647401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":71647401},{"type":"v1","addr":"172.21.15.195:6837","nonce":71647401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2915155705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2915155705},{"type":"v1","addr":"172.21.15.115:6837","nonce":2915155705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6839/2574695134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":2574695134},{"type":"v1","addr":"172.21.15.195:6839","nonce":2574695134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24500,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6839/2238031796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2238031796},{"type":"v1","addr":"172.21.15.115:6839","nonce":2238031796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-12T03:26:52.209 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 13 2024-09-12T03:26:52.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:52.210+0000 7f88467a2640 1 -- 172.21.15.64:0/500423996 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f882c0a2c00 msgr2=0x7f882c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:52.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:52.210+0000 7f88467a2640 1 --2- 172.21.15.64:0/500423996 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f882c0a2c00 0x7f882c0a50c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f883c00bfd0 tx=0x7f883c00b390 comp rx=0 tx=0).stop 2024-09-12T03:26:52.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:52.210+0000 7f88467a2640 1 -- 172.21.15.64:0/500423996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88401010d0 msgr2=0x7f884019a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:52.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:52.210+0000 7f88467a2640 1 --2- 172.21.15.64:0/500423996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88401010d0 0x7f884019a8e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f883000e9f0 tx=0x7f883000eeb0 comp rx=0 tx=0).stop 2024-09-12T03:26:52.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:52.211+0000 7f88467a2640 1 -- 172.21.15.64:0/500423996 shutdown_connections 2024-09-12T03:26:52.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:52.211+0000 7f88467a2640 1 --2- 172.21.15.64:0/500423996 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f882c0a2c00 0x7f882c0a50c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:52.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:52.211+0000 7f88467a2640 1 --2- 172.21.15.64:0/500423996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f884010e2d0 0x7f884019ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:52.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:52.211+0000 7f88467a2640 1 --2- 172.21.15.64:0/500423996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88401010d0 0x7f884019a8e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:52.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:52.211+0000 7f88467a2640 1 --2- 172.21.15.64:0/500423996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8840100720 0x7f884019a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:52.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:52.211+0000 7f88467a2640 1 -- 172.21.15.64:0/500423996 >> 172.21.15.64:0/500423996 conn(0x7f88400fc630 msgr2=0x7f88400fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:52.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:52.211+0000 7f88467a2640 1 -- 172.21.15.64:0/500423996 shutdown_connections 2024-09-12T03:26:52.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:52.211+0000 7f88467a2640 1 -- 172.21.15.64:0/500423996 wait complete. 2024-09-12T03:26:52.864 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-12T03:26:53.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:52 smithi115 ceph-mon[34801]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:53.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:52 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:53.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:52 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:53.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:52 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:53.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:52 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:53.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:52 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/500423996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:53.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:52 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:53.143 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:53.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:52 smithi195 ceph-mon[34786]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:53.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:52 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:53.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:52 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:53.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:52 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:53.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:52 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:53.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:52 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/500423996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:53.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:52 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:53.265 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:52 smithi064 ceph-mon[30952]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:53.265 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:52 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:53.265 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:52 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:53.265 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:52 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:53.265 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:52 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:53.265 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:52 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/500423996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:53.265 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:52 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:53.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.628+0000 7f256f2e4640 1 -- 172.21.15.64:0/1554250072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f256810c170 msgr2=0x7f256810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:53.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.628+0000 7f256f2e4640 1 --2- 172.21.15.64:0/1554250072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f256810c170 0x7f256810e610 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2560009ca0 tx=0x7f256002a950 comp rx=0 tx=0).stop 2024-09-12T03:26:53.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.628+0000 7f256f2e4640 1 -- 172.21.15.64:0/1554250072 shutdown_connections 2024-09-12T03:26:53.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.628+0000 7f256f2e4640 1 --2- 172.21.15.64:0/1554250072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f256810c170 0x7f256810e610 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:53.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.628+0000 7f256f2e4640 1 --2- 172.21.15.64:0/1554250072 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2568103660 0x7f256810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:53.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.628+0000 7f256f2e4640 1 --2- 172.21.15.64:0/1554250072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2568102d40 0x7f2568103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:53.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.628+0000 7f256f2e4640 1 -- 172.21.15.64:0/1554250072 >> 172.21.15.64:0/1554250072 conn(0x7f25680fc7a0 msgr2=0x7f25680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:53.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.628+0000 7f256f2e4640 1 -- 172.21.15.64:0/1554250072 shutdown_connections 2024-09-12T03:26:53.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.628+0000 7f256f2e4640 1 -- 172.21.15.64:0/1554250072 wait complete. 2024-09-12T03:26:53.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.628+0000 7f256f2e4640 1 Processor -- start 2024-09-12T03:26:53.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.628+0000 7f256f2e4640 1 -- start start 2024-09-12T03:26:53.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.629+0000 7f256f2e4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2568102d40 0x7f25681983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:53.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.629+0000 7f256f2e4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2568103660 0x7f25681988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:53.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.629+0000 7f256f2e4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f256810c170 0x7f256819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:53.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.629+0000 7f256f2e4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2568111e20 con 0x7f2568102d40 2024-09-12T03:26:53.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.629+0000 7f256f2e4640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2568111ca0 con 0x7f2568103660 2024-09-12T03:26:53.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.629+0000 7f256f2e4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2568111fa0 con 0x7f256810c170 2024-09-12T03:26:53.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.629+0000 7f256d080640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2568102d40 0x7f25681983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:53.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.629+0000 7f256d080640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2568102d40 0x7f25681983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43200/0 (socket says 172.21.15.64:43200) 2024-09-12T03:26:53.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.629+0000 7f256d080640 1 -- 172.21.15.64:0/4110290037 learned_addr learned my addr 172.21.15.64:0/4110290037 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:26:53.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.629+0000 7f256c87f640 1 --2- 172.21.15.64:0/4110290037 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2568103660 0x7f25681988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:53.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.629+0000 7f256d881640 1 --2- 172.21.15.64:0/4110290037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f256810c170 0x7f256819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:53.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.629+0000 7f256d080640 1 -- 172.21.15.64:0/4110290037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f256810c170 msgr2=0x7f256819cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:53.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.629+0000 7f256d080640 1 --2- 172.21.15.64:0/4110290037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f256810c170 0x7f256819cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:53.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.629+0000 7f256d080640 1 -- 172.21.15.64:0/4110290037 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2568103660 msgr2=0x7f25681988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:53.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.629+0000 7f256d080640 1 --2- 172.21.15.64:0/4110290037 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2568103660 0x7f25681988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:53.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.629+0000 7f256d080640 1 -- 172.21.15.64:0/4110290037 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f256819d350 con 0x7f2568102d40 2024-09-12T03:26:53.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.629+0000 7f256d080640 1 --2- 172.21.15.64:0/4110290037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2568102d40 0x7f25681983a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f256400d950 tx=0x7f256400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:53.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.629+0000 7f255a7fc640 1 -- 172.21.15.64:0/4110290037 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2564017070 con 0x7f2568102d40 2024-09-12T03:26:53.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.629+0000 7f255a7fc640 1 -- 172.21.15.64:0/4110290037 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2564009c30 con 0x7f2568102d40 2024-09-12T03:26:53.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.630+0000 7f256f2e4640 1 -- 172.21.15.64:0/4110290037 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f256819d640 con 0x7f2568102d40 2024-09-12T03:26:53.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.630+0000 7f255a7fc640 1 -- 172.21.15.64:0/4110290037 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256400f620 con 0x7f2568102d40 2024-09-12T03:26:53.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.630+0000 7f256f2e4640 1 -- 172.21.15.64:0/4110290037 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25681a4f20 con 0x7f2568102d40 2024-09-12T03:26:53.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.631+0000 7f256f2e4640 1 -- 172.21.15.64:0/4110290037 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2568103120 con 0x7f2568102d40 2024-09-12T03:26:53.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.632+0000 7f255a7fc640 1 -- 172.21.15.64:0/4110290037 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f25640109a0 con 0x7f2568102d40 2024-09-12T03:26:53.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.632+0000 7f255a7fc640 1 --2- 172.21.15.64:0/4110290037 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f25540a2c00 0x7f25540a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:53.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.632+0000 7f256c87f640 1 --2- 172.21.15.64:0/4110290037 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f25540a2c00 0x7f25540a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:53.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.632+0000 7f255a7fc640 1 -- 172.21.15.64:0/4110290037 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f25640ce6c0 con 0x7f2568102d40 2024-09-12T03:26:53.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.633+0000 7f256c87f640 1 --2- 172.21.15.64:0/4110290037 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f25540a2c00 0x7f25540a50c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f25680fedc0 tx=0x7f255c00b2e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:53.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.637+0000 7f255a7fc640 1 -- 172.21.15.64:0/4110290037 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2564009de0 con 0x7f2568102d40 2024-09-12T03:26:53.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.815+0000 7f256f2e4640 1 -- 172.21.15.64:0/4110290037 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f2568106c50 con 0x7f2568102d40 2024-09-12T03:26:53.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.815+0000 7f255a7fc640 1 -- 172.21.15.64:0/4110290037 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v77) ==== 84+0+65 (secure 0 0 0) 0x7f256409b470 con 0x7f2568102d40 2024-09-12T03:26:53.815 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:26:53.816 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2024-09-12T03:26:53.816 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:26:53.816 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 1, 2024-09-12T03:26:53.816 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": ".mgr" 2024-09-12T03:26:53.816 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:26:53.816 INFO:teuthology.orchestra.run.smithi064.stdout:] 2024-09-12T03:26:53.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.818+0000 7f256f2e4640 1 -- 172.21.15.64:0/4110290037 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f25540a2c00 msgr2=0x7f25540a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:53.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.818+0000 7f256f2e4640 1 --2- 172.21.15.64:0/4110290037 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f25540a2c00 0x7f25540a50c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f25680fedc0 tx=0x7f255c00b2e0 comp rx=0 tx=0).stop 2024-09-12T03:26:53.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.818+0000 7f256f2e4640 1 -- 172.21.15.64:0/4110290037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2568102d40 msgr2=0x7f25681983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:53.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.818+0000 7f256f2e4640 1 --2- 172.21.15.64:0/4110290037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2568102d40 0x7f25681983a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f256400d950 tx=0x7f256400de10 comp rx=0 tx=0).stop 2024-09-12T03:26:53.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.818+0000 7f256f2e4640 1 -- 172.21.15.64:0/4110290037 shutdown_connections 2024-09-12T03:26:53.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.818+0000 7f256f2e4640 1 --2- 172.21.15.64:0/4110290037 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f25540a2c00 0x7f25540a50c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:53.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.818+0000 7f256f2e4640 1 --2- 172.21.15.64:0/4110290037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f256810c170 0x7f256819cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:53.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.818+0000 7f256f2e4640 1 --2- 172.21.15.64:0/4110290037 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2568103660 0x7f25681988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:53.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.818+0000 7f256f2e4640 1 --2- 172.21.15.64:0/4110290037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2568102d40 0x7f25681983a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:53.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.818+0000 7f256f2e4640 1 -- 172.21.15.64:0/4110290037 >> 172.21.15.64:0/4110290037 conn(0x7f25680fc7a0 msgr2=0x7f25680fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:53.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.818+0000 7f256f2e4640 1 -- 172.21.15.64:0/4110290037 shutdown_connections 2024-09-12T03:26:53.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:53.818+0000 7f256f2e4640 1 -- 172.21.15.64:0/4110290037 wait complete. 2024-09-12T03:26:54.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:53 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:54.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:53 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:54.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:53 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:54.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:53 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:26:54.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:53 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:54.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:53 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/4110290037' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T03:26:54.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:53 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:54.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:53 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:54.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:53 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:54.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:53 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:26:54.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:53 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:54.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:53 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/4110290037' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T03:26:54.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:53 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:54.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:53 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:54.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:53 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:54.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:53 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:26:54.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:53 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:26:54.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:53 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/4110290037' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T03:26:54.437 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-12T03:26:54.437 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-12T03:26:54.734 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:55.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:54 smithi064 ceph-mon[30952]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:55.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:54 smithi195 ceph-mon[34786]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:55.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.206+0000 7f54ee87e640 1 -- 172.21.15.64:0/679555724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54e8100ca0 msgr2=0x7f54e8106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:55.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.206+0000 7f54ee87e640 1 --2- 172.21.15.64:0/679555724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54e8100ca0 0x7f54e8106840 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f54dc009930 tx=0x7f54dc02a420 comp rx=0 tx=0).stop 2024-09-12T03:26:55.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.206+0000 7f54ee87e640 1 -- 172.21.15.64:0/679555724 shutdown_connections 2024-09-12T03:26:55.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.206+0000 7f54ee87e640 1 --2- 172.21.15.64:0/679555724 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54e8106f50 0x7f54e810eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:55.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.206+0000 7f54ee87e640 1 --2- 172.21.15.64:0/679555724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54e8100ca0 0x7f54e8106840 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:55.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.206+0000 7f54ee87e640 1 --2- 172.21.15.64:0/679555724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54e81002f0 0x7f54e81006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:55.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.206+0000 7f54ee87e640 1 -- 172.21.15.64:0/679555724 >> 172.21.15.64:0/679555724 conn(0x7f54e8077210 msgr2=0x7f54e80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:55.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.206+0000 7f54ee87e640 1 -- 172.21.15.64:0/679555724 shutdown_connections 2024-09-12T03:26:55.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.206+0000 7f54ee87e640 1 -- 172.21.15.64:0/679555724 wait complete. 2024-09-12T03:26:55.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.207+0000 7f54ee87e640 1 Processor -- start 2024-09-12T03:26:55.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.207+0000 7f54ee87e640 1 -- start start 2024-09-12T03:26:55.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.207+0000 7f54ee87e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54e81002f0 0x7f54e8101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:55.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.207+0000 7f54ee87e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54e8100ca0 0x7f54e81017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:55.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.207+0000 7f54ee87e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54e8106f50 0x7f54e8101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:55.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.207+0000 7f54ee87e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f54e8111e50 con 0x7f54e8100ca0 2024-09-12T03:26:55.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.207+0000 7f54ee87e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f54e8111cd0 con 0x7f54e81002f0 2024-09-12T03:26:55.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.207+0000 7f54ee87e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f54e8111fd0 con 0x7f54e8106f50 2024-09-12T03:26:55.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.207+0000 7f54e77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54e8100ca0 0x7f54e81017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:55.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.207+0000 7f54e77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54e8100ca0 0x7f54e81017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43214/0 (socket says 172.21.15.64:43214) 2024-09-12T03:26:55.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.207+0000 7f54e77fe640 1 -- 172.21.15.64:0/4256422491 learned_addr learned my addr 172.21.15.64:0/4256422491 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:26:55.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.207+0000 7f54e7fff640 1 --2- 172.21.15.64:0/4256422491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54e81002f0 0x7f54e8101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:55.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.207+0000 7f54ece1b640 1 --2- 172.21.15.64:0/4256422491 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54e8106f50 0x7f54e8101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:55.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.207+0000 7f54e77fe640 1 -- 172.21.15.64:0/4256422491 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54e8106f50 msgr2=0x7f54e8101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:55.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.207+0000 7f54e77fe640 1 --2- 172.21.15.64:0/4256422491 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54e8106f50 0x7f54e8101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:55.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.207+0000 7f54e77fe640 1 -- 172.21.15.64:0/4256422491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54e81002f0 msgr2=0x7f54e8101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:55.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.207+0000 7f54e77fe640 1 --2- 172.21.15.64:0/4256422491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54e81002f0 0x7f54e8101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:55.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.207+0000 7f54e77fe640 1 -- 172.21.15.64:0/4256422491 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e81a68b0 con 0x7f54e8100ca0 2024-09-12T03:26:55.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.207+0000 7f54e7fff640 1 --2- 172.21.15.64:0/4256422491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54e81002f0 0x7f54e8101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:26:55.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.208+0000 7f54e77fe640 1 --2- 172.21.15.64:0/4256422491 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54e8100ca0 0x7f54e81017b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f54dc031440 tx=0x7f54dc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:55.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.208+0000 7f54e57fa640 1 -- 172.21.15.64:0/4256422491 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54dc03f030 con 0x7f54e8100ca0 2024-09-12T03:26:55.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.208+0000 7f54ee87e640 1 -- 172.21.15.64:0/4256422491 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54e81a6b40 con 0x7f54e8100ca0 2024-09-12T03:26:55.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.208+0000 7f54e57fa640 1 -- 172.21.15.64:0/4256422491 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54dc003aa0 con 0x7f54e8100ca0 2024-09-12T03:26:55.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.208+0000 7f54e57fa640 1 -- 172.21.15.64:0/4256422491 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54dc035070 con 0x7f54e8100ca0 2024-09-12T03:26:55.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.208+0000 7f54ee87e640 1 -- 172.21.15.64:0/4256422491 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54e81a7020 con 0x7f54e8100ca0 2024-09-12T03:26:55.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.209+0000 7f54ee87e640 1 -- 172.21.15.64:0/4256422491 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54e81006d0 con 0x7f54e8100ca0 2024-09-12T03:26:55.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.210+0000 7f54e57fa640 1 -- 172.21.15.64:0/4256422491 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f54dc042030 con 0x7f54e8100ca0 2024-09-12T03:26:55.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.211+0000 7f54e57fa640 1 --2- 172.21.15.64:0/4256422491 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f54d00a2b80 0x7f54d00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:55.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.211+0000 7f54e7fff640 1 --2- 172.21.15.64:0/4256422491 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f54d00a2b80 0x7f54d00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:55.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.211+0000 7f54e57fa640 1 -- 172.21.15.64:0/4256422491 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f54dc0ef7f0 con 0x7f54e8100ca0 2024-09-12T03:26:55.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.212+0000 7f54e7fff640 1 --2- 172.21.15.64:0/4256422491 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f54d00a2b80 0x7f54d00a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f54d8009b40 tx=0x7f54d8005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:55.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.214+0000 7f54e57fa640 1 -- 172.21.15.64:0/4256422491 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f54dc0bcb50 con 0x7f54e8100ca0 2024-09-12T03:26:55.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.392+0000 7f54ee87e640 1 -- 172.21.15.64:0/4256422491 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f54e810a1a0 con 0x7f54e8100ca0 2024-09-12T03:26:55.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:54 smithi115 ceph-mon[34801]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:55.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.939+0000 7f54e57fa640 1 -- 172.21.15.64:0/4256422491 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v78) ==== 138+0+0 (secure 0 0 0) 0x7f54dc0bc450 con 0x7f54e8100ca0 2024-09-12T03:26:55.940 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' created 2024-09-12T03:26:55.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.941+0000 7f54ee87e640 1 -- 172.21.15.64:0/4256422491 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f54d00a2b80 msgr2=0x7f54d00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:55.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.941+0000 7f54ee87e640 1 --2- 172.21.15.64:0/4256422491 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f54d00a2b80 0x7f54d00a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f54d8009b40 tx=0x7f54d8005e50 comp rx=0 tx=0).stop 2024-09-12T03:26:55.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.941+0000 7f54ee87e640 1 -- 172.21.15.64:0/4256422491 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54e8100ca0 msgr2=0x7f54e81017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:55.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.941+0000 7f54ee87e640 1 --2- 172.21.15.64:0/4256422491 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54e8100ca0 0x7f54e81017b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f54dc031440 tx=0x7f54dc004270 comp rx=0 tx=0).stop 2024-09-12T03:26:55.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.942+0000 7f54ee87e640 1 -- 172.21.15.64:0/4256422491 shutdown_connections 2024-09-12T03:26:55.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.942+0000 7f54ee87e640 1 --2- 172.21.15.64:0/4256422491 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f54d00a2b80 0x7f54d00a5040 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:55.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.942+0000 7f54ee87e640 1 --2- 172.21.15.64:0/4256422491 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54e8106f50 0x7f54e8101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:55.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.942+0000 7f54ee87e640 1 --2- 172.21.15.64:0/4256422491 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54e8100ca0 0x7f54e81017b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:55.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.942+0000 7f54ee87e640 1 --2- 172.21.15.64:0/4256422491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54e81002f0 0x7f54e8101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:55.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.942+0000 7f54ee87e640 1 -- 172.21.15.64:0/4256422491 >> 172.21.15.64:0/4256422491 conn(0x7f54e8077210 msgr2=0x7f54e8105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:55.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.942+0000 7f54ee87e640 1 -- 172.21.15.64:0/4256422491 shutdown_connections 2024-09-12T03:26:55.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:55.942+0000 7f54ee87e640 1 -- 172.21.15.64:0/4256422491 wait complete. 2024-09-12T03:26:56.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:55 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/4256422491' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T03:26:56.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:55 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/4256422491' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T03:26:56.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:55 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/4256422491' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T03:26:56.623 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-12T03:26:56.882 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:57.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:56 smithi195 ceph-mon[34786]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:57.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:56 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/4256422491' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T03:26:57.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:56 smithi195 ceph-mon[34786]: osdmap e78: 12 total, 12 up, 12 in 2024-09-12T03:26:57.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:56 smithi064 ceph-mon[30952]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:57.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:56 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/4256422491' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T03:26:57.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:56 smithi064 ceph-mon[30952]: osdmap e78: 12 total, 12 up, 12 in 2024-09-12T03:26:57.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.370+0000 7f651c134640 1 -- 172.21.15.64:0/370241770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6514102570 msgr2=0x7f65141029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:57.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.370+0000 7f651c134640 1 --2- 172.21.15.64:0/370241770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6514102570 0x7f65141029b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6508009900 tx=0x7f650802a480 comp rx=0 tx=0).stop 2024-09-12T03:26:57.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.370+0000 7f651c134640 1 -- 172.21.15.64:0/370241770 shutdown_connections 2024-09-12T03:26:57.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.370+0000 7f651c134640 1 --2- 172.21.15.64:0/370241770 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6514102ef0 0x7f651410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:57.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.370+0000 7f651c134640 1 --2- 172.21.15.64:0/370241770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6514102570 0x7f65141029b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:57.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.370+0000 7f651c134640 1 --2- 172.21.15.64:0/370241770 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6514100e20 0x7f6514101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:57.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.370+0000 7f651c134640 1 -- 172.21.15.64:0/370241770 >> 172.21.15.64:0/370241770 conn(0x7f65140fc7d0 msgr2=0x7f65140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:57.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.370+0000 7f651c134640 1 -- 172.21.15.64:0/370241770 shutdown_connections 2024-09-12T03:26:57.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.370+0000 7f651c134640 1 -- 172.21.15.64:0/370241770 wait complete. 2024-09-12T03:26:57.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.371+0000 7f651c134640 1 Processor -- start 2024-09-12T03:26:57.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.371+0000 7f651c134640 1 -- start start 2024-09-12T03:26:57.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.371+0000 7f651c134640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6514100e20 0x7f651419a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:57.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.371+0000 7f651c134640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6514102570 0x7f651419aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:57.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.371+0000 7f651c134640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6514102ef0 0x7f651419ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:57.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.371+0000 7f651c134640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6514110e30 con 0x7f6514102570 2024-09-12T03:26:57.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.371+0000 7f651c134640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6514110cb0 con 0x7f6514102ef0 2024-09-12T03:26:57.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.371+0000 7f651c134640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6514110fb0 con 0x7f6514100e20 2024-09-12T03:26:57.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.371+0000 7f65196cf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6514102570 0x7f651419aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:57.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.371+0000 7f65196cf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6514102570 0x7f651419aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43222/0 (socket says 172.21.15.64:43222) 2024-09-12T03:26:57.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.371+0000 7f65196cf640 1 -- 172.21.15.64:0/333512942 learned_addr learned my addr 172.21.15.64:0/333512942 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:26:57.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.371+0000 7f6519ed0640 1 --2- 172.21.15.64:0/333512942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6514100e20 0x7f651419a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:57.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.372+0000 7f65196cf640 1 -- 172.21.15.64:0/333512942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6514100e20 msgr2=0x7f651419a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:57.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.372+0000 7f65196cf640 1 --2- 172.21.15.64:0/333512942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6514100e20 0x7f651419a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:57.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.372+0000 7f65196cf640 1 -- 172.21.15.64:0/333512942 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6514102ef0 msgr2=0x7f651419ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:57.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.372+0000 7f65196cf640 1 --2- 172.21.15.64:0/333512942 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6514102ef0 0x7f651419ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:57.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.372+0000 7f65196cf640 1 -- 172.21.15.64:0/333512942 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f651419f550 con 0x7f6514102570 2024-09-12T03:26:57.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.372+0000 7f6519ed0640 1 --2- 172.21.15.64:0/333512942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6514100e20 0x7f651419a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:26:57.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.372+0000 7f65196cf640 1 --2- 172.21.15.64:0/333512942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6514102570 0x7f651419aae0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6508006010 tx=0x7f65080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:57.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.372+0000 7f6506ffd640 1 -- 172.21.15.64:0/333512942 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650803f070 con 0x7f6514102570 2024-09-12T03:26:57.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.372+0000 7f651c134640 1 -- 172.21.15.64:0/333512942 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f651419f7e0 con 0x7f6514102570 2024-09-12T03:26:57.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.372+0000 7f6506ffd640 1 -- 172.21.15.64:0/333512942 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6508003aa0 con 0x7f6514102570 2024-09-12T03:26:57.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.372+0000 7f6506ffd640 1 -- 172.21.15.64:0/333512942 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6508035030 con 0x7f6514102570 2024-09-12T03:26:57.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.372+0000 7f651c134640 1 -- 172.21.15.64:0/333512942 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65141a7120 con 0x7f6514102570 2024-09-12T03:26:57.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.373+0000 7f651c134640 1 -- 172.21.15.64:0/333512942 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6514101200 con 0x7f6514102570 2024-09-12T03:26:57.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.374+0000 7f6506ffd640 1 -- 172.21.15.64:0/333512942 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6508042070 con 0x7f6514102570 2024-09-12T03:26:57.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.375+0000 7f6506ffd640 1 --2- 172.21.15.64:0/333512942 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f65000a2b80 0x7f65000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:57.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.375+0000 7f6506ffd640 1 -- 172.21.15.64:0/333512942 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7497+0+0 (secure 0 0 0) 0x7f65080ef9c0 con 0x7f6514102570 2024-09-12T03:26:57.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.375+0000 7f6519ed0640 1 --2- 172.21.15.64:0/333512942 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f65000a2b80 0x7f65000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:57.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.375+0000 7f6519ed0640 1 --2- 172.21.15.64:0/333512942 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f65000a2b80 0x7f65000a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f65100058c0 tx=0x7f6510005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:57.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.378+0000 7f6506ffd640 1 -- 172.21.15.64:0/333512942 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f65080bcc50 con 0x7f6514102570 2024-09-12T03:26:57.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:56 smithi115 ceph-mon[34801]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:57.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:56 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/4256422491' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T03:26:57.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:56 smithi115 ceph-mon[34801]: osdmap e78: 12 total, 12 up, 12 in 2024-09-12T03:26:57.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.553+0000 7f651c134640 1 -- 172.21.15.64:0/333512942 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f65141a72f0 con 0x7f6514102570 2024-09-12T03:26:57.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.989+0000 7f6506ffd640 1 -- 172.21.15.64:0/333512942 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v80) ==== 171+0+0 (secure 0 0 0) 0x7f65080bc550 con 0x7f6514102570 2024-09-12T03:26:57.990 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' created 2024-09-12T03:26:57.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.992+0000 7f651c134640 1 -- 172.21.15.64:0/333512942 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f65000a2b80 msgr2=0x7f65000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:57.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.992+0000 7f651c134640 1 --2- 172.21.15.64:0/333512942 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f65000a2b80 0x7f65000a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f65100058c0 tx=0x7f6510005d30 comp rx=0 tx=0).stop 2024-09-12T03:26:57.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.992+0000 7f651c134640 1 -- 172.21.15.64:0/333512942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6514102570 msgr2=0x7f651419aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:57.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.992+0000 7f651c134640 1 --2- 172.21.15.64:0/333512942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6514102570 0x7f651419aae0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6508006010 tx=0x7f65080042f0 comp rx=0 tx=0).stop 2024-09-12T03:26:57.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.992+0000 7f651c134640 1 -- 172.21.15.64:0/333512942 shutdown_connections 2024-09-12T03:26:57.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.992+0000 7f651c134640 1 --2- 172.21.15.64:0/333512942 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f65000a2b80 0x7f65000a5040 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:57.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.992+0000 7f651c134640 1 --2- 172.21.15.64:0/333512942 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6514102ef0 0x7f651419ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:57.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.992+0000 7f651c134640 1 --2- 172.21.15.64:0/333512942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6514102570 0x7f651419aae0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:57.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.992+0000 7f651c134640 1 --2- 172.21.15.64:0/333512942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6514100e20 0x7f651419a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:57.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.992+0000 7f651c134640 1 -- 172.21.15.64:0/333512942 >> 172.21.15.64:0/333512942 conn(0x7f65140fc7d0 msgr2=0x7f6514069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:57.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.992+0000 7f651c134640 1 -- 172.21.15.64:0/333512942 shutdown_connections 2024-09-12T03:26:57.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:57.992+0000 7f651c134640 1 -- 172.21.15.64:0/333512942 wait complete. 2024-09-12T03:26:58.277 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:57 smithi064 ceph-mon[30952]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T03:26:58.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:57 smithi115 ceph-mon[34801]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T03:26:58.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:57 smithi195 ceph-mon[34786]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T03:26:58.712 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-12T03:26:58.975 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:59.258 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:58 smithi064 ceph-mon[30952]: pgmap v274: 33 pgs: 22 creating+peering, 5 active+clean, 6 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:59.258 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:58 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/333512942' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-12T03:26:59.258 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:58 smithi064 ceph-mon[30952]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T03:26:59.259 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:58 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/333512942' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-12T03:26:59.259 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:26:58 smithi064 ceph-mon[30952]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T03:26:59.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:58 smithi115 ceph-mon[34801]: pgmap v274: 33 pgs: 22 creating+peering, 5 active+clean, 6 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:59.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:58 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/333512942' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-12T03:26:59.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:58 smithi115 ceph-mon[34801]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T03:26:59.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:58 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/333512942' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-12T03:26:59.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:26:58 smithi115 ceph-mon[34801]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T03:26:59.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.425+0000 7fc83b4ec640 1 -- 172.21.15.64:0/3402476600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc834106f10 msgr2=0x7fc83410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:59.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.425+0000 7fc83b4ec640 1 --2- 172.21.15.64:0/3402476600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc834106f10 0x7fc83410ea90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc82c009ca0 tx=0x7fc82c02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.425+0000 7fc83b4ec640 1 -- 172.21.15.64:0/3402476600 shutdown_connections 2024-09-12T03:26:59.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.425+0000 7fc83b4ec640 1 --2- 172.21.15.64:0/3402476600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc834106f10 0x7fc83410ea90 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.425+0000 7fc83b4ec640 1 --2- 172.21.15.64:0/3402476600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc834102790 0x7fc8341067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.425+0000 7fc83b4ec640 1 --2- 172.21.15.64:0/3402476600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc834101de0 0x7fc8341021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.425+0000 7fc83b4ec640 1 -- 172.21.15.64:0/3402476600 >> 172.21.15.64:0/3402476600 conn(0x7fc8340fd8c0 msgr2=0x7fc8340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:59.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.425+0000 7fc83b4ec640 1 -- 172.21.15.64:0/3402476600 shutdown_connections 2024-09-12T03:26:59.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.425+0000 7fc83b4ec640 1 -- 172.21.15.64:0/3402476600 wait complete. 2024-09-12T03:26:59.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.426+0000 7fc83b4ec640 1 Processor -- start 2024-09-12T03:26:59.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.426+0000 7fc83b4ec640 1 -- start start 2024-09-12T03:26:59.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.426+0000 7fc83b4ec640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc834101de0 0x7fc834198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:59.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.426+0000 7fc83b4ec640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc834102790 0x7fc8341989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:59.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.426+0000 7fc83b4ec640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc834106f10 0x7fc83419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:59.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.426+0000 7fc83b4ec640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc834111ec0 con 0x7fc834102790 2024-09-12T03:26:59.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.426+0000 7fc83b4ec640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc834111d40 con 0x7fc834101de0 2024-09-12T03:26:59.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.426+0000 7fc83b4ec640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc834112040 con 0x7fc834106f10 2024-09-12T03:26:59.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.426+0000 7fc838a87640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc834102790 0x7fc8341989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:59.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.426+0000 7fc838a87640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc834102790 0x7fc8341989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45698/0 (socket says 172.21.15.64:45698) 2024-09-12T03:26:59.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.426+0000 7fc838a87640 1 -- 172.21.15.64:0/2747517271 learned_addr learned my addr 172.21.15.64:0/2747517271 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:26:59.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.426+0000 7fc839a89640 1 --2- 172.21.15.64:0/2747517271 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc834106f10 0x7fc83419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:59.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.426+0000 7fc839288640 1 --2- 172.21.15.64:0/2747517271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc834101de0 0x7fc834198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:59.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.426+0000 7fc838a87640 1 -- 172.21.15.64:0/2747517271 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc834106f10 msgr2=0x7fc83419cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:59.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.426+0000 7fc838a87640 1 --2- 172.21.15.64:0/2747517271 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc834106f10 0x7fc83419cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.426+0000 7fc838a87640 1 -- 172.21.15.64:0/2747517271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc834101de0 msgr2=0x7fc834198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:59.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.426+0000 7fc838a87640 1 --2- 172.21.15.64:0/2747517271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc834101de0 0x7fc834198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.426+0000 7fc838a87640 1 -- 172.21.15.64:0/2747517271 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc83419d430 con 0x7fc834102790 2024-09-12T03:26:59.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.427+0000 7fc838a87640 1 --2- 172.21.15.64:0/2747517271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc834102790 0x7fc8341989c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc83000b4a0 tx=0x7fc83000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:59.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.427+0000 7fc8267fc640 1 -- 172.21.15.64:0/2747517271 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc830014070 con 0x7fc834102790 2024-09-12T03:26:59.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.427+0000 7fc8267fc640 1 -- 172.21.15.64:0/2747517271 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc830010ac0 con 0x7fc834102790 2024-09-12T03:26:59.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.427+0000 7fc83b4ec640 1 -- 172.21.15.64:0/2747517271 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc83419d720 con 0x7fc834102790 2024-09-12T03:26:59.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.427+0000 7fc8267fc640 1 -- 172.21.15.64:0/2747517271 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc830017c30 con 0x7fc834102790 2024-09-12T03:26:59.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.427+0000 7fc83b4ec640 1 -- 172.21.15.64:0/2747517271 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8341a5000 con 0x7fc834102790 2024-09-12T03:26:59.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.428+0000 7fc83b4ec640 1 -- 172.21.15.64:0/2747517271 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8341021c0 con 0x7fc834102790 2024-09-12T03:26:59.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.429+0000 7fc8267fc640 1 -- 172.21.15.64:0/2747517271 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc830017070 con 0x7fc834102790 2024-09-12T03:26:59.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.430+0000 7fc8267fc640 1 --2- 172.21.15.64:0/2747517271 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fc8200a2c00 0x7fc8200a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:59.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.430+0000 7fc839288640 1 --2- 172.21.15.64:0/2747517271 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fc8200a2c00 0x7fc8200a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:59.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.430+0000 7fc8267fc640 1 -- 172.21.15.64:0/2747517271 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7879+0+0 (secure 0 0 0) 0x7fc8300ce670 con 0x7fc834102790 2024-09-12T03:26:59.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.430+0000 7fc839288640 1 --2- 172.21.15.64:0/2747517271 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fc8200a2c00 0x7fc8200a50c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc82800a810 tx=0x7fc828008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:59.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.433+0000 7fc8267fc640 1 -- 172.21.15.64:0/2747517271 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc83009b780 con 0x7fc834102790 2024-09-12T03:26:59.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:58 smithi195 ceph-mon[34786]: pgmap v274: 33 pgs: 22 creating+peering, 5 active+clean, 6 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:26:59.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:58 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/333512942' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-12T03:26:59.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:58 smithi195 ceph-mon[34786]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T03:26:59.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:58 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/333512942' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-12T03:26:59.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:26:58 smithi195 ceph-mon[34786]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T03:26:59.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.646+0000 7fc83b4ec640 1 -- 172.21.15.64:0/2747517271 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fc8341997e0 con 0x7fc834102790 2024-09-12T03:26:59.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:26:59.999+0000 7fc8267fc640 1 -- 172.21.15.64:0/2747517271 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fc83009b080 con 0x7fc834102790 2024-09-12T03:27:00.000 INFO:teuthology.orchestra.run.smithi064.stderr:new fs with metadata pool 2 and data pool 3 2024-09-12T03:27:00.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:00.002+0000 7fc83b4ec640 1 -- 172.21.15.64:0/2747517271 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fc8200a2c00 msgr2=0x7fc8200a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:00.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:00.002+0000 7fc83b4ec640 1 --2- 172.21.15.64:0/2747517271 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fc8200a2c00 0x7fc8200a50c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc82800a810 tx=0x7fc828008040 comp rx=0 tx=0).stop 2024-09-12T03:27:00.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:00.002+0000 7fc83b4ec640 1 -- 172.21.15.64:0/2747517271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc834102790 msgr2=0x7fc8341989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:00.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:00.002+0000 7fc83b4ec640 1 --2- 172.21.15.64:0/2747517271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc834102790 0x7fc8341989c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc83000b4a0 tx=0x7fc83000b960 comp rx=0 tx=0).stop 2024-09-12T03:27:00.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:00.002+0000 7fc83b4ec640 1 -- 172.21.15.64:0/2747517271 shutdown_connections 2024-09-12T03:27:00.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:00.002+0000 7fc83b4ec640 1 --2- 172.21.15.64:0/2747517271 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fc8200a2c00 0x7fc8200a50c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:00.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:00.002+0000 7fc83b4ec640 1 --2- 172.21.15.64:0/2747517271 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc834106f10 0x7fc83419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:00.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:00.002+0000 7fc83b4ec640 1 --2- 172.21.15.64:0/2747517271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc834102790 0x7fc8341989c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:00.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:00.002+0000 7fc83b4ec640 1 --2- 172.21.15.64:0/2747517271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc834101de0 0x7fc834198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:00.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:00.002+0000 7fc83b4ec640 1 -- 172.21.15.64:0/2747517271 >> 172.21.15.64:0/2747517271 conn(0x7fc8340fd8c0 msgr2=0x7fc8340fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:00.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:00.002+0000 7fc83b4ec640 1 -- 172.21.15.64:0/2747517271 shutdown_connections 2024-09-12T03:27:00.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:00.002+0000 7fc83b4ec640 1 -- 172.21.15.64:0/2747517271 wait complete. 2024-09-12T03:27:00.269 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:00 smithi064 ceph-mon[30952]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T03:27:00.269 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:00 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/2747517271' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T03:27:00.269 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:00 smithi064 ceph-mon[30952]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-12T03:27:00.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:00 smithi115 ceph-mon[34801]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T03:27:00.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:00 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/2747517271' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T03:27:00.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:00 smithi115 ceph-mon[34801]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-12T03:27:00.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:00 smithi195 ceph-mon[34786]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T03:27:00.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:00 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/2747517271' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T03:27:00.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:00 smithi195 ceph-mon[34786]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-12T03:27:00.569 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-12T03:27:00.569 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-12T03:27:00.830 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:01.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:01 smithi115 ceph-mon[34801]: pgmap v277: 97 pgs: 24 creating+peering, 11 active+clean, 62 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:01.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:01 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/2747517271' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T03:27:01.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:01 smithi115 ceph-mon[34801]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T03:27:01.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:01 smithi115 ceph-mon[34801]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-12T03:27:01.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:01 smithi115 ceph-mon[34801]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-12T03:27:01.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:01 smithi115 ceph-mon[34801]: mds.? [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] up:active 2024-09-12T03:27:01.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:01 smithi115 ceph-mon[34801]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:27:01.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:01 smithi195 ceph-mon[34786]: pgmap v277: 97 pgs: 24 creating+peering, 11 active+clean, 62 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:01.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:01 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/2747517271' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T03:27:01.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:01 smithi195 ceph-mon[34786]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T03:27:01.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:01 smithi195 ceph-mon[34786]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-12T03:27:01.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:01 smithi195 ceph-mon[34786]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-12T03:27:01.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:01 smithi195 ceph-mon[34786]: mds.? [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] up:active 2024-09-12T03:27:01.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:01 smithi195 ceph-mon[34786]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:27:01.249 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:01 smithi064 ceph-mon[30952]: pgmap v277: 97 pgs: 24 creating+peering, 11 active+clean, 62 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:01.249 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:01 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/2747517271' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T03:27:01.249 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:01 smithi064 ceph-mon[30952]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T03:27:01.250 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:01 smithi064 ceph-mon[30952]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-12T03:27:01.250 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:01 smithi064 ceph-mon[30952]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-12T03:27:01.250 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:01 smithi064 ceph-mon[30952]: mds.? [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] up:active 2024-09-12T03:27:01.250 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:01 smithi064 ceph-mon[30952]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:27:01.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.337+0000 7feaf0049640 1 -- 172.21.15.64:0/2350989680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feae8102790 msgr2=0x7feae81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:01.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.337+0000 7feaf0049640 1 --2- 172.21.15.64:0/2350989680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feae8102790 0x7feae81067e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fead8009900 tx=0x7fead802a410 comp rx=0 tx=0).stop 2024-09-12T03:27:01.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.337+0000 7feaf0049640 1 -- 172.21.15.64:0/2350989680 shutdown_connections 2024-09-12T03:27:01.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.337+0000 7feaf0049640 1 --2- 172.21.15.64:0/2350989680 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feae8106f10 0x7feae810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.337+0000 7feaf0049640 1 --2- 172.21.15.64:0/2350989680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feae8102790 0x7feae81067e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.337+0000 7feaf0049640 1 --2- 172.21.15.64:0/2350989680 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feae8101de0 0x7feae81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.337+0000 7feaf0049640 1 -- 172.21.15.64:0/2350989680 >> 172.21.15.64:0/2350989680 conn(0x7feae80fd8c0 msgr2=0x7feae80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:01.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.337+0000 7feaf0049640 1 -- 172.21.15.64:0/2350989680 shutdown_connections 2024-09-12T03:27:01.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.337+0000 7feaf0049640 1 -- 172.21.15.64:0/2350989680 wait complete. 2024-09-12T03:27:01.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.338+0000 7feaf0049640 1 Processor -- start 2024-09-12T03:27:01.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.338+0000 7feaf0049640 1 -- start start 2024-09-12T03:27:01.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.338+0000 7feaf0049640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feae8101de0 0x7feae81983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.338+0000 7feaf0049640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feae8102790 0x7feae81988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.338+0000 7feaf0049640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feae8106f10 0x7feae819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.338+0000 7feaf0049640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7feae8111e20 con 0x7feae8102790 2024-09-12T03:27:01.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.338+0000 7feaf0049640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7feae8111ca0 con 0x7feae8101de0 2024-09-12T03:27:01.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.338+0000 7feaf0049640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7feae8111fa0 con 0x7feae8106f10 2024-09-12T03:27:01.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.338+0000 7feaed5e4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feae8102790 0x7feae81988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.338+0000 7feaed5e4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feae8102790 0x7feae81988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45714/0 (socket says 172.21.15.64:45714) 2024-09-12T03:27:01.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.338+0000 7feaed5e4640 1 -- 172.21.15.64:0/643210514 learned_addr learned my addr 172.21.15.64:0/643210514 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:27:01.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.338+0000 7feaedde5640 1 --2- 172.21.15.64:0/643210514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feae8101de0 0x7feae81983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.338+0000 7feaee5e6640 1 --2- 172.21.15.64:0/643210514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feae8106f10 0x7feae819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.339+0000 7feaed5e4640 1 -- 172.21.15.64:0/643210514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feae8106f10 msgr2=0x7feae819cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:01.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.339+0000 7feaed5e4640 1 --2- 172.21.15.64:0/643210514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feae8106f10 0x7feae819cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.339+0000 7feaed5e4640 1 -- 172.21.15.64:0/643210514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feae8101de0 msgr2=0x7feae81983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:01.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.339+0000 7feaed5e4640 1 --2- 172.21.15.64:0/643210514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feae8101de0 0x7feae81983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.339+0000 7feaed5e4640 1 -- 172.21.15.64:0/643210514 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feae819d350 con 0x7feae8102790 2024-09-12T03:27:01.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.339+0000 7feaee5e6640 1 --2- 172.21.15.64:0/643210514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feae8106f10 0x7feae819cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:27:01.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.339+0000 7feaedde5640 1 --2- 172.21.15.64:0/643210514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feae8101de0 0x7feae81983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:27:01.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.339+0000 7feaed5e4640 1 --2- 172.21.15.64:0/643210514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feae8102790 0x7feae81988e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fead8005940 tx=0x7fead8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:01.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.339+0000 7feadeffd640 1 -- 172.21.15.64:0/643210514 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead803f030 con 0x7feae8102790 2024-09-12T03:27:01.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.339+0000 7feaf0049640 1 -- 172.21.15.64:0/643210514 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feae819d5e0 con 0x7feae8102790 2024-09-12T03:27:01.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.339+0000 7feadeffd640 1 -- 172.21.15.64:0/643210514 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fead8003aa0 con 0x7feae8102790 2024-09-12T03:27:01.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.339+0000 7feadeffd640 1 -- 172.21.15.64:0/643210514 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead8035070 con 0x7feae8102790 2024-09-12T03:27:01.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.339+0000 7feaf0049640 1 -- 172.21.15.64:0/643210514 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feae81a4f20 con 0x7feae8102790 2024-09-12T03:27:01.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.340+0000 7feaf0049640 1 -- 172.21.15.64:0/643210514 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feae81032c0 con 0x7feae8102790 2024-09-12T03:27:01.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.341+0000 7feadeffd640 1 -- 172.21.15.64:0/643210514 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fead8042030 con 0x7feae8102790 2024-09-12T03:27:01.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.342+0000 7feadeffd640 1 --2- 172.21.15.64:0/643210514 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fead40a2b80 0x7fead40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.342+0000 7feaedde5640 1 --2- 172.21.15.64:0/643210514 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fead40a2b80 0x7fead40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.342+0000 7feadeffd640 1 -- 172.21.15.64:0/643210514 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7fead80ef920 con 0x7feae8102790 2024-09-12T03:27:01.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.343+0000 7feaedde5640 1 --2- 172.21.15.64:0/643210514 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fead40a2b80 0x7fead40a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7feae4005d00 tx=0x7feae400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:01.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.345+0000 7feadeffd640 1 -- 172.21.15.64:0/643210514 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fead80bc9d0 con 0x7feae8102790 2024-09-12T03:27:01.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:01.531+0000 7feaf0049640 1 -- 172.21.15.64:0/643210514 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7feae810a180 con 0x7feae8102790 2024-09-12T03:27:02.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:02.020+0000 7feadeffd640 1 -- 172.21.15.64:0/643210514 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v83) ==== 157+0+0 (secure 0 0 0) 0x7fead80bc2d0 con 0x7feae8102790 2024-09-12T03:27:02.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:02.022+0000 7feaf0049640 1 -- 172.21.15.64:0/643210514 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fead40a2b80 msgr2=0x7fead40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:02.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:02.022+0000 7feaf0049640 1 --2- 172.21.15.64:0/643210514 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fead40a2b80 0x7fead40a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7feae4005d00 tx=0x7feae400b410 comp rx=0 tx=0).stop 2024-09-12T03:27:02.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:02.022+0000 7feaf0049640 1 -- 172.21.15.64:0/643210514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feae8102790 msgr2=0x7feae81988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:02.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:02.022+0000 7feaf0049640 1 --2- 172.21.15.64:0/643210514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feae8102790 0x7feae81988e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fead8005940 tx=0x7fead8004270 comp rx=0 tx=0).stop 2024-09-12T03:27:02.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:02.022+0000 7feaf0049640 1 -- 172.21.15.64:0/643210514 shutdown_connections 2024-09-12T03:27:02.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:02.022+0000 7feaf0049640 1 --2- 172.21.15.64:0/643210514 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fead40a2b80 0x7fead40a5040 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:02.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:02.022+0000 7feaf0049640 1 --2- 172.21.15.64:0/643210514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feae8106f10 0x7feae819cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:02.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:02.022+0000 7feaf0049640 1 --2- 172.21.15.64:0/643210514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feae8102790 0x7feae81988e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:02.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:02.022+0000 7feaf0049640 1 --2- 172.21.15.64:0/643210514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feae8101de0 0x7feae81983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:02.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:02.022+0000 7feaf0049640 1 -- 172.21.15.64:0/643210514 >> 172.21.15.64:0/643210514 conn(0x7feae80fd8c0 msgr2=0x7feae80fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:02.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:02.023+0000 7feaf0049640 1 -- 172.21.15.64:0/643210514 shutdown_connections 2024-09-12T03:27:02.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:02.023+0000 7feaf0049640 1 -- 172.21.15.64:0/643210514 wait complete. 2024-09-12T03:27:02.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:02 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/643210514' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-12T03:27:02.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:02 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/643210514' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-12T03:27:02.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:02 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/643210514' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-12T03:27:02.744 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-12T03:27:03.003 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:03.266 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:03 smithi064 ceph-mon[30952]: pgmap v279: 97 pgs: 2 creating+peering, 66 active+clean, 29 unknown; 578 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s wr, 7 op/s 2024-09-12T03:27:03.266 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:03 smithi064 ceph-mon[30952]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T03:27:03.266 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:03 smithi064 ceph-mon[30952]: Cluster is now healthy 2024-09-12T03:27:03.266 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:03 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/643210514' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-12T03:27:03.266 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:03 smithi064 ceph-mon[30952]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T03:27:03.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:03 smithi115 ceph-mon[34801]: pgmap v279: 97 pgs: 2 creating+peering, 66 active+clean, 29 unknown; 578 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s wr, 7 op/s 2024-09-12T03:27:03.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:03 smithi115 ceph-mon[34801]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T03:27:03.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:03 smithi115 ceph-mon[34801]: Cluster is now healthy 2024-09-12T03:27:03.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:03 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/643210514' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-12T03:27:03.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:03 smithi115 ceph-mon[34801]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T03:27:03.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:03 smithi195 ceph-mon[34786]: pgmap v279: 97 pgs: 2 creating+peering, 66 active+clean, 29 unknown; 578 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s wr, 7 op/s 2024-09-12T03:27:03.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:03 smithi195 ceph-mon[34786]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T03:27:03.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:03 smithi195 ceph-mon[34786]: Cluster is now healthy 2024-09-12T03:27:03.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:03 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/643210514' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-12T03:27:03.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:03 smithi195 ceph-mon[34786]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T03:27:03.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:03.509+0000 7f8390171640 1 -- 172.21.15.64:0/3129148868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8388102570 msgr2=0x7f83881029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:03.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:03.509+0000 7f8390171640 1 --2- 172.21.15.64:0/3129148868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8388102570 0x7f83881029b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8384009900 tx=0x7f838402a480 comp rx=0 tx=0).stop 2024-09-12T03:27:03.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:03.509+0000 7f8390171640 1 -- 172.21.15.64:0/3129148868 shutdown_connections 2024-09-12T03:27:03.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:03.509+0000 7f8390171640 1 --2- 172.21.15.64:0/3129148868 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8388102ef0 0x7f838810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:03.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:03.509+0000 7f8390171640 1 --2- 172.21.15.64:0/3129148868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8388102570 0x7f83881029b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:03.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:03.509+0000 7f8390171640 1 --2- 172.21.15.64:0/3129148868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8388100e20 0x7f8388101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:03.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:03.509+0000 7f8390171640 1 -- 172.21.15.64:0/3129148868 >> 172.21.15.64:0/3129148868 conn(0x7f83880fc7d0 msgr2=0x7f83880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:03.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:03.509+0000 7f8390171640 1 -- 172.21.15.64:0/3129148868 shutdown_connections 2024-09-12T03:27:03.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:03.509+0000 7f8390171640 1 -- 172.21.15.64:0/3129148868 wait complete. 2024-09-12T03:27:03.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:03.510+0000 7f8390171640 1 Processor -- start 2024-09-12T03:27:03.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:03.510+0000 7f8390171640 1 -- start start 2024-09-12T03:27:03.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:03.510+0000 7f8390171640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8388100e20 0x7f838810e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:03.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:03.510+0000 7f8390171640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8388102570 0x7f838810ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:03.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:03.510+0000 7f8390171640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8388102ef0 0x7f838810ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:03.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:03.510+0000 7f8390171640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8388111f50 con 0x7f8388102570 2024-09-12T03:27:03.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:03.510+0000 7f8390171640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8388111dd0 con 0x7f8388102ef0 2024-09-12T03:27:03.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:03.510+0000 7f8390171640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f83881120d0 con 0x7f8388100e20 2024-09-12T03:27:03.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:03.510+0000 7f838d70c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8388102570 0x7f838810ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:03.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:03.510+0000 7f838d70c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8388102570 0x7f838810ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45728/0 (socket says 172.21.15.64:45728) 2024-09-12T03:27:03.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:03.510+0000 7f838d70c640 1 -- 172.21.15.64:0/2769620309 learned_addr learned my addr 172.21.15.64:0/2769620309 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:27:03.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:03.510+0000 7f838df0d640 1 --2- 172.21.15.64:0/2769620309 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8388100e20 0x7f838810e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:03.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:03.511+0000 7f838d70c640 1 -- 172.21.15.64:0/2769620309 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8388100e20 msgr2=0x7f838810e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:03.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:03.511+0000 7f838d70c640 1 --2- 172.21.15.64:0/2769620309 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8388100e20 0x7f838810e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:03.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:03.511+0000 7f838d70c640 1 -- 172.21.15.64:0/2769620309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8388102ef0 msgr2=0x7f838810ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:03.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:03.511+0000 7f838d70c640 1 --2- 172.21.15.64:0/2769620309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8388102ef0 0x7f838810ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:03.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:03.511+0000 7f838d70c640 1 -- 172.21.15.64:0/2769620309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83881a6970 con 0x7f8388102570 2024-09-12T03:27:03.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:03.511+0000 7f838d70c640 1 --2- 172.21.15.64:0/2769620309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8388102570 0x7f838810ea10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8384006010 tx=0x7f8384004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:03.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:03.511+0000 7f837affd640 1 -- 172.21.15.64:0/2769620309 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838403f030 con 0x7f8388102570 2024-09-12T03:27:03.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:03.511+0000 7f8390171640 1 -- 172.21.15.64:0/2769620309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83881a6c00 con 0x7f8388102570 2024-09-12T03:27:03.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:03.511+0000 7f837affd640 1 -- 172.21.15.64:0/2769620309 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8384003aa0 con 0x7f8388102570 2024-09-12T03:27:03.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:03.511+0000 7f837affd640 1 -- 172.21.15.64:0/2769620309 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8384035070 con 0x7f8388102570 2024-09-12T03:27:03.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:03.511+0000 7f8390171640 1 -- 172.21.15.64:0/2769620309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83881a7130 con 0x7f8388102570 2024-09-12T03:27:03.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:03.512+0000 7f8390171640 1 -- 172.21.15.64:0/2769620309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8388101200 con 0x7f8388102570 2024-09-12T03:27:03.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:03.513+0000 7f837affd640 1 -- 172.21.15.64:0/2769620309 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8384042030 con 0x7f8388102570 2024-09-12T03:27:03.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:03.514+0000 7f837affd640 1 --2- 172.21.15.64:0/2769620309 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f83740a2b80 0x7f83740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:03.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:03.514+0000 7f837affd640 1 -- 172.21.15.64:0/2769620309 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8277+0+0 (secure 0 0 0) 0x7f83840efcf0 con 0x7f8388102570 2024-09-12T03:27:03.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:03.514+0000 7f838df0d640 1 --2- 172.21.15.64:0/2769620309 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f83740a2b80 0x7f83740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:03.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:03.514+0000 7f838df0d640 1 --2- 172.21.15.64:0/2769620309 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f83740a2b80 0x7f83740a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f837c005fd0 tx=0x7f837c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:03.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:03.517+0000 7f837affd640 1 -- 172.21.15.64:0/2769620309 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f83840bcc20 con 0x7f8388102570 2024-09-12T03:27:03.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:03.697+0000 7f8390171640 1 -- 172.21.15.64:0/2769620309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f838810fa00 con 0x7f8388102570 2024-09-12T03:27:03.893 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 12 03:27:03 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-b[64111]: 2024-09-12T03:27:03.569+0000 7f439e946640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T03:27:04.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:04 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/2769620309' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T03:27:04.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:04 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/2769620309' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T03:27:04.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:04 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/2769620309' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T03:27:05.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:05.029+0000 7f837affd640 1 -- 172.21.15.64:0/2769620309 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v85) ==== 228+0+0 (secure 0 0 0) 0x7f838810fa00 con 0x7f8388102570 2024-09-12T03:27:05.030 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data_ec' created 2024-09-12T03:27:05.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:05.032+0000 7f8390171640 1 -- 172.21.15.64:0/2769620309 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f83740a2b80 msgr2=0x7f83740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:05.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:05.032+0000 7f8390171640 1 --2- 172.21.15.64:0/2769620309 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f83740a2b80 0x7f83740a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f837c005fd0 tx=0x7f837c009500 comp rx=0 tx=0).stop 2024-09-12T03:27:05.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:05.032+0000 7f8390171640 1 -- 172.21.15.64:0/2769620309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8388102570 msgr2=0x7f838810ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:05.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:05.032+0000 7f8390171640 1 --2- 172.21.15.64:0/2769620309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8388102570 0x7f838810ea10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8384006010 tx=0x7f8384004270 comp rx=0 tx=0).stop 2024-09-12T03:27:05.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:05.032+0000 7f8390171640 1 -- 172.21.15.64:0/2769620309 shutdown_connections 2024-09-12T03:27:05.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:05.032+0000 7f8390171640 1 --2- 172.21.15.64:0/2769620309 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f83740a2b80 0x7f83740a5040 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:05.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:05.032+0000 7f8390171640 1 --2- 172.21.15.64:0/2769620309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8388102ef0 0x7f838810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:05.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:05.032+0000 7f8390171640 1 --2- 172.21.15.64:0/2769620309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8388102570 0x7f838810ea10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:05.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:05.032+0000 7f8390171640 1 --2- 172.21.15.64:0/2769620309 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8388100e20 0x7f838810e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:05.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:05.032+0000 7f8390171640 1 -- 172.21.15.64:0/2769620309 >> 172.21.15.64:0/2769620309 conn(0x7f83880fc7d0 msgr2=0x7f83880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:05.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:05.032+0000 7f8390171640 1 -- 172.21.15.64:0/2769620309 shutdown_connections 2024-09-12T03:27:05.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:05.032+0000 7f8390171640 1 -- 172.21.15.64:0/2769620309 wait complete. 2024-09-12T03:27:05.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:05 smithi064 ceph-mon[30952]: pgmap v281: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2024-09-12T03:27:05.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:05 smithi064 ceph-mon[30952]: osdmap e84: 12 total, 12 up, 12 in 2024-09-12T03:27:05.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:05 smithi115 ceph-mon[34801]: pgmap v281: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2024-09-12T03:27:05.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:05 smithi115 ceph-mon[34801]: osdmap e84: 12 total, 12 up, 12 in 2024-09-12T03:27:05.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:05 smithi195 ceph-mon[34786]: pgmap v281: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2024-09-12T03:27:05.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:05 smithi195 ceph-mon[34786]: osdmap e84: 12 total, 12 up, 12 in 2024-09-12T03:27:05.717 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-12T03:27:05.977 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:06.360 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:06 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/2769620309' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T03:27:06.360 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:06 smithi064 ceph-mon[30952]: osdmap e85: 12 total, 12 up, 12 in 2024-09-12T03:27:06.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:06 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/2769620309' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T03:27:06.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:06 smithi115 ceph-mon[34801]: osdmap e85: 12 total, 12 up, 12 in 2024-09-12T03:27:06.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:06 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/2769620309' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T03:27:06.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:06 smithi195 ceph-mon[34786]: osdmap e85: 12 total, 12 up, 12 in 2024-09-12T03:27:06.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.448+0000 7f2e61995640 1 -- 172.21.15.64:0/1480849019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e5c102790 msgr2=0x7f2e5c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:06.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.448+0000 7f2e61995640 1 --2- 172.21.15.64:0/1480849019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e5c102790 0x7f2e5c1067e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2e50009900 tx=0x7f2e5002a410 comp rx=0 tx=0).stop 2024-09-12T03:27:06.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.448+0000 7f2e61995640 1 -- 172.21.15.64:0/1480849019 shutdown_connections 2024-09-12T03:27:06.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.448+0000 7f2e61995640 1 --2- 172.21.15.64:0/1480849019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e5c106f10 0x7f2e5c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:06.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.448+0000 7f2e61995640 1 --2- 172.21.15.64:0/1480849019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e5c102790 0x7f2e5c1067e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:06.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.448+0000 7f2e61995640 1 --2- 172.21.15.64:0/1480849019 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e5c101de0 0x7f2e5c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:06.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.448+0000 7f2e61995640 1 -- 172.21.15.64:0/1480849019 >> 172.21.15.64:0/1480849019 conn(0x7f2e5c0fd8c0 msgr2=0x7f2e5c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:06.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.448+0000 7f2e61995640 1 -- 172.21.15.64:0/1480849019 shutdown_connections 2024-09-12T03:27:06.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.448+0000 7f2e61995640 1 -- 172.21.15.64:0/1480849019 wait complete. 2024-09-12T03:27:06.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.448+0000 7f2e61995640 1 Processor -- start 2024-09-12T03:27:06.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.448+0000 7f2e61995640 1 -- start start 2024-09-12T03:27:06.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.449+0000 7f2e61995640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e5c101de0 0x7f2e5c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:06.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.449+0000 7f2e61995640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e5c102790 0x7f2e5c1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:06.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.449+0000 7f2e61995640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e5c106f10 0x7f2e5c19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:06.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.449+0000 7f2e61995640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c111e20 con 0x7f2e5c101de0 2024-09-12T03:27:06.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.449+0000 7f2e61995640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c111ca0 con 0x7f2e5c106f10 2024-09-12T03:27:06.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.449+0000 7f2e61995640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c111fa0 con 0x7f2e5c102790 2024-09-12T03:27:06.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.449+0000 7f2e5affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e5c101de0 0x7f2e5c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:06.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.449+0000 7f2e5affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e5c101de0 0x7f2e5c1983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45740/0 (socket says 172.21.15.64:45740) 2024-09-12T03:27:06.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.449+0000 7f2e5affd640 1 -- 172.21.15.64:0/1742932991 learned_addr learned my addr 172.21.15.64:0/1742932991 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:27:06.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.449+0000 7f2e5a7fc640 1 --2- 172.21.15.64:0/1742932991 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e5c102790 0x7f2e5c1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:06.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.449+0000 7f2e5b7fe640 1 --2- 172.21.15.64:0/1742932991 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e5c106f10 0x7f2e5c19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:06.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.449+0000 7f2e5affd640 1 -- 172.21.15.64:0/1742932991 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e5c102790 msgr2=0x7f2e5c1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:06.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.449+0000 7f2e5affd640 1 --2- 172.21.15.64:0/1742932991 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e5c102790 0x7f2e5c1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:06.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.449+0000 7f2e5affd640 1 -- 172.21.15.64:0/1742932991 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e5c106f10 msgr2=0x7f2e5c19cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:06.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.449+0000 7f2e5affd640 1 --2- 172.21.15.64:0/1742932991 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e5c106f10 0x7f2e5c19cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:06.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.449+0000 7f2e5affd640 1 -- 172.21.15.64:0/1742932991 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e5c19d350 con 0x7f2e5c101de0 2024-09-12T03:27:06.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.449+0000 7f2e5b7fe640 1 --2- 172.21.15.64:0/1742932991 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e5c106f10 0x7f2e5c19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:27:06.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.449+0000 7f2e5affd640 1 --2- 172.21.15.64:0/1742932991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e5c101de0 0x7f2e5c1983a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c00c890 tx=0x7f2e4c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:06.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.449+0000 7f2e60993640 1 -- 172.21.15.64:0/1742932991 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c004220 con 0x7f2e5c101de0 2024-09-12T03:27:06.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.449+0000 7f2e60993640 1 -- 172.21.15.64:0/1742932991 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e4c0043c0 con 0x7f2e5c101de0 2024-09-12T03:27:06.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.449+0000 7f2e60993640 1 -- 172.21.15.64:0/1742932991 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c004a00 con 0x7f2e5c101de0 2024-09-12T03:27:06.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.449+0000 7f2e61995640 1 -- 172.21.15.64:0/1742932991 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e5c19d640 con 0x7f2e5c101de0 2024-09-12T03:27:06.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.450+0000 7f2e61995640 1 -- 172.21.15.64:0/1742932991 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e5c1a4f20 con 0x7f2e5c101de0 2024-09-12T03:27:06.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.450+0000 7f2e61995640 1 -- 172.21.15.64:0/1742932991 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e5c1021c0 con 0x7f2e5c101de0 2024-09-12T03:27:06.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.452+0000 7f2e60993640 1 -- 172.21.15.64:0/1742932991 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2e4c004ba0 con 0x7f2e5c101de0 2024-09-12T03:27:06.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.453+0000 7f2e60993640 1 --2- 172.21.15.64:0/1742932991 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f2e440a2b80 0x7f2e440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:06.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.453+0000 7f2e5a7fc640 1 --2- 172.21.15.64:0/1742932991 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f2e440a2b80 0x7f2e440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:06.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.453+0000 7f2e60993640 1 -- 172.21.15.64:0/1742932991 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8770+0+0 (secure 0 0 0) 0x7f2e4c0cf350 con 0x7f2e5c101de0 2024-09-12T03:27:06.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.453+0000 7f2e5a7fc640 1 --2- 172.21.15.64:0/1742932991 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f2e440a2b80 0x7f2e440a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2e50031440 tx=0x7f2e50005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:06.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.457+0000 7f2e60993640 1 -- 172.21.15.64:0/1742932991 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2e4c014030 con 0x7f2e5c101de0 2024-09-12T03:27:06.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:06.632+0000 7f2e61995640 1 -- 172.21.15.64:0/1742932991 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f2e5c199700 con 0x7f2e5c101de0 2024-09-12T03:27:07.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:07.039+0000 7f2e60993640 1 -- 172.21.15.64:0/1742932991 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v87) ==== 169+0+0 (secure 0 0 0) 0x7f2e5c199700 con 0x7f2e5c101de0 2024-09-12T03:27:07.039 INFO:teuthology.orchestra.run.smithi064.stderr:set pool 4 allow_ec_overwrites to true 2024-09-12T03:27:07.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:07.042+0000 7f2e61995640 1 -- 172.21.15.64:0/1742932991 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f2e440a2b80 msgr2=0x7f2e440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:07.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:07.042+0000 7f2e61995640 1 --2- 172.21.15.64:0/1742932991 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f2e440a2b80 0x7f2e440a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2e50031440 tx=0x7f2e50005b50 comp rx=0 tx=0).stop 2024-09-12T03:27:07.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:07.042+0000 7f2e61995640 1 -- 172.21.15.64:0/1742932991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e5c101de0 msgr2=0x7f2e5c1983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:07.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:07.042+0000 7f2e61995640 1 --2- 172.21.15.64:0/1742932991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e5c101de0 0x7f2e5c1983a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c00c890 tx=0x7f2e4c00cd50 comp rx=0 tx=0).stop 2024-09-12T03:27:07.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:07.042+0000 7f2e61995640 1 -- 172.21.15.64:0/1742932991 shutdown_connections 2024-09-12T03:27:07.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:07.042+0000 7f2e61995640 1 --2- 172.21.15.64:0/1742932991 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f2e440a2b80 0x7f2e440a5040 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:07.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:07.042+0000 7f2e61995640 1 --2- 172.21.15.64:0/1742932991 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e5c106f10 0x7f2e5c19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:07.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:07.042+0000 7f2e61995640 1 --2- 172.21.15.64:0/1742932991 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e5c102790 0x7f2e5c1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:07.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:07.042+0000 7f2e61995640 1 --2- 172.21.15.64:0/1742932991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e5c101de0 0x7f2e5c1983a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:07.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:07.042+0000 7f2e61995640 1 -- 172.21.15.64:0/1742932991 >> 172.21.15.64:0/1742932991 conn(0x7f2e5c0fd8c0 msgr2=0x7f2e5c0fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:07.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:07.042+0000 7f2e61995640 1 -- 172.21.15.64:0/1742932991 shutdown_connections 2024-09-12T03:27:07.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:07.042+0000 7f2e61995640 1 -- 172.21.15.64:0/1742932991 wait complete. 2024-09-12T03:27:07.358 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:07 smithi064 ceph-mon[30952]: pgmap v284: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2024-09-12T03:27:07.358 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:07 smithi064 ceph-mon[30952]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T03:27:07.358 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:07 smithi064 ceph-mon[30952]: osdmap e86: 12 total, 12 up, 12 in 2024-09-12T03:27:07.358 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:07 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/1742932991' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-12T03:27:07.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:07 smithi115 ceph-mon[34801]: pgmap v284: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2024-09-12T03:27:07.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:07 smithi115 ceph-mon[34801]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T03:27:07.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:07 smithi115 ceph-mon[34801]: osdmap e86: 12 total, 12 up, 12 in 2024-09-12T03:27:07.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:07 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/1742932991' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-12T03:27:07.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:07 smithi195 ceph-mon[34786]: pgmap v284: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2024-09-12T03:27:07.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:07 smithi195 ceph-mon[34786]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T03:27:07.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:07 smithi195 ceph-mon[34786]: osdmap e86: 12 total, 12 up, 12 in 2024-09-12T03:27:07.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:07 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/1742932991' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-12T03:27:07.683 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-12T03:27:07.947 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:08.370 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:08 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/1742932991' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-12T03:27:08.371 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:08 smithi064 ceph-mon[30952]: osdmap e87: 12 total, 12 up, 12 in 2024-09-12T03:27:08.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:08 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/1742932991' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-12T03:27:08.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:08 smithi115 ceph-mon[34801]: osdmap e87: 12 total, 12 up, 12 in 2024-09-12T03:27:08.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:08 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/1742932991' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-12T03:27:08.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:08 smithi195 ceph-mon[34786]: osdmap e87: 12 total, 12 up, 12 in 2024-09-12T03:27:08.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.454+0000 7fe91753e640 1 -- 172.21.15.64:0/1377673811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe910106a80 msgr2=0x7fe910106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:08.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.454+0000 7fe91753e640 1 --2- 172.21.15.64:0/1377673811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe910106a80 0x7fe910106e60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe90c009930 tx=0x7fe90c02a420 comp rx=0 tx=0).stop 2024-09-12T03:27:08.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.454+0000 7fe91753e640 1 -- 172.21.15.64:0/1377673811 shutdown_connections 2024-09-12T03:27:08.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.454+0000 7fe91753e640 1 --2- 172.21.15.64:0/1377673811 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe910101420 0x7fe91010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:08.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.454+0000 7fe91753e640 1 --2- 172.21.15.64:0/1377673811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe910100a80 0x7fe910100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:08.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.454+0000 7fe91753e640 1 --2- 172.21.15.64:0/1377673811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe910106a80 0x7fe910106e60 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:08.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.454+0000 7fe91753e640 1 -- 172.21.15.64:0/1377673811 >> 172.21.15.64:0/1377673811 conn(0x7fe9100fc7a0 msgr2=0x7fe9100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:08.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.454+0000 7fe91753e640 1 -- 172.21.15.64:0/1377673811 shutdown_connections 2024-09-12T03:27:08.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.454+0000 7fe91753e640 1 -- 172.21.15.64:0/1377673811 wait complete. 2024-09-12T03:27:08.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.455+0000 7fe91753e640 1 Processor -- start 2024-09-12T03:27:08.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.455+0000 7fe91753e640 1 -- start start 2024-09-12T03:27:08.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.455+0000 7fe91753e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe910100a80 0x7fe910198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:08.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.455+0000 7fe91753e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe910101420 0x7fe910198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:08.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.455+0000 7fe91753e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe910106a80 0x7fe91019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:08.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.455+0000 7fe91753e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe910110dd0 con 0x7fe910106a80 2024-09-12T03:27:08.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.455+0000 7fe91753e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe910110c50 con 0x7fe910100a80 2024-09-12T03:27:08.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.455+0000 7fe91753e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe910110f50 con 0x7fe910101420 2024-09-12T03:27:08.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.455+0000 7fe915adb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe910106a80 0x7fe91019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:08.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.455+0000 7fe915adb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe910106a80 0x7fe91019cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56326/0 (socket says 172.21.15.64:56326) 2024-09-12T03:27:08.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.455+0000 7fe915adb640 1 -- 172.21.15.64:0/2233096125 learned_addr learned my addr 172.21.15.64:0/2233096125 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:27:08.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.455+0000 7fe9152da640 1 --2- 172.21.15.64:0/2233096125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe910100a80 0x7fe910198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:08.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.455+0000 7fe914ad9640 1 --2- 172.21.15.64:0/2233096125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe910101420 0x7fe910198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:08.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.455+0000 7fe915adb640 1 -- 172.21.15.64:0/2233096125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe910101420 msgr2=0x7fe910198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:08.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.455+0000 7fe915adb640 1 --2- 172.21.15.64:0/2233096125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe910101420 0x7fe910198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:08.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.455+0000 7fe915adb640 1 -- 172.21.15.64:0/2233096125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe910100a80 msgr2=0x7fe910198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:08.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.455+0000 7fe915adb640 1 --2- 172.21.15.64:0/2233096125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe910100a80 0x7fe910198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:08.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.455+0000 7fe915adb640 1 -- 172.21.15.64:0/2233096125 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe91019d4b0 con 0x7fe910106a80 2024-09-12T03:27:08.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.455+0000 7fe914ad9640 1 --2- 172.21.15.64:0/2233096125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe910101420 0x7fe910198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:27:08.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.456+0000 7fe915adb640 1 --2- 172.21.15.64:0/2233096125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe910106a80 0x7fe91019cdd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe90800b970 tx=0x7fe90800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:08.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.456+0000 7fe9067fc640 1 -- 172.21.15.64:0/2233096125 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe908019070 con 0x7fe910106a80 2024-09-12T03:27:08.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.456+0000 7fe91753e640 1 -- 172.21.15.64:0/2233096125 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe91019d7a0 con 0x7fe910106a80 2024-09-12T03:27:08.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.456+0000 7fe9067fc640 1 -- 172.21.15.64:0/2233096125 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe908014440 con 0x7fe910106a80 2024-09-12T03:27:08.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.456+0000 7fe9067fc640 1 -- 172.21.15.64:0/2233096125 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9080075d0 con 0x7fe910106a80 2024-09-12T03:27:08.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.456+0000 7fe91753e640 1 -- 172.21.15.64:0/2233096125 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9101a5080 con 0x7fe910106a80 2024-09-12T03:27:08.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.457+0000 7fe91753e640 1 -- 172.21.15.64:0/2233096125 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe910100f60 con 0x7fe910106a80 2024-09-12T03:27:08.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.459+0000 7fe9067fc640 1 -- 172.21.15.64:0/2233096125 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe90801c070 con 0x7fe910106a80 2024-09-12T03:27:08.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.460+0000 7fe9067fc640 1 --2- 172.21.15.64:0/2233096125 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe8fc0a2b80 0x7fe8fc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:08.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.460+0000 7fe9152da640 1 --2- 172.21.15.64:0/2233096125 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe8fc0a2b80 0x7fe8fc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:08.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.460+0000 7fe9067fc640 1 -- 172.21.15.64:0/2233096125 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8770+0+0 (secure 0 0 0) 0x7fe9080ced90 con 0x7fe910106a80 2024-09-12T03:27:08.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.460+0000 7fe9152da640 1 --2- 172.21.15.64:0/2233096125 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe8fc0a2b80 0x7fe8fc0a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe90c000c00 tx=0x7fe90c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:08.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.463+0000 7fe9067fc640 1 -- 172.21.15.64:0/2233096125 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe90809bad0 con 0x7fe910106a80 2024-09-12T03:27:08.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:08.679+0000 7fe91753e640 1 -- 172.21.15.64:0/2233096125 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fe9101a5320 con 0x7fe910106a80 2024-09-12T03:27:09.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:09.057+0000 7fe9067fc640 1 -- 172.21.15.64:0/2233096125 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v16) ==== 137+0+0 (secure 0 0 0) 0x7fe90809b3d0 con 0x7fe910106a80 2024-09-12T03:27:09.057 INFO:teuthology.orchestra.run.smithi064.stderr:added data pool 4 to fsmap 2024-09-12T03:27:09.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:09.059+0000 7fe91753e640 1 -- 172.21.15.64:0/2233096125 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe8fc0a2b80 msgr2=0x7fe8fc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:09.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:09.059+0000 7fe91753e640 1 --2- 172.21.15.64:0/2233096125 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe8fc0a2b80 0x7fe8fc0a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe90c000c00 tx=0x7fe90c031400 comp rx=0 tx=0).stop 2024-09-12T03:27:09.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:09.059+0000 7fe91753e640 1 -- 172.21.15.64:0/2233096125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe910106a80 msgr2=0x7fe91019cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:09.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:09.059+0000 7fe91753e640 1 --2- 172.21.15.64:0/2233096125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe910106a80 0x7fe91019cdd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe90800b970 tx=0x7fe90800be30 comp rx=0 tx=0).stop 2024-09-12T03:27:09.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:09.059+0000 7fe91753e640 1 -- 172.21.15.64:0/2233096125 shutdown_connections 2024-09-12T03:27:09.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:09.059+0000 7fe91753e640 1 --2- 172.21.15.64:0/2233096125 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe8fc0a2b80 0x7fe8fc0a5040 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:09.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:09.059+0000 7fe91753e640 1 --2- 172.21.15.64:0/2233096125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe910106a80 0x7fe91019cdd0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:09.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:09.059+0000 7fe91753e640 1 --2- 172.21.15.64:0/2233096125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe910101420 0x7fe910198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:09.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:09.059+0000 7fe91753e640 1 --2- 172.21.15.64:0/2233096125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe910100a80 0x7fe910198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:09.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:09.059+0000 7fe91753e640 1 -- 172.21.15.64:0/2233096125 >> 172.21.15.64:0/2233096125 conn(0x7fe9100fc7a0 msgr2=0x7fe910104990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:09.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:09.059+0000 7fe91753e640 1 -- 172.21.15.64:0/2233096125 shutdown_connections 2024-09-12T03:27:09.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:09.059+0000 7fe91753e640 1 -- 172.21.15.64:0/2233096125 wait complete. 2024-09-12T03:27:09.367 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:09 smithi064 ceph-mon[30952]: pgmap v287: 129 pgs: 10 unknown, 119 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:09.367 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:09 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/2233096125' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-12T03:27:09.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:09 smithi115 ceph-mon[34801]: pgmap v287: 129 pgs: 10 unknown, 119 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:09.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:09 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/2233096125' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-12T03:27:09.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:09 smithi195 ceph-mon[34786]: pgmap v287: 129 pgs: 10 unknown, 119 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:09.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:09 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/2233096125' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-12T03:27:09.712 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:27:09.976 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:10.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:10 smithi115 ceph-mon[34801]: osdmap e88: 12 total, 12 up, 12 in 2024-09-12T03:27:10.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:10 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/2233096125' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-12T03:27:10.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:10 smithi115 ceph-mon[34801]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:27:10.406 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:10 smithi064 ceph-mon[30952]: osdmap e88: 12 total, 12 up, 12 in 2024-09-12T03:27:10.406 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:10 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/2233096125' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-12T03:27:10.406 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:10 smithi064 ceph-mon[30952]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:27:10.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:10 smithi195 ceph-mon[34786]: osdmap e88: 12 total, 12 up, 12 in 2024-09-12T03:27:10.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:10 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/2233096125' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-12T03:27:10.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:10 smithi195 ceph-mon[34786]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:27:10.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.486+0000 7f63e7d37640 1 -- 172.21.15.64:0/3436122652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63e0101de0 msgr2=0x7f63e01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:10.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.486+0000 7f63e7d37640 1 --2- 172.21.15.64:0/3436122652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63e0101de0 0x7f63e01021c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f63d4009ca0 tx=0x7f63d402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:27:10.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.486+0000 7f63e7d37640 1 -- 172.21.15.64:0/3436122652 shutdown_connections 2024-09-12T03:27:10.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.486+0000 7f63e7d37640 1 --2- 172.21.15.64:0/3436122652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f63e0106f10 0x7f63e010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:10.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.486+0000 7f63e7d37640 1 --2- 172.21.15.64:0/3436122652 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f63e0102790 0x7f63e01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:10.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.486+0000 7f63e7d37640 1 --2- 172.21.15.64:0/3436122652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63e0101de0 0x7f63e01021c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:10.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.486+0000 7f63e7d37640 1 -- 172.21.15.64:0/3436122652 >> 172.21.15.64:0/3436122652 conn(0x7f63e00fd8c0 msgr2=0x7f63e00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:10.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.486+0000 7f63e7d37640 1 -- 172.21.15.64:0/3436122652 shutdown_connections 2024-09-12T03:27:10.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.486+0000 7f63e7d37640 1 -- 172.21.15.64:0/3436122652 wait complete. 2024-09-12T03:27:10.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.487+0000 7f63e7d37640 1 Processor -- start 2024-09-12T03:27:10.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.487+0000 7f63e7d37640 1 -- start start 2024-09-12T03:27:10.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.487+0000 7f63e7d37640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f63e0101de0 0x7f63e0198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:10.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.487+0000 7f63e7d37640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f63e0102790 0x7f63e0198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:10.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.487+0000 7f63e7d37640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63e0106f10 0x7f63e019cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:10.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.487+0000 7f63e7d37640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f63e0111e20 con 0x7f63e0106f10 2024-09-12T03:27:10.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.487+0000 7f63e7d37640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f63e0111ca0 con 0x7f63e0102790 2024-09-12T03:27:10.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.487+0000 7f63e7d37640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f63e0111fa0 con 0x7f63e0101de0 2024-09-12T03:27:10.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.487+0000 7f63e62d4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63e0106f10 0x7f63e019cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:10.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.487+0000 7f63e62d4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63e0106f10 0x7f63e019cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56332/0 (socket says 172.21.15.64:56332) 2024-09-12T03:27:10.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.487+0000 7f63e62d4640 1 -- 172.21.15.64:0/618708129 learned_addr learned my addr 172.21.15.64:0/618708129 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:27:10.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.487+0000 7f63e5ad3640 1 --2- 172.21.15.64:0/618708129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f63e0101de0 0x7f63e0198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:10.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.488+0000 7f63e62d4640 1 -- 172.21.15.64:0/618708129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f63e0101de0 msgr2=0x7f63e0198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:10.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.488+0000 7f63e62d4640 1 --2- 172.21.15.64:0/618708129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f63e0101de0 0x7f63e0198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:10.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.488+0000 7f63e52d2640 1 --2- 172.21.15.64:0/618708129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f63e0102790 0x7f63e0198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:10.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.488+0000 7f63e62d4640 1 -- 172.21.15.64:0/618708129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f63e0102790 msgr2=0x7f63e0198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:10.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.488+0000 7f63e62d4640 1 --2- 172.21.15.64:0/618708129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f63e0102790 0x7f63e0198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:10.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.488+0000 7f63e62d4640 1 -- 172.21.15.64:0/618708129 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e019d200 con 0x7f63e0106f10 2024-09-12T03:27:10.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.488+0000 7f63e52d2640 1 --2- 172.21.15.64:0/618708129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f63e0102790 0x7f63e0198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:27:10.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.488+0000 7f63e62d4640 1 --2- 172.21.15.64:0/618708129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63e0106f10 0x7f63e019cb20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f63d800c970 tx=0x7f63d800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:10.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.488+0000 7f63deffd640 1 -- 172.21.15.64:0/618708129 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d8007bb0 con 0x7f63e0106f10 2024-09-12T03:27:10.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.488+0000 7f63deffd640 1 -- 172.21.15.64:0/618708129 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63d8007d50 con 0x7f63e0106f10 2024-09-12T03:27:10.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.488+0000 7f63e7d37640 1 -- 172.21.15.64:0/618708129 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63e019d4f0 con 0x7f63e0106f10 2024-09-12T03:27:10.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.488+0000 7f63deffd640 1 -- 172.21.15.64:0/618708129 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d8018700 con 0x7f63e0106f10 2024-09-12T03:27:10.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.488+0000 7f63e7d37640 1 -- 172.21.15.64:0/618708129 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63e01a4dd0 con 0x7f63e0106f10 2024-09-12T03:27:10.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.489+0000 7f63e7d37640 1 -- 172.21.15.64:0/618708129 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63e01021c0 con 0x7f63e0106f10 2024-09-12T03:27:10.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.490+0000 7f63deffd640 1 -- 172.21.15.64:0/618708129 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f63d800fa20 con 0x7f63e0106f10 2024-09-12T03:27:10.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.491+0000 7f63deffd640 1 --2- 172.21.15.64:0/618708129 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f63c80a2b80 0x7f63c80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:10.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.491+0000 7f63e5ad3640 1 --2- 172.21.15.64:0/618708129 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f63c80a2b80 0x7f63c80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:10.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.491+0000 7f63deffd640 1 -- 172.21.15.64:0/618708129 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f63d80cf2f0 con 0x7f63e0106f10 2024-09-12T03:27:10.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.491+0000 7f63e5ad3640 1 --2- 172.21.15.64:0/618708129 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f63c80a2b80 0x7f63c80a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f63d4009a60 tx=0x7f63d4031000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:10.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.495+0000 7f63deffd640 1 -- 172.21.15.64:0/618708129 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f63d8014030 con 0x7f63e0106f10 2024-09-12T03:27:10.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.708+0000 7f63e7d37640 1 -- 172.21.15.64:0/618708129 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63e002d7c0 con 0x7f63e0106f10 2024-09-12T03:27:10.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.708+0000 7f63deffd640 1 -- 172.21.15.64:0/618708129 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+11848 (secure 0 0 0) 0x7f63d809c010 con 0x7f63e0106f10 2024-09-12T03:27:10.709 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:27:10.709 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":16,"btime":"2024-09-12T03:27:09:051020+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2580078738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2580078738},{"type":"v1","addr":"172.21.15.64:6835","nonce":2580078738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2333415004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2333415004},{"type":"v1","addr":"172.21.15.64:6837","nonce":2333415004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/3030618395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":3030618395},{"type":"v1","addr":"172.21.15.64:6839","nonce":3030618395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3333057625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3333057625},{"type":"v1","addr":"172.21.15.64:6841","nonce":3333057625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2991933266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2991933266},{"type":"v1","addr":"172.21.15.195:6833","nonce":2991933266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1138960346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1138960346},{"type":"v1","addr":"172.21.15.115:6833","nonce":1138960346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/332737916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":332737916},{"type":"v1","addr":"172.21.15.115:6835","nonce":332737916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2531045583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2531045583},{"type":"v1","addr":"172.21.15.195:6835","nonce":2531045583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/71647401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":71647401},{"type":"v1","addr":"172.21.15.195:6837","nonce":71647401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2915155705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2915155705},{"type":"v1","addr":"172.21.15.115:6837","nonce":2915155705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6839/2574695134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":2574695134},{"type":"v1","addr":"172.21.15.195:6839","nonce":2574695134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:26:59.647657+0000","modified":"2024-09-12T03:27:08.680950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6839/2238031796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2238031796},{"type":"v1","addr":"172.21.15.115:6839","nonce":2238031796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24500,"qdb_cluster":[24500]},"id":1}]} 2024-09-12T03:27:10.709 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 16 2024-09-12T03:27:10.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.710+0000 7f63e7d37640 1 -- 172.21.15.64:0/618708129 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f63c80a2b80 msgr2=0x7f63c80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:10.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.710+0000 7f63e7d37640 1 --2- 172.21.15.64:0/618708129 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f63c80a2b80 0x7f63c80a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f63d4009a60 tx=0x7f63d4031000 comp rx=0 tx=0).stop 2024-09-12T03:27:10.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.710+0000 7f63e7d37640 1 -- 172.21.15.64:0/618708129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63e0106f10 msgr2=0x7f63e019cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:10.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.710+0000 7f63e7d37640 1 --2- 172.21.15.64:0/618708129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63e0106f10 0x7f63e019cb20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f63d800c970 tx=0x7f63d800ce30 comp rx=0 tx=0).stop 2024-09-12T03:27:10.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.711+0000 7f63e7d37640 1 -- 172.21.15.64:0/618708129 shutdown_connections 2024-09-12T03:27:10.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.711+0000 7f63e7d37640 1 --2- 172.21.15.64:0/618708129 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f63c80a2b80 0x7f63c80a5040 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:10.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.711+0000 7f63e7d37640 1 --2- 172.21.15.64:0/618708129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63e0106f10 0x7f63e019cb20 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:10.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.711+0000 7f63e7d37640 1 --2- 172.21.15.64:0/618708129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f63e0102790 0x7f63e0198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:10.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.711+0000 7f63e7d37640 1 --2- 172.21.15.64:0/618708129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f63e0101de0 0x7f63e0198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:10.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.711+0000 7f63e7d37640 1 -- 172.21.15.64:0/618708129 >> 172.21.15.64:0/618708129 conn(0x7f63e00fd8c0 msgr2=0x7f63e00fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:10.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.711+0000 7f63e7d37640 1 -- 172.21.15.64:0/618708129 shutdown_connections 2024-09-12T03:27:10.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:10.711+0000 7f63e7d37640 1 -- 172.21.15.64:0/618708129 wait complete. 2024-09-12T03:27:11.336 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:11 smithi064 ceph-mon[30952]: pgmap v289: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:11.336 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:11 smithi064 ceph-mon[30952]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-12T03:27:11.336 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:11 smithi064 ceph-mon[30952]: Cluster is now healthy 2024-09-12T03:27:11.336 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:11 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/618708129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:27:11.368 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:27:11.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:11 smithi115 ceph-mon[34801]: pgmap v289: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:11.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:11 smithi115 ceph-mon[34801]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-12T03:27:11.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:11 smithi115 ceph-mon[34801]: Cluster is now healthy 2024-09-12T03:27:11.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:11 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/618708129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:27:11.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:11 smithi195 ceph-mon[34786]: pgmap v289: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:11.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:11 smithi195 ceph-mon[34786]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-12T03:27:11.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:11 smithi195 ceph-mon[34786]: Cluster is now healthy 2024-09-12T03:27:11.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:11 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/618708129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:27:11.627 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:12.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.105+0000 7fe898cc6640 1 -- 172.21.15.64:0/920456366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe894101420 msgr2=0x7fe89410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:12.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.105+0000 7fe898cc6640 1 --2- 172.21.15.64:0/920456366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe894101420 0x7fe89410d950 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe88c009930 tx=0x7fe88c02a420 comp rx=0 tx=0).stop 2024-09-12T03:27:12.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.105+0000 7fe898cc6640 1 -- 172.21.15.64:0/920456366 shutdown_connections 2024-09-12T03:27:12.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.105+0000 7fe898cc6640 1 --2- 172.21.15.64:0/920456366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe894101420 0x7fe89410d950 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:12.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.105+0000 7fe898cc6640 1 --2- 172.21.15.64:0/920456366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe894100a80 0x7fe894100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:12.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.105+0000 7fe898cc6640 1 --2- 172.21.15.64:0/920456366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe894106a80 0x7fe894106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:12.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.105+0000 7fe898cc6640 1 -- 172.21.15.64:0/920456366 >> 172.21.15.64:0/920456366 conn(0x7fe8940fc7a0 msgr2=0x7fe8940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:12.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.105+0000 7fe898cc6640 1 -- 172.21.15.64:0/920456366 shutdown_connections 2024-09-12T03:27:12.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.105+0000 7fe898cc6640 1 -- 172.21.15.64:0/920456366 wait complete. 2024-09-12T03:27:12.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.106+0000 7fe898cc6640 1 Processor -- start 2024-09-12T03:27:12.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.106+0000 7fe898cc6640 1 -- start start 2024-09-12T03:27:12.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.106+0000 7fe898cc6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe894100a80 0x7fe894198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:12.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.106+0000 7fe898cc6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe894101420 0x7fe8941988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:12.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.106+0000 7fe898cc6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe894106a80 0x7fe89419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:12.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.106+0000 7fe898cc6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe894110d50 con 0x7fe894101420 2024-09-12T03:27:12.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.106+0000 7fe898cc6640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe894110bd0 con 0x7fe894100a80 2024-09-12T03:27:12.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.106+0000 7fe898cc6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe894110ed0 con 0x7fe894106a80 2024-09-12T03:27:12.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.106+0000 7fe891d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe894101420 0x7fe8941988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:12.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.106+0000 7fe891d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe894101420 0x7fe8941988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56354/0 (socket says 172.21.15.64:56354) 2024-09-12T03:27:12.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.106+0000 7fe891d9b640 1 -- 172.21.15.64:0/520322643 learned_addr learned my addr 172.21.15.64:0/520322643 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:27:12.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.106+0000 7fe892d9d640 1 --2- 172.21.15.64:0/520322643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe894106a80 0x7fe89419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:12.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.106+0000 7fe89259c640 1 --2- 172.21.15.64:0/520322643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe894100a80 0x7fe894198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:12.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.106+0000 7fe891d9b640 1 -- 172.21.15.64:0/520322643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe894106a80 msgr2=0x7fe89419cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:12.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.106+0000 7fe891d9b640 1 --2- 172.21.15.64:0/520322643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe894106a80 0x7fe89419cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:12.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.106+0000 7fe891d9b640 1 -- 172.21.15.64:0/520322643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe894100a80 msgr2=0x7fe894198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:12.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.106+0000 7fe891d9b640 1 --2- 172.21.15.64:0/520322643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe894100a80 0x7fe894198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:12.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.106+0000 7fe891d9b640 1 -- 172.21.15.64:0/520322643 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe89419d320 con 0x7fe894101420 2024-09-12T03:27:12.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.107+0000 7fe89259c640 1 --2- 172.21.15.64:0/520322643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe894100a80 0x7fe894198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:27:12.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.107+0000 7fe891d9b640 1 --2- 172.21.15.64:0/520322643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe894101420 0x7fe8941988b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe88400b4a0 tx=0x7fe88400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:12.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.107+0000 7fe88b7fe640 1 -- 172.21.15.64:0/520322643 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe884014070 con 0x7fe894101420 2024-09-12T03:27:12.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.107+0000 7fe898cc6640 1 -- 172.21.15.64:0/520322643 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe89419d610 con 0x7fe894101420 2024-09-12T03:27:12.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.107+0000 7fe88b7fe640 1 -- 172.21.15.64:0/520322643 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe884010ac0 con 0x7fe894101420 2024-09-12T03:27:12.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.107+0000 7fe88b7fe640 1 -- 172.21.15.64:0/520322643 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe884017c30 con 0x7fe894101420 2024-09-12T03:27:12.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.107+0000 7fe898cc6640 1 -- 172.21.15.64:0/520322643 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8941a4ef0 con 0x7fe894101420 2024-09-12T03:27:12.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.108+0000 7fe898cc6640 1 -- 172.21.15.64:0/520322643 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe894100ee0 con 0x7fe894101420 2024-09-12T03:27:12.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.109+0000 7fe88b7fe640 1 -- 172.21.15.64:0/520322643 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe884017070 con 0x7fe894101420 2024-09-12T03:27:12.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.110+0000 7fe88b7fe640 1 --2- 172.21.15.64:0/520322643 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe8800a2c00 0x7fe8800a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:12.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.110+0000 7fe89259c640 1 --2- 172.21.15.64:0/520322643 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe8800a2c00 0x7fe8800a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:12.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.110+0000 7fe88b7fe640 1 -- 172.21.15.64:0/520322643 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fe8840ce8b0 con 0x7fe894101420 2024-09-12T03:27:12.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.110+0000 7fe89259c640 1 --2- 172.21.15.64:0/520322643 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe8800a2c00 0x7fe8800a50c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe87c005fd0 tx=0x7fe87c009520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:12.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.114+0000 7fe88b7fe640 1 -- 172.21.15.64:0/520322643 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe88409b5d0 con 0x7fe894101420 2024-09-12T03:27:12.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.290+0000 7fe898cc6640 1 -- 172.21.15.64:0/520322643 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe894109a90 con 0x7fe894101420 2024-09-12T03:27:12.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.292+0000 7fe88b7fe640 1 -- 172.21.15.64:0/520322643 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24209 (secure 0 0 0) 0x7fe88409aed0 con 0x7fe894101420 2024-09-12T03:27:12.292 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:27:12.292 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":88,"fsid":"b0812460-70b5-11ef-bcea-c7b262605968","created":"2024-09-12T03:19:15.900322+0000","modified":"2024-09-12T03:27:08.680818+0000","last_up_change":"2024-09-12T03:26:06.122971+0000","last_in_change":"2024-09-12T03:25:43.130689+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:22:21.915870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:26:55.826348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:26:57.984319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T03:27:04.460292+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"326271c9-1f28-4ce6-b3be-fcc57479d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6803","nonce":1307832590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6805","nonce":1307832590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6809","nonce":1307832590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6807","nonce":1307832590}]},"public_addr":"172.21.15.64:6803/1307832590","cluster_addr":"172.21.15.64:6805/1307832590","heartbeat_back_addr":"172.21.15.64:6809/1307832590","heartbeat_front_addr":"172.21.15.64:6807/1307832590","state":["exists","up"]},{"osd":1,"uuid":"7dca0c28-c234-423c-936e-2351d63f918e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6801","nonce":3123728622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6803","nonce":3123728622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6807","nonce":3123728622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6805","nonce":3123728622}]},"public_addr":"172.21.15.115:6801/3123728622","cluster_addr":"172.21.15.115:6803/3123728622","heartbeat_back_addr":"172.21.15.115:6807/3123728622","heartbeat_front_addr":"172.21.15.115:6805/3123728622","state":["exists","up"]},{"osd":2,"uuid":"4b3a3c7a-aec6-455a-9a9a-1ac9cc900f24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6801","nonce":1664125451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6803","nonce":1664125451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6807","nonce":1664125451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6805","nonce":1664125451}]},"public_addr":"172.21.15.195:6801/1664125451","cluster_addr":"172.21.15.195:6803/1664125451","heartbeat_back_addr":"172.21.15.195:6807/1664125451","heartbeat_front_addr":"172.21.15.195:6805/1664125451","state":["exists","up"]},{"osd":3,"uuid":"8768c339-2c1e-4294-8b78-3c2adab6750f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6811","nonce":3935517185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6813","nonce":3935517185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6817","nonce":3935517185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6815","nonce":3935517185}]},"public_addr":"172.21.15.64:6811/3935517185","cluster_addr":"172.21.15.64:6813/3935517185","heartbeat_back_addr":"172.21.15.64:6817/3935517185","heartbeat_front_addr":"172.21.15.64:6815/3935517185","state":["exists","up"]},{"osd":4,"uuid":"123aac47-a8ae-47c7-93a7-0dd293356322","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6809","nonce":446665227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6811","nonce":446665227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6815","nonce":446665227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6813","nonce":446665227}]},"public_addr":"172.21.15.115:6809/446665227","cluster_addr":"172.21.15.115:6811/446665227","heartbeat_back_addr":"172.21.15.115:6815/446665227","heartbeat_front_addr":"172.21.15.115:6813/446665227","state":["exists","up"]},{"osd":5,"uuid":"c90bcaed-e7a8-4852-8677-b01dc73ea44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6809","nonce":1187293668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6811","nonce":1187293668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6815","nonce":1187293668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6813","nonce":1187293668}]},"public_addr":"172.21.15.195:6809/1187293668","cluster_addr":"172.21.15.195:6811/1187293668","heartbeat_back_addr":"172.21.15.195:6815/1187293668","heartbeat_front_addr":"172.21.15.195:6813/1187293668","state":["exists","up"]},{"osd":6,"uuid":"fd292547-6695-4889-8867-e74318bbb00f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6819","nonce":560343274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6821","nonce":560343274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6825","nonce":560343274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6823","nonce":560343274}]},"public_addr":"172.21.15.64:6819/560343274","cluster_addr":"172.21.15.64:6821/560343274","heartbeat_back_addr":"172.21.15.64:6825/560343274","heartbeat_front_addr":"172.21.15.64:6823/560343274","state":["exists","up"]},{"osd":7,"uuid":"ad7c4151-cd98-4113-879b-ca2b7c83c7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6817","nonce":3075734174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6819","nonce":3075734174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6823","nonce":3075734174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6821","nonce":3075734174}]},"public_addr":"172.21.15.115:6817/3075734174","cluster_addr":"172.21.15.115:6819/3075734174","heartbeat_back_addr":"172.21.15.115:6823/3075734174","heartbeat_front_addr":"172.21.15.115:6821/3075734174","state":["exists","up"]},{"osd":8,"uuid":"6de50049-c28f-4f21-b123-05e55bf59127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6817","nonce":2184742494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6819","nonce":2184742494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6823","nonce":2184742494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6821","nonce":2184742494}]},"public_addr":"172.21.15.195:6817/2184742494","cluster_addr":"172.21.15.195:6819/2184742494","heartbeat_back_addr":"172.21.15.195:6823/2184742494","heartbeat_front_addr":"172.21.15.195:6821/2184742494","state":["exists","up"]},{"osd":9,"uuid":"36ab792c-58f2-4beb-9024-7f3d1e3785dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6827","nonce":561728112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6829","nonce":561728112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6833","nonce":561728112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6831","nonce":561728112}]},"public_addr":"172.21.15.64:6827/561728112","cluster_addr":"172.21.15.64:6829/561728112","heartbeat_back_addr":"172.21.15.64:6833/561728112","heartbeat_front_addr":"172.21.15.64:6831/561728112","state":["exists","up"]},{"osd":10,"uuid":"d1f7881c-ec3f-466e-9cea-1d5cb441d970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6825","nonce":2977887627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6827","nonce":2977887627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6831","nonce":2977887627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6829","nonce":2977887627}]},"public_addr":"172.21.15.115:6825/2977887627","cluster_addr":"172.21.15.115:6827/2977887627","heartbeat_back_addr":"172.21.15.115:6831/2977887627","heartbeat_front_addr":"172.21.15.115:6829/2977887627","state":["exists","up"]},{"osd":11,"uuid":"7c46c028-22c7-450f-9927-2144a7a09e9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6825","nonce":591599352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6827","nonce":591599352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6831","nonce":591599352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6829","nonce":591599352}]},"public_addr":"172.21.15.195:6825/591599352","cluster_addr":"172.21.15.195:6827/591599352","heartbeat_back_addr":"172.21.15.195:6831/591599352","heartbeat_front_addr":"172.21.15.195:6829/591599352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:29.151454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:54.308909+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:18.680995+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:44.687548+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:10.166131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:35.026132+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:24:00.242966+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:24:24.600848+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:24:49.563060+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:25:15.025730+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:25:39.830995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:26:04.394276+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6801/68603379":"2024-09-13T03:20:07.411102+0000","172.21.15.64:0/3827202979":"2024-09-13T03:20:07.411102+0000","172.21.15.64:6800/68603379":"2024-09-13T03:20:07.411102+0000","172.21.15.64:0/1217136221":"2024-09-13T03:20:07.411102+0000","172.21.15.64:0/3079679964":"2024-09-13T03:19:36.839431+0000","172.21.15.64:6801/3186640984":"2024-09-13T03:19:36.839431+0000","172.21.15.64:0/3334629354":"2024-09-13T03:20:07.411102+0000","172.21.15.64:6800/3186640984":"2024-09-13T03:19:36.839431+0000","172.21.15.64:0/3564418115":"2024-09-13T03:19:36.839431+0000","172.21.15.64:0/686137382":"2024-09-13T03:19:36.839431+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:27:12.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.294+0000 7fe898cc6640 1 -- 172.21.15.64:0/520322643 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe8800a2c00 msgr2=0x7fe8800a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:12.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.294+0000 7fe898cc6640 1 --2- 172.21.15.64:0/520322643 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe8800a2c00 0x7fe8800a50c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe87c005fd0 tx=0x7fe87c009520 comp rx=0 tx=0).stop 2024-09-12T03:27:12.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.294+0000 7fe898cc6640 1 -- 172.21.15.64:0/520322643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe894101420 msgr2=0x7fe8941988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:12.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.294+0000 7fe898cc6640 1 --2- 172.21.15.64:0/520322643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe894101420 0x7fe8941988b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe88400b4a0 tx=0x7fe88400b960 comp rx=0 tx=0).stop 2024-09-12T03:27:12.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.294+0000 7fe898cc6640 1 -- 172.21.15.64:0/520322643 shutdown_connections 2024-09-12T03:27:12.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.294+0000 7fe898cc6640 1 --2- 172.21.15.64:0/520322643 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe8800a2c00 0x7fe8800a50c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:12.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.294+0000 7fe898cc6640 1 --2- 172.21.15.64:0/520322643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe894106a80 0x7fe89419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:12.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.294+0000 7fe898cc6640 1 --2- 172.21.15.64:0/520322643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe894101420 0x7fe8941988b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:12.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.294+0000 7fe898cc6640 1 --2- 172.21.15.64:0/520322643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe894100a80 0x7fe894198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:12.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.294+0000 7fe898cc6640 1 -- 172.21.15.64:0/520322643 >> 172.21.15.64:0/520322643 conn(0x7fe8940fc7a0 msgr2=0x7fe89406aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:12.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.294+0000 7fe898cc6640 1 -- 172.21.15.64:0/520322643 shutdown_connections 2024-09-12T03:27:12.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:12.294+0000 7fe898cc6640 1 -- 172.21.15.64:0/520322643 wait complete. 2024-09-12T03:27:12.926 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:27:13.185 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:13.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:13 smithi064 ceph-mon[30952]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:13.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:13 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/520322643' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:27:13.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:13 smithi115 ceph-mon[34801]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:13.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:13 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/520322643' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:27:13.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:13 smithi195 ceph-mon[34786]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:13.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:13 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/520322643' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:27:13.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.673+0000 7faf2812e640 1 -- 172.21.15.64:0/1711937679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf20106f10 msgr2=0x7faf2010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:13.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.673+0000 7faf2812e640 1 --2- 172.21.15.64:0/1711937679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf20106f10 0x7faf2010ea90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7faf18009ca0 tx=0x7faf1802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:27:13.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.673+0000 7faf2812e640 1 -- 172.21.15.64:0/1711937679 shutdown_connections 2024-09-12T03:27:13.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.673+0000 7faf2812e640 1 --2- 172.21.15.64:0/1711937679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf20106f10 0x7faf2010ea90 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:13.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.673+0000 7faf2812e640 1 --2- 172.21.15.64:0/1711937679 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf20102790 0x7faf201067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:13.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.673+0000 7faf2812e640 1 --2- 172.21.15.64:0/1711937679 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faf20101de0 0x7faf201021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:13.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.674+0000 7faf2812e640 1 -- 172.21.15.64:0/1711937679 >> 172.21.15.64:0/1711937679 conn(0x7faf200fd8c0 msgr2=0x7faf200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:13.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.674+0000 7faf2812e640 1 -- 172.21.15.64:0/1711937679 shutdown_connections 2024-09-12T03:27:13.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.674+0000 7faf2812e640 1 -- 172.21.15.64:0/1711937679 wait complete. 2024-09-12T03:27:13.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.674+0000 7faf2812e640 1 Processor -- start 2024-09-12T03:27:13.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.674+0000 7faf2812e640 1 -- start start 2024-09-12T03:27:13.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.675+0000 7faf2812e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faf20101de0 0x7faf201982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:13.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.675+0000 7faf2812e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf20102790 0x7faf201987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:13.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.675+0000 7faf2812e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf20106f10 0x7faf2019cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:13.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.675+0000 7faf2812e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7faf20111e20 con 0x7faf20106f10 2024-09-12T03:27:13.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.675+0000 7faf2812e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7faf20111ca0 con 0x7faf20102790 2024-09-12T03:27:13.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.675+0000 7faf2812e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7faf20111fa0 con 0x7faf20101de0 2024-09-12T03:27:13.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.675+0000 7faf266cb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf20106f10 0x7faf2019cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:13.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.675+0000 7faf266cb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf20106f10 0x7faf2019cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56376/0 (socket says 172.21.15.64:56376) 2024-09-12T03:27:13.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.675+0000 7faf266cb640 1 -- 172.21.15.64:0/396280615 learned_addr learned my addr 172.21.15.64:0/396280615 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:27:13.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.675+0000 7faf25eca640 1 --2- 172.21.15.64:0/396280615 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faf20101de0 0x7faf201982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:13.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.675+0000 7faf256c9640 1 --2- 172.21.15.64:0/396280615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf20102790 0x7faf201987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:13.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.675+0000 7faf266cb640 1 -- 172.21.15.64:0/396280615 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faf20101de0 msgr2=0x7faf201982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:13.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.675+0000 7faf266cb640 1 --2- 172.21.15.64:0/396280615 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faf20101de0 0x7faf201982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:13.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.675+0000 7faf266cb640 1 -- 172.21.15.64:0/396280615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf20102790 msgr2=0x7faf201987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:13.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.675+0000 7faf266cb640 1 --2- 172.21.15.64:0/396280615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf20102790 0x7faf201987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:13.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.675+0000 7faf266cb640 1 -- 172.21.15.64:0/396280615 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf2019d250 con 0x7faf20106f10 2024-09-12T03:27:13.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.675+0000 7faf25eca640 1 --2- 172.21.15.64:0/396280615 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faf20101de0 0x7faf201982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:27:13.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.675+0000 7faf266cb640 1 --2- 172.21.15.64:0/396280615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf20106f10 0x7faf2019cb70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7faf18009dc0 tx=0x7faf18032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:13.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.676+0000 7faf16ffd640 1 -- 172.21.15.64:0/396280615 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf18003cf0 con 0x7faf20106f10 2024-09-12T03:27:13.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.676+0000 7faf16ffd640 1 -- 172.21.15.64:0/396280615 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf180044f0 con 0x7faf20106f10 2024-09-12T03:27:13.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.676+0000 7faf2812e640 1 -- 172.21.15.64:0/396280615 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf2019d4e0 con 0x7faf20106f10 2024-09-12T03:27:13.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.676+0000 7faf16ffd640 1 -- 172.21.15.64:0/396280615 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf18030bf0 con 0x7faf20106f10 2024-09-12T03:27:13.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.676+0000 7faf2812e640 1 -- 172.21.15.64:0/396280615 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf201a4e20 con 0x7faf20106f10 2024-09-12T03:27:13.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.677+0000 7faf2812e640 1 -- 172.21.15.64:0/396280615 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf201021c0 con 0x7faf20106f10 2024-09-12T03:27:13.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.678+0000 7faf16ffd640 1 -- 172.21.15.64:0/396280615 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7faf18004040 con 0x7faf20106f10 2024-09-12T03:27:13.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.679+0000 7faf16ffd640 1 --2- 172.21.15.64:0/396280615 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7faf0c0a2b80 0x7faf0c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:13.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.679+0000 7faf25eca640 1 --2- 172.21.15.64:0/396280615 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7faf0c0a2b80 0x7faf0c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:13.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.679+0000 7faf16ffd640 1 -- 172.21.15.64:0/396280615 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7faf180f0a80 con 0x7faf20106f10 2024-09-12T03:27:13.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.679+0000 7faf25eca640 1 --2- 172.21.15.64:0/396280615 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7faf0c0a2b80 0x7faf0c0a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7faf1c00abe0 tx=0x7faf1c00a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:13.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.682+0000 7faf16ffd640 1 -- 172.21.15.64:0/396280615 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faf180bd7a0 con 0x7faf20106f10 2024-09-12T03:27:13.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.857+0000 7faf2812e640 1 -- 172.21.15.64:0/396280615 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faf2010a1b0 con 0x7faf20106f10 2024-09-12T03:27:13.860 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:27:13.860 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":88,"fsid":"b0812460-70b5-11ef-bcea-c7b262605968","created":"2024-09-12T03:19:15.900322+0000","modified":"2024-09-12T03:27:08.680818+0000","last_up_change":"2024-09-12T03:26:06.122971+0000","last_in_change":"2024-09-12T03:25:43.130689+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:22:21.915870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:26:55.826348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:26:57.984319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T03:27:04.460292+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"326271c9-1f28-4ce6-b3be-fcc57479d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6803","nonce":1307832590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6805","nonce":1307832590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6809","nonce":1307832590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6807","nonce":1307832590}]},"public_addr":"172.21.15.64:6803/1307832590","cluster_addr":"172.21.15.64:6805/1307832590","heartbeat_back_addr":"172.21.15.64:6809/1307832590","heartbeat_front_addr":"172.21.15.64:6807/1307832590","state":["exists","up"]},{"osd":1,"uuid":"7dca0c28-c234-423c-936e-2351d63f918e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6801","nonce":3123728622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6803","nonce":3123728622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6807","nonce":3123728622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6805","nonce":3123728622}]},"public_addr":"172.21.15.115:6801/3123728622","cluster_addr":"172.21.15.115:6803/3123728622","heartbeat_back_addr":"172.21.15.115:6807/3123728622","heartbeat_front_addr":"172.21.15.115:6805/3123728622","state":["exists","up"]},{"osd":2,"uuid":"4b3a3c7a-aec6-455a-9a9a-1ac9cc900f24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6801","nonce":1664125451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6803","nonce":1664125451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6807","nonce":1664125451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6805","nonce":1664125451}]},"public_addr":"172.21.15.195:6801/1664125451","cluster_addr":"172.21.15.195:6803/1664125451","heartbeat_back_addr":"172.21.15.195:6807/1664125451","heartbeat_front_addr":"172.21.15.195:6805/1664125451","state":["exists","up"]},{"osd":3,"uuid":"8768c339-2c1e-4294-8b78-3c2adab6750f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6811","nonce":3935517185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6813","nonce":3935517185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6817","nonce":3935517185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6815","nonce":3935517185}]},"public_addr":"172.21.15.64:6811/3935517185","cluster_addr":"172.21.15.64:6813/3935517185","heartbeat_back_addr":"172.21.15.64:6817/3935517185","heartbeat_front_addr":"172.21.15.64:6815/3935517185","state":["exists","up"]},{"osd":4,"uuid":"123aac47-a8ae-47c7-93a7-0dd293356322","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6809","nonce":446665227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6811","nonce":446665227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6815","nonce":446665227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6813","nonce":446665227}]},"public_addr":"172.21.15.115:6809/446665227","cluster_addr":"172.21.15.115:6811/446665227","heartbeat_back_addr":"172.21.15.115:6815/446665227","heartbeat_front_addr":"172.21.15.115:6813/446665227","state":["exists","up"]},{"osd":5,"uuid":"c90bcaed-e7a8-4852-8677-b01dc73ea44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6809","nonce":1187293668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6811","nonce":1187293668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6815","nonce":1187293668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6813","nonce":1187293668}]},"public_addr":"172.21.15.195:6809/1187293668","cluster_addr":"172.21.15.195:6811/1187293668","heartbeat_back_addr":"172.21.15.195:6815/1187293668","heartbeat_front_addr":"172.21.15.195:6813/1187293668","state":["exists","up"]},{"osd":6,"uuid":"fd292547-6695-4889-8867-e74318bbb00f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6819","nonce":560343274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6821","nonce":560343274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6825","nonce":560343274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6823","nonce":560343274}]},"public_addr":"172.21.15.64:6819/560343274","cluster_addr":"172.21.15.64:6821/560343274","heartbeat_back_addr":"172.21.15.64:6825/560343274","heartbeat_front_addr":"172.21.15.64:6823/560343274","state":["exists","up"]},{"osd":7,"uuid":"ad7c4151-cd98-4113-879b-ca2b7c83c7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6817","nonce":3075734174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6819","nonce":3075734174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6823","nonce":3075734174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6821","nonce":3075734174}]},"public_addr":"172.21.15.115:6817/3075734174","cluster_addr":"172.21.15.115:6819/3075734174","heartbeat_back_addr":"172.21.15.115:6823/3075734174","heartbeat_front_addr":"172.21.15.115:6821/3075734174","state":["exists","up"]},{"osd":8,"uuid":"6de50049-c28f-4f21-b123-05e55bf59127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6817","nonce":2184742494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6819","nonce":2184742494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6823","nonce":2184742494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6821","nonce":2184742494}]},"public_addr":"172.21.15.195:6817/2184742494","cluster_addr":"172.21.15.195:6819/2184742494","heartbeat_back_addr":"172.21.15.195:6823/2184742494","heartbeat_front_addr":"172.21.15.195:6821/2184742494","state":["exists","up"]},{"osd":9,"uuid":"36ab792c-58f2-4beb-9024-7f3d1e3785dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6827","nonce":561728112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6829","nonce":561728112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6833","nonce":561728112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6831","nonce":561728112}]},"public_addr":"172.21.15.64:6827/561728112","cluster_addr":"172.21.15.64:6829/561728112","heartbeat_back_addr":"172.21.15.64:6833/561728112","heartbeat_front_addr":"172.21.15.64:6831/561728112","state":["exists","up"]},{"osd":10,"uuid":"d1f7881c-ec3f-466e-9cea-1d5cb441d970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6825","nonce":2977887627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6827","nonce":2977887627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6831","nonce":2977887627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6829","nonce":2977887627}]},"public_addr":"172.21.15.115:6825/2977887627","cluster_addr":"172.21.15.115:6827/2977887627","heartbeat_back_addr":"172.21.15.115:6831/2977887627","heartbeat_front_addr":"172.21.15.115:6829/2977887627","state":["exists","up"]},{"osd":11,"uuid":"7c46c028-22c7-450f-9927-2144a7a09e9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6825","nonce":591599352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6827","nonce":591599352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6831","nonce":591599352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6829","nonce":591599352}]},"public_addr":"172.21.15.195:6825/591599352","cluster_addr":"172.21.15.195:6827/591599352","heartbeat_back_addr":"172.21.15.195:6831/591599352","heartbeat_front_addr":"172.21.15.195:6829/591599352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:29.151454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:54.308909+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:18.680995+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:44.687548+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:10.166131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:35.026132+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:24:00.242966+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:24:24.600848+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:24:49.563060+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:25:15.025730+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:25:39.830995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:26:04.394276+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6801/68603379":"2024-09-13T03:20:07.411102+0000","172.21.15.64:0/3827202979":"2024-09-13T03:20:07.411102+0000","172.21.15.64:6800/68603379":"2024-09-13T03:20:07.411102+0000","172.21.15.64:0/1217136221":"2024-09-13T03:20:07.411102+0000","172.21.15.64:0/3079679964":"2024-09-13T03:19:36.839431+0000","172.21.15.64:6801/3186640984":"2024-09-13T03:19:36.839431+0000","172.21.15.64:0/3334629354":"2024-09-13T03:20:07.411102+0000","172.21.15.64:6800/3186640984":"2024-09-13T03:19:36.839431+0000","172.21.15.64:0/3564418115":"2024-09-13T03:19:36.839431+0000","172.21.15.64:0/686137382":"2024-09-13T03:19:36.839431+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:27:13.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.859+0000 7faf16ffd640 1 -- 172.21.15.64:0/396280615 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24209 (secure 0 0 0) 0x7faf180bd0a0 con 0x7faf20106f10 2024-09-12T03:27:13.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.861+0000 7faf2812e640 1 -- 172.21.15.64:0/396280615 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7faf0c0a2b80 msgr2=0x7faf0c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:13.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.861+0000 7faf2812e640 1 --2- 172.21.15.64:0/396280615 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7faf0c0a2b80 0x7faf0c0a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7faf1c00abe0 tx=0x7faf1c00a5c0 comp rx=0 tx=0).stop 2024-09-12T03:27:13.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.861+0000 7faf2812e640 1 -- 172.21.15.64:0/396280615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf20106f10 msgr2=0x7faf2019cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:13.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.861+0000 7faf2812e640 1 --2- 172.21.15.64:0/396280615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf20106f10 0x7faf2019cb70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7faf18009dc0 tx=0x7faf18032ff0 comp rx=0 tx=0).stop 2024-09-12T03:27:13.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.861+0000 7faf2812e640 1 -- 172.21.15.64:0/396280615 shutdown_connections 2024-09-12T03:27:13.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.861+0000 7faf2812e640 1 --2- 172.21.15.64:0/396280615 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7faf0c0a2b80 0x7faf0c0a5040 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:13.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.861+0000 7faf2812e640 1 --2- 172.21.15.64:0/396280615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf20106f10 0x7faf2019cb70 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:13.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.861+0000 7faf2812e640 1 --2- 172.21.15.64:0/396280615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf20102790 0x7faf201987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:13.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.861+0000 7faf2812e640 1 --2- 172.21.15.64:0/396280615 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faf20101de0 0x7faf201982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:13.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.861+0000 7faf2812e640 1 -- 172.21.15.64:0/396280615 >> 172.21.15.64:0/396280615 conn(0x7faf200fd8c0 msgr2=0x7faf200feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:13.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.862+0000 7faf2812e640 1 -- 172.21.15.64:0/396280615 shutdown_connections 2024-09-12T03:27:13.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:13.862+0000 7faf2812e640 1 -- 172.21.15.64:0/396280615 wait complete. 2024-09-12T03:27:14.160 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:14 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/396280615' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:27:14.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:14 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/396280615' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:27:14.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:14 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/396280615' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:27:14.513 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:27:14.768 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:15.171 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:15 smithi064 ceph-mon[30952]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:15.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.253+0000 7fc265321640 1 -- 172.21.15.64:0/1821222242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc260106f10 msgr2=0x7fc26010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:15.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.253+0000 7fc265321640 1 --2- 172.21.15.64:0/1821222242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc260106f10 0x7fc26010ea90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc258009ca0 tx=0x7fc25802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:27:15.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.253+0000 7fc265321640 1 -- 172.21.15.64:0/1821222242 shutdown_connections 2024-09-12T03:27:15.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.253+0000 7fc265321640 1 --2- 172.21.15.64:0/1821222242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc260106f10 0x7fc26010ea90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:15.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.253+0000 7fc265321640 1 --2- 172.21.15.64:0/1821222242 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc260102790 0x7fc2601067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:15.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.253+0000 7fc265321640 1 --2- 172.21.15.64:0/1821222242 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc260101de0 0x7fc2601021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:15.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.253+0000 7fc265321640 1 -- 172.21.15.64:0/1821222242 >> 172.21.15.64:0/1821222242 conn(0x7fc2600fd8c0 msgr2=0x7fc2600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:15.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.253+0000 7fc265321640 1 -- 172.21.15.64:0/1821222242 shutdown_connections 2024-09-12T03:27:15.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.253+0000 7fc265321640 1 -- 172.21.15.64:0/1821222242 wait complete. 2024-09-12T03:27:15.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.254+0000 7fc265321640 1 Processor -- start 2024-09-12T03:27:15.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.254+0000 7fc265321640 1 -- start start 2024-09-12T03:27:15.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.254+0000 7fc265321640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc260101de0 0x7fc26019a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:15.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.254+0000 7fc265321640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc260102790 0x7fc26019aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:15.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.254+0000 7fc265321640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc260106f10 0x7fc26019ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:15.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.254+0000 7fc265321640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc260111e20 con 0x7fc260102790 2024-09-12T03:27:15.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.254+0000 7fc265321640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc260111ca0 con 0x7fc260106f10 2024-09-12T03:27:15.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.254+0000 7fc265321640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc260111fa0 con 0x7fc260101de0 2024-09-12T03:27:15.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.254+0000 7fc25e7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc260102790 0x7fc26019aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:15.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.254+0000 7fc25e7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc260102790 0x7fc26019aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56394/0 (socket says 172.21.15.64:56394) 2024-09-12T03:27:15.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.254+0000 7fc25e7fc640 1 -- 172.21.15.64:0/3100880771 learned_addr learned my addr 172.21.15.64:0/3100880771 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:27:15.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.254+0000 7fc25f7fe640 1 --2- 172.21.15.64:0/3100880771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc260106f10 0x7fc26019ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:15.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.254+0000 7fc25effd640 1 --2- 172.21.15.64:0/3100880771 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc260101de0 0x7fc26019a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:15.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.254+0000 7fc25e7fc640 1 -- 172.21.15.64:0/3100880771 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc260101de0 msgr2=0x7fc26019a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:15.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.254+0000 7fc25e7fc640 1 --2- 172.21.15.64:0/3100880771 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc260101de0 0x7fc26019a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:15.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.254+0000 7fc25e7fc640 1 -- 172.21.15.64:0/3100880771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc260106f10 msgr2=0x7fc26019ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:15.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.254+0000 7fc25e7fc640 1 --2- 172.21.15.64:0/3100880771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc260106f10 0x7fc26019ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:15.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.254+0000 7fc25e7fc640 1 -- 172.21.15.64:0/3100880771 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc26019f4c0 con 0x7fc260102790 2024-09-12T03:27:15.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.255+0000 7fc25effd640 1 --2- 172.21.15.64:0/3100880771 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc260101de0 0x7fc26019a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:27:15.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.255+0000 7fc25e7fc640 1 --2- 172.21.15.64:0/3100880771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc260102790 0x7fc26019aa50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc25000d950 tx=0x7fc25000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:15.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.255+0000 7fc257fff640 1 -- 172.21.15.64:0/3100880771 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc250017070 con 0x7fc260102790 2024-09-12T03:27:15.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.255+0000 7fc257fff640 1 -- 172.21.15.64:0/3100880771 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc250009c30 con 0x7fc260102790 2024-09-12T03:27:15.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.255+0000 7fc257fff640 1 -- 172.21.15.64:0/3100880771 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25000f620 con 0x7fc260102790 2024-09-12T03:27:15.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.255+0000 7fc265321640 1 -- 172.21.15.64:0/3100880771 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc26019f7b0 con 0x7fc260102790 2024-09-12T03:27:15.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.255+0000 7fc265321640 1 -- 172.21.15.64:0/3100880771 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2601a7090 con 0x7fc260102790 2024-09-12T03:27:15.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.256+0000 7fc265321640 1 -- 172.21.15.64:0/3100880771 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2601021c0 con 0x7fc260102790 2024-09-12T03:27:15.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.257+0000 7fc257fff640 1 -- 172.21.15.64:0/3100880771 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc2500109a0 con 0x7fc260102790 2024-09-12T03:27:15.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.258+0000 7fc257fff640 1 --2- 172.21.15.64:0/3100880771 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fc24c0a2b80 0x7fc24c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:15.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.258+0000 7fc25effd640 1 --2- 172.21.15.64:0/3100880771 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fc24c0a2b80 0x7fc24c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:15.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.258+0000 7fc257fff640 1 -- 172.21.15.64:0/3100880771 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fc2500cee60 con 0x7fc260102790 2024-09-12T03:27:15.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.258+0000 7fc25effd640 1 --2- 172.21.15.64:0/3100880771 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fc24c0a2b80 0x7fc24c0a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc248009b40 tx=0x7fc248005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:15.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.262+0000 7fc257fff640 1 -- 172.21.15.64:0/3100880771 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc250009de0 con 0x7fc260102790 2024-09-12T03:27:15.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:15 smithi115 ceph-mon[34801]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:15.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:15 smithi195 ceph-mon[34786]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:15.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.476+0000 7fc265321640 1 -- 172.21.15.64:0/3100880771 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2601a72f0 con 0x7fc260102790 2024-09-12T03:27:15.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.477+0000 7fc257fff640 1 -- 172.21.15.64:0/3100880771 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+11848 (secure 0 0 0) 0x7fc25009bb80 con 0x7fc260102790 2024-09-12T03:27:15.477 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:27:15.477 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":16,"btime":"2024-09-12T03:27:09:051020+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2580078738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2580078738},{"type":"v1","addr":"172.21.15.64:6835","nonce":2580078738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2333415004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2333415004},{"type":"v1","addr":"172.21.15.64:6837","nonce":2333415004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/3030618395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":3030618395},{"type":"v1","addr":"172.21.15.64:6839","nonce":3030618395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3333057625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3333057625},{"type":"v1","addr":"172.21.15.64:6841","nonce":3333057625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2991933266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2991933266},{"type":"v1","addr":"172.21.15.195:6833","nonce":2991933266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1138960346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1138960346},{"type":"v1","addr":"172.21.15.115:6833","nonce":1138960346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/332737916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":332737916},{"type":"v1","addr":"172.21.15.115:6835","nonce":332737916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2531045583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2531045583},{"type":"v1","addr":"172.21.15.195:6835","nonce":2531045583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/71647401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":71647401},{"type":"v1","addr":"172.21.15.195:6837","nonce":71647401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2915155705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2915155705},{"type":"v1","addr":"172.21.15.115:6837","nonce":2915155705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6839/2574695134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":2574695134},{"type":"v1","addr":"172.21.15.195:6839","nonce":2574695134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:26:59.647657+0000","modified":"2024-09-12T03:27:08.680950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6839/2238031796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2238031796},{"type":"v1","addr":"172.21.15.115:6839","nonce":2238031796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24500,"qdb_cluster":[24500]},"id":1}]} 2024-09-12T03:27:15.477 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 16 2024-09-12T03:27:15.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.479+0000 7fc265321640 1 -- 172.21.15.64:0/3100880771 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fc24c0a2b80 msgr2=0x7fc24c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:15.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.479+0000 7fc265321640 1 --2- 172.21.15.64:0/3100880771 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fc24c0a2b80 0x7fc24c0a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc248009b40 tx=0x7fc248005e50 comp rx=0 tx=0).stop 2024-09-12T03:27:15.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.479+0000 7fc265321640 1 -- 172.21.15.64:0/3100880771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc260102790 msgr2=0x7fc26019aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:15.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.479+0000 7fc265321640 1 --2- 172.21.15.64:0/3100880771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc260102790 0x7fc26019aa50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc25000d950 tx=0x7fc25000de10 comp rx=0 tx=0).stop 2024-09-12T03:27:15.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.479+0000 7fc265321640 1 -- 172.21.15.64:0/3100880771 shutdown_connections 2024-09-12T03:27:15.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.479+0000 7fc265321640 1 --2- 172.21.15.64:0/3100880771 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fc24c0a2b80 0x7fc24c0a5040 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:15.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.479+0000 7fc265321640 1 --2- 172.21.15.64:0/3100880771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc260106f10 0x7fc26019ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:15.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.479+0000 7fc265321640 1 --2- 172.21.15.64:0/3100880771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc260102790 0x7fc26019aa50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:15.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.479+0000 7fc265321640 1 --2- 172.21.15.64:0/3100880771 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc260101de0 0x7fc26019a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:15.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.479+0000 7fc265321640 1 -- 172.21.15.64:0/3100880771 >> 172.21.15.64:0/3100880771 conn(0x7fc2600fd8c0 msgr2=0x7fc2600febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:15.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.479+0000 7fc265321640 1 -- 172.21.15.64:0/3100880771 shutdown_connections 2024-09-12T03:27:15.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:15.479+0000 7fc265321640 1 -- 172.21.15.64:0/3100880771 wait complete. 2024-09-12T03:27:16.142 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:27:16.167 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:16 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/3100880771' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:27:16.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:16 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/3100880771' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:27:16.397 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:16.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:16 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/3100880771' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:27:16.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:16.866+0000 7f6ff69d9640 1 -- 172.21.15.64:0/4024666471 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ff0101df0 msgr2=0x7f6ff01021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:16.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:16.866+0000 7f6ff69d9640 1 --2- 172.21.15.64:0/4024666471 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ff0101df0 0x7f6ff01021d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0009900 tx=0x7f6fe002a410 comp rx=0 tx=0).stop 2024-09-12T03:27:16.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:16.866+0000 7f6ff69d9640 1 -- 172.21.15.64:0/4024666471 shutdown_connections 2024-09-12T03:27:16.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:16.866+0000 7f6ff69d9640 1 --2- 172.21.15.64:0/4024666471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ff0106f20 0x7f6ff010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:16.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:16.866+0000 7f6ff69d9640 1 --2- 172.21.15.64:0/4024666471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ff01027a0 0x7f6ff01067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:16.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:16.866+0000 7f6ff69d9640 1 --2- 172.21.15.64:0/4024666471 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ff0101df0 0x7f6ff01021d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:16.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:16.866+0000 7f6ff69d9640 1 -- 172.21.15.64:0/4024666471 >> 172.21.15.64:0/4024666471 conn(0x7f6ff00fd8b0 msgr2=0x7f6ff00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:16.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:16.866+0000 7f6ff69d9640 1 -- 172.21.15.64:0/4024666471 shutdown_connections 2024-09-12T03:27:16.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:16.866+0000 7f6ff69d9640 1 -- 172.21.15.64:0/4024666471 wait complete. 2024-09-12T03:27:16.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:16.867+0000 7f6ff69d9640 1 Processor -- start 2024-09-12T03:27:16.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:16.867+0000 7f6ff69d9640 1 -- start start 2024-09-12T03:27:16.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:16.867+0000 7f6ff69d9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ff0101df0 0x7f6ff00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:16.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:16.867+0000 7f6ff69d9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ff01027a0 0x7f6ff0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:16.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:16.867+0000 7f6ff69d9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ff0106f20 0x7f6ff0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:16.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:16.867+0000 7f6ff69d9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff0111ed0 con 0x7f6ff0101df0 2024-09-12T03:27:16.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:16.867+0000 7f6ff69d9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff0111d50 con 0x7f6ff01027a0 2024-09-12T03:27:16.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:16.867+0000 7f6ff69d9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff0112050 con 0x7f6ff0106f20 2024-09-12T03:27:16.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:16.868+0000 7f6feffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ff0101df0 0x7f6ff00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:16.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:16.868+0000 7f6feffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ff0101df0 0x7f6ff00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56412/0 (socket says 172.21.15.64:56412) 2024-09-12T03:27:16.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:16.868+0000 7f6feffff640 1 -- 172.21.15.64:0/2653954376 learned_addr learned my addr 172.21.15.64:0/2653954376 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:27:16.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:16.868+0000 7f6ff4f76640 1 --2- 172.21.15.64:0/2653954376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ff0106f20 0x7f6ff0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:16.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:16.868+0000 7f6fef7fe640 1 --2- 172.21.15.64:0/2653954376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ff01027a0 0x7f6ff0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:16.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:16.868+0000 7f6feffff640 1 -- 172.21.15.64:0/2653954376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ff0106f20 msgr2=0x7f6ff0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:16.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:16.868+0000 7f6feffff640 1 --2- 172.21.15.64:0/2653954376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ff0106f20 0x7f6ff0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:16.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:16.868+0000 7f6feffff640 1 -- 172.21.15.64:0/2653954376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ff01027a0 msgr2=0x7f6ff0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:16.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:16.868+0000 7f6feffff640 1 --2- 172.21.15.64:0/2653954376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ff01027a0 0x7f6ff0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:16.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:16.868+0000 7f6feffff640 1 -- 172.21.15.64:0/2653954376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ff01a6950 con 0x7f6ff0101df0 2024-09-12T03:27:16.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:16.868+0000 7f6fef7fe640 1 --2- 172.21.15.64:0/2653954376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ff01027a0 0x7f6ff0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:27:16.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:16.868+0000 7f6feffff640 1 --2- 172.21.15.64:0/2653954376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ff0101df0 0x7f6ff00744f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0006010 tx=0x7f6fe0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:16.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:16.868+0000 7f6fed7fa640 1 -- 172.21.15.64:0/2653954376 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe003f030 con 0x7f6ff0101df0 2024-09-12T03:27:16.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:16.868+0000 7f6ff69d9640 1 -- 172.21.15.64:0/2653954376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ff01a6be0 con 0x7f6ff0101df0 2024-09-12T03:27:16.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:16.868+0000 7f6fed7fa640 1 -- 172.21.15.64:0/2653954376 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6fe00035f0 con 0x7f6ff0101df0 2024-09-12T03:27:16.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:16.868+0000 7f6fed7fa640 1 -- 172.21.15.64:0/2653954376 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0035070 con 0x7f6ff0101df0 2024-09-12T03:27:16.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:16.868+0000 7f6ff69d9640 1 -- 172.21.15.64:0/2653954376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ff01a70c0 con 0x7f6ff0101df0 2024-09-12T03:27:16.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:16.869+0000 7f6ff69d9640 1 -- 172.21.15.64:0/2653954376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ff0102250 con 0x7f6ff0101df0 2024-09-12T03:27:16.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:16.870+0000 7f6fed7fa640 1 -- 172.21.15.64:0/2653954376 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6fe0003b10 con 0x7f6ff0101df0 2024-09-12T03:27:16.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:16.871+0000 7f6fed7fa640 1 --2- 172.21.15.64:0/2653954376 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f6fd80a2b80 0x7f6fd80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:16.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:16.871+0000 7f6fef7fe640 1 --2- 172.21.15.64:0/2653954376 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f6fd80a2b80 0x7f6fd80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:16.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:16.871+0000 7f6fed7fa640 1 -- 172.21.15.64:0/2653954376 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f6fe00efc80 con 0x7f6ff0101df0 2024-09-12T03:27:16.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:16.872+0000 7f6fef7fe640 1 --2- 172.21.15.64:0/2653954376 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f6fd80a2b80 0x7f6fd80a5040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4006fd0 tx=0x7f6fe4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:16.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:16.875+0000 7f6fed7fa640 1 -- 172.21.15.64:0/2653954376 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6fe00bc9a0 con 0x7f6ff0101df0 2024-09-12T03:27:17.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:17.052+0000 7f6ff69d9640 1 -- 172.21.15.64:0/2653954376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6ff010a100 con 0x7f6ff0101df0 2024-09-12T03:27:17.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:17.054+0000 7f6fed7fa640 1 -- 172.21.15.64:0/2653954376 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24209 (secure 0 0 0) 0x7f6fe00bc2a0 con 0x7f6ff0101df0 2024-09-12T03:27:17.054 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:27:17.054 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":88,"fsid":"b0812460-70b5-11ef-bcea-c7b262605968","created":"2024-09-12T03:19:15.900322+0000","modified":"2024-09-12T03:27:08.680818+0000","last_up_change":"2024-09-12T03:26:06.122971+0000","last_in_change":"2024-09-12T03:25:43.130689+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:22:21.915870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:26:55.826348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:26:57.984319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T03:27:04.460292+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"326271c9-1f28-4ce6-b3be-fcc57479d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6803","nonce":1307832590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6805","nonce":1307832590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6809","nonce":1307832590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6807","nonce":1307832590}]},"public_addr":"172.21.15.64:6803/1307832590","cluster_addr":"172.21.15.64:6805/1307832590","heartbeat_back_addr":"172.21.15.64:6809/1307832590","heartbeat_front_addr":"172.21.15.64:6807/1307832590","state":["exists","up"]},{"osd":1,"uuid":"7dca0c28-c234-423c-936e-2351d63f918e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6801","nonce":3123728622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6803","nonce":3123728622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6807","nonce":3123728622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6805","nonce":3123728622}]},"public_addr":"172.21.15.115:6801/3123728622","cluster_addr":"172.21.15.115:6803/3123728622","heartbeat_back_addr":"172.21.15.115:6807/3123728622","heartbeat_front_addr":"172.21.15.115:6805/3123728622","state":["exists","up"]},{"osd":2,"uuid":"4b3a3c7a-aec6-455a-9a9a-1ac9cc900f24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6801","nonce":1664125451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6803","nonce":1664125451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6807","nonce":1664125451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6805","nonce":1664125451}]},"public_addr":"172.21.15.195:6801/1664125451","cluster_addr":"172.21.15.195:6803/1664125451","heartbeat_back_addr":"172.21.15.195:6807/1664125451","heartbeat_front_addr":"172.21.15.195:6805/1664125451","state":["exists","up"]},{"osd":3,"uuid":"8768c339-2c1e-4294-8b78-3c2adab6750f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6811","nonce":3935517185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6813","nonce":3935517185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6817","nonce":3935517185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6815","nonce":3935517185}]},"public_addr":"172.21.15.64:6811/3935517185","cluster_addr":"172.21.15.64:6813/3935517185","heartbeat_back_addr":"172.21.15.64:6817/3935517185","heartbeat_front_addr":"172.21.15.64:6815/3935517185","state":["exists","up"]},{"osd":4,"uuid":"123aac47-a8ae-47c7-93a7-0dd293356322","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6809","nonce":446665227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6811","nonce":446665227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6815","nonce":446665227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6813","nonce":446665227}]},"public_addr":"172.21.15.115:6809/446665227","cluster_addr":"172.21.15.115:6811/446665227","heartbeat_back_addr":"172.21.15.115:6815/446665227","heartbeat_front_addr":"172.21.15.115:6813/446665227","state":["exists","up"]},{"osd":5,"uuid":"c90bcaed-e7a8-4852-8677-b01dc73ea44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6809","nonce":1187293668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6811","nonce":1187293668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6815","nonce":1187293668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6813","nonce":1187293668}]},"public_addr":"172.21.15.195:6809/1187293668","cluster_addr":"172.21.15.195:6811/1187293668","heartbeat_back_addr":"172.21.15.195:6815/1187293668","heartbeat_front_addr":"172.21.15.195:6813/1187293668","state":["exists","up"]},{"osd":6,"uuid":"fd292547-6695-4889-8867-e74318bbb00f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6819","nonce":560343274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6821","nonce":560343274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6825","nonce":560343274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6823","nonce":560343274}]},"public_addr":"172.21.15.64:6819/560343274","cluster_addr":"172.21.15.64:6821/560343274","heartbeat_back_addr":"172.21.15.64:6825/560343274","heartbeat_front_addr":"172.21.15.64:6823/560343274","state":["exists","up"]},{"osd":7,"uuid":"ad7c4151-cd98-4113-879b-ca2b7c83c7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6817","nonce":3075734174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6819","nonce":3075734174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6823","nonce":3075734174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6821","nonce":3075734174}]},"public_addr":"172.21.15.115:6817/3075734174","cluster_addr":"172.21.15.115:6819/3075734174","heartbeat_back_addr":"172.21.15.115:6823/3075734174","heartbeat_front_addr":"172.21.15.115:6821/3075734174","state":["exists","up"]},{"osd":8,"uuid":"6de50049-c28f-4f21-b123-05e55bf59127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6817","nonce":2184742494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6819","nonce":2184742494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6823","nonce":2184742494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6821","nonce":2184742494}]},"public_addr":"172.21.15.195:6817/2184742494","cluster_addr":"172.21.15.195:6819/2184742494","heartbeat_back_addr":"172.21.15.195:6823/2184742494","heartbeat_front_addr":"172.21.15.195:6821/2184742494","state":["exists","up"]},{"osd":9,"uuid":"36ab792c-58f2-4beb-9024-7f3d1e3785dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6827","nonce":561728112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6829","nonce":561728112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6833","nonce":561728112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6831","nonce":561728112}]},"public_addr":"172.21.15.64:6827/561728112","cluster_addr":"172.21.15.64:6829/561728112","heartbeat_back_addr":"172.21.15.64:6833/561728112","heartbeat_front_addr":"172.21.15.64:6831/561728112","state":["exists","up"]},{"osd":10,"uuid":"d1f7881c-ec3f-466e-9cea-1d5cb441d970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6825","nonce":2977887627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6827","nonce":2977887627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6831","nonce":2977887627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6829","nonce":2977887627}]},"public_addr":"172.21.15.115:6825/2977887627","cluster_addr":"172.21.15.115:6827/2977887627","heartbeat_back_addr":"172.21.15.115:6831/2977887627","heartbeat_front_addr":"172.21.15.115:6829/2977887627","state":["exists","up"]},{"osd":11,"uuid":"7c46c028-22c7-450f-9927-2144a7a09e9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6825","nonce":591599352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6827","nonce":591599352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6831","nonce":591599352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6829","nonce":591599352}]},"public_addr":"172.21.15.195:6825/591599352","cluster_addr":"172.21.15.195:6827/591599352","heartbeat_back_addr":"172.21.15.195:6831/591599352","heartbeat_front_addr":"172.21.15.195:6829/591599352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:29.151454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:54.308909+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:18.680995+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:44.687548+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:10.166131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:35.026132+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:24:00.242966+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:24:24.600848+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:24:49.563060+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:25:15.025730+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:25:39.830995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:26:04.394276+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6801/68603379":"2024-09-13T03:20:07.411102+0000","172.21.15.64:0/3827202979":"2024-09-13T03:20:07.411102+0000","172.21.15.64:6800/68603379":"2024-09-13T03:20:07.411102+0000","172.21.15.64:0/1217136221":"2024-09-13T03:20:07.411102+0000","172.21.15.64:0/3079679964":"2024-09-13T03:19:36.839431+0000","172.21.15.64:6801/3186640984":"2024-09-13T03:19:36.839431+0000","172.21.15.64:0/3334629354":"2024-09-13T03:20:07.411102+0000","172.21.15.64:6800/3186640984":"2024-09-13T03:19:36.839431+0000","172.21.15.64:0/3564418115":"2024-09-13T03:19:36.839431+0000","172.21.15.64:0/686137382":"2024-09-13T03:19:36.839431+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:27:17.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:17.056+0000 7f6ff69d9640 1 -- 172.21.15.64:0/2653954376 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f6fd80a2b80 msgr2=0x7f6fd80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:17.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:17.056+0000 7f6ff69d9640 1 --2- 172.21.15.64:0/2653954376 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f6fd80a2b80 0x7f6fd80a5040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4006fd0 tx=0x7f6fe4008040 comp rx=0 tx=0).stop 2024-09-12T03:27:17.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:17.056+0000 7f6ff69d9640 1 -- 172.21.15.64:0/2653954376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ff0101df0 msgr2=0x7f6ff00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:17.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:17.056+0000 7f6ff69d9640 1 --2- 172.21.15.64:0/2653954376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ff0101df0 0x7f6ff00744f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0006010 tx=0x7f6fe0004270 comp rx=0 tx=0).stop 2024-09-12T03:27:17.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:17.057+0000 7f6ff69d9640 1 -- 172.21.15.64:0/2653954376 shutdown_connections 2024-09-12T03:27:17.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:17.057+0000 7f6ff69d9640 1 --2- 172.21.15.64:0/2653954376 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f6fd80a2b80 0x7f6fd80a5040 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:17.057+0000 7f6ff69d9640 1 --2- 172.21.15.64:0/2653954376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ff0106f20 0x7f6ff0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:17.057+0000 7f6ff69d9640 1 --2- 172.21.15.64:0/2653954376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ff01027a0 0x7f6ff0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:17.057+0000 7f6ff69d9640 1 --2- 172.21.15.64:0/2653954376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ff0101df0 0x7f6ff00744f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:17.057+0000 7f6ff69d9640 1 -- 172.21.15.64:0/2653954376 >> 172.21.15.64:0/2653954376 conn(0x7f6ff00fd8b0 msgr2=0x7f6ff00fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:17.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:17.057+0000 7f6ff69d9640 1 -- 172.21.15.64:0/2653954376 shutdown_connections 2024-09-12T03:27:17.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:17.057+0000 7f6ff69d9640 1 -- 172.21.15.64:0/2653954376 wait complete. 2024-09-12T03:27:17.334 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:17 smithi115 ceph-mon[34801]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:17.334 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:17 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/2653954376' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:27:17.363 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:17 smithi064 ceph-mon[30952]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:17.364 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:17 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/2653954376' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:27:17.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:17 smithi195 ceph-mon[34786]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:17.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:17 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/2653954376' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:27:17.794 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-12T03:26:59.647657+0000', 'modified': '2024-09-12T03:27:08.680950+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24500}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24500': {'gid': 24500, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.115:6839/2238031796', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6838', 'nonce': 2238031796}, {'type': 'v1', 'addr': '172.21.15.115:6839', 'nonce': 2238031796}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24500, 'qdb_cluster': [24500]} 2024-09-12T03:27:17.795 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-12T03:27:17.795 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-12T03:27:18.099 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:18.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.595+0000 7f40cb36d640 1 -- 172.21.15.64:0/1724667911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40c4102570 msgr2=0x7f40c41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:18.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.595+0000 7f40cb36d640 1 --2- 172.21.15.64:0/1724667911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40c4102570 0x7f40c41029b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f40b8009900 tx=0x7f40b802a480 comp rx=0 tx=0).stop 2024-09-12T03:27:18.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.595+0000 7f40cb36d640 1 -- 172.21.15.64:0/1724667911 shutdown_connections 2024-09-12T03:27:18.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.595+0000 7f40cb36d640 1 --2- 172.21.15.64:0/1724667911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40c4102ef0 0x7f40c410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.595+0000 7f40cb36d640 1 --2- 172.21.15.64:0/1724667911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40c4102570 0x7f40c41029b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.595+0000 7f40cb36d640 1 --2- 172.21.15.64:0/1724667911 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40c4100e20 0x7f40c4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.595+0000 7f40cb36d640 1 -- 172.21.15.64:0/1724667911 >> 172.21.15.64:0/1724667911 conn(0x7f40c40fc7d0 msgr2=0x7f40c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:18.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.595+0000 7f40cb36d640 1 -- 172.21.15.64:0/1724667911 shutdown_connections 2024-09-12T03:27:18.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.595+0000 7f40cb36d640 1 -- 172.21.15.64:0/1724667911 wait complete. 2024-09-12T03:27:18.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.596+0000 7f40cb36d640 1 Processor -- start 2024-09-12T03:27:18.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.596+0000 7f40cb36d640 1 -- start start 2024-09-12T03:27:18.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.596+0000 7f40cb36d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40c4100e20 0x7f40c4197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:18.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.596+0000 7f40cb36d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40c4102570 0x7f40c4197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:18.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.596+0000 7f40cb36d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40c4102ef0 0x7f40c4197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:18.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.596+0000 7f40cb36d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f40c4110ed0 con 0x7f40c4102ef0 2024-09-12T03:27:18.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.596+0000 7f40cb36d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f40c4110d50 con 0x7f40c4100e20 2024-09-12T03:27:18.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.596+0000 7f40cb36d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f40c4111050 con 0x7f40c4102570 2024-09-12T03:27:18.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.596+0000 7f40c990a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40c4102ef0 0x7f40c4197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:18.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.596+0000 7f40c990a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40c4102ef0 0x7f40c4197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35390/0 (socket says 172.21.15.64:35390) 2024-09-12T03:27:18.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.596+0000 7f40c990a640 1 -- 172.21.15.64:0/2296120801 learned_addr learned my addr 172.21.15.64:0/2296120801 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:27:18.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.596+0000 7f40c8908640 1 --2- 172.21.15.64:0/2296120801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40c4102570 0x7f40c4197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:18.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.596+0000 7f40c9109640 1 --2- 172.21.15.64:0/2296120801 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40c4100e20 0x7f40c4197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:18.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.596+0000 7f40c990a640 1 -- 172.21.15.64:0/2296120801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40c4102570 msgr2=0x7f40c4197670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:18.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.596+0000 7f40c990a640 1 --2- 172.21.15.64:0/2296120801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40c4102570 0x7f40c4197670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.596+0000 7f40c990a640 1 -- 172.21.15.64:0/2296120801 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40c4100e20 msgr2=0x7f40c4197130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:18.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.596+0000 7f40c990a640 1 --2- 172.21.15.64:0/2296120801 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40c4100e20 0x7f40c4197130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.596+0000 7f40c990a640 1 -- 172.21.15.64:0/2296120801 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40c41a69a0 con 0x7f40c4102ef0 2024-09-12T03:27:18.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.597+0000 7f40c990a640 1 --2- 172.21.15.64:0/2296120801 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40c4102ef0 0x7f40c4197bb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f40bc009fd0 tx=0x7f40bc00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:18.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.597+0000 7f40c27fc640 1 -- 172.21.15.64:0/2296120801 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40bc011830 con 0x7f40c4102ef0 2024-09-12T03:27:18.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.597+0000 7f40cb36d640 1 -- 172.21.15.64:0/2296120801 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40c41a6c90 con 0x7f40c4102ef0 2024-09-12T03:27:18.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.597+0000 7f40c27fc640 1 -- 172.21.15.64:0/2296120801 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40bc016070 con 0x7f40c4102ef0 2024-09-12T03:27:18.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.597+0000 7f40c27fc640 1 -- 172.21.15.64:0/2296120801 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40bc010de0 con 0x7f40c4102ef0 2024-09-12T03:27:18.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.597+0000 7f40cb36d640 1 -- 172.21.15.64:0/2296120801 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40c41a7200 con 0x7f40c4102ef0 2024-09-12T03:27:18.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.599+0000 7f40c27fc640 1 -- 172.21.15.64:0/2296120801 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f40bc0119d0 con 0x7f40c4102ef0 2024-09-12T03:27:18.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.600+0000 7f40cd746740 1 Processor -- start 2024-09-12T03:27:18.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.600+0000 7f40cd746740 1 -- start start 2024-09-12T03:27:18.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.600+0000 7f40cd746740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f225109d0 0x560f22510db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:18.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.600+0000 7f40cd746740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f225112f0 0x560f2251b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:18.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.600+0000 7f40cd746740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f2251bbb0 0x560f2251dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:18.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.600+0000 7f40cd746740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560f22477090 con 0x560f225112f0 2024-09-12T03:27:18.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.600+0000 7f40cd746740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560f2241ea70 con 0x560f2251bbb0 2024-09-12T03:27:18.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.600+0000 7f40cd746740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560f22473050 con 0x560f225109d0 2024-09-12T03:27:18.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.600+0000 7f40c9109640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f225112f0 0x560f2251b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:18.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.600+0000 7f40c9109640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f225112f0 0x560f2251b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35404/0 (socket says 172.21.15.64:35404) 2024-09-12T03:27:18.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.600+0000 7f40c9109640 1 -- 172.21.15.64:0/2436203248 learned_addr learned my addr 172.21.15.64:0/2436203248 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:27:18.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.600+0000 7f40c8908640 1 --2- 172.21.15.64:0/2436203248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f225109d0 0x560f22510db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:18.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.600+0000 7f40c27fc640 1 --2- 172.21.15.64:0/2296120801 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f40ac0a2b80 0x7f40ac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:18.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.600+0000 7f40c8908640 1 --2- 172.21.15.64:0/2436203248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f2251bbb0 0x560f2251dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:18.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.600+0000 7f40c990a640 1 --2- 172.21.15.64:0/2296120801 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f40ac0a2b80 0x7f40ac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:18.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.600+0000 7f40c9109640 1 -- 172.21.15.64:0/2436203248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f225109d0 msgr2=0x560f22510db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:18.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.600+0000 7f40c9109640 1 --2- 172.21.15.64:0/2436203248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f225109d0 0x560f22510db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.600+0000 7f40c9109640 1 -- 172.21.15.64:0/2436203248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f2251bbb0 msgr2=0x560f2251dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:18.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.600+0000 7f40c9109640 1 --2- 172.21.15.64:0/2436203248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f2251bbb0 0x560f2251dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.600+0000 7f40c9109640 1 -- 172.21.15.64:0/2436203248 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f2242b130 con 0x560f225112f0 2024-09-12T03:27:18.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.601+0000 7f40c27fc640 1 -- 172.21.15.64:0/2296120801 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f40bc0cea20 con 0x7f40c4102ef0 2024-09-12T03:27:18.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.601+0000 7f40c8908640 1 --2- 172.21.15.64:0/2436203248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f2251bbb0 0x560f2251dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:27:18.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.601+0000 7f40c9109640 1 --2- 172.21.15.64:0/2436203248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f225112f0 0x560f2251b670 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f40b4009fa0 tx=0x7f40b400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a7570d09eafa756 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:18.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.601+0000 7f40c990a640 1 --2- 172.21.15.64:0/2296120801 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f40ac0a2b80 0x7f40ac0a5040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f40bc0a3780 tx=0x7f40bc06e100 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:18.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.601+0000 7f40ab7fe640 1 -- 172.21.15.64:0/2436203248 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b401a070 con 0x560f225112f0 2024-09-12T03:27:18.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.601+0000 7f40ab7fe640 1 -- 172.21.15.64:0/2436203248 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40b40158c0 con 0x560f225112f0 2024-09-12T03:27:18.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.601+0000 7f40ab7fe640 1 -- 172.21.15.64:0/2436203248 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b400a8c0 con 0x560f225112f0 2024-09-12T03:27:18.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.602+0000 7f40cd746740 1 -- 172.21.15.64:0/2436203248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f225112f0 msgr2=0x560f2251b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:18.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.602+0000 7f40cd746740 1 --2- 172.21.15.64:0/2436203248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f225112f0 0x560f2251b670 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f40b4009fa0 tx=0x7f40b400c8b0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.602+0000 7f40cd746740 1 -- 172.21.15.64:0/2436203248 shutdown_connections 2024-09-12T03:27:18.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.602+0000 7f40cd746740 1 --2- 172.21.15.64:0/2436203248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f2251bbb0 0x560f2251dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.602+0000 7f40cd746740 1 --2- 172.21.15.64:0/2436203248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f225112f0 0x560f2251b670 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.602+0000 7f40cd746740 1 --2- 172.21.15.64:0/2436203248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f225109d0 0x560f22510db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.602+0000 7f40cd746740 1 -- 172.21.15.64:0/2436203248 >> 172.21.15.64:0/2436203248 conn(0x560f2250cf60 msgr2=0x560f2250d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:18.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.602+0000 7f40cd746740 1 -- 172.21.15.64:0/2436203248 shutdown_connections 2024-09-12T03:27:18.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.602+0000 7f40cd746740 1 -- 172.21.15.64:0/2436203248 wait complete. 2024-09-12T03:27:18.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.602+0000 7f40cd746740 1 -- start start 2024-09-12T03:27:18.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.602+0000 7f40cd746740 1 Processor -- start 2024-09-12T03:27:18.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.603+0000 7f40cd746740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f225109d0 0x560f2251b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:18.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.603+0000 7f40cd746740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f225112f0 0x560f22518750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:18.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.603+0000 7f40cd746740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f2251bbb0 0x560f22518c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:18.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.603+0000 7f40cd746740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560f2251e780 con 0x560f225109d0 2024-09-12T03:27:18.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.603+0000 7f40cd746740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560f22431f20 con 0x560f225112f0 2024-09-12T03:27:18.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.603+0000 7f40cd746740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560f2251e900 con 0x560f2251bbb0 2024-09-12T03:27:18.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.603+0000 7f40cd746740 10 client.0 _ll_register_callbacks cb 0x560f223fb030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:18.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.603+0000 7f40c8908640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f225109d0 0x560f2251b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:18.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.603+0000 7f40c8908640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f225109d0 0x560f2251b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35416/0 (socket says 172.21.15.64:35416) 2024-09-12T03:27:18.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.603+0000 7f40c8908640 1 -- 172.21.15.64:0/2250436782 learned_addr learned my addr 172.21.15.64:0/2250436782 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:27:18.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.603+0000 7f40c9109640 1 --2- 172.21.15.64:0/2250436782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f225112f0 0x560f22518750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:18.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.603+0000 7f40c8908640 1 --2- 172.21.15.64:0/2250436782 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f2251bbb0 0x560f22518c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:18.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.603+0000 7f40c8908640 1 -- 172.21.15.64:0/2250436782 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f2251bbb0 msgr2=0x560f22518c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:18.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.603+0000 7f40c8908640 1 --2- 172.21.15.64:0/2250436782 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f2251bbb0 0x560f22518c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.603+0000 7f40c8908640 1 -- 172.21.15.64:0/2250436782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f225112f0 msgr2=0x560f22518750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:18.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.603+0000 7f40c8908640 1 --2- 172.21.15.64:0/2250436782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f225112f0 0x560f22518750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.603+0000 7f40c8908640 1 -- 172.21.15.64:0/2250436782 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f221e82b0 con 0x560f225109d0 2024-09-12T03:27:18.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.603+0000 7f40c9109640 1 --2- 172.21.15.64:0/2250436782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f225112f0 0x560f22518750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:27:18.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.603+0000 7f40c8908640 1 --2- 172.21.15.64:0/2250436782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f225109d0 0x560f2251b400 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f40b8009900 tx=0x7f40b8032bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2432e50681e92fcf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:18.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.603+0000 7f40a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:27:18.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.604+0000 7f40a8ff9640 1 -- 172.21.15.64:0/2250436782 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b8035040 con 0x560f225109d0 2024-09-12T03:27:18.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.604+0000 7f40a8ff9640 1 -- 172.21.15.64:0/2250436782 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f40b80035f0 con 0x560f225109d0 2024-09-12T03:27:18.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.604+0000 7f40cd746740 1 -- 172.21.15.64:0/2250436782 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40b40158c0 con 0x560f225109d0 2024-09-12T03:27:18.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.604+0000 7f40a8ff9640 1 -- 172.21.15.64:0/2250436782 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f40b804f050 con 0x560f225109d0 2024-09-12T03:27:18.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.604+0000 7f40cd746740 10 client.14715 fetch_fsmap learned FSMap version 16 2024-09-12T03:27:18.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.604+0000 7f40cd746740 1 -- 172.21.15.64:0/2250436782 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) -- 0x560f2252a050 con 0x560f225109d0 2024-09-12T03:27:18.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.604+0000 7f40a8ff9640 1 -- 172.21.15.64:0/2250436782 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40b40158c0 con 0x560f225109d0 2024-09-12T03:27:18.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.604+0000 7f40a8ff9640 1 -- 172.21.15.64:0/2250436782 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 16) ==== 8350+0+0 (secure 0 0 0) 0x7f40b803d900 con 0x560f225109d0 2024-09-12T03:27:18.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.604+0000 7f40cd746740 10 client.14715 fetch_fsmap finished waiting for FSMap version 16 2024-09-12T03:27:18.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.604+0000 7f40cd746740 10 client.14715 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T03:27:18.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.604+0000 7f40cd746740 20 client.14715 populate_metadata read hostname 'smithi064' 2024-09-12T03:27:18.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.604+0000 7f40cd746740 1 --2- 172.21.15.64:0/2250436782 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x560f2252c680 0x560f2252daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:18.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.604+0000 7f40cd746740 4 client.14715 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:27:18.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.604+0000 7f40cd746740 1 -- 172.21.15.64:0/2250436782 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f22484600 con 0x560f2252c680 2024-09-12T03:27:18.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.605+0000 7f40c9109640 1 --2- 172.21.15.64:0/2250436782 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x560f2252c680 0x560f2252daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:18.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.605+0000 7f40c9109640 1 --2- 172.21.15.64:0/2250436782 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x560f2252c680 0x560f2252daf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5200f0be2fa9683 server_cookie=476afd9ce4b5f63d in_seq=0 out_seq=0 2024-09-12T03:27:18.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.605+0000 7f40a8ff9640 10 client.14715 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:27:18.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.606+0000 7f40a8ff9640 1 -- 172.21.15.64:0/2250436782 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560f22484600 con 0x560f2252c680 2024-09-12T03:27:18.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.606+0000 7f40a8ff9640 10 client.14715 handle_command_reply: tid=0 2024-09-12T03:27:18.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.606+0000 7f40cd746740 1 client.14715 shutdown 2024-09-12T03:27:18.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.606+0000 7f40cd746740 1 -- 172.21.15.64:0/2250436782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f225109d0 msgr2=0x560f2251b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:18.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.606+0000 7f40cd746740 1 --2- 172.21.15.64:0/2250436782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f225109d0 0x560f2251b400 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f40b8009900 tx=0x7f40b8032bb0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.606+0000 7f40cd746740 1 -- 172.21.15.64:0/2250436782 shutdown_connections 2024-09-12T03:27:18.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.606+0000 7f40cd746740 1 --2- 172.21.15.64:0/2250436782 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x560f2252c680 0x560f2252daf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.606+0000 7f40a8ff9640 0 client.14715 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:27:18.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.606+0000 7f40cd746740 1 --2- 172.21.15.64:0/2250436782 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f2251bbb0 0x560f22518c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.606+0000 7f40cd746740 1 --2- 172.21.15.64:0/2250436782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f225112f0 0x560f22518750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.606+0000 7f40cd746740 1 --2- 172.21.15.64:0/2250436782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f225109d0 0x560f2251b400 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.606+0000 7f40cd746740 1 -- 172.21.15.64:0/2250436782 >> 172.21.15.64:0/2250436782 conn(0x560f2250cf60 msgr2=0x560f22512c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:18.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.606+0000 7f40cd746740 1 -- 172.21.15.64:0/2250436782 shutdown_connections 2024-09-12T03:27:18.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.606+0000 7f40cd746740 1 -- 172.21.15.64:0/2250436782 wait complete. 2024-09-12T03:27:18.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.606+0000 7f40cd746740 20 client.14715 trim_cache size 0 max 16384 2024-09-12T03:27:18.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.624+0000 7f40cd746740 1 Processor -- start 2024-09-12T03:27:18.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.624+0000 7f40cd746740 1 -- start start 2024-09-12T03:27:18.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.624+0000 7f40cd746740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f225109d0 0x560f2252daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:18.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.624+0000 7f40cd746740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f225112f0 0x560f22518f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:18.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.624+0000 7f40cd746740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f2251bbb0 0x560f22518030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:18.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.624+0000 7f40cd746740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560f2252be30 con 0x560f2251bbb0 2024-09-12T03:27:18.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.624+0000 7f40cd746740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560f2252c500 con 0x560f225109d0 2024-09-12T03:27:18.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.624+0000 7f40cd746740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560f2252cbd0 con 0x560f225112f0 2024-09-12T03:27:18.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.624+0000 7f40c8908640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f2251bbb0 0x560f22518030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:18.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.624+0000 7f40c8908640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f2251bbb0 0x560f22518030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35424/0 (socket says 172.21.15.64:35424) 2024-09-12T03:27:18.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.624+0000 7f40c8908640 1 -- 172.21.15.64:0/877427291 learned_addr learned my addr 172.21.15.64:0/877427291 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:27:18.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.625+0000 7f40c8908640 1 --2- 172.21.15.64:0/877427291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f225109d0 0x560f2252daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:18.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.625+0000 7f40c9109640 1 --2- 172.21.15.64:0/877427291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f225112f0 0x560f22518f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:18.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.625+0000 7f40c8908640 1 -- 172.21.15.64:0/877427291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f225112f0 msgr2=0x560f22518f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:18.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.625+0000 7f40c8908640 1 --2- 172.21.15.64:0/877427291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f225112f0 0x560f22518f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.625+0000 7f40c8908640 1 -- 172.21.15.64:0/877427291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f225109d0 msgr2=0x560f2252daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:18.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.625+0000 7f40c8908640 1 --2- 172.21.15.64:0/877427291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f225109d0 0x560f2252daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.625+0000 7f40c8908640 1 -- 172.21.15.64:0/877427291 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f2250ca00 con 0x560f2251bbb0 2024-09-12T03:27:18.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.625+0000 7f40c8908640 1 --2- 172.21.15.64:0/877427291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f2251bbb0 0x560f22518030 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f40b803de70 tx=0x7f40b8009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e5633100142c15 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:18.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.625+0000 7f40aaffd640 1 -- 172.21.15.64:0/877427291 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b8031a50 con 0x560f2251bbb0 2024-09-12T03:27:18.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.625+0000 7f40aaffd640 1 -- 172.21.15.64:0/877427291 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40b803d900 con 0x560f2251bbb0 2024-09-12T03:27:18.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.625+0000 7f40aaffd640 1 -- 172.21.15.64:0/877427291 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b803ca70 con 0x560f2251bbb0 2024-09-12T03:27:18.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.626+0000 7f40cd746740 1 -- 172.21.15.64:0/877427291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f2251bbb0 msgr2=0x560f22518030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:18.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.626+0000 7f40cd746740 1 --2- 172.21.15.64:0/877427291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f2251bbb0 0x560f22518030 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f40b803de70 tx=0x7f40b8009560 comp rx=0 tx=0).stop 2024-09-12T03:27:18.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.626+0000 7f40cd746740 1 -- 172.21.15.64:0/877427291 shutdown_connections 2024-09-12T03:27:18.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.626+0000 7f40cd746740 1 --2- 172.21.15.64:0/877427291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f2251bbb0 0x560f22518030 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.626+0000 7f40cd746740 1 --2- 172.21.15.64:0/877427291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f225112f0 0x560f22518f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.626+0000 7f40cd746740 1 --2- 172.21.15.64:0/877427291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f225109d0 0x560f2252daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.626+0000 7f40cd746740 1 -- 172.21.15.64:0/877427291 >> 172.21.15.64:0/877427291 conn(0x560f2250cf60 msgr2=0x560f22519ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:18.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.626+0000 7f40cd746740 1 -- 172.21.15.64:0/877427291 shutdown_connections 2024-09-12T03:27:18.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.626+0000 7f40cd746740 1 -- 172.21.15.64:0/877427291 wait complete. 2024-09-12T03:27:18.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.626+0000 7f40cd746740 1 -- start start 2024-09-12T03:27:18.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.626+0000 7f40cd746740 1 Processor -- start 2024-09-12T03:27:18.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.627+0000 7f40cd746740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f225109d0 0x560f22536130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:18.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.627+0000 7f40cd746740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f225112f0 0x560f22536670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:18.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.627+0000 7f40cd746740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f2251bbb0 0x560f225316b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:18.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.627+0000 7f40cd746740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560f2252d970 con 0x560f225109d0 2024-09-12T03:27:18.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.627+0000 7f40cd746740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560f2252d2a0 con 0x560f225112f0 2024-09-12T03:27:18.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.627+0000 7f40cd746740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560f2252e040 con 0x560f2251bbb0 2024-09-12T03:27:18.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.627+0000 7f40cd746740 10 client.0 _ll_register_callbacks cb 0x7f40b8031830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:18.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.627+0000 7f40c8908640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f225109d0 0x560f22536130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:18.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.627+0000 7f40c8908640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f225109d0 0x560f22536130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35428/0 (socket says 172.21.15.64:35428) 2024-09-12T03:27:18.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.627+0000 7f40c8908640 1 -- 172.21.15.64:0/827078097 learned_addr learned my addr 172.21.15.64:0/827078097 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:27:18.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.627+0000 7f40c9109640 1 --2- 172.21.15.64:0/827078097 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f225112f0 0x560f22536670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:18.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.627+0000 7f40c8908640 1 --2- 172.21.15.64:0/827078097 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f2251bbb0 0x560f225316b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:18.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.627+0000 7f40c8908640 1 -- 172.21.15.64:0/827078097 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f2251bbb0 msgr2=0x560f225316b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:18.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.627+0000 7f40c8908640 1 --2- 172.21.15.64:0/827078097 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f2251bbb0 0x560f225316b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.627+0000 7f40c8908640 1 -- 172.21.15.64:0/827078097 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f225112f0 msgr2=0x560f22536670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:18.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.627+0000 7f40c8908640 1 --2- 172.21.15.64:0/827078097 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f225112f0 0x560f22536670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.627+0000 7f40c8908640 1 -- 172.21.15.64:0/827078097 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f22531ee0 con 0x560f225109d0 2024-09-12T03:27:18.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.627+0000 7f40c9109640 1 --2- 172.21.15.64:0/827078097 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f225112f0 0x560f22536670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:27:18.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.627+0000 7f40c8908640 1 --2- 172.21.15.64:0/827078097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f225109d0 0x560f22536130 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f40b80096c0 tx=0x7f40b8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4032dee096ec639 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:18.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.627+0000 7f40a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:27:18.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.628+0000 7f40a8ff9640 1 -- 172.21.15.64:0/827078097 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b8035070 con 0x560f225109d0 2024-09-12T03:27:18.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.628+0000 7f40a8ff9640 1 -- 172.21.15.64:0/827078097 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f40b8031a50 con 0x560f225109d0 2024-09-12T03:27:18.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.628+0000 7f40cd746740 1 -- 172.21.15.64:0/827078097 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40b803d900 con 0x560f225109d0 2024-09-12T03:27:18.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.628+0000 7f40a8ff9640 1 -- 172.21.15.64:0/827078097 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f40b8031c50 con 0x560f225109d0 2024-09-12T03:27:18.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.628+0000 7f40cd746740 10 client.14721 fetch_fsmap learned FSMap version 16 2024-09-12T03:27:18.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.628+0000 7f40cd746740 1 -- 172.21.15.64:0/827078097 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) -- 0x560f225969f0 con 0x560f225109d0 2024-09-12T03:27:18.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.628+0000 7f40a8ff9640 1 -- 172.21.15.64:0/827078097 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40b803d900 con 0x560f225109d0 2024-09-12T03:27:18.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.628+0000 7f40a8ff9640 1 -- 172.21.15.64:0/827078097 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 16) ==== 8350+0+0 (secure 0 0 0) 0x7f40b8003680 con 0x560f225109d0 2024-09-12T03:27:18.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.628+0000 7f40cd746740 10 client.14721 fetch_fsmap finished waiting for FSMap version 16 2024-09-12T03:27:18.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.628+0000 7f40cd746740 10 client.14721 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T03:27:18.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.628+0000 7f40cd746740 20 client.14721 populate_metadata read hostname 'smithi064' 2024-09-12T03:27:18.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.628+0000 7f40cd746740 1 --2- 172.21.15.64:0/827078097 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x560f2252c680 0x560f22599f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:18.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.628+0000 7f40cd746740 4 client.14721 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:27:18.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.628+0000 7f40cd746740 1 -- 172.21.15.64:0/827078097 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "status"}) -- 0x560f2247eb10 con 0x560f2252c680 2024-09-12T03:27:18.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.629+0000 7f40c9109640 1 --2- 172.21.15.64:0/827078097 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x560f2252c680 0x560f22599f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:18.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.629+0000 7f40c9109640 1 --2- 172.21.15.64:0/827078097 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x560f2252c680 0x560f22599f60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e16c6398204f4b4d server_cookie=4c6b4de9ffe498a0 in_seq=0 out_seq=0 2024-09-12T03:27:18.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.629+0000 7f40a8ff9640 10 client.14721 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:27:18.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.629+0000 7f40a8ff9640 1 -- 172.21.15.64:0/827078097 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x560f2247eb10 con 0x560f2252c680 2024-09-12T03:27:18.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.629+0000 7f40a8ff9640 10 client.14721 handle_command_reply: tid=0 2024-09-12T03:27:18.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.629+0000 7f40cd746740 1 client.14721 shutdown 2024-09-12T03:27:18.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.629+0000 7f40cd746740 1 -- 172.21.15.64:0/827078097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f225109d0 msgr2=0x560f22536130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:18.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.629+0000 7f40cd746740 1 --2- 172.21.15.64:0/827078097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f225109d0 0x560f22536130 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f40b80096c0 tx=0x7f40b8000bc0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.629+0000 7f40cd746740 1 -- 172.21.15.64:0/827078097 shutdown_connections 2024-09-12T03:27:18.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.629+0000 7f40cd746740 1 --2- 172.21.15.64:0/827078097 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x560f2252c680 0x560f22599f60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.629+0000 7f40cd746740 1 --2- 172.21.15.64:0/827078097 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f2251bbb0 0x560f225316b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.629+0000 7f40a8ff9640 0 client.14721 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:27:18.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.629+0000 7f40cd746740 1 --2- 172.21.15.64:0/827078097 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560f225112f0 0x560f22536670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.629+0000 7f40cd746740 1 --2- 172.21.15.64:0/827078097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f225109d0 0x560f22536130 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.629+0000 7f40cd746740 1 -- 172.21.15.64:0/827078097 >> 172.21.15.64:0/827078097 conn(0x560f2250cf60 msgr2=0x560f2250c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:18.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.630+0000 7f40cd746740 1 -- 172.21.15.64:0/827078097 shutdown_connections 2024-09-12T03:27:18.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.630+0000 7f40cd746740 1 -- 172.21.15.64:0/827078097 wait complete. 2024-09-12T03:27:18.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.630+0000 7f40cd746740 20 client.14721 trim_cache size 0 max 16384 2024-09-12T03:27:18.635 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T03:27:18.635 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "b0812460-70b5-11ef-bcea-c7b262605968", 2024-09-12T03:27:18.635 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2024-09-12T03:27:18.635 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24500, 2024-09-12T03:27:18.635 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-09-12T03:27:18.635 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-09-12T03:27:18.636 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-09-12T03:27:18.636 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 18.628445892999999, 2024-09-12T03:27:18.636 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 16, 2024-09-12T03:27:18.636 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 88, 2024-09-12T03:27:18.636 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 88, 2024-09-12T03:27:18.636 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 30.546896045 2024-09-12T03:27:18.636 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T03:27:18.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.630+0000 7f40cb36d640 1 -- 172.21.15.64:0/2296120801 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f40ac0a2b80 msgr2=0x7f40ac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:18.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.630+0000 7f40cb36d640 1 --2- 172.21.15.64:0/2296120801 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f40ac0a2b80 0x7f40ac0a5040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f40bc0a3780 tx=0x7f40bc06e100 comp rx=0 tx=0).stop 2024-09-12T03:27:18.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.630+0000 7f40cb36d640 1 -- 172.21.15.64:0/2296120801 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40c4102ef0 msgr2=0x7f40c4197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:18.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.630+0000 7f40cb36d640 1 --2- 172.21.15.64:0/2296120801 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40c4102ef0 0x7f40c4197bb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f40bc009fd0 tx=0x7f40bc00b8e0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.630+0000 7f40cb36d640 1 -- 172.21.15.64:0/2296120801 shutdown_connections 2024-09-12T03:27:18.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.630+0000 7f40cb36d640 1 --2- 172.21.15.64:0/2296120801 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f40ac0a2b80 0x7f40ac0a5040 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.630+0000 7f40cb36d640 1 --2- 172.21.15.64:0/2296120801 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40c4102ef0 0x7f40c4197bb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.630+0000 7f40cb36d640 1 --2- 172.21.15.64:0/2296120801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40c4102570 0x7f40c4197670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.630+0000 7f40cb36d640 1 --2- 172.21.15.64:0/2296120801 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40c4100e20 0x7f40c4197130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:18.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.630+0000 7f40cb36d640 1 -- 172.21.15.64:0/2296120801 >> 172.21.15.64:0/2296120801 conn(0x7f40c40fc7d0 msgr2=0x7f40c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:18.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.630+0000 7f40cb36d640 1 -- 172.21.15.64:0/2296120801 shutdown_connections 2024-09-12T03:27:18.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:18.630+0000 7f40cb36d640 1 -- 172.21.15.64:0/2296120801 wait complete. 2024-09-12T03:27:19.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:19 smithi115 ceph-mon[34801]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:19.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:19 smithi195 ceph-mon[34786]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:19.518 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:19 smithi064 ceph-mon[30952]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:19.548 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-12T03:27:19.548 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-12T03:27:19.548 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-12T03:27:19.806 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:20.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.270+0000 7f63f5010640 1 -- 172.21.15.64:0/847518670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63f0102570 msgr2=0x7f63f01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:20.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.270+0000 7f63f5010640 1 --2- 172.21.15.64:0/847518670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63f0102570 0x7f63f01029b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f63d8009900 tx=0x7f63d802a480 comp rx=0 tx=0).stop 2024-09-12T03:27:20.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.270+0000 7f63f5010640 1 -- 172.21.15.64:0/847518670 shutdown_connections 2024-09-12T03:27:20.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.270+0000 7f63f5010640 1 --2- 172.21.15.64:0/847518670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f63f0102ef0 0x7f63f010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:20.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.270+0000 7f63f5010640 1 --2- 172.21.15.64:0/847518670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63f0102570 0x7f63f01029b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:20.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.270+0000 7f63f5010640 1 --2- 172.21.15.64:0/847518670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f63f0100e20 0x7f63f0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:20.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.270+0000 7f63f5010640 1 -- 172.21.15.64:0/847518670 >> 172.21.15.64:0/847518670 conn(0x7f63f00fc7d0 msgr2=0x7f63f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:20.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.270+0000 7f63f5010640 1 -- 172.21.15.64:0/847518670 shutdown_connections 2024-09-12T03:27:20.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.270+0000 7f63f5010640 1 -- 172.21.15.64:0/847518670 wait complete. 2024-09-12T03:27:20.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.271+0000 7f63f5010640 1 Processor -- start 2024-09-12T03:27:20.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.271+0000 7f63f5010640 1 -- start start 2024-09-12T03:27:20.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.272+0000 7f63f5010640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63f0100e20 0x7f63f0198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:20.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.272+0000 7f63f5010640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f63f0102570 0x7f63f01989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:20.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.272+0000 7f63f5010640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f63f0102ef0 0x7f63f019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:20.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.272+0000 7f63f5010640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f63f0111fa0 con 0x7f63f0100e20 2024-09-12T03:27:20.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.272+0000 7f63f5010640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f63f0111e20 con 0x7f63f0102570 2024-09-12T03:27:20.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.272+0000 7f63f5010640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f63f0112120 con 0x7f63f0102ef0 2024-09-12T03:27:20.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.272+0000 7f63eed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63f0100e20 0x7f63f0198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:20.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.272+0000 7f63eed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63f0100e20 0x7f63f0198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35452/0 (socket says 172.21.15.64:35452) 2024-09-12T03:27:20.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.272+0000 7f63eed9d640 1 -- 172.21.15.64:0/1956804590 learned_addr learned my addr 172.21.15.64:0/1956804590 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:27:20.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.272+0000 7f63ef59e640 1 --2- 172.21.15.64:0/1956804590 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f63f0102ef0 0x7f63f019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:20.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.272+0000 7f63ee59c640 1 --2- 172.21.15.64:0/1956804590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f63f0102570 0x7f63f01989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:20.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.272+0000 7f63eed9d640 1 -- 172.21.15.64:0/1956804590 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f63f0102ef0 msgr2=0x7f63f019cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:20.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.272+0000 7f63eed9d640 1 --2- 172.21.15.64:0/1956804590 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f63f0102ef0 0x7f63f019cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:20.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.272+0000 7f63eed9d640 1 -- 172.21.15.64:0/1956804590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f63f0102570 msgr2=0x7f63f01989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:20.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.272+0000 7f63eed9d640 1 --2- 172.21.15.64:0/1956804590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f63f0102570 0x7f63f01989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:20.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.272+0000 7f63eed9d640 1 -- 172.21.15.64:0/1956804590 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63f019d410 con 0x7f63f0100e20 2024-09-12T03:27:20.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.272+0000 7f63ee59c640 1 --2- 172.21.15.64:0/1956804590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f63f0102570 0x7f63f01989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:27:20.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.272+0000 7f63eed9d640 1 --2- 172.21.15.64:0/1956804590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63f0100e20 0x7f63f0198460 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f63e800b4a0 tx=0x7f63e800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:20.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.273+0000 7f63e7fff640 1 -- 172.21.15.64:0/1956804590 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e8014070 con 0x7f63f0100e20 2024-09-12T03:27:20.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.273+0000 7f63f5010640 1 -- 172.21.15.64:0/1956804590 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63f019d700 con 0x7f63f0100e20 2024-09-12T03:27:20.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.273+0000 7f63e7fff640 1 -- 172.21.15.64:0/1956804590 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63e8010ac0 con 0x7f63f0100e20 2024-09-12T03:27:20.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.273+0000 7f63e7fff640 1 -- 172.21.15.64:0/1956804590 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e8017c30 con 0x7f63f0100e20 2024-09-12T03:27:20.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.273+0000 7f63f5010640 1 -- 172.21.15.64:0/1956804590 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63f01a4fe0 con 0x7f63f0100e20 2024-09-12T03:27:20.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.274+0000 7f63f5010640 1 -- 172.21.15.64:0/1956804590 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63f0101200 con 0x7f63f0100e20 2024-09-12T03:27:20.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.275+0000 7f63e7fff640 1 -- 172.21.15.64:0/1956804590 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f63e8017070 con 0x7f63f0100e20 2024-09-12T03:27:20.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.276+0000 7f63e7fff640 1 --2- 172.21.15.64:0/1956804590 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f63dc0a2b80 0x7f63dc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:20.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.276+0000 7f63ee59c640 1 --2- 172.21.15.64:0/1956804590 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f63dc0a2b80 0x7f63dc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:20.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.276+0000 7f63e7fff640 1 -- 172.21.15.64:0/1956804590 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f63e80ce950 con 0x7f63f0100e20 2024-09-12T03:27:20.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.276+0000 7f63ee59c640 1 --2- 172.21.15.64:0/1956804590 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f63dc0a2b80 0x7f63dc0a5040 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f63d8000c00 tx=0x7f63d8005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:20.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.280+0000 7f63e7fff640 1 -- 172.21.15.64:0/1956804590 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f63e809b670 con 0x7f63f0100e20 2024-09-12T03:27:20.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.490+0000 7f63f5010640 1 -- 172.21.15.64:0/1956804590 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f63f0109090 con 0x7f63f0100e20 2024-09-12T03:27:20.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.490+0000 7f63e7fff640 1 -- 172.21.15.64:0/1956804590 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+88 (secure 0 0 0) 0x7f63e809af70 con 0x7f63f0100e20 2024-09-12T03:27:20.490 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-12T03:27:20.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.492+0000 7f63f5010640 1 -- 172.21.15.64:0/1956804590 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f63dc0a2b80 msgr2=0x7f63dc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:20.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.492+0000 7f63f5010640 1 --2- 172.21.15.64:0/1956804590 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f63dc0a2b80 0x7f63dc0a5040 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f63d8000c00 tx=0x7f63d8005a40 comp rx=0 tx=0).stop 2024-09-12T03:27:20.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.492+0000 7f63f5010640 1 -- 172.21.15.64:0/1956804590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63f0100e20 msgr2=0x7f63f0198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:20.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.492+0000 7f63f5010640 1 --2- 172.21.15.64:0/1956804590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63f0100e20 0x7f63f0198460 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f63e800b4a0 tx=0x7f63e800b960 comp rx=0 tx=0).stop 2024-09-12T03:27:20.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.493+0000 7f63f5010640 1 -- 172.21.15.64:0/1956804590 shutdown_connections 2024-09-12T03:27:20.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.493+0000 7f63f5010640 1 --2- 172.21.15.64:0/1956804590 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f63dc0a2b80 0x7f63dc0a5040 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:20.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.493+0000 7f63f5010640 1 --2- 172.21.15.64:0/1956804590 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f63f0102ef0 0x7f63f019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:20.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.493+0000 7f63f5010640 1 --2- 172.21.15.64:0/1956804590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f63f0102570 0x7f63f01989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:20.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.493+0000 7f63f5010640 1 --2- 172.21.15.64:0/1956804590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63f0100e20 0x7f63f0198460 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:20.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.493+0000 7f63f5010640 1 -- 172.21.15.64:0/1956804590 >> 172.21.15.64:0/1956804590 conn(0x7f63f00fc7d0 msgr2=0x7f63f01068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:20.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.493+0000 7f63f5010640 1 -- 172.21.15.64:0/1956804590 shutdown_connections 2024-09-12T03:27:20.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:20.493+0000 7f63f5010640 1 -- 172.21.15.64:0/1956804590 wait complete. 2024-09-12T03:27:21.193 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:21 smithi064 ceph-mon[30952]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:21.194 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:21 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/1956804590' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T03:27:21.221 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-12T03:27:21.222 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T03:27:21.222 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2024-09-12T03:27:21.222 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-12T03:27:21.222 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-12T03:27:21.222 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-12T03:27:21.222 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T03:27:21.222 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T03:27:21.222 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-12T03:27:21.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:27:21.223 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2024-09-12T03:27:21.240 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T03:27:21.240 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T03:27:21.241 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T03:27:21.241 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:27:21.241 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2024-09-12T03:27:21.241 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:27:21.241 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:27:21.241 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:27:21.241 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f0 2024-09-12T03:27:21.241 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:27:21.241 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:27:21.241 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f1 2024-09-12T03:27:21.241 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:27:21.241 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2024-09-12T03:27:21.242 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T03:27:21.242 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2024-09-12T03:27:21.242 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-12T03:27:21.242 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 42302sec preferred_lft 42302sec 2024-09-12T03:27:21.242 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link noprefixroute 2024-09-12T03:27:21.242 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:27:21.242 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-12T03:27:21.243 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:27:21.243 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-09-12T03:27:21.243 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link add name ceph-brx type bridge 2024-09-12T03:27:21.243 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip addr flush dev ceph-brx 2024-09-12T03:27:21.243 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set ceph-brx up 2024-09-12T03:27:21.243 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-12T03:27:21.243 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-09-12T03:27:21.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:27:21.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:21 smithi115 ceph-mon[34801]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:21.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:21 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/1956804590' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T03:27:21.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:27:21.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:27:21.449 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T03:27:21.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:21 smithi195 ceph-mon[34786]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:21.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:21 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/1956804590' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T03:27:21.475 INFO:teuthology.orchestra.run.smithi064.stdout:1 2024-09-12T03:27:21.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:27:21.477 DEBUG:teuthology.orchestra.run.smithi064:> route 2024-09-12T03:27:21.551 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2024-09-12T03:27:21.551 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T03:27:21.552 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-12T03:27:21.552 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-12T03:27:21.552 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T03:27:21.552 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:27:21.552 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-09-12T03:27:21.552 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-12T03:27:21.552 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-12T03:27:21.553 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-12T03:27:21.553 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-09-12T03:27:21.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:27:21.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:27:21.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:27:21.842 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2024-09-12T03:27:21.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:27:21.859 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list-id 2024-09-12T03:27:21.915 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:27:21.916 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-09-12T03:27:21.916 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T03:27:21.916 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-12T03:27:21.916 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-09-12T03:27:22.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:27:22.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:27:22.036 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-12T03:27:22.037 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:27:22.037 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-09-12T03:27:22.037 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-12T03:27:22.037 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-12T03:27:22.037 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-12T03:27:22.037 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-12T03:27:22.037 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-12T03:27:22.037 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-09-12T03:27:22.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:27:22.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:27:22.167 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:27:22.167 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-09-12T03:27:22.167 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.0 up 2024-09-12T03:27:22.167 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set dev brx.0 master ceph-brx 2024-09-12T03:27:22.167 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-09-12T03:27:22.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:27:22.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:27:22.247 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-12T03:27:22.247 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:27:22.247 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-12T03:27:22.263 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-12T03:27:22.263 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:27:22.264 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-12T03:27:22.320 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2024-09-12T03:27:22.386 DEBUG:teuthology.orchestra.run.smithi064:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T03:27:22.444 INFO:teuthology.orchestra.run.smithi064.stdout:/proc 2024-09-12T03:27:22.444 INFO:teuthology.orchestra.run.smithi064.stdout:/sys 2024-09-12T03:27:22.444 INFO:teuthology.orchestra.run.smithi064.stdout:/dev 2024-09-12T03:27:22.444 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/security 2024-09-12T03:27:22.444 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/shm 2024-09-12T03:27:22.444 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/pts 2024-09-12T03:27:22.445 INFO:teuthology.orchestra.run.smithi064.stdout:/run 2024-09-12T03:27:22.445 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/cgroup 2024-09-12T03:27:22.445 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/pstore 2024-09-12T03:27:22.445 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/bpf 2024-09-12T03:27:22.445 INFO:teuthology.orchestra.run.smithi064.stdout:/ 2024-09-12T03:27:22.445 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/selinux 2024-09-12T03:27:22.445 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T03:27:22.445 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/mqueue 2024-09-12T03:27:22.445 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/hugepages 2024-09-12T03:27:22.445 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/debug 2024-09-12T03:27:22.445 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/tracing 2024-09-12T03:27:22.445 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/fuse/connections 2024-09-12T03:27:22.445 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/config 2024-09-12T03:27:22.446 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-sysctl.service 2024-09-12T03:27:22.446 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-12T03:27:22.446 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-12T03:27:22.446 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/nfs/rpc_pipefs 2024-09-12T03:27:22.446 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/1000 2024-09-12T03:27:22.446 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph 2024-09-12T03:27:22.446 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay 2024-09-12T03:27:22.446 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/ee33a1d8e4eaa1e034e23579453d7974193472acfcc3f767e0dd3d4df36a3590/merged 2024-09-12T03:27:22.446 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/659cd9feec649f7e9ef720891b9b8dee9cd5f2da02847139e453f34cea210194/merged 2024-09-12T03:27:22.446 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/0 2024-09-12T03:27:22.446 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T03:27:22.446 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/6657365ee9908bf70289e1e2b4eba2994c383e00f4e1271d1910290a4e352dfb/merged 2024-09-12T03:27:22.446 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/d63e2a8e350f2e9f6e0b5aaea9b095d7e0b824ac35353c9dfb67c8bbf5259935/merged 2024-09-12T03:27:22.446 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/d01078f963b9aca8da0d3791bbb79a913c48a122c7a98b85055d0e0b00bdae45/merged 2024-09-12T03:27:22.447 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/12ca51601f86349c78af34a3c17cb3ca77bb030d12b3c8f618a5a555c1466e7e/merged 2024-09-12T03:27:22.447 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/29a3719ab7c8a64396f80d3a1c6095fbc7965ffe2b1e0730f1b22eacf0ef7da5/merged 2024-09-12T03:27:22.447 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/7870e610b0a225b40a938f58f36210d45d2c00dfdaf3db8248049e82bc99279a/merged 2024-09-12T03:27:22.447 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/a27f9d3c9d12f78c7f07ff9d3dee29c8d26c31554950b6dc66444fc30bd57047/merged 2024-09-12T03:27:22.447 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/e71f1074293252655b8551c7acdc167b8003259d1051bf1071b7e92e0506ef0a/merged 2024-09-12T03:27:22.447 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns 2024-09-12T03:27:22.447 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T03:27:22.447 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T03:27:22.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:27:22.447 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2024-09-12T03:27:22.501 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-12T03:27:22.501 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-12T03:27:22.547 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2024-09-12T03:27:22.572 DEBUG:teuthology.orchestra.run.smithi064:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T03:27:22.629 INFO:teuthology.orchestra.run.smithi064.stdout:/proc 2024-09-12T03:27:22.630 INFO:teuthology.orchestra.run.smithi064.stdout:/sys 2024-09-12T03:27:22.630 INFO:teuthology.orchestra.run.smithi064.stdout:/dev 2024-09-12T03:27:22.630 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/security 2024-09-12T03:27:22.630 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/shm 2024-09-12T03:27:22.630 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/pts 2024-09-12T03:27:22.630 INFO:teuthology.orchestra.run.smithi064.stdout:/run 2024-09-12T03:27:22.630 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/cgroup 2024-09-12T03:27:22.630 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/pstore 2024-09-12T03:27:22.630 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/bpf 2024-09-12T03:27:22.630 INFO:teuthology.orchestra.run.smithi064.stdout:/ 2024-09-12T03:27:22.630 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/selinux 2024-09-12T03:27:22.630 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T03:27:22.631 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/mqueue 2024-09-12T03:27:22.631 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/hugepages 2024-09-12T03:27:22.631 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/debug 2024-09-12T03:27:22.631 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/tracing 2024-09-12T03:27:22.631 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/fuse/connections 2024-09-12T03:27:22.631 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/config 2024-09-12T03:27:22.631 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-sysctl.service 2024-09-12T03:27:22.631 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-12T03:27:22.631 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-12T03:27:22.631 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/nfs/rpc_pipefs 2024-09-12T03:27:22.631 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/1000 2024-09-12T03:27:22.631 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph 2024-09-12T03:27:22.631 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay 2024-09-12T03:27:22.632 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/ee33a1d8e4eaa1e034e23579453d7974193472acfcc3f767e0dd3d4df36a3590/merged 2024-09-12T03:27:22.632 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/659cd9feec649f7e9ef720891b9b8dee9cd5f2da02847139e453f34cea210194/merged 2024-09-12T03:27:22.632 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/0 2024-09-12T03:27:22.632 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T03:27:22.632 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/6657365ee9908bf70289e1e2b4eba2994c383e00f4e1271d1910290a4e352dfb/merged 2024-09-12T03:27:22.632 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/d63e2a8e350f2e9f6e0b5aaea9b095d7e0b824ac35353c9dfb67c8bbf5259935/merged 2024-09-12T03:27:22.632 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/d01078f963b9aca8da0d3791bbb79a913c48a122c7a98b85055d0e0b00bdae45/merged 2024-09-12T03:27:22.632 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/12ca51601f86349c78af34a3c17cb3ca77bb030d12b3c8f618a5a555c1466e7e/merged 2024-09-12T03:27:22.632 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/29a3719ab7c8a64396f80d3a1c6095fbc7965ffe2b1e0730f1b22eacf0ef7da5/merged 2024-09-12T03:27:22.632 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/7870e610b0a225b40a938f58f36210d45d2c00dfdaf3db8248049e82bc99279a/merged 2024-09-12T03:27:22.632 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/a27f9d3c9d12f78c7f07ff9d3dee29c8d26c31554950b6dc66444fc30bd57047/merged 2024-09-12T03:27:22.632 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/e71f1074293252655b8551c7acdc167b8003259d1051bf1071b7e92e0506ef0a/merged 2024-09-12T03:27:22.632 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns 2024-09-12T03:27:22.633 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T03:27:22.633 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T03:27:22.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:27:22.633 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2024-09-12T03:27:22.660 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi064.stderr:2024-09-12T03:27:22.660+0000 7faeb811f480 -1 init, newargv = 0x5586957e7ce0 newargc=15 2024-09-12T03:27:22.660 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi064.stderr:ceph-fuse[75353]: starting ceph client 2024-09-12T03:27:22.665 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi064.stderr:ceph-fuse[75353]: starting fuse 2024-09-12T03:27:22.676 INFO:teuthology.orchestra.run.smithi064.stdout:96 2024-09-12T03:27:22.676 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [96] 2024-09-12T03:27:22.677 DEBUG:teuthology.orchestra.run.smithi064:> sudo stdin-killer -- python3 -c ' 2024-09-12T03:27:22.677 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2024-09-12T03:27:22.677 DEBUG:teuthology.orchestra.run.smithi064:> import re 2024-09-12T03:27:22.677 DEBUG:teuthology.orchestra.run.smithi064:> import os 2024-09-12T03:27:22.677 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2024-09-12T03:27:22.677 DEBUG:teuthology.orchestra.run.smithi064:> 2024-09-12T03:27:22.677 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2024-09-12T03:27:22.677 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-12T03:27:22.677 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2024-09-12T03:27:22.677 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-12T03:27:22.677 DEBUG:teuthology.orchestra.run.smithi064:> 2024-09-12T03:27:22.677 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2024-09-12T03:27:22.678 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2024-09-12T03:27:22.678 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2024-09-12T03:27:22.678 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2024-09-12T03:27:22.678 DEBUG:teuthology.orchestra.run.smithi064:> 2024-09-12T03:27:22.678 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2024-09-12T03:27:22.678 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-12T03:27:22.678 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-12T03:27:22.678 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-12T03:27:22.678 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2024-09-12T03:27:22.678 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2024-09-12T03:27:22.678 DEBUG:teuthology.orchestra.run.smithi064:> return f 2024-09-12T03:27:22.678 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-12T03:27:22.678 DEBUG:teuthology.orchestra.run.smithi064:> 2024-09-12T03:27:22.678 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.admin")) 2024-09-12T03:27:22.678 DEBUG:teuthology.orchestra.run.smithi064:> ' 2024-09-12T03:27:22.797 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.admin.75353.asok 2024-09-12T03:27:22.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:27:22.811 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.75353.asok 2024-09-12T03:27:22.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:27:22.811 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.75353.asok status 2024-09-12T03:27:22.946 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T03:27:22.946 INFO:teuthology.orchestra.run.smithi064.stdout: "metadata": { 2024-09-12T03:27:22.946 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", 2024-09-12T03:27:22.946 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T03:27:22.946 INFO:teuthology.orchestra.run.smithi064.stdout: "entity_id": "admin", 2024-09-12T03:27:22.946 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": "smithi064", 2024-09-12T03:27:22.946 INFO:teuthology.orchestra.run.smithi064.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-12T03:27:22.946 INFO:teuthology.orchestra.run.smithi064.stdout: "pid": "75353", 2024-09-12T03:27:22.946 INFO:teuthology.orchestra.run.smithi064.stdout: "root": "/" 2024-09-12T03:27:22.947 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:27:22.947 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_count": 0, 2024-09-12T03:27:22.947 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_pinned_count": 0, 2024-09-12T03:27:22.947 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 14733, 2024-09-12T03:27:22.947 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2024-09-12T03:27:22.947 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2024-09-12T03:27:22.947 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2024-09-12T03:27:22.947 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 14733 2024-09-12T03:27:22.947 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:27:22.947 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2024-09-12T03:27:22.947 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2024-09-12T03:27:22.947 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2024-09-12T03:27:22.947 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 956652389 2024-09-12T03:27:22.948 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:27:22.948 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:27:22.948 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2024-09-12T03:27:22.948 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2024-09-12T03:27:22.948 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2024-09-12T03:27:22.948 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 956652389 2024-09-12T03:27:22.948 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:27:22.948 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.14733 192.168.0.1:0/956652389", 2024-09-12T03:27:22.948 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "192.168.0.1:0/956652389", 2024-09-12T03:27:22.948 INFO:teuthology.orchestra.run.smithi064.stdout: "inode_count": 1, 2024-09-12T03:27:22.948 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_epoch": 16, 2024-09-12T03:27:22.948 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch": 88, 2024-09-12T03:27:22.949 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch_barrier": 0, 2024-09-12T03:27:22.949 INFO:teuthology.orchestra.run.smithi064.stdout: "blocklisted": false, 2024-09-12T03:27:22.949 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs" 2024-09-12T03:27:22.949 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T03:27:22.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:27:22.954 DEBUG:teuthology.orchestra.run.smithi064:> stat --file-system '--printf=%T 2024-09-12T03:27:22.954 DEBUG:teuthology.orchestra.run.smithi064:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-12T03:27:22.971 INFO:teuthology.orchestra.run.smithi064.stdout:fuseblk 2024-09-12T03:27:22.971 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-12T03:27:22.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:27:22.971 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-12T03:27:23.040 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-12T03:27:23.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:27:23.147 INFO:teuthology.orchestra.run.smithi064.stdout:# file: . 2024-09-12T03:27:23.147 INFO:teuthology.orchestra.run.smithi064.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-12T03:27:23.147 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:27:23.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:27:23.153 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-12T03:27:23.154 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:27:23.171 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-12T03:27:23.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:23 smithi115 ceph-mon[34801]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:23.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:23 smithi195 ceph-mon[34786]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:23.467 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:23.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:23 smithi064 ceph-mon[30952]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:23.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.929+0000 7f306c540640 1 -- 172.21.15.64:0/1984475435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3064101de0 msgr2=0x7f30641021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:23.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.929+0000 7f306c540640 1 --2- 172.21.15.64:0/1984475435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3064101de0 0x7f30641021c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3060009900 tx=0x7f306002a3f0 comp rx=0 tx=0).stop 2024-09-12T03:27:23.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.929+0000 7f306c540640 1 -- 172.21.15.64:0/1984475435 shutdown_connections 2024-09-12T03:27:23.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.929+0000 7f306c540640 1 --2- 172.21.15.64:0/1984475435 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3064106f10 0x7f306410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:23.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.929+0000 7f306c540640 1 --2- 172.21.15.64:0/1984475435 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3064102790 0x7f30641067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:23.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.929+0000 7f306c540640 1 --2- 172.21.15.64:0/1984475435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3064101de0 0x7f30641021c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:23.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.929+0000 7f306c540640 1 -- 172.21.15.64:0/1984475435 >> 172.21.15.64:0/1984475435 conn(0x7f30640fd8c0 msgr2=0x7f30640ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:23.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.929+0000 7f306c540640 1 -- 172.21.15.64:0/1984475435 shutdown_connections 2024-09-12T03:27:23.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.929+0000 7f306c540640 1 -- 172.21.15.64:0/1984475435 wait complete. 2024-09-12T03:27:23.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.930+0000 7f306c540640 1 Processor -- start 2024-09-12T03:27:23.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.930+0000 7f306c540640 1 -- start start 2024-09-12T03:27:23.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.930+0000 7f306c540640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3064101de0 0x7f30641982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:23.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.930+0000 7f306c540640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3064102790 0x7f3064198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:23.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.930+0000 7f306c540640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3064106f10 0x7f306419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:23.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.930+0000 7f306c540640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3064111e20 con 0x7f3064101de0 2024-09-12T03:27:23.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.930+0000 7f306c540640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3064111ca0 con 0x7f3064102790 2024-09-12T03:27:23.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.930+0000 7f306c540640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3064111fa0 con 0x7f3064106f10 2024-09-12T03:27:23.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.930+0000 7f306a2dc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3064101de0 0x7f30641982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:23.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.930+0000 7f306a2dc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3064101de0 0x7f30641982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35474/0 (socket says 172.21.15.64:35474) 2024-09-12T03:27:23.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.930+0000 7f306a2dc640 1 -- 172.21.15.64:0/2495518741 learned_addr learned my addr 172.21.15.64:0/2495518741 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:27:23.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.931+0000 7f3069adb640 1 --2- 172.21.15.64:0/2495518741 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3064102790 0x7f3064198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:23.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.931+0000 7f306aadd640 1 --2- 172.21.15.64:0/2495518741 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3064106f10 0x7f306419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:23.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.931+0000 7f306a2dc640 1 -- 172.21.15.64:0/2495518741 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3064106f10 msgr2=0x7f306419cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:23.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.931+0000 7f306a2dc640 1 --2- 172.21.15.64:0/2495518741 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3064106f10 0x7f306419cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:23.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.931+0000 7f306a2dc640 1 -- 172.21.15.64:0/2495518741 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3064102790 msgr2=0x7f3064198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:23.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.931+0000 7f306a2dc640 1 --2- 172.21.15.64:0/2495518741 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3064102790 0x7f3064198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:23.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.931+0000 7f306a2dc640 1 -- 172.21.15.64:0/2495518741 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f306419d290 con 0x7f3064101de0 2024-09-12T03:27:23.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.931+0000 7f3069adb640 1 --2- 172.21.15.64:0/2495518741 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3064102790 0x7f3064198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:27:23.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.931+0000 7f306aadd640 1 --2- 172.21.15.64:0/2495518741 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3064106f10 0x7f306419cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:27:23.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.931+0000 7f306a2dc640 1 --2- 172.21.15.64:0/2495518741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3064101de0 0x7f30641982e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3060005940 tx=0x7f30600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:23.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.931+0000 7f305b7fe640 1 -- 172.21.15.64:0/2495518741 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306003f030 con 0x7f3064101de0 2024-09-12T03:27:23.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.931+0000 7f305b7fe640 1 -- 172.21.15.64:0/2495518741 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3060003cb0 con 0x7f3064101de0 2024-09-12T03:27:23.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.931+0000 7f306c540640 1 -- 172.21.15.64:0/2495518741 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f306419d520 con 0x7f3064101de0 2024-09-12T03:27:23.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.931+0000 7f305b7fe640 1 -- 172.21.15.64:0/2495518741 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3060035070 con 0x7f3064101de0 2024-09-12T03:27:23.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.931+0000 7f306c540640 1 -- 172.21.15.64:0/2495518741 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30641a4e60 con 0x7f3064101de0 2024-09-12T03:27:23.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.933+0000 7f306c540640 1 -- 172.21.15.64:0/2495518741 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30641032c0 con 0x7f3064101de0 2024-09-12T03:27:23.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.933+0000 7f305b7fe640 1 -- 172.21.15.64:0/2495518741 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3060042030 con 0x7f3064101de0 2024-09-12T03:27:23.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.934+0000 7f305b7fe640 1 --2- 172.21.15.64:0/2495518741 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f30500a2b80 0x7f30500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:23.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.934+0000 7f3069adb640 1 --2- 172.21.15.64:0/2495518741 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f30500a2b80 0x7f30500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:23.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.934+0000 7f305b7fe640 1 -- 172.21.15.64:0/2495518741 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f30600efaf0 con 0x7f3064101de0 2024-09-12T03:27:23.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.935+0000 7f3069adb640 1 --2- 172.21.15.64:0/2495518741 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f30500a2b80 0x7f30500a5040 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f30640fdce0 tx=0x7f305400a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:23.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:23.938+0000 7f305b7fe640 1 -- 172.21.15.64:0/2495518741 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f30600bc810 con 0x7f3064101de0 2024-09-12T03:27:24.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:24.130+0000 7f306c540640 1 -- 172.21.15.64:0/2495518741 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f306410a0f0 con 0x7f3064101de0 2024-09-12T03:27:24.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:24.130+0000 7f305b7fe640 1 -- 172.21.15.64:0/2495518741 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) ==== 81+0+587 (secure 0 0 0) 0x7f30600bc110 con 0x7f3064101de0 2024-09-12T03:27:24.131 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/68603379 2024-09-13T03:20:07.411102+0000 2024-09-12T03:27:24.131 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3827202979 2024-09-13T03:20:07.411102+0000 2024-09-12T03:27:24.131 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/68603379 2024-09-13T03:20:07.411102+0000 2024-09-12T03:27:24.131 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1217136221 2024-09-13T03:20:07.411102+0000 2024-09-12T03:27:24.132 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3079679964 2024-09-13T03:19:36.839431+0000 2024-09-12T03:27:24.132 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/3186640984 2024-09-13T03:19:36.839431+0000 2024-09-12T03:27:24.132 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3334629354 2024-09-13T03:20:07.411102+0000 2024-09-12T03:27:24.132 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/3186640984 2024-09-13T03:19:36.839431+0000 2024-09-12T03:27:24.132 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3564418115 2024-09-13T03:19:36.839431+0000 2024-09-12T03:27:24.132 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/686137382 2024-09-13T03:19:36.839431+0000 2024-09-12T03:27:24.132 INFO:teuthology.orchestra.run.smithi064.stderr:listed 10 entries 2024-09-12T03:27:24.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:24.132+0000 7f306c540640 1 -- 172.21.15.64:0/2495518741 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f30500a2b80 msgr2=0x7f30500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:24.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:24.132+0000 7f306c540640 1 --2- 172.21.15.64:0/2495518741 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f30500a2b80 0x7f30500a5040 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f30640fdce0 tx=0x7f305400a5c0 comp rx=0 tx=0).stop 2024-09-12T03:27:24.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:24.132+0000 7f306c540640 1 -- 172.21.15.64:0/2495518741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3064101de0 msgr2=0x7f30641982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:24.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:24.132+0000 7f306c540640 1 --2- 172.21.15.64:0/2495518741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3064101de0 0x7f30641982e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3060005940 tx=0x7f30600042f0 comp rx=0 tx=0).stop 2024-09-12T03:27:24.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:24.133+0000 7f306c540640 1 -- 172.21.15.64:0/2495518741 shutdown_connections 2024-09-12T03:27:24.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:24.133+0000 7f306c540640 1 --2- 172.21.15.64:0/2495518741 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f30500a2b80 0x7f30500a5040 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:24.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:24.133+0000 7f306c540640 1 --2- 172.21.15.64:0/2495518741 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3064106f10 0x7f306419cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:24.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:24.133+0000 7f306c540640 1 --2- 172.21.15.64:0/2495518741 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3064102790 0x7f3064198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:24.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:24.133+0000 7f306c540640 1 --2- 172.21.15.64:0/2495518741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3064101de0 0x7f30641982e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:24.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:24.133+0000 7f306c540640 1 -- 172.21.15.64:0/2495518741 >> 172.21.15.64:0/2495518741 conn(0x7f30640fd8c0 msgr2=0x7f30640febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:24.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:24.133+0000 7f306c540640 1 -- 172.21.15.64:0/2495518741 shutdown_connections 2024-09-12T03:27:24.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:24.133+0000 7f306c540640 1 -- 172.21.15.64:0/2495518741 wait complete. 2024-09-12T03:27:24.855 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-12T03:27:24.855 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:27:24.878 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-12T03:27:25.176 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:25.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:25 smithi064 ceph-mon[30952]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:25.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:25 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/2495518741' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:27:25.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:25 smithi115 ceph-mon[34801]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:25.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:25 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/2495518741' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:27:25.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:25 smithi195 ceph-mon[34786]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:25.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:25 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/2495518741' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:27:25.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.635+0000 7fce0eaf0640 1 -- 172.21.15.64:0/1436942426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce08101420 msgr2=0x7fce0810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:25.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.635+0000 7fce0eaf0640 1 --2- 172.21.15.64:0/1436942426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce08101420 0x7fce0810d950 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fce00009930 tx=0x7fce0002a420 comp rx=0 tx=0).stop 2024-09-12T03:27:25.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.635+0000 7fce0eaf0640 1 -- 172.21.15.64:0/1436942426 shutdown_connections 2024-09-12T03:27:25.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.635+0000 7fce0eaf0640 1 --2- 172.21.15.64:0/1436942426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce08101420 0x7fce0810d950 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.635+0000 7fce0eaf0640 1 --2- 172.21.15.64:0/1436942426 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce08100a80 0x7fce08100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.635+0000 7fce0eaf0640 1 --2- 172.21.15.64:0/1436942426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce08106a80 0x7fce08106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.635+0000 7fce0eaf0640 1 -- 172.21.15.64:0/1436942426 >> 172.21.15.64:0/1436942426 conn(0x7fce080fc7a0 msgr2=0x7fce080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:25.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.635+0000 7fce0eaf0640 1 -- 172.21.15.64:0/1436942426 shutdown_connections 2024-09-12T03:27:25.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.635+0000 7fce0eaf0640 1 -- 172.21.15.64:0/1436942426 wait complete. 2024-09-12T03:27:25.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.636+0000 7fce0eaf0640 1 Processor -- start 2024-09-12T03:27:25.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.636+0000 7fce0eaf0640 1 -- start start 2024-09-12T03:27:25.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.636+0000 7fce0eaf0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce08100a80 0x7fce08197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:25.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.636+0000 7fce0eaf0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce08101420 0x7fce081977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:25.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.636+0000 7fce0eaf0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce08106a80 0x7fce0819bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:25.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.636+0000 7fce0eaf0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fce08110d30 con 0x7fce08106a80 2024-09-12T03:27:25.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.636+0000 7fce0eaf0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fce08110bb0 con 0x7fce08100a80 2024-09-12T03:27:25.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.636+0000 7fce0eaf0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fce08110eb0 con 0x7fce08101420 2024-09-12T03:27:25.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.636+0000 7fce0d08d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce08106a80 0x7fce0819bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:25.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.636+0000 7fce0d08d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce08106a80 0x7fce0819bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35494/0 (socket says 172.21.15.64:35494) 2024-09-12T03:27:25.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.636+0000 7fce0d08d640 1 -- 172.21.15.64:0/1087972606 learned_addr learned my addr 172.21.15.64:0/1087972606 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:27:25.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.637+0000 7fce0c88c640 1 --2- 172.21.15.64:0/1087972606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce08100a80 0x7fce08197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:25.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.637+0000 7fce07fff640 1 --2- 172.21.15.64:0/1087972606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce08101420 0x7fce081977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:25.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.637+0000 7fce0d08d640 1 -- 172.21.15.64:0/1087972606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce08101420 msgr2=0x7fce081977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:25.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.637+0000 7fce0d08d640 1 --2- 172.21.15.64:0/1087972606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce08101420 0x7fce081977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.637+0000 7fce0d08d640 1 -- 172.21.15.64:0/1087972606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce08100a80 msgr2=0x7fce08197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:25.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.637+0000 7fce0d08d640 1 --2- 172.21.15.64:0/1087972606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce08100a80 0x7fce08197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.637+0000 7fce0d08d640 1 -- 172.21.15.64:0/1087972606 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce0819c220 con 0x7fce08106a80 2024-09-12T03:27:25.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.637+0000 7fce0d08d640 1 --2- 172.21.15.64:0/1087972606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce08106a80 0x7fce0819bb40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fce00005940 tx=0x7fce00004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:25.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.637+0000 7fce05ffb640 1 -- 172.21.15.64:0/1087972606 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0003f030 con 0x7fce08106a80 2024-09-12T03:27:25.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.637+0000 7fce05ffb640 1 -- 172.21.15.64:0/1087972606 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce00003aa0 con 0x7fce08106a80 2024-09-12T03:27:25.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.637+0000 7fce0eaf0640 1 -- 172.21.15.64:0/1087972606 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce0819c4b0 con 0x7fce08106a80 2024-09-12T03:27:25.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.637+0000 7fce05ffb640 1 -- 172.21.15.64:0/1087972606 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce00035070 con 0x7fce08106a80 2024-09-12T03:27:25.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.637+0000 7fce0eaf0640 1 -- 172.21.15.64:0/1087972606 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce081a3df0 con 0x7fce08106a80 2024-09-12T03:27:25.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.638+0000 7fce0eaf0640 1 -- 172.21.15.64:0/1087972606 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce08100ee0 con 0x7fce08106a80 2024-09-12T03:27:25.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.639+0000 7fce05ffb640 1 -- 172.21.15.64:0/1087972606 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fce00042030 con 0x7fce08106a80 2024-09-12T03:27:25.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.640+0000 7fce05ffb640 1 --2- 172.21.15.64:0/1087972606 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fcdf40a2c00 0x7fcdf40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:25.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.640+0000 7fce0c88c640 1 --2- 172.21.15.64:0/1087972606 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fcdf40a2c00 0x7fcdf40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:25.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.640+0000 7fce05ffb640 1 -- 172.21.15.64:0/1087972606 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fce000efea0 con 0x7fce08106a80 2024-09-12T03:27:25.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.641+0000 7fce0c88c640 1 --2- 172.21.15.64:0/1087972606 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fcdf40a2c00 0x7fcdf40a50c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0005fd0 tx=0x7fcdf0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:25.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.643+0000 7fce05ffb640 1 -- 172.21.15.64:0/1087972606 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fce000bcbc0 con 0x7fce08106a80 2024-09-12T03:27:25.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.830+0000 7fce0eaf0640 1 -- 172.21.15.64:0/1087972606 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fce081985d0 con 0x7fce08106a80 2024-09-12T03:27:25.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.830+0000 7fce05ffb640 1 -- 172.21.15.64:0/1087972606 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) ==== 81+0+587 (secure 0 0 0) 0x7fce081985d0 con 0x7fce08106a80 2024-09-12T03:27:25.831 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/68603379 2024-09-13T03:20:07.411102+0000 2024-09-12T03:27:25.831 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3827202979 2024-09-13T03:20:07.411102+0000 2024-09-12T03:27:25.832 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/68603379 2024-09-13T03:20:07.411102+0000 2024-09-12T03:27:25.832 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1217136221 2024-09-13T03:20:07.411102+0000 2024-09-12T03:27:25.832 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3079679964 2024-09-13T03:19:36.839431+0000 2024-09-12T03:27:25.832 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/3186640984 2024-09-13T03:19:36.839431+0000 2024-09-12T03:27:25.832 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3334629354 2024-09-13T03:20:07.411102+0000 2024-09-12T03:27:25.832 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/3186640984 2024-09-13T03:19:36.839431+0000 2024-09-12T03:27:25.832 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3564418115 2024-09-13T03:19:36.839431+0000 2024-09-12T03:27:25.832 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/686137382 2024-09-13T03:19:36.839431+0000 2024-09-12T03:27:25.832 INFO:teuthology.orchestra.run.smithi064.stderr:listed 10 entries 2024-09-12T03:27:25.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.833+0000 7fce0eaf0640 1 -- 172.21.15.64:0/1087972606 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fcdf40a2c00 msgr2=0x7fcdf40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:25.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.833+0000 7fce0eaf0640 1 --2- 172.21.15.64:0/1087972606 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fcdf40a2c00 0x7fcdf40a50c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0005fd0 tx=0x7fcdf0005850 comp rx=0 tx=0).stop 2024-09-12T03:27:25.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.833+0000 7fce0eaf0640 1 -- 172.21.15.64:0/1087972606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce08106a80 msgr2=0x7fce0819bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:25.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.833+0000 7fce0eaf0640 1 --2- 172.21.15.64:0/1087972606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce08106a80 0x7fce0819bb40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fce00005940 tx=0x7fce00004270 comp rx=0 tx=0).stop 2024-09-12T03:27:25.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.833+0000 7fce0eaf0640 1 -- 172.21.15.64:0/1087972606 shutdown_connections 2024-09-12T03:27:25.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.833+0000 7fce0eaf0640 1 --2- 172.21.15.64:0/1087972606 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fcdf40a2c00 0x7fcdf40a50c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.833+0000 7fce0eaf0640 1 --2- 172.21.15.64:0/1087972606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce08106a80 0x7fce0819bb40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.833+0000 7fce0eaf0640 1 --2- 172.21.15.64:0/1087972606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce08101420 0x7fce081977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.833+0000 7fce0eaf0640 1 --2- 172.21.15.64:0/1087972606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce08100a80 0x7fce08197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.833+0000 7fce0eaf0640 1 -- 172.21.15.64:0/1087972606 >> 172.21.15.64:0/1087972606 conn(0x7fce080fc7a0 msgr2=0x7fce080fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:25.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.833+0000 7fce0eaf0640 1 -- 172.21.15.64:0/1087972606 shutdown_connections 2024-09-12T03:27:25.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:25.833+0000 7fce0eaf0640 1 -- 172.21.15.64:0/1087972606 wait complete. 2024-09-12T03:27:26.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:26 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/1087972606' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:27:26.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:26 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/1087972606' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:27:26.501 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:26 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/1087972606' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:27:26.528 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi064.front.sepia.ceph.com... 2024-09-12T03:27:26.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:27:26.529 DEBUG:teuthology.orchestra.run.smithi064:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-12T03:27:26.554 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi064.stderr:ceph-fuse[75353]: fuse finished with error 0 and tester_r 0 2024-09-12T03:27:26.556 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:27:26.645 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2024-09-12T03:27:26.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:27:26.645 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-12T03:27:26.673 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:27:26.969 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:27.352 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:27 smithi064 ceph-mon[30952]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:27.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:27 smithi115 ceph-mon[34801]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:27.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.433+0000 7ff137060640 1 -- 172.21.15.64:0/2860987210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1301027a0 msgr2=0x7ff1301067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:27.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.433+0000 7ff137060640 1 --2- 172.21.15.64:0/2860987210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1301027a0 0x7ff1301067f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff120009ca0 tx=0x7ff12002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.433+0000 7ff137060640 1 -- 172.21.15.64:0/2860987210 shutdown_connections 2024-09-12T03:27:27.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.433+0000 7ff137060640 1 --2- 172.21.15.64:0/2860987210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff130106f20 0x7ff13010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.433+0000 7ff137060640 1 --2- 172.21.15.64:0/2860987210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1301027a0 0x7ff1301067f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.433+0000 7ff137060640 1 --2- 172.21.15.64:0/2860987210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff130101df0 0x7ff1301021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.433+0000 7ff137060640 1 -- 172.21.15.64:0/2860987210 >> 172.21.15.64:0/2860987210 conn(0x7ff1300fd8b0 msgr2=0x7ff1300ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:27.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.433+0000 7ff137060640 1 -- 172.21.15.64:0/2860987210 shutdown_connections 2024-09-12T03:27:27.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.433+0000 7ff137060640 1 -- 172.21.15.64:0/2860987210 wait complete. 2024-09-12T03:27:27.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.434+0000 7ff137060640 1 Processor -- start 2024-09-12T03:27:27.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.434+0000 7ff137060640 1 -- start start 2024-09-12T03:27:27.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.434+0000 7ff137060640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff130101df0 0x7ff1301983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:27.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.434+0000 7ff137060640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1301027a0 0x7ff130198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:27.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.434+0000 7ff137060640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff130106f20 0x7ff13019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:27.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.434+0000 7ff137060640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff130111e70 con 0x7ff130101df0 2024-09-12T03:27:27.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.434+0000 7ff137060640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff130111cf0 con 0x7ff1301027a0 2024-09-12T03:27:27.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.434+0000 7ff137060640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff130111ff0 con 0x7ff130106f20 2024-09-12T03:27:27.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.434+0000 7ff134dfc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff130101df0 0x7ff1301983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:27.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.434+0000 7ff134dfc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff130101df0 0x7ff1301983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35516/0 (socket says 172.21.15.64:35516) 2024-09-12T03:27:27.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.434+0000 7ff134dfc640 1 -- 172.21.15.64:0/2783698546 learned_addr learned my addr 172.21.15.64:0/2783698546 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:27:27.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.434+0000 7ff1355fd640 1 --2- 172.21.15.64:0/2783698546 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff130106f20 0x7ff13019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:27.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.434+0000 7ff12ffff640 1 --2- 172.21.15.64:0/2783698546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1301027a0 0x7ff130198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:27.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.434+0000 7ff134dfc640 1 -- 172.21.15.64:0/2783698546 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff130106f20 msgr2=0x7ff13019ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:27.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.434+0000 7ff134dfc640 1 --2- 172.21.15.64:0/2783698546 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff130106f20 0x7ff13019ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.434+0000 7ff134dfc640 1 -- 172.21.15.64:0/2783698546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1301027a0 msgr2=0x7ff130198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:27.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.434+0000 7ff134dfc640 1 --2- 172.21.15.64:0/2783698546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1301027a0 0x7ff130198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.434+0000 7ff134dfc640 1 -- 172.21.15.64:0/2783698546 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff13019d3a0 con 0x7ff130101df0 2024-09-12T03:27:27.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.434+0000 7ff12ffff640 1 --2- 172.21.15.64:0/2783698546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1301027a0 0x7ff130198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:27:27.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.435+0000 7ff134dfc640 1 --2- 172.21.15.64:0/2783698546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff130101df0 0x7ff1301983f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff12800c970 tx=0x7ff12800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:27.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.435+0000 7ff12dffb640 1 -- 172.21.15.64:0/2783698546 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff128007bb0 con 0x7ff130101df0 2024-09-12T03:27:27.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.435+0000 7ff12dffb640 1 -- 172.21.15.64:0/2783698546 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff128007d50 con 0x7ff130101df0 2024-09-12T03:27:27.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.435+0000 7ff137060640 1 -- 172.21.15.64:0/2783698546 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff13019d690 con 0x7ff130101df0 2024-09-12T03:27:27.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.435+0000 7ff12dffb640 1 -- 172.21.15.64:0/2783698546 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12800f450 con 0x7ff130101df0 2024-09-12T03:27:27.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.435+0000 7ff137060640 1 -- 172.21.15.64:0/2783698546 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1301a4f70 con 0x7ff130101df0 2024-09-12T03:27:27.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.436+0000 7ff137060640 1 -- 172.21.15.64:0/2783698546 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1301021d0 con 0x7ff130101df0 2024-09-12T03:27:27.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.437+0000 7ff12dffb640 1 -- 172.21.15.64:0/2783698546 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff128004f20 con 0x7ff130101df0 2024-09-12T03:27:27.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.438+0000 7ff12dffb640 1 --2- 172.21.15.64:0/2783698546 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff11c0a2b80 0x7ff11c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:27.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.438+0000 7ff12dffb640 1 -- 172.21.15.64:0/2783698546 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7ff1280cf0b0 con 0x7ff130101df0 2024-09-12T03:27:27.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.438+0000 7ff12ffff640 1 --2- 172.21.15.64:0/2783698546 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff11c0a2b80 0x7ff11c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:27.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.439+0000 7ff12ffff640 1 --2- 172.21.15.64:0/2783698546 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff11c0a2b80 0x7ff11c0a5040 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff120006010 tx=0x7ff1200093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:27.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.442+0000 7ff12dffb640 1 -- 172.21.15.64:0/2783698546 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff128014030 con 0x7ff130101df0 2024-09-12T03:27:27.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:27 smithi195 ceph-mon[34786]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:27:27.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.621+0000 7ff137060640 1 -- 172.21.15.64:0/2783698546 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff13010a160 con 0x7ff130101df0 2024-09-12T03:27:27.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.622+0000 7ff12dffb640 1 -- 172.21.15.64:0/2783698546 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24209 (secure 0 0 0) 0x7ff12809bdd0 con 0x7ff130101df0 2024-09-12T03:27:27.624 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:27:27.624 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":88,"fsid":"b0812460-70b5-11ef-bcea-c7b262605968","created":"2024-09-12T03:19:15.900322+0000","modified":"2024-09-12T03:27:08.680818+0000","last_up_change":"2024-09-12T03:26:06.122971+0000","last_in_change":"2024-09-12T03:25:43.130689+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:22:21.915870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:26:55.826348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:26:57.984319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T03:27:04.460292+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"326271c9-1f28-4ce6-b3be-fcc57479d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6803","nonce":1307832590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6805","nonce":1307832590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6809","nonce":1307832590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6807","nonce":1307832590}]},"public_addr":"172.21.15.64:6803/1307832590","cluster_addr":"172.21.15.64:6805/1307832590","heartbeat_back_addr":"172.21.15.64:6809/1307832590","heartbeat_front_addr":"172.21.15.64:6807/1307832590","state":["exists","up"]},{"osd":1,"uuid":"7dca0c28-c234-423c-936e-2351d63f918e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6801","nonce":3123728622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6803","nonce":3123728622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6807","nonce":3123728622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6805","nonce":3123728622}]},"public_addr":"172.21.15.115:6801/3123728622","cluster_addr":"172.21.15.115:6803/3123728622","heartbeat_back_addr":"172.21.15.115:6807/3123728622","heartbeat_front_addr":"172.21.15.115:6805/3123728622","state":["exists","up"]},{"osd":2,"uuid":"4b3a3c7a-aec6-455a-9a9a-1ac9cc900f24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6801","nonce":1664125451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6803","nonce":1664125451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6807","nonce":1664125451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6805","nonce":1664125451}]},"public_addr":"172.21.15.195:6801/1664125451","cluster_addr":"172.21.15.195:6803/1664125451","heartbeat_back_addr":"172.21.15.195:6807/1664125451","heartbeat_front_addr":"172.21.15.195:6805/1664125451","state":["exists","up"]},{"osd":3,"uuid":"8768c339-2c1e-4294-8b78-3c2adab6750f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6811","nonce":3935517185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6813","nonce":3935517185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6817","nonce":3935517185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6815","nonce":3935517185}]},"public_addr":"172.21.15.64:6811/3935517185","cluster_addr":"172.21.15.64:6813/3935517185","heartbeat_back_addr":"172.21.15.64:6817/3935517185","heartbeat_front_addr":"172.21.15.64:6815/3935517185","state":["exists","up"]},{"osd":4,"uuid":"123aac47-a8ae-47c7-93a7-0dd293356322","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6809","nonce":446665227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6811","nonce":446665227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6815","nonce":446665227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6813","nonce":446665227}]},"public_addr":"172.21.15.115:6809/446665227","cluster_addr":"172.21.15.115:6811/446665227","heartbeat_back_addr":"172.21.15.115:6815/446665227","heartbeat_front_addr":"172.21.15.115:6813/446665227","state":["exists","up"]},{"osd":5,"uuid":"c90bcaed-e7a8-4852-8677-b01dc73ea44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6809","nonce":1187293668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6811","nonce":1187293668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6815","nonce":1187293668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6813","nonce":1187293668}]},"public_addr":"172.21.15.195:6809/1187293668","cluster_addr":"172.21.15.195:6811/1187293668","heartbeat_back_addr":"172.21.15.195:6815/1187293668","heartbeat_front_addr":"172.21.15.195:6813/1187293668","state":["exists","up"]},{"osd":6,"uuid":"fd292547-6695-4889-8867-e74318bbb00f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6819","nonce":560343274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6821","nonce":560343274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6825","nonce":560343274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6823","nonce":560343274}]},"public_addr":"172.21.15.64:6819/560343274","cluster_addr":"172.21.15.64:6821/560343274","heartbeat_back_addr":"172.21.15.64:6825/560343274","heartbeat_front_addr":"172.21.15.64:6823/560343274","state":["exists","up"]},{"osd":7,"uuid":"ad7c4151-cd98-4113-879b-ca2b7c83c7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6817","nonce":3075734174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6819","nonce":3075734174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6823","nonce":3075734174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6821","nonce":3075734174}]},"public_addr":"172.21.15.115:6817/3075734174","cluster_addr":"172.21.15.115:6819/3075734174","heartbeat_back_addr":"172.21.15.115:6823/3075734174","heartbeat_front_addr":"172.21.15.115:6821/3075734174","state":["exists","up"]},{"osd":8,"uuid":"6de50049-c28f-4f21-b123-05e55bf59127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6817","nonce":2184742494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6819","nonce":2184742494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6823","nonce":2184742494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6821","nonce":2184742494}]},"public_addr":"172.21.15.195:6817/2184742494","cluster_addr":"172.21.15.195:6819/2184742494","heartbeat_back_addr":"172.21.15.195:6823/2184742494","heartbeat_front_addr":"172.21.15.195:6821/2184742494","state":["exists","up"]},{"osd":9,"uuid":"36ab792c-58f2-4beb-9024-7f3d1e3785dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6827","nonce":561728112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6829","nonce":561728112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6833","nonce":561728112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6831","nonce":561728112}]},"public_addr":"172.21.15.64:6827/561728112","cluster_addr":"172.21.15.64:6829/561728112","heartbeat_back_addr":"172.21.15.64:6833/561728112","heartbeat_front_addr":"172.21.15.64:6831/561728112","state":["exists","up"]},{"osd":10,"uuid":"d1f7881c-ec3f-466e-9cea-1d5cb441d970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6825","nonce":2977887627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6827","nonce":2977887627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6831","nonce":2977887627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6829","nonce":2977887627}]},"public_addr":"172.21.15.115:6825/2977887627","cluster_addr":"172.21.15.115:6827/2977887627","heartbeat_back_addr":"172.21.15.115:6831/2977887627","heartbeat_front_addr":"172.21.15.115:6829/2977887627","state":["exists","up"]},{"osd":11,"uuid":"7c46c028-22c7-450f-9927-2144a7a09e9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6825","nonce":591599352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6827","nonce":591599352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6831","nonce":591599352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6829","nonce":591599352}]},"public_addr":"172.21.15.195:6825/591599352","cluster_addr":"172.21.15.195:6827/591599352","heartbeat_back_addr":"172.21.15.195:6831/591599352","heartbeat_front_addr":"172.21.15.195:6829/591599352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:29.151454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:54.308909+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:18.680995+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:44.687548+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:10.166131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:35.026132+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:24:00.242966+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:24:24.600848+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:24:49.563060+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:25:15.025730+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:25:39.830995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:26:04.394276+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6801/68603379":"2024-09-13T03:20:07.411102+0000","172.21.15.64:0/3827202979":"2024-09-13T03:20:07.411102+0000","172.21.15.64:6800/68603379":"2024-09-13T03:20:07.411102+0000","172.21.15.64:0/1217136221":"2024-09-13T03:20:07.411102+0000","172.21.15.64:0/3079679964":"2024-09-13T03:19:36.839431+0000","172.21.15.64:6801/3186640984":"2024-09-13T03:19:36.839431+0000","172.21.15.64:0/3334629354":"2024-09-13T03:20:07.411102+0000","172.21.15.64:6800/3186640984":"2024-09-13T03:19:36.839431+0000","172.21.15.64:0/3564418115":"2024-09-13T03:19:36.839431+0000","172.21.15.64:0/686137382":"2024-09-13T03:19:36.839431+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:27:27.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.625+0000 7ff137060640 1 -- 172.21.15.64:0/2783698546 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff11c0a2b80 msgr2=0x7ff11c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:27.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.625+0000 7ff137060640 1 --2- 172.21.15.64:0/2783698546 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff11c0a2b80 0x7ff11c0a5040 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff120006010 tx=0x7ff1200093f0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.625+0000 7ff137060640 1 -- 172.21.15.64:0/2783698546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff130101df0 msgr2=0x7ff1301983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:27.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.625+0000 7ff137060640 1 --2- 172.21.15.64:0/2783698546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff130101df0 0x7ff1301983f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff12800c970 tx=0x7ff12800ce30 comp rx=0 tx=0).stop 2024-09-12T03:27:27.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.625+0000 7ff137060640 1 -- 172.21.15.64:0/2783698546 shutdown_connections 2024-09-12T03:27:27.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.625+0000 7ff137060640 1 --2- 172.21.15.64:0/2783698546 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff11c0a2b80 0x7ff11c0a5040 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.625+0000 7ff137060640 1 --2- 172.21.15.64:0/2783698546 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff130106f20 0x7ff13019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.625+0000 7ff137060640 1 --2- 172.21.15.64:0/2783698546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1301027a0 0x7ff130198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.625+0000 7ff137060640 1 --2- 172.21.15.64:0/2783698546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff130101df0 0x7ff1301983f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.625+0000 7ff137060640 1 -- 172.21.15.64:0/2783698546 >> 172.21.15.64:0/2783698546 conn(0x7ff1300fd8b0 msgr2=0x7ff1300fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:27.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.625+0000 7ff137060640 1 -- 172.21.15.64:0/2783698546 shutdown_connections 2024-09-12T03:27:27.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:27.625+0000 7ff137060640 1 -- 172.21.15.64:0/2783698546 wait complete. 2024-09-12T03:27:28.361 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:28 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/2783698546' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:27:28.364 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:27:28.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:28 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/2783698546' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:27:28.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:28 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/2783698546' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:27:28.623 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:29.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.086+0000 7f104d452640 1 -- 172.21.15.64:0/2894438967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f104810c1a0 msgr2=0x7f104810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:29.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.086+0000 7f104d452640 1 --2- 172.21.15.64:0/2894438967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f104810c1a0 0x7f104810e610 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1040009900 tx=0x7f104002a480 comp rx=0 tx=0).stop 2024-09-12T03:27:29.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.087+0000 7f104d452640 1 -- 172.21.15.64:0/2894438967 shutdown_connections 2024-09-12T03:27:29.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.087+0000 7f104d452640 1 --2- 172.21.15.64:0/2894438967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f104810c1a0 0x7f104810e610 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.087+0000 7f104d452640 1 --2- 172.21.15.64:0/2894438967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1048103720 0x7f104810bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.087+0000 7f104d452640 1 --2- 172.21.15.64:0/2894438967 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1048102e00 0x7f10481031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.087+0000 7f104d452640 1 -- 172.21.15.64:0/2894438967 >> 172.21.15.64:0/2894438967 conn(0x7f10480fc7a0 msgr2=0x7f10480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:29.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.087+0000 7f104d452640 1 -- 172.21.15.64:0/2894438967 shutdown_connections 2024-09-12T03:27:29.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.087+0000 7f104d452640 1 -- 172.21.15.64:0/2894438967 wait complete. 2024-09-12T03:27:29.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.087+0000 7f104d452640 1 Processor -- start 2024-09-12T03:27:29.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.087+0000 7f104d452640 1 -- start start 2024-09-12T03:27:29.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.088+0000 7f104d452640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1048102e00 0x7f1048198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:29.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.088+0000 7f104d452640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1048103720 0x7f1048198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:29.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.088+0000 7f104d452640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f104810c1a0 0x7f104819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:29.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.088+0000 7f104d452640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1048111e20 con 0x7f1048102e00 2024-09-12T03:27:29.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.088+0000 7f104d452640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1048111ca0 con 0x7f1048103720 2024-09-12T03:27:29.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.088+0000 7f104d452640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1048111fa0 con 0x7f104810c1a0 2024-09-12T03:27:29.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.088+0000 7f1046ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1048102e00 0x7f1048198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:29.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.088+0000 7f1046ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1048102e00 0x7f1048198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41518/0 (socket says 172.21.15.64:41518) 2024-09-12T03:27:29.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.088+0000 7f1046ffd640 1 -- 172.21.15.64:0/1551951005 learned_addr learned my addr 172.21.15.64:0/1551951005 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:27:29.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.088+0000 7f10467fc640 1 --2- 172.21.15.64:0/1551951005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1048103720 0x7f1048198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:29.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.088+0000 7f10477fe640 1 --2- 172.21.15.64:0/1551951005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f104810c1a0 0x7f104819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:29.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.088+0000 7f1046ffd640 1 -- 172.21.15.64:0/1551951005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f104810c1a0 msgr2=0x7f104819cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:29.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.088+0000 7f1046ffd640 1 --2- 172.21.15.64:0/1551951005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f104810c1a0 0x7f104819cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.088+0000 7f1046ffd640 1 -- 172.21.15.64:0/1551951005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1048103720 msgr2=0x7f1048198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:29.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.088+0000 7f1046ffd640 1 --2- 172.21.15.64:0/1551951005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1048103720 0x7f1048198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.088+0000 7f1046ffd640 1 -- 172.21.15.64:0/1551951005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f104819d2b0 con 0x7f1048102e00 2024-09-12T03:27:29.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.088+0000 7f1046ffd640 1 --2- 172.21.15.64:0/1551951005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1048102e00 0x7f1048198300 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f103800db80 tx=0x7f103800b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:29.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.088+0000 7f102ffff640 1 -- 172.21.15.64:0/1551951005 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1038017070 con 0x7f1048102e00 2024-09-12T03:27:29.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.088+0000 7f102ffff640 1 -- 172.21.15.64:0/1551951005 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1038003b70 con 0x7f1048102e00 2024-09-12T03:27:29.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.088+0000 7f102ffff640 1 -- 172.21.15.64:0/1551951005 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10380104c0 con 0x7f1048102e00 2024-09-12T03:27:29.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.088+0000 7f104d452640 1 -- 172.21.15.64:0/1551951005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f104819d5a0 con 0x7f1048102e00 2024-09-12T03:27:29.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.089+0000 7f104d452640 1 -- 172.21.15.64:0/1551951005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10481a4e80 con 0x7f1048102e00 2024-09-12T03:27:29.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.089+0000 7f104d452640 1 -- 172.21.15.64:0/1551951005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10481031e0 con 0x7f1048102e00 2024-09-12T03:27:29.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.091+0000 7f102ffff640 1 -- 172.21.15.64:0/1551951005 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1038004020 con 0x7f1048102e00 2024-09-12T03:27:29.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.092+0000 7f102ffff640 1 --2- 172.21.15.64:0/1551951005 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f10300a2c00 0x7f10300a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:29.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.092+0000 7f10467fc640 1 --2- 172.21.15.64:0/1551951005 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f10300a2c00 0x7f10300a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:29.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.092+0000 7f102ffff640 1 -- 172.21.15.64:0/1551951005 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f10380cf390 con 0x7f1048102e00 2024-09-12T03:27:29.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.092+0000 7f10467fc640 1 --2- 172.21.15.64:0/1551951005 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f10300a2c00 0x7f10300a50c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f103c005fd0 tx=0x7f103c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:29.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.096+0000 7f102ffff640 1 -- 172.21.15.64:0/1551951005 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f103809c0b0 con 0x7f1048102e00 2024-09-12T03:27:29.271 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:29 smithi064 ceph-mon[30952]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-12T03:27:29.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.274+0000 7f104d452640 1 -- 172.21.15.64:0/1551951005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1048199660 con 0x7f1048102e00 2024-09-12T03:27:29.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.275+0000 7f102ffff640 1 -- 172.21.15.64:0/1551951005 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24209 (secure 0 0 0) 0x7f103809b9b0 con 0x7f1048102e00 2024-09-12T03:27:29.276 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:27:29.277 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":88,"fsid":"b0812460-70b5-11ef-bcea-c7b262605968","created":"2024-09-12T03:19:15.900322+0000","modified":"2024-09-12T03:27:08.680818+0000","last_up_change":"2024-09-12T03:26:06.122971+0000","last_in_change":"2024-09-12T03:25:43.130689+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:22:21.915870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:26:55.826348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:26:57.984319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T03:27:04.460292+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"326271c9-1f28-4ce6-b3be-fcc57479d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6803","nonce":1307832590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6805","nonce":1307832590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6809","nonce":1307832590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6807","nonce":1307832590}]},"public_addr":"172.21.15.64:6803/1307832590","cluster_addr":"172.21.15.64:6805/1307832590","heartbeat_back_addr":"172.21.15.64:6809/1307832590","heartbeat_front_addr":"172.21.15.64:6807/1307832590","state":["exists","up"]},{"osd":1,"uuid":"7dca0c28-c234-423c-936e-2351d63f918e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6801","nonce":3123728622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6803","nonce":3123728622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6807","nonce":3123728622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6805","nonce":3123728622}]},"public_addr":"172.21.15.115:6801/3123728622","cluster_addr":"172.21.15.115:6803/3123728622","heartbeat_back_addr":"172.21.15.115:6807/3123728622","heartbeat_front_addr":"172.21.15.115:6805/3123728622","state":["exists","up"]},{"osd":2,"uuid":"4b3a3c7a-aec6-455a-9a9a-1ac9cc900f24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6801","nonce":1664125451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6803","nonce":1664125451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6807","nonce":1664125451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6805","nonce":1664125451}]},"public_addr":"172.21.15.195:6801/1664125451","cluster_addr":"172.21.15.195:6803/1664125451","heartbeat_back_addr":"172.21.15.195:6807/1664125451","heartbeat_front_addr":"172.21.15.195:6805/1664125451","state":["exists","up"]},{"osd":3,"uuid":"8768c339-2c1e-4294-8b78-3c2adab6750f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6811","nonce":3935517185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6813","nonce":3935517185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6817","nonce":3935517185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6815","nonce":3935517185}]},"public_addr":"172.21.15.64:6811/3935517185","cluster_addr":"172.21.15.64:6813/3935517185","heartbeat_back_addr":"172.21.15.64:6817/3935517185","heartbeat_front_addr":"172.21.15.64:6815/3935517185","state":["exists","up"]},{"osd":4,"uuid":"123aac47-a8ae-47c7-93a7-0dd293356322","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6809","nonce":446665227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6811","nonce":446665227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6815","nonce":446665227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6813","nonce":446665227}]},"public_addr":"172.21.15.115:6809/446665227","cluster_addr":"172.21.15.115:6811/446665227","heartbeat_back_addr":"172.21.15.115:6815/446665227","heartbeat_front_addr":"172.21.15.115:6813/446665227","state":["exists","up"]},{"osd":5,"uuid":"c90bcaed-e7a8-4852-8677-b01dc73ea44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6809","nonce":1187293668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6811","nonce":1187293668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6815","nonce":1187293668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6813","nonce":1187293668}]},"public_addr":"172.21.15.195:6809/1187293668","cluster_addr":"172.21.15.195:6811/1187293668","heartbeat_back_addr":"172.21.15.195:6815/1187293668","heartbeat_front_addr":"172.21.15.195:6813/1187293668","state":["exists","up"]},{"osd":6,"uuid":"fd292547-6695-4889-8867-e74318bbb00f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6819","nonce":560343274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6821","nonce":560343274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6825","nonce":560343274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6823","nonce":560343274}]},"public_addr":"172.21.15.64:6819/560343274","cluster_addr":"172.21.15.64:6821/560343274","heartbeat_back_addr":"172.21.15.64:6825/560343274","heartbeat_front_addr":"172.21.15.64:6823/560343274","state":["exists","up"]},{"osd":7,"uuid":"ad7c4151-cd98-4113-879b-ca2b7c83c7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6817","nonce":3075734174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6819","nonce":3075734174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6823","nonce":3075734174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6821","nonce":3075734174}]},"public_addr":"172.21.15.115:6817/3075734174","cluster_addr":"172.21.15.115:6819/3075734174","heartbeat_back_addr":"172.21.15.115:6823/3075734174","heartbeat_front_addr":"172.21.15.115:6821/3075734174","state":["exists","up"]},{"osd":8,"uuid":"6de50049-c28f-4f21-b123-05e55bf59127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6817","nonce":2184742494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6819","nonce":2184742494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6823","nonce":2184742494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6821","nonce":2184742494}]},"public_addr":"172.21.15.195:6817/2184742494","cluster_addr":"172.21.15.195:6819/2184742494","heartbeat_back_addr":"172.21.15.195:6823/2184742494","heartbeat_front_addr":"172.21.15.195:6821/2184742494","state":["exists","up"]},{"osd":9,"uuid":"36ab792c-58f2-4beb-9024-7f3d1e3785dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6827","nonce":561728112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6829","nonce":561728112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6833","nonce":561728112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6831","nonce":561728112}]},"public_addr":"172.21.15.64:6827/561728112","cluster_addr":"172.21.15.64:6829/561728112","heartbeat_back_addr":"172.21.15.64:6833/561728112","heartbeat_front_addr":"172.21.15.64:6831/561728112","state":["exists","up"]},{"osd":10,"uuid":"d1f7881c-ec3f-466e-9cea-1d5cb441d970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6825","nonce":2977887627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6827","nonce":2977887627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6831","nonce":2977887627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6829","nonce":2977887627}]},"public_addr":"172.21.15.115:6825/2977887627","cluster_addr":"172.21.15.115:6827/2977887627","heartbeat_back_addr":"172.21.15.115:6831/2977887627","heartbeat_front_addr":"172.21.15.115:6829/2977887627","state":["exists","up"]},{"osd":11,"uuid":"7c46c028-22c7-450f-9927-2144a7a09e9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6825","nonce":591599352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6827","nonce":591599352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6831","nonce":591599352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6829","nonce":591599352}]},"public_addr":"172.21.15.195:6825/591599352","cluster_addr":"172.21.15.195:6827/591599352","heartbeat_back_addr":"172.21.15.195:6831/591599352","heartbeat_front_addr":"172.21.15.195:6829/591599352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:29.151454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:54.308909+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:18.680995+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:44.687548+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:10.166131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:35.026132+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:24:00.242966+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:24:24.600848+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:24:49.563060+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:25:15.025730+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:25:39.830995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:26:04.394276+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6801/68603379":"2024-09-13T03:20:07.411102+0000","172.21.15.64:0/3827202979":"2024-09-13T03:20:07.411102+0000","172.21.15.64:6800/68603379":"2024-09-13T03:20:07.411102+0000","172.21.15.64:0/1217136221":"2024-09-13T03:20:07.411102+0000","172.21.15.64:0/3079679964":"2024-09-13T03:19:36.839431+0000","172.21.15.64:6801/3186640984":"2024-09-13T03:19:36.839431+0000","172.21.15.64:0/3334629354":"2024-09-13T03:20:07.411102+0000","172.21.15.64:6800/3186640984":"2024-09-13T03:19:36.839431+0000","172.21.15.64:0/3564418115":"2024-09-13T03:19:36.839431+0000","172.21.15.64:0/686137382":"2024-09-13T03:19:36.839431+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:27:29.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.278+0000 7f104d452640 1 -- 172.21.15.64:0/1551951005 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f10300a2c00 msgr2=0x7f10300a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:29.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.278+0000 7f104d452640 1 --2- 172.21.15.64:0/1551951005 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f10300a2c00 0x7f10300a50c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f103c005fd0 tx=0x7f103c005850 comp rx=0 tx=0).stop 2024-09-12T03:27:29.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.278+0000 7f104d452640 1 -- 172.21.15.64:0/1551951005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1048102e00 msgr2=0x7f1048198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:29.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.278+0000 7f104d452640 1 --2- 172.21.15.64:0/1551951005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1048102e00 0x7f1048198300 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f103800db80 tx=0x7f103800b630 comp rx=0 tx=0).stop 2024-09-12T03:27:29.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.278+0000 7f104d452640 1 -- 172.21.15.64:0/1551951005 shutdown_connections 2024-09-12T03:27:29.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.278+0000 7f104d452640 1 --2- 172.21.15.64:0/1551951005 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f10300a2c00 0x7f10300a50c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.278+0000 7f104d452640 1 --2- 172.21.15.64:0/1551951005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f104810c1a0 0x7f104819cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.278+0000 7f104d452640 1 --2- 172.21.15.64:0/1551951005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1048103720 0x7f1048198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.278+0000 7f104d452640 1 --2- 172.21.15.64:0/1551951005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1048102e00 0x7f1048198300 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:29.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.278+0000 7f104d452640 1 -- 172.21.15.64:0/1551951005 >> 172.21.15.64:0/1551951005 conn(0x7f10480fc7a0 msgr2=0x7f10480fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:29.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.278+0000 7f104d452640 1 -- 172.21.15.64:0/1551951005 shutdown_connections 2024-09-12T03:27:29.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:29.278+0000 7f104d452640 1 -- 172.21.15.64:0/1551951005 wait complete. 2024-09-12T03:27:29.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:29 smithi115 ceph-mon[34801]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-12T03:27:29.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:29 smithi195 ceph-mon[34786]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-12T03:27:30.050 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-12T03:27:30.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:30 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/1551951005' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:27:30.313 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:30.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:30 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/1551951005' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:27:30.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:30 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/1551951005' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:27:30.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:30.782+0000 7f87a4f14640 1 -- 172.21.15.64:0/844219823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87a0102790 msgr2=0x7f87a01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:30.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:30.782+0000 7f87a4f14640 1 --2- 172.21.15.64:0/844219823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87a0102790 0x7f87a01067e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8788009900 tx=0x7f878802a410 comp rx=0 tx=0).stop 2024-09-12T03:27:30.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:30.783+0000 7f87a4f14640 1 -- 172.21.15.64:0/844219823 shutdown_connections 2024-09-12T03:27:30.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:30.783+0000 7f87a4f14640 1 --2- 172.21.15.64:0/844219823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87a0106f10 0x7f87a010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:30.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:30.783+0000 7f87a4f14640 1 --2- 172.21.15.64:0/844219823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87a0102790 0x7f87a01067e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:30.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:30.783+0000 7f87a4f14640 1 --2- 172.21.15.64:0/844219823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87a0101de0 0x7f87a01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:30.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:30.783+0000 7f87a4f14640 1 -- 172.21.15.64:0/844219823 >> 172.21.15.64:0/844219823 conn(0x7f87a00fd8c0 msgr2=0x7f87a00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:30.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:30.783+0000 7f87a4f14640 1 -- 172.21.15.64:0/844219823 shutdown_connections 2024-09-12T03:27:30.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:30.783+0000 7f87a4f14640 1 -- 172.21.15.64:0/844219823 wait complete. 2024-09-12T03:27:30.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:30.783+0000 7f87a4f14640 1 Processor -- start 2024-09-12T03:27:30.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:30.783+0000 7f87a4f14640 1 -- start start 2024-09-12T03:27:30.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:30.784+0000 7f87a4f14640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87a0101de0 0x7f87a01983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:30.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:30.784+0000 7f87a4f14640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87a0102790 0x7f87a0198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:30.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:30.784+0000 7f87a4f14640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87a0106f10 0x7f87a019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:30.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:30.784+0000 7f87a4f14640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f87a0111e20 con 0x7f87a0101de0 2024-09-12T03:27:30.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:30.784+0000 7f87a4f14640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f87a0111ca0 con 0x7f87a0106f10 2024-09-12T03:27:30.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:30.784+0000 7f87a4f14640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f87a0111fa0 con 0x7f87a0102790 2024-09-12T03:27:30.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:30.784+0000 7f879e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87a0101de0 0x7f87a01983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:30.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:30.784+0000 7f8797fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87a0102790 0x7f87a0198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:30.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:30.784+0000 7f879e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87a0101de0 0x7f87a01983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41524/0 (socket says 172.21.15.64:41524) 2024-09-12T03:27:30.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:30.784+0000 7f879e59c640 1 -- 172.21.15.64:0/1777508714 learned_addr learned my addr 172.21.15.64:0/1777508714 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:27:30.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:30.784+0000 7f879ed9d640 1 --2- 172.21.15.64:0/1777508714 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87a0106f10 0x7f87a019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:30.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:30.784+0000 7f879e59c640 1 -- 172.21.15.64:0/1777508714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87a0102790 msgr2=0x7f87a0198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:30.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:30.784+0000 7f879e59c640 1 --2- 172.21.15.64:0/1777508714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87a0102790 0x7f87a0198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:30.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:30.784+0000 7f879e59c640 1 -- 172.21.15.64:0/1777508714 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87a0106f10 msgr2=0x7f87a019cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:30.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:30.784+0000 7f879e59c640 1 --2- 172.21.15.64:0/1777508714 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87a0106f10 0x7f87a019cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:30.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:30.784+0000 7f879e59c640 1 -- 172.21.15.64:0/1777508714 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a019d370 con 0x7f87a0101de0 2024-09-12T03:27:30.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:30.784+0000 7f879e59c640 1 --2- 172.21.15.64:0/1777508714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87a0101de0 0x7f87a01983c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f879000d770 tx=0x7f879000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:30.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:30.784+0000 7f87977fe640 1 -- 172.21.15.64:0/1777508714 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8790017070 con 0x7f87a0101de0 2024-09-12T03:27:30.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:30.785+0000 7f87a4f14640 1 -- 172.21.15.64:0/1777508714 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87a019d660 con 0x7f87a0101de0 2024-09-12T03:27:30.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:30.785+0000 7f87977fe640 1 -- 172.21.15.64:0/1777508714 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87900054d0 con 0x7f87a0101de0 2024-09-12T03:27:30.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:30.785+0000 7f87977fe640 1 -- 172.21.15.64:0/1777508714 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8790004cf0 con 0x7f87a0101de0 2024-09-12T03:27:30.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:30.785+0000 7f87a4f14640 1 -- 172.21.15.64:0/1777508714 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87a01a4f40 con 0x7f87a0101de0 2024-09-12T03:27:30.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:30.786+0000 7f87a4f14640 1 -- 172.21.15.64:0/1777508714 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87a01021c0 con 0x7f87a0101de0 2024-09-12T03:27:30.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:30.787+0000 7f87977fe640 1 -- 172.21.15.64:0/1777508714 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8790004e90 con 0x7f87a0101de0 2024-09-12T03:27:30.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:30.788+0000 7f87977fe640 1 --2- 172.21.15.64:0/1777508714 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f878c0a2b80 0x7f878c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:30.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:30.788+0000 7f8797fff640 1 --2- 172.21.15.64:0/1777508714 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f878c0a2b80 0x7f878c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:30.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:30.788+0000 7f87977fe640 1 -- 172.21.15.64:0/1777508714 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f87900cecf0 con 0x7f87a0101de0 2024-09-12T03:27:30.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:30.788+0000 7f8797fff640 1 --2- 172.21.15.64:0/1777508714 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f878c0a2b80 0x7f878c0a5040 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8788031440 tx=0x7f8788005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:30.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:30.792+0000 7f87977fe640 1 -- 172.21.15.64:0/1777508714 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f87900d3070 con 0x7f87a0101de0 2024-09-12T03:27:31.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:31.003+0000 7f87a4f14640 1 -- 172.21.15.64:0/1777508714 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f87a010a150 con 0x7f87a0101de0 2024-09-12T03:27:31.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:31.104+0000 7f87977fe640 1 -- 172.21.15.64:0/1777508714 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f879009ba10 con 0x7f87a0101de0 2024-09-12T03:27:31.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:31.106+0000 7f87a4f14640 1 -- 172.21.15.64:0/1777508714 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f878c0a2b80 msgr2=0x7f878c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:31.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:31.106+0000 7f87a4f14640 1 --2- 172.21.15.64:0/1777508714 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f878c0a2b80 0x7f878c0a5040 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8788031440 tx=0x7f8788005b50 comp rx=0 tx=0).stop 2024-09-12T03:27:31.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:31.106+0000 7f87a4f14640 1 -- 172.21.15.64:0/1777508714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87a0101de0 msgr2=0x7f87a01983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:31.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:31.106+0000 7f87a4f14640 1 --2- 172.21.15.64:0/1777508714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87a0101de0 0x7f87a01983c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f879000d770 tx=0x7f879000dc30 comp rx=0 tx=0).stop 2024-09-12T03:27:31.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:31.106+0000 7f87a4f14640 1 -- 172.21.15.64:0/1777508714 shutdown_connections 2024-09-12T03:27:31.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:31.106+0000 7f87a4f14640 1 --2- 172.21.15.64:0/1777508714 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f878c0a2b80 0x7f878c0a5040 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:31.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:31.106+0000 7f87a4f14640 1 --2- 172.21.15.64:0/1777508714 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87a0106f10 0x7f87a019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:31.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:31.106+0000 7f87a4f14640 1 --2- 172.21.15.64:0/1777508714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87a0102790 0x7f87a0198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:31.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:31.106+0000 7f87a4f14640 1 --2- 172.21.15.64:0/1777508714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87a0101de0 0x7f87a01983c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:31.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:31.106+0000 7f87a4f14640 1 -- 172.21.15.64:0/1777508714 >> 172.21.15.64:0/1777508714 conn(0x7f87a00fd8c0 msgr2=0x7f87a00fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:31.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:31.106+0000 7f87a4f14640 1 -- 172.21.15.64:0/1777508714 shutdown_connections 2024-09-12T03:27:31.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:31.106+0000 7f87a4f14640 1 -- 172.21.15.64:0/1777508714 wait complete. 2024-09-12T03:27:31.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:31 smithi064 ceph-mon[30952]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-12T03:27:31.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:31 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/1777508714' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T03:27:31.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:31 smithi115 ceph-mon[34801]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-12T03:27:31.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:31 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/1777508714' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T03:27:31.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:31 smithi195 ceph-mon[34786]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-12T03:27:31.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:31 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/1777508714' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T03:27:32.116 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-09-12T03:27:32.116 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-09-12T03:27:32.381 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:32.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:32 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/1777508714' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T03:27:32.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:32 smithi115 ceph-mon[34801]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:27:32.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:32 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/1777508714' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T03:27:32.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:32 smithi195 ceph-mon[34786]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:27:32.502 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:32 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/1777508714' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T03:27:32.503 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:32 smithi064 ceph-mon[30952]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:27:32.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:32.864+0000 7fbe94890640 1 -- 172.21.15.64:0/488338379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe90102da0 msgr2=0x7fbe90103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:32.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:32.864+0000 7fbe94890640 1 --2- 172.21.15.64:0/488338379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe90102da0 0x7fbe90103180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbe80009900 tx=0x7fbe8002a480 comp rx=0 tx=0).stop 2024-09-12T03:27:32.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:32.864+0000 7fbe94890640 1 -- 172.21.15.64:0/488338379 shutdown_connections 2024-09-12T03:27:32.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:32.864+0000 7fbe94890640 1 --2- 172.21.15.64:0/488338379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe9010c180 0x7fbe9010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:32.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:32.864+0000 7fbe94890640 1 --2- 172.21.15.64:0/488338379 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe901036c0 0x7fbe9010bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:32.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:32.864+0000 7fbe94890640 1 --2- 172.21.15.64:0/488338379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe90102da0 0x7fbe90103180 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:32.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:32.864+0000 7fbe94890640 1 -- 172.21.15.64:0/488338379 >> 172.21.15.64:0/488338379 conn(0x7fbe900fc7a0 msgr2=0x7fbe900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:32.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:32.864+0000 7fbe94890640 1 -- 172.21.15.64:0/488338379 shutdown_connections 2024-09-12T03:27:32.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:32.864+0000 7fbe94890640 1 -- 172.21.15.64:0/488338379 wait complete. 2024-09-12T03:27:32.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:32.865+0000 7fbe94890640 1 Processor -- start 2024-09-12T03:27:32.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:32.865+0000 7fbe94890640 1 -- start start 2024-09-12T03:27:32.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:32.865+0000 7fbe94890640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe90102da0 0x7fbe90100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:32.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:32.865+0000 7fbe94890640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe901036c0 0x7fbe90100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:32.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:32.865+0000 7fbe94890640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe9010c180 0x7fbe90101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:32.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:32.865+0000 7fbe94890640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbe90111e10 con 0x7fbe9010c180 2024-09-12T03:27:32.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:32.865+0000 7fbe94890640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbe90111c90 con 0x7fbe90102da0 2024-09-12T03:27:32.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:32.865+0000 7fbe94890640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbe90111f90 con 0x7fbe901036c0 2024-09-12T03:27:32.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:32.865+0000 7fbe8ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe9010c180 0x7fbe90101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:32.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:32.865+0000 7fbe8ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe9010c180 0x7fbe90101480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41544/0 (socket says 172.21.15.64:41544) 2024-09-12T03:27:32.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:32.865+0000 7fbe8ed9d640 1 -- 172.21.15.64:0/963983784 learned_addr learned my addr 172.21.15.64:0/963983784 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:27:32.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:32.865+0000 7fbe8dd9b640 1 --2- 172.21.15.64:0/963983784 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe901036c0 0x7fbe90100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:32.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:32.865+0000 7fbe8e59c640 1 --2- 172.21.15.64:0/963983784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe90102da0 0x7fbe90100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:32.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:32.866+0000 7fbe8ed9d640 1 -- 172.21.15.64:0/963983784 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe901036c0 msgr2=0x7fbe90100f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:32.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:32.866+0000 7fbe8ed9d640 1 --2- 172.21.15.64:0/963983784 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe901036c0 0x7fbe90100f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:32.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:32.866+0000 7fbe8ed9d640 1 -- 172.21.15.64:0/963983784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe90102da0 msgr2=0x7fbe90100a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:32.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:32.866+0000 7fbe8ed9d640 1 --2- 172.21.15.64:0/963983784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe90102da0 0x7fbe90100a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:32.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:32.866+0000 7fbe8ed9d640 1 -- 172.21.15.64:0/963983784 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe901a6870 con 0x7fbe9010c180 2024-09-12T03:27:32.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:32.866+0000 7fbe8ed9d640 1 --2- 172.21.15.64:0/963983784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe9010c180 0x7fbe90101480 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbe8800cad0 tx=0x7fbe8800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:32.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:32.866+0000 7fbe7f7fe640 1 -- 172.21.15.64:0/963983784 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8800ae20 con 0x7fbe9010c180 2024-09-12T03:27:32.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:32.866+0000 7fbe7f7fe640 1 -- 172.21.15.64:0/963983784 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe88005020 con 0x7fbe9010c180 2024-09-12T03:27:32.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:32.866+0000 7fbe94890640 1 -- 172.21.15.64:0/963983784 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe901a6b60 con 0x7fbe9010c180 2024-09-12T03:27:32.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:32.866+0000 7fbe7f7fe640 1 -- 172.21.15.64:0/963983784 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8800f6b0 con 0x7fbe9010c180 2024-09-12T03:27:32.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:32.866+0000 7fbe94890640 1 -- 172.21.15.64:0/963983784 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe901a70a0 con 0x7fbe9010c180 2024-09-12T03:27:32.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:32.867+0000 7fbe94890640 1 -- 172.21.15.64:0/963983784 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe90103180 con 0x7fbe9010c180 2024-09-12T03:27:32.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:32.868+0000 7fbe7f7fe640 1 -- 172.21.15.64:0/963983784 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbe880051d0 con 0x7fbe9010c180 2024-09-12T03:27:32.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:32.869+0000 7fbe7f7fe640 1 --2- 172.21.15.64:0/963983784 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fbe780a2c00 0x7fbe780a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:32.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:32.869+0000 7fbe8e59c640 1 --2- 172.21.15.64:0/963983784 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fbe780a2c00 0x7fbe780a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:32.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:32.869+0000 7fbe7f7fe640 1 -- 172.21.15.64:0/963983784 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fbe880cf190 con 0x7fbe9010c180 2024-09-12T03:27:32.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:32.870+0000 7fbe8e59c640 1 --2- 172.21.15.64:0/963983784 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fbe780a2c00 0x7fbe780a50c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbe80031440 tx=0x7fbe80005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:32.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:32.873+0000 7fbe7f7fe640 1 -- 172.21.15.64:0/963983784 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbe88014030 con 0x7fbe9010c180 2024-09-12T03:27:33.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:33.084+0000 7fbe94890640 1 -- 172.21.15.64:0/963983784 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7fbe90107220 con 0x7fbe9010c180 2024-09-12T03:27:33.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:33.115+0000 7fbe7f7fe640 1 -- 172.21.15.64:0/963983784 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7fbe8809b860 con 0x7fbe9010c180 2024-09-12T03:27:33.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:33.117+0000 7fbe94890640 1 -- 172.21.15.64:0/963983784 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fbe780a2c00 msgr2=0x7fbe780a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:33.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:33.118+0000 7fbe94890640 1 --2- 172.21.15.64:0/963983784 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fbe780a2c00 0x7fbe780a50c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbe80031440 tx=0x7fbe80005b50 comp rx=0 tx=0).stop 2024-09-12T03:27:33.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:33.118+0000 7fbe94890640 1 -- 172.21.15.64:0/963983784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe9010c180 msgr2=0x7fbe90101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:33.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:33.118+0000 7fbe94890640 1 --2- 172.21.15.64:0/963983784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe9010c180 0x7fbe90101480 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbe8800cad0 tx=0x7fbe8800cf90 comp rx=0 tx=0).stop 2024-09-12T03:27:33.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:33.118+0000 7fbe94890640 1 -- 172.21.15.64:0/963983784 shutdown_connections 2024-09-12T03:27:33.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:33.118+0000 7fbe94890640 1 --2- 172.21.15.64:0/963983784 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fbe780a2c00 0x7fbe780a50c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:33.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:33.118+0000 7fbe94890640 1 --2- 172.21.15.64:0/963983784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe9010c180 0x7fbe90101480 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:33.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:33.118+0000 7fbe94890640 1 --2- 172.21.15.64:0/963983784 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe901036c0 0x7fbe90100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:33.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:33.118+0000 7fbe94890640 1 --2- 172.21.15.64:0/963983784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe90102da0 0x7fbe90100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:33.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:33.118+0000 7fbe94890640 1 -- 172.21.15.64:0/963983784 >> 172.21.15.64:0/963983784 conn(0x7fbe900fc7a0 msgr2=0x7fbe900fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:33.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:33.118+0000 7fbe94890640 1 -- 172.21.15.64:0/963983784 shutdown_connections 2024-09-12T03:27:33.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:33.118+0000 7fbe94890640 1 -- 172.21.15.64:0/963983784 wait complete. 2024-09-12T03:27:33.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:33 smithi115 ceph-mon[34801]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-09-12T03:27:33.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:33 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/963983784' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-12T03:27:33.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:33 smithi195 ceph-mon[34786]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-09-12T03:27:33.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:33 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/963983784' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-12T03:27:33.521 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:33 smithi064 ceph-mon[30952]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-09-12T03:27:33.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:33 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/963983784' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-12T03:27:33.911 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-12T03:27:34.185 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:34.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:34 smithi115 ceph-mon[34801]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T03:27:34.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:34 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/963983784' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-12T03:27:34.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:34 smithi115 ceph-mon[34801]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T03:27:34.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:34 smithi115 ceph-mon[34801]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:27:34.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:34 smithi115 ceph-mon[34801]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-12T03:27:34.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:34 smithi115 ceph-mon[34801]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-12T03:27:34.421 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:34 smithi064 ceph-mon[30952]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T03:27:34.421 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:34 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/963983784' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-12T03:27:34.422 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:34 smithi064 ceph-mon[30952]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T03:27:34.422 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:34 smithi064 ceph-mon[30952]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:27:34.422 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:34 smithi064 ceph-mon[30952]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-12T03:27:34.422 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:34 smithi064 ceph-mon[30952]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-12T03:27:34.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:34 smithi195 ceph-mon[34786]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T03:27:34.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:34 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/963983784' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-12T03:27:34.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:34 smithi195 ceph-mon[34786]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T03:27:34.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:34 smithi195 ceph-mon[34786]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:27:34.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:34 smithi195 ceph-mon[34786]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-12T03:27:34.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:34 smithi195 ceph-mon[34786]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-12T03:27:34.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.638+0000 7f82a2de4640 1 -- 172.21.15.64:0/1078811252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f829c100b30 msgr2=0x7f829c100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:34.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.638+0000 7f82a2de4640 1 --2- 172.21.15.64:0/1078811252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f829c100b30 0x7f829c100f10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8290009900 tx=0x7f829002a470 comp rx=0 tx=0).stop 2024-09-12T03:27:34.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.638+0000 7f82a2de4640 1 -- 172.21.15.64:0/1078811252 shutdown_connections 2024-09-12T03:27:34.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.638+0000 7f82a2de4640 1 --2- 172.21.15.64:0/1078811252 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f829c105f40 0x7f829c1063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:34.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.638+0000 7f82a2de4640 1 --2- 172.21.15.64:0/1078811252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f829c101450 0x7f829c105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:34.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.638+0000 7f82a2de4640 1 --2- 172.21.15.64:0/1078811252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f829c100b30 0x7f829c100f10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:34.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.638+0000 7f82a2de4640 1 -- 172.21.15.64:0/1078811252 >> 172.21.15.64:0/1078811252 conn(0x7f829c0fc7a0 msgr2=0x7f829c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:34.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.638+0000 7f82a2de4640 1 -- 172.21.15.64:0/1078811252 shutdown_connections 2024-09-12T03:27:34.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.638+0000 7f82a2de4640 1 -- 172.21.15.64:0/1078811252 wait complete. 2024-09-12T03:27:34.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.639+0000 7f82a2de4640 1 Processor -- start 2024-09-12T03:27:34.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.639+0000 7f82a2de4640 1 -- start start 2024-09-12T03:27:34.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.639+0000 7f82a2de4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f829c100b30 0x7f829c19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:34.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.639+0000 7f82a2de4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f829c101450 0x7f829c19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:34.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.639+0000 7f82a2de4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f829c105f40 0x7f829c1a33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:34.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.639+0000 7f82a2de4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f829c118480 con 0x7f829c105f40 2024-09-12T03:27:34.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.639+0000 7f82a2de4640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f829c118300 con 0x7f829c101450 2024-09-12T03:27:34.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.639+0000 7f82a2de4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f829c118600 con 0x7f829c100b30 2024-09-12T03:27:34.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.639+0000 7f82a1381640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f829c105f40 0x7f829c1a33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:34.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.639+0000 7f82a1381640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f829c105f40 0x7f829c1a33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41558/0 (socket says 172.21.15.64:41558) 2024-09-12T03:27:34.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.639+0000 7f829bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f829c101450 0x7f829c19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:34.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.639+0000 7f82a1381640 1 -- 172.21.15.64:0/2408306819 learned_addr learned my addr 172.21.15.64:0/2408306819 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:27:34.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.639+0000 7f82a0b80640 1 --2- 172.21.15.64:0/2408306819 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f829c100b30 0x7f829c19eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:34.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.640+0000 7f82a1381640 1 -- 172.21.15.64:0/2408306819 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f829c100b30 msgr2=0x7f829c19eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:34.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.640+0000 7f82a1381640 1 --2- 172.21.15.64:0/2408306819 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f829c100b30 0x7f829c19eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:34.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.640+0000 7f82a1381640 1 -- 172.21.15.64:0/2408306819 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f829c101450 msgr2=0x7f829c19f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:34.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.640+0000 7f82a1381640 1 --2- 172.21.15.64:0/2408306819 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f829c101450 0x7f829c19f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:34.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.640+0000 7f82a1381640 1 -- 172.21.15.64:0/2408306819 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f829c1a3af0 con 0x7f829c105f40 2024-09-12T03:27:34.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.640+0000 7f82a0b80640 1 --2- 172.21.15.64:0/2408306819 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f829c100b30 0x7f829c19eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:27:34.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.640+0000 7f82a1381640 1 --2- 172.21.15.64:0/2408306819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f829c105f40 0x7f829c1a33c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f829400b970 tx=0x7f829400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:34.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.640+0000 7f8299ffb640 1 -- 172.21.15.64:0/2408306819 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8294019070 con 0x7f829c105f40 2024-09-12T03:27:34.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.640+0000 7f8299ffb640 1 -- 172.21.15.64:0/2408306819 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8294014440 con 0x7f829c105f40 2024-09-12T03:27:34.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.640+0000 7f82a2de4640 1 -- 172.21.15.64:0/2408306819 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f829c1a3de0 con 0x7f829c105f40 2024-09-12T03:27:34.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.640+0000 7f8299ffb640 1 -- 172.21.15.64:0/2408306819 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82940075d0 con 0x7f829c105f40 2024-09-12T03:27:34.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.640+0000 7f82a2de4640 1 -- 172.21.15.64:0/2408306819 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f829c1ab670 con 0x7f829c105f40 2024-09-12T03:27:34.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.641+0000 7f82a2de4640 1 -- 172.21.15.64:0/2408306819 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f829c105920 con 0x7f829c105f40 2024-09-12T03:27:34.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.642+0000 7f8299ffb640 1 -- 172.21.15.64:0/2408306819 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f829401c070 con 0x7f829c105f40 2024-09-12T03:27:34.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.643+0000 7f8299ffb640 1 --2- 172.21.15.64:0/2408306819 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f828c0a2b80 0x7f828c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:34.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.643+0000 7f82a0b80640 1 --2- 172.21.15.64:0/2408306819 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f828c0a2b80 0x7f828c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:34.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.643+0000 7f8299ffb640 1 -- 172.21.15.64:0/2408306819 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f82940ce850 con 0x7f829c105f40 2024-09-12T03:27:34.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.644+0000 7f82a0b80640 1 --2- 172.21.15.64:0/2408306819 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f828c0a2b80 0x7f828c0a5040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8290031440 tx=0x7f8290005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:34.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.647+0000 7f8299ffb640 1 -- 172.21.15.64:0/2408306819 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f829409af20 con 0x7f829c105f40 2024-09-12T03:27:34.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:34.860+0000 7f82a2de4640 1 -- 172.21.15.64:0/2408306819 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f829c1ab840 con 0x7f829c105f40 2024-09-12T03:27:35.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:35.132+0000 7f8299ffb640 1 -- 172.21.15.64:0/2408306819 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f829409f220 con 0x7f829c105f40 2024-09-12T03:27:35.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:35.134+0000 7f82a2de4640 1 -- 172.21.15.64:0/2408306819 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f828c0a2b80 msgr2=0x7f828c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:35.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:35.134+0000 7f82a2de4640 1 --2- 172.21.15.64:0/2408306819 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f828c0a2b80 0x7f828c0a5040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8290031440 tx=0x7f8290005a20 comp rx=0 tx=0).stop 2024-09-12T03:27:35.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:35.134+0000 7f82a2de4640 1 -- 172.21.15.64:0/2408306819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f829c105f40 msgr2=0x7f829c1a33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:35.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:35.134+0000 7f82a2de4640 1 --2- 172.21.15.64:0/2408306819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f829c105f40 0x7f829c1a33c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f829400b970 tx=0x7f829400be30 comp rx=0 tx=0).stop 2024-09-12T03:27:35.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:35.134+0000 7f82a2de4640 1 -- 172.21.15.64:0/2408306819 shutdown_connections 2024-09-12T03:27:35.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:35.134+0000 7f82a2de4640 1 --2- 172.21.15.64:0/2408306819 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f828c0a2b80 0x7f828c0a5040 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:35.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:35.134+0000 7f82a2de4640 1 --2- 172.21.15.64:0/2408306819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f829c105f40 0x7f829c1a33c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:35.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:35.134+0000 7f82a2de4640 1 --2- 172.21.15.64:0/2408306819 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f829c101450 0x7f829c19f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:35.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:35.134+0000 7f82a2de4640 1 --2- 172.21.15.64:0/2408306819 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f829c100b30 0x7f829c19eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:35.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:35.134+0000 7f82a2de4640 1 -- 172.21.15.64:0/2408306819 >> 172.21.15.64:0/2408306819 conn(0x7f829c0fc7a0 msgr2=0x7f829c10f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:35.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:35.134+0000 7f82a2de4640 1 -- 172.21.15.64:0/2408306819 shutdown_connections 2024-09-12T03:27:35.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:35.134+0000 7f82a2de4640 1 -- 172.21.15.64:0/2408306819 wait complete. 2024-09-12T03:27:35.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:35 smithi115 ceph-mon[34801]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-09-12T03:27:35.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:35 smithi115 ceph-mon[34801]: mds.? [v2:172.21.15.195:6838/2574695134,v1:172.21.15.195:6839/2574695134] up:active 2024-09-12T03:27:35.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:35 smithi115 ceph-mon[34801]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T03:27:35.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:35 smithi115 ceph-mon[34801]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T03:27:35.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:35 smithi115 ceph-mon[34801]: Cluster is now healthy 2024-09-12T03:27:35.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:35 smithi115 ceph-mon[34801]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-12T03:27:35.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:35 smithi115 ceph-mon[34801]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-12T03:27:35.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:35 smithi115 ceph-mon[34801]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-12T03:27:35.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:35 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/2408306819' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T03:27:35.431 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:35 smithi064 ceph-mon[30952]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-09-12T03:27:35.431 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:35 smithi064 ceph-mon[30952]: mds.? [v2:172.21.15.195:6838/2574695134,v1:172.21.15.195:6839/2574695134] up:active 2024-09-12T03:27:35.431 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:35 smithi064 ceph-mon[30952]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T03:27:35.431 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:35 smithi064 ceph-mon[30952]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T03:27:35.431 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:35 smithi064 ceph-mon[30952]: Cluster is now healthy 2024-09-12T03:27:35.431 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:35 smithi064 ceph-mon[30952]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-12T03:27:35.431 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:35 smithi064 ceph-mon[30952]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-12T03:27:35.431 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:35 smithi064 ceph-mon[30952]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-12T03:27:35.431 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:35 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/2408306819' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T03:27:35.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:35 smithi195 ceph-mon[34786]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-09-12T03:27:35.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:35 smithi195 ceph-mon[34786]: mds.? [v2:172.21.15.195:6838/2574695134,v1:172.21.15.195:6839/2574695134] up:active 2024-09-12T03:27:35.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:35 smithi195 ceph-mon[34786]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T03:27:35.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:35 smithi195 ceph-mon[34786]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T03:27:35.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:35 smithi195 ceph-mon[34786]: Cluster is now healthy 2024-09-12T03:27:35.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:35 smithi195 ceph-mon[34786]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-12T03:27:35.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:35 smithi195 ceph-mon[34786]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-12T03:27:35.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:35 smithi195 ceph-mon[34786]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-12T03:27:35.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:35 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/2408306819' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T03:27:35.831 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-12T03:27:36.098 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:36.392 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:36 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/2408306819' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T03:27:36.392 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:36 smithi064 ceph-mon[30952]: mds.? [v2:172.21.15.115:6836/2915155705,v1:172.21.15.115:6837/2915155705] up:active 2024-09-12T03:27:36.392 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:36 smithi064 ceph-mon[30952]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-12T03:27:36.392 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:36 smithi064 ceph-mon[30952]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:27:36.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:36 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/2408306819' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T03:27:36.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:36 smithi115 ceph-mon[34801]: mds.? [v2:172.21.15.115:6836/2915155705,v1:172.21.15.115:6837/2915155705] up:active 2024-09-12T03:27:36.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:36 smithi115 ceph-mon[34801]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-12T03:27:36.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:36 smithi115 ceph-mon[34801]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:27:36.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:36 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/2408306819' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T03:27:36.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:36 smithi195 ceph-mon[34786]: mds.? [v2:172.21.15.115:6836/2915155705,v1:172.21.15.115:6837/2915155705] up:active 2024-09-12T03:27:36.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:36 smithi195 ceph-mon[34786]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-12T03:27:36.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:36 smithi195 ceph-mon[34786]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:27:36.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:36.643+0000 7f20cc34c640 1 -- 172.21.15.64:0/2492941095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20c4102570 msgr2=0x7f20c41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:36.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:36.643+0000 7f20cc34c640 1 --2- 172.21.15.64:0/2492941095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20c4102570 0x7f20c41029b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f20b8009900 tx=0x7f20b802a480 comp rx=0 tx=0).stop 2024-09-12T03:27:36.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:36.643+0000 7f20cc34c640 1 -- 172.21.15.64:0/2492941095 shutdown_connections 2024-09-12T03:27:36.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:36.643+0000 7f20cc34c640 1 --2- 172.21.15.64:0/2492941095 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20c4102ef0 0x7f20c410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:36.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:36.643+0000 7f20cc34c640 1 --2- 172.21.15.64:0/2492941095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20c4102570 0x7f20c41029b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:36.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:36.643+0000 7f20cc34c640 1 --2- 172.21.15.64:0/2492941095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20c4100e20 0x7f20c4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:36.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:36.643+0000 7f20cc34c640 1 -- 172.21.15.64:0/2492941095 >> 172.21.15.64:0/2492941095 conn(0x7f20c40fc7d0 msgr2=0x7f20c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:36.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:36.643+0000 7f20cc34c640 1 -- 172.21.15.64:0/2492941095 shutdown_connections 2024-09-12T03:27:36.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:36.643+0000 7f20cc34c640 1 -- 172.21.15.64:0/2492941095 wait complete. 2024-09-12T03:27:36.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:36.643+0000 7f20cc34c640 1 Processor -- start 2024-09-12T03:27:36.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:36.644+0000 7f20cc34c640 1 -- start start 2024-09-12T03:27:36.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:36.644+0000 7f20cc34c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20c4100e20 0x7f20c40fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:36.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:36.644+0000 7f20cc34c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20c4102570 0x7f20c40fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:36.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:36.644+0000 7f20cc34c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20c4102ef0 0x7f20c40fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:36.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:36.644+0000 7f20cc34c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f20c4110e30 con 0x7f20c4102570 2024-09-12T03:27:36.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:36.644+0000 7f20cc34c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f20c4110cb0 con 0x7f20c4100e20 2024-09-12T03:27:36.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:36.644+0000 7f20cc34c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f20c4110fb0 con 0x7f20c4102ef0 2024-09-12T03:27:36.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:36.644+0000 7f20c98e7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20c4102570 0x7f20c40fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:36.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:36.644+0000 7f20c98e7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20c4102570 0x7f20c40fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41572/0 (socket says 172.21.15.64:41572) 2024-09-12T03:27:36.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:36.644+0000 7f20c98e7640 1 -- 172.21.15.64:0/24645988 learned_addr learned my addr 172.21.15.64:0/24645988 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:27:36.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:36.644+0000 7f20ca0e8640 1 --2- 172.21.15.64:0/24645988 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20c4100e20 0x7f20c40fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:36.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:36.644+0000 7f20ca8e9640 1 --2- 172.21.15.64:0/24645988 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20c4102ef0 0x7f20c40fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:36.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:36.644+0000 7f20c98e7640 1 -- 172.21.15.64:0/24645988 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20c4102ef0 msgr2=0x7f20c40fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:36.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:36.644+0000 7f20c98e7640 1 --2- 172.21.15.64:0/24645988 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20c4102ef0 0x7f20c40fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:36.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:36.644+0000 7f20c98e7640 1 -- 172.21.15.64:0/24645988 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20c4100e20 msgr2=0x7f20c40fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:36.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:36.644+0000 7f20c98e7640 1 --2- 172.21.15.64:0/24645988 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20c4100e20 0x7f20c40fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:36.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:36.644+0000 7f20c98e7640 1 -- 172.21.15.64:0/24645988 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20c41a6960 con 0x7f20c4102570 2024-09-12T03:27:36.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:36.644+0000 7f20c98e7640 1 --2- 172.21.15.64:0/24645988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20c4102570 0x7f20c40fd1c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f20b8006010 tx=0x7f20b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:36.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:36.645+0000 7f20c37fe640 1 -- 172.21.15.64:0/24645988 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b803f030 con 0x7f20c4102570 2024-09-12T03:27:36.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:36.645+0000 7f20c37fe640 1 -- 172.21.15.64:0/24645988 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20b8003cb0 con 0x7f20c4102570 2024-09-12T03:27:36.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:36.645+0000 7f20cc34c640 1 -- 172.21.15.64:0/24645988 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20c41a6bf0 con 0x7f20c4102570 2024-09-12T03:27:36.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:36.645+0000 7f20c37fe640 1 -- 172.21.15.64:0/24645988 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b8035070 con 0x7f20c4102570 2024-09-12T03:27:36.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:36.645+0000 7f20cc34c640 1 -- 172.21.15.64:0/24645988 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20c41a70d0 con 0x7f20c4102570 2024-09-12T03:27:36.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:36.646+0000 7f20cc34c640 1 -- 172.21.15.64:0/24645988 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20c4101200 con 0x7f20c4102570 2024-09-12T03:27:36.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:36.647+0000 7f20c37fe640 1 -- 172.21.15.64:0/24645988 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f20b8042030 con 0x7f20c4102570 2024-09-12T03:27:36.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:36.647+0000 7f20c37fe640 1 --2- 172.21.15.64:0/24645988 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f20ac0a2b80 0x7f20ac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:36.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:36.648+0000 7f20ca0e8640 1 --2- 172.21.15.64:0/24645988 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f20ac0a2b80 0x7f20ac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:36.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:36.648+0000 7f20c37fe640 1 -- 172.21.15.64:0/24645988 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f20b80ef750 con 0x7f20c4102570 2024-09-12T03:27:36.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:36.648+0000 7f20ca0e8640 1 --2- 172.21.15.64:0/24645988 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f20ac0a2b80 0x7f20ac0a5040 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f20b4005fd0 tx=0x7f20b4009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:36.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:36.652+0000 7f20c37fe640 1 -- 172.21.15.64:0/24645988 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f20b80bc470 con 0x7f20c4102570 2024-09-12T03:27:36.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:36.863+0000 7f20cc34c640 1 -- 172.21.15.64:0/24645988 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f20c410a0a0 con 0x7f20c4102570 2024-09-12T03:27:37.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:37.139+0000 7f20c37fe640 1 -- 172.21.15.64:0/24645988 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v24) ==== 115+0+0 (secure 0 0 0) 0x7f20b80bbd70 con 0x7f20c4102570 2024-09-12T03:27:37.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:37.141+0000 7f20cc34c640 1 -- 172.21.15.64:0/24645988 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f20ac0a2b80 msgr2=0x7f20ac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:37.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:37.141+0000 7f20cc34c640 1 --2- 172.21.15.64:0/24645988 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f20ac0a2b80 0x7f20ac0a5040 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f20b4005fd0 tx=0x7f20b4009680 comp rx=0 tx=0).stop 2024-09-12T03:27:37.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:37.141+0000 7f20cc34c640 1 -- 172.21.15.64:0/24645988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20c4102570 msgr2=0x7f20c40fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:37.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:37.141+0000 7f20cc34c640 1 --2- 172.21.15.64:0/24645988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20c4102570 0x7f20c40fd1c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f20b8006010 tx=0x7f20b80042f0 comp rx=0 tx=0).stop 2024-09-12T03:27:37.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:37.142+0000 7f20cc34c640 1 -- 172.21.15.64:0/24645988 shutdown_connections 2024-09-12T03:27:37.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:37.142+0000 7f20cc34c640 1 --2- 172.21.15.64:0/24645988 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f20ac0a2b80 0x7f20ac0a5040 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:37.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:37.142+0000 7f20cc34c640 1 --2- 172.21.15.64:0/24645988 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20c4102ef0 0x7f20c40fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:37.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:37.142+0000 7f20cc34c640 1 --2- 172.21.15.64:0/24645988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20c4102570 0x7f20c40fd1c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:37.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:37.142+0000 7f20cc34c640 1 --2- 172.21.15.64:0/24645988 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20c4100e20 0x7f20c40fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:37.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:37.142+0000 7f20cc34c640 1 -- 172.21.15.64:0/24645988 >> 172.21.15.64:0/24645988 conn(0x7f20c40fc7d0 msgr2=0x7f20c406a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:37.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:37.142+0000 7f20cc34c640 1 -- 172.21.15.64:0/24645988 shutdown_connections 2024-09-12T03:27:37.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:37.142+0000 7f20cc34c640 1 -- 172.21.15.64:0/24645988 wait complete. 2024-09-12T03:27:37.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:37 smithi195 ceph-mon[34786]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-09-12T03:27:37.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:37 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/24645988' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T03:27:37.502 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:37 smithi064 ceph-mon[30952]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-09-12T03:27:37.503 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:37 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/24645988' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T03:27:37.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:37 smithi115 ceph-mon[34801]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-09-12T03:27:37.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:37 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/24645988' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T03:27:37.947 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-09-12T03:27:37.947 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-12T03:27:38.213 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:38.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:38 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/24645988' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T03:27:38.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:38 smithi064 ceph-mon[30952]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:27:38.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:38 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/24645988' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T03:27:38.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:38 smithi195 ceph-mon[34786]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:27:38.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:38 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/24645988' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T03:27:38.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:38 smithi115 ceph-mon[34801]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:27:38.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.735+0000 7f88aed57640 1 -- 172.21.15.64:0/4000428705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88a8101de0 msgr2=0x7f88a81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:38.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.735+0000 7f88aed57640 1 --2- 172.21.15.64:0/4000428705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88a8101de0 0x7f88a81021c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f88a4009900 tx=0x7f88a402a410 comp rx=0 tx=0).stop 2024-09-12T03:27:38.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.735+0000 7f88aed57640 1 -- 172.21.15.64:0/4000428705 shutdown_connections 2024-09-12T03:27:38.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.736+0000 7f88aed57640 1 --2- 172.21.15.64:0/4000428705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88a8106f10 0x7f88a810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.736+0000 7f88aed57640 1 --2- 172.21.15.64:0/4000428705 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88a8102790 0x7f88a81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.736+0000 7f88aed57640 1 --2- 172.21.15.64:0/4000428705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88a8101de0 0x7f88a81021c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.736+0000 7f88aed57640 1 -- 172.21.15.64:0/4000428705 >> 172.21.15.64:0/4000428705 conn(0x7f88a80fd8c0 msgr2=0x7f88a80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:38.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.736+0000 7f88aed57640 1 -- 172.21.15.64:0/4000428705 shutdown_connections 2024-09-12T03:27:38.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.736+0000 7f88aed57640 1 -- 172.21.15.64:0/4000428705 wait complete. 2024-09-12T03:27:38.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.736+0000 7f88aed57640 1 Processor -- start 2024-09-12T03:27:38.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.736+0000 7f88aed57640 1 -- start start 2024-09-12T03:27:38.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.736+0000 7f88aed57640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88a8101de0 0x7f88a8198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:38.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.736+0000 7f88aed57640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88a8102790 0x7f88a81988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:38.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.736+0000 7f88aed57640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88a8106f10 0x7f88a819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:38.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.736+0000 7f88aed57640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f88a8111e20 con 0x7f88a8102790 2024-09-12T03:27:38.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.736+0000 7f88aed57640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f88a8111ca0 con 0x7f88a8101de0 2024-09-12T03:27:38.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.736+0000 7f88aed57640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f88a8111fa0 con 0x7f88a8106f10 2024-09-12T03:27:38.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.737+0000 7f889ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88a8102790 0x7f88a81988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:38.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.737+0000 7f889ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88a8102790 0x7f88a81988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49528/0 (socket says 172.21.15.64:49528) 2024-09-12T03:27:38.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.737+0000 7f88acaf3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88a8101de0 0x7f88a8198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:38.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.737+0000 7f889ffff640 1 -- 172.21.15.64:0/580995851 learned_addr learned my addr 172.21.15.64:0/580995851 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:27:38.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.737+0000 7f88ad2f4640 1 --2- 172.21.15.64:0/580995851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88a8106f10 0x7f88a819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:38.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.737+0000 7f889ffff640 1 -- 172.21.15.64:0/580995851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88a8106f10 msgr2=0x7f88a819cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:38.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.737+0000 7f889ffff640 1 --2- 172.21.15.64:0/580995851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88a8106f10 0x7f88a819cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.737+0000 7f889ffff640 1 -- 172.21.15.64:0/580995851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88a8101de0 msgr2=0x7f88a8198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:38.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.737+0000 7f889ffff640 1 --2- 172.21.15.64:0/580995851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88a8101de0 0x7f88a8198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.737+0000 7f889ffff640 1 -- 172.21.15.64:0/580995851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a819d330 con 0x7f88a8102790 2024-09-12T03:27:38.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.737+0000 7f889ffff640 1 --2- 172.21.15.64:0/580995851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88a8102790 0x7f88a81988c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f889800c970 tx=0x7f889800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:38.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.737+0000 7f889dffb640 1 -- 172.21.15.64:0/580995851 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8898007bb0 con 0x7f88a8102790 2024-09-12T03:27:38.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.737+0000 7f88aed57640 1 -- 172.21.15.64:0/580995851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88a819d620 con 0x7f88a8102790 2024-09-12T03:27:38.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.737+0000 7f889dffb640 1 -- 172.21.15.64:0/580995851 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8898007d50 con 0x7f88a8102790 2024-09-12T03:27:38.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.737+0000 7f889dffb640 1 -- 172.21.15.64:0/580995851 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889800f450 con 0x7f88a8102790 2024-09-12T03:27:38.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.738+0000 7f88aed57640 1 -- 172.21.15.64:0/580995851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88a81a4f00 con 0x7f88a8102790 2024-09-12T03:27:38.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.739+0000 7f88aed57640 1 -- 172.21.15.64:0/580995851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88a81021c0 con 0x7f88a8102790 2024-09-12T03:27:38.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.739+0000 7f889dffb640 1 -- 172.21.15.64:0/580995851 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8898004f20 con 0x7f88a8102790 2024-09-12T03:27:38.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.740+0000 7f889dffb640 1 --2- 172.21.15.64:0/580995851 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f88900a2b80 0x7f88900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:38.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.740+0000 7f88acaf3640 1 --2- 172.21.15.64:0/580995851 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f88900a2b80 0x7f88900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:38.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.740+0000 7f889dffb640 1 -- 172.21.15.64:0/580995851 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f88980cef50 con 0x7f88a8102790 2024-09-12T03:27:38.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.741+0000 7f88acaf3640 1 --2- 172.21.15.64:0/580995851 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f88900a2b80 0x7f88900a5040 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f88a4031440 tx=0x7f88a4005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:38.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.744+0000 7f889dffb640 1 -- 172.21.15.64:0/580995851 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8898014030 con 0x7f88a8102790 2024-09-12T03:27:38.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.942+0000 7f88aed57640 1 -- 172.21.15.64:0/580995851 --> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f88a8062630 con 0x7f88900a2b80 2024-09-12T03:27:38.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.966+0000 7f889dffb640 1 -- 172.21.15.64:0/580995851 <== mgr.14150 v2:172.21.15.64:6800/759931848 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f88a8062630 con 0x7f88900a2b80 2024-09-12T03:27:38.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.968+0000 7f88aed57640 1 -- 172.21.15.64:0/580995851 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f88900a2b80 msgr2=0x7f88900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:38.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.968+0000 7f88aed57640 1 --2- 172.21.15.64:0/580995851 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f88900a2b80 0x7f88900a5040 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f88a4031440 tx=0x7f88a4005b50 comp rx=0 tx=0).stop 2024-09-12T03:27:38.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.968+0000 7f88aed57640 1 -- 172.21.15.64:0/580995851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88a8102790 msgr2=0x7f88a81988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:38.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.968+0000 7f88aed57640 1 --2- 172.21.15.64:0/580995851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88a8102790 0x7f88a81988c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f889800c970 tx=0x7f889800ce30 comp rx=0 tx=0).stop 2024-09-12T03:27:38.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.969+0000 7f88aed57640 1 -- 172.21.15.64:0/580995851 shutdown_connections 2024-09-12T03:27:38.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.969+0000 7f88aed57640 1 --2- 172.21.15.64:0/580995851 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f88900a2b80 0x7f88900a5040 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.969+0000 7f88aed57640 1 --2- 172.21.15.64:0/580995851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88a8106f10 0x7f88a819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.969+0000 7f88aed57640 1 --2- 172.21.15.64:0/580995851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88a8102790 0x7f88a81988c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.969+0000 7f88aed57640 1 --2- 172.21.15.64:0/580995851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88a8101de0 0x7f88a8198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:38.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.969+0000 7f88aed57640 1 -- 172.21.15.64:0/580995851 >> 172.21.15.64:0/580995851 conn(0x7f88a80fd8c0 msgr2=0x7f88a80fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:38.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.969+0000 7f88aed57640 1 -- 172.21.15.64:0/580995851 shutdown_connections 2024-09-12T03:27:38.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:38.969+0000 7f88aed57640 1 -- 172.21.15.64:0/580995851 wait complete. 2024-09-12T03:27:39.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:39 smithi064 ceph-mon[30952]: pgmap v303: 129 pgs: 129 active+clean; 583 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-12T03:27:39.272 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:27:38 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:27:38.948+0000 7f4415ee9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:27:39.272 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:27:38 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:27:38.948+0000 7f4415ee9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:27:39.272 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:27:38 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:27:38.948+0000 7f4415ee9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:27:39.272 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:27:38 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:27:38.948+0000 7f4415ee9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:27:39.272 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:27:38 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:27:38.948+0000 7f4415ee9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:27:39.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:39 smithi195 ceph-mon[34786]: pgmap v303: 129 pgs: 129 active+clean; 583 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-12T03:27:39.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:39 smithi115 ceph-mon[34801]: pgmap v303: 129 pgs: 129 active+clean; 583 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-12T03:27:39.766 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-12T03:27:40.028 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:40.445 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:40 smithi064 ceph-mon[30952]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:27:40.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:40 smithi195 ceph-mon[34786]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:27:40.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.531+0000 7fe886e58640 1 -- 172.21.15.64:0/4264542534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe880100480 msgr2=0x7fe880100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:40.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.531+0000 7fe886e58640 1 --2- 172.21.15.64:0/4264542534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe880100480 0x7fe880100860 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe87400aa10 tx=0x7fe87402b720 comp rx=0 tx=0).stop 2024-09-12T03:27:40.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.531+0000 7fe886e58640 1 -- 172.21.15.64:0/4264542534 shutdown_connections 2024-09-12T03:27:40.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.531+0000 7fe886e58640 1 --2- 172.21.15.64:0/4264542534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8801055b0 0x7fe88010d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.531+0000 7fe886e58640 1 --2- 172.21.15.64:0/4264542534 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe880100e30 0x7fe880104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.531+0000 7fe886e58640 1 --2- 172.21.15.64:0/4264542534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe880100480 0x7fe880100860 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.531+0000 7fe886e58640 1 -- 172.21.15.64:0/4264542534 >> 172.21.15.64:0/4264542534 conn(0x7fe880077210 msgr2=0x7fe8800fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:40.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.531+0000 7fe886e58640 1 -- 172.21.15.64:0/4264542534 shutdown_connections 2024-09-12T03:27:40.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.531+0000 7fe886e58640 1 -- 172.21.15.64:0/4264542534 wait complete. 2024-09-12T03:27:40.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.532+0000 7fe886e58640 1 Processor -- start 2024-09-12T03:27:40.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.532+0000 7fe886e58640 1 -- start start 2024-09-12T03:27:40.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.532+0000 7fe886e58640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe880100480 0x7fe8801983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:40.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.532+0000 7fe886e58640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe880100e30 0x7fe8801988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:40.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.532+0000 7fe886e58640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8801055b0 0x7fe88019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:40.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.532+0000 7fe886e58640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe88010fc40 con 0x7fe880100e30 2024-09-12T03:27:40.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.532+0000 7fe886e58640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe88010fac0 con 0x7fe8801055b0 2024-09-12T03:27:40.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.532+0000 7fe886e58640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe88010fdc0 con 0x7fe880100480 2024-09-12T03:27:40.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.532+0000 7fe87ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe880100e30 0x7fe8801988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:40.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.532+0000 7fe87ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe880100e30 0x7fe8801988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49568/0 (socket says 172.21.15.64:49568) 2024-09-12T03:27:40.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.532+0000 7fe87ffff640 1 -- 172.21.15.64:0/275428063 learned_addr learned my addr 172.21.15.64:0/275428063 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:27:40.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.533+0000 7fe884bf4640 1 --2- 172.21.15.64:0/275428063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe880100480 0x7fe8801983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:40.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.533+0000 7fe8853f5640 1 --2- 172.21.15.64:0/275428063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8801055b0 0x7fe88019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:40.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.533+0000 7fe87ffff640 1 -- 172.21.15.64:0/275428063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe880100480 msgr2=0x7fe8801983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:40.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.533+0000 7fe87ffff640 1 --2- 172.21.15.64:0/275428063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe880100480 0x7fe8801983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.533+0000 7fe87ffff640 1 -- 172.21.15.64:0/275428063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8801055b0 msgr2=0x7fe88019cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:40.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.533+0000 7fe87ffff640 1 --2- 172.21.15.64:0/275428063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8801055b0 0x7fe88019cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.533+0000 7fe87ffff640 1 -- 172.21.15.64:0/275428063 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe88019d360 con 0x7fe880100e30 2024-09-12T03:27:40.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.533+0000 7fe8853f5640 1 --2- 172.21.15.64:0/275428063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8801055b0 0x7fe88019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:27:40.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.533+0000 7fe884bf4640 1 --2- 172.21.15.64:0/275428063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe880100480 0x7fe8801983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:27:40.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.533+0000 7fe87ffff640 1 --2- 172.21.15.64:0/275428063 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe880100e30 0x7fe8801988f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe86c00d770 tx=0x7fe86c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:40.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.533+0000 7fe87dffb640 1 -- 172.21.15.64:0/275428063 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c017070 con 0x7fe880100e30 2024-09-12T03:27:40.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.533+0000 7fe87dffb640 1 -- 172.21.15.64:0/275428063 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe86c0054d0 con 0x7fe880100e30 2024-09-12T03:27:40.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.533+0000 7fe886e58640 1 -- 172.21.15.64:0/275428063 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe88019d650 con 0x7fe880100e30 2024-09-12T03:27:40.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.533+0000 7fe87dffb640 1 -- 172.21.15.64:0/275428063 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c004cf0 con 0x7fe880100e30 2024-09-12T03:27:40.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.533+0000 7fe886e58640 1 -- 172.21.15.64:0/275428063 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8801a4f30 con 0x7fe880100e30 2024-09-12T03:27:40.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.534+0000 7fe886e58640 1 -- 172.21.15.64:0/275428063 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe880100860 con 0x7fe880100e30 2024-09-12T03:27:40.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.535+0000 7fe87dffb640 1 -- 172.21.15.64:0/275428063 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe86c004e90 con 0x7fe880100e30 2024-09-12T03:27:40.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.536+0000 7fe87dffb640 1 --2- 172.21.15.64:0/275428063 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe8700a2c10 0x7fe8700a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:40.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.536+0000 7fe884bf4640 1 --2- 172.21.15.64:0/275428063 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe8700a2c10 0x7fe8700a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:40.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.536+0000 7fe87dffb640 1 -- 172.21.15.64:0/275428063 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fe86c0cef30 con 0x7fe880100e30 2024-09-12T03:27:40.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.536+0000 7fe884bf4640 1 --2- 172.21.15.64:0/275428063 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe8700a2c10 0x7fe8700a50d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe874030550 tx=0x7fe874005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:40.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.540+0000 7fe87dffb640 1 -- 172.21.15.64:0/275428063 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe86c09baf0 con 0x7fe880100e30 2024-09-12T03:27:40.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:40 smithi115 ceph-mon[34801]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:27:40.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.743+0000 7fe886e58640 1 -- 172.21.15.64:0/275428063 --> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fe880062630 con 0x7fe8700a2c10 2024-09-12T03:27:40.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.769+0000 7fe87dffb640 1 -- 172.21.15.64:0/275428063 <== mgr.14150 v2:172.21.15.64:6800/759931848 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe880062630 con 0x7fe8700a2c10 2024-09-12T03:27:40.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.771+0000 7fe886e58640 1 -- 172.21.15.64:0/275428063 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe8700a2c10 msgr2=0x7fe8700a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:40.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.771+0000 7fe886e58640 1 --2- 172.21.15.64:0/275428063 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe8700a2c10 0x7fe8700a50d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe874030550 tx=0x7fe874005a20 comp rx=0 tx=0).stop 2024-09-12T03:27:40.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.771+0000 7fe886e58640 1 -- 172.21.15.64:0/275428063 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe880100e30 msgr2=0x7fe8801988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:40.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.771+0000 7fe886e58640 1 --2- 172.21.15.64:0/275428063 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe880100e30 0x7fe8801988f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe86c00d770 tx=0x7fe86c00dc30 comp rx=0 tx=0).stop 2024-09-12T03:27:40.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.771+0000 7fe886e58640 1 -- 172.21.15.64:0/275428063 shutdown_connections 2024-09-12T03:27:40.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.771+0000 7fe886e58640 1 --2- 172.21.15.64:0/275428063 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fe8700a2c10 0x7fe8700a50d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.771+0000 7fe886e58640 1 --2- 172.21.15.64:0/275428063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8801055b0 0x7fe88019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.771+0000 7fe886e58640 1 --2- 172.21.15.64:0/275428063 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe880100e30 0x7fe8801988f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.771+0000 7fe886e58640 1 --2- 172.21.15.64:0/275428063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe880100480 0x7fe8801983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:40.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.771+0000 7fe886e58640 1 -- 172.21.15.64:0/275428063 >> 172.21.15.64:0/275428063 conn(0x7fe880077210 msgr2=0x7fe8801038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:40.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.771+0000 7fe886e58640 1 -- 172.21.15.64:0/275428063 shutdown_connections 2024-09-12T03:27:40.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:40.771+0000 7fe886e58640 1 -- 172.21.15.64:0/275428063 wait complete. 2024-09-12T03:27:41.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:41 smithi195 ceph-mon[34786]: pgmap v304: 129 pgs: 129 active+clean; 584 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-09-12T03:27:41.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:41 smithi195 ceph-mon[34786]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-12T03:27:41.521 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:41 smithi064 ceph-mon[30952]: pgmap v304: 129 pgs: 129 active+clean; 584 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-09-12T03:27:41.521 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:41 smithi064 ceph-mon[30952]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-12T03:27:41.555 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-12T03:27:41.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:41 smithi115 ceph-mon[34801]: pgmap v304: 129 pgs: 129 active+clean; 584 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-09-12T03:27:41.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:41 smithi115 ceph-mon[34801]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-12T03:27:41.819 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:42.240 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:42 smithi064 ceph-mon[30952]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:27:42.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.329+0000 7f649144f640 1 -- 172.21.15.64:0/3357255341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f648c102ef0 msgr2=0x7f648c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:42.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.329+0000 7f649144f640 1 --2- 172.21.15.64:0/3357255341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f648c102ef0 0x7f648c10da40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6484009ca0 tx=0x7f648402a950 comp rx=0 tx=0).stop 2024-09-12T03:27:42.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.329+0000 7f649144f640 1 -- 172.21.15.64:0/3357255341 shutdown_connections 2024-09-12T03:27:42.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.329+0000 7f649144f640 1 --2- 172.21.15.64:0/3357255341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f648c102ef0 0x7f648c10da40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:42.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.329+0000 7f649144f640 1 --2- 172.21.15.64:0/3357255341 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f648c102570 0x7f648c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:42.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.329+0000 7f649144f640 1 --2- 172.21.15.64:0/3357255341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f648c100e20 0x7f648c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:42.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.329+0000 7f649144f640 1 -- 172.21.15.64:0/3357255341 >> 172.21.15.64:0/3357255341 conn(0x7f648c0fc7d0 msgr2=0x7f648c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:42.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.329+0000 7f649144f640 1 -- 172.21.15.64:0/3357255341 shutdown_connections 2024-09-12T03:27:42.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.329+0000 7f649144f640 1 -- 172.21.15.64:0/3357255341 wait complete. 2024-09-12T03:27:42.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.329+0000 7f649144f640 1 Processor -- start 2024-09-12T03:27:42.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.330+0000 7f649144f640 1 -- start start 2024-09-12T03:27:42.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.330+0000 7f649144f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f648c100e20 0x7f648c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:42.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.330+0000 7f649144f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f648c102570 0x7f648c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:42.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.330+0000 7f649144f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f648c102ef0 0x7f648c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:42.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.330+0000 7f649144f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f648c111f50 con 0x7f648c102570 2024-09-12T03:27:42.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.330+0000 7f649144f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f648c111dd0 con 0x7f648c100e20 2024-09-12T03:27:42.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.330+0000 7f649144f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f648c1120d0 con 0x7f648c102ef0 2024-09-12T03:27:42.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.330+0000 7f648a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f648c102570 0x7f648c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:42.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.330+0000 7f648a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f648c102570 0x7f648c198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49586/0 (socket says 172.21.15.64:49586) 2024-09-12T03:27:42.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.330+0000 7f648a7fc640 1 -- 172.21.15.64:0/1298969616 learned_addr learned my addr 172.21.15.64:0/1298969616 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:27:42.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.330+0000 7f648affd640 1 --2- 172.21.15.64:0/1298969616 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f648c100e20 0x7f648c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:42.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.330+0000 7f648b7fe640 1 --2- 172.21.15.64:0/1298969616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f648c102ef0 0x7f648c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:42.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.330+0000 7f648a7fc640 1 -- 172.21.15.64:0/1298969616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f648c102ef0 msgr2=0x7f648c19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:42.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.330+0000 7f648a7fc640 1 --2- 172.21.15.64:0/1298969616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f648c102ef0 0x7f648c19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:42.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.330+0000 7f648a7fc640 1 -- 172.21.15.64:0/1298969616 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f648c100e20 msgr2=0x7f648c198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:42.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.330+0000 7f648a7fc640 1 --2- 172.21.15.64:0/1298969616 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f648c100e20 0x7f648c198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:42.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.330+0000 7f648a7fc640 1 -- 172.21.15.64:0/1298969616 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f648c19d3e0 con 0x7f648c102570 2024-09-12T03:27:42.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.330+0000 7f648b7fe640 1 --2- 172.21.15.64:0/1298969616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f648c102ef0 0x7f648c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:27:42.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.330+0000 7f648a7fc640 1 --2- 172.21.15.64:0/1298969616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f648c102570 0x7f648c198970 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f648000b4a0 tx=0x7f648000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:42.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.331+0000 7f6473fff640 1 -- 172.21.15.64:0/1298969616 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6480014070 con 0x7f648c102570 2024-09-12T03:27:42.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.331+0000 7f649144f640 1 -- 172.21.15.64:0/1298969616 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f648c19d6d0 con 0x7f648c102570 2024-09-12T03:27:42.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.331+0000 7f6473fff640 1 -- 172.21.15.64:0/1298969616 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6480010ac0 con 0x7f648c102570 2024-09-12T03:27:42.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.331+0000 7f6473fff640 1 -- 172.21.15.64:0/1298969616 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6480017c30 con 0x7f648c102570 2024-09-12T03:27:42.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.331+0000 7f649144f640 1 -- 172.21.15.64:0/1298969616 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f648c1a4fb0 con 0x7f648c102570 2024-09-12T03:27:42.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.332+0000 7f649144f640 1 -- 172.21.15.64:0/1298969616 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f648c101200 con 0x7f648c102570 2024-09-12T03:27:42.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.333+0000 7f6473fff640 1 -- 172.21.15.64:0/1298969616 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6480017070 con 0x7f648c102570 2024-09-12T03:27:42.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.334+0000 7f6473fff640 1 --2- 172.21.15.64:0/1298969616 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f64740a2c10 0x7f64740a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:42.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.334+0000 7f648affd640 1 --2- 172.21.15.64:0/1298969616 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f64740a2c10 0x7f64740a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:42.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.334+0000 7f6473fff640 1 -- 172.21.15.64:0/1298969616 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f64800cea90 con 0x7f648c102570 2024-09-12T03:27:42.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.334+0000 7f648affd640 1 --2- 172.21.15.64:0/1298969616 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f64740a2c10 0x7f64740a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f647c005fd0 tx=0x7f647c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:42.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.338+0000 7f6473fff640 1 -- 172.21.15.64:0/1298969616 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f648009b760 con 0x7f648c102570 2024-09-12T03:27:42.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:42 smithi195 ceph-mon[34786]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:27:42.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.542+0000 7f649144f640 1 -- 172.21.15.64:0/1298969616 --> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f648c062630 con 0x7f64740a2c10 2024-09-12T03:27:42.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.547+0000 7f6473fff640 1 -- 172.21.15.64:0/1298969616 <== mgr.14150 v2:172.21.15.64:6800/759931848 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f648c062630 con 0x7f64740a2c10 2024-09-12T03:27:42.547 INFO:teuthology.orchestra.run.smithi064.stdout:/volumes/qa/sv_0/2158fa2b-94db-46f4-9312-ea8d5f733d5c 2024-09-12T03:27:42.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.549+0000 7f649144f640 1 -- 172.21.15.64:0/1298969616 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f64740a2c10 msgr2=0x7f64740a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:42.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.549+0000 7f649144f640 1 --2- 172.21.15.64:0/1298969616 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f64740a2c10 0x7f64740a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f647c005fd0 tx=0x7f647c005850 comp rx=0 tx=0).stop 2024-09-12T03:27:42.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.549+0000 7f649144f640 1 -- 172.21.15.64:0/1298969616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f648c102570 msgr2=0x7f648c198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:42.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.549+0000 7f649144f640 1 --2- 172.21.15.64:0/1298969616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f648c102570 0x7f648c198970 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f648000b4a0 tx=0x7f648000b960 comp rx=0 tx=0).stop 2024-09-12T03:27:42.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.549+0000 7f649144f640 1 -- 172.21.15.64:0/1298969616 shutdown_connections 2024-09-12T03:27:42.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.549+0000 7f649144f640 1 --2- 172.21.15.64:0/1298969616 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f64740a2c10 0x7f64740a50d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:42.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.549+0000 7f649144f640 1 --2- 172.21.15.64:0/1298969616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f648c102ef0 0x7f648c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:42.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.549+0000 7f649144f640 1 --2- 172.21.15.64:0/1298969616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f648c102570 0x7f648c198970 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:42.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.549+0000 7f649144f640 1 --2- 172.21.15.64:0/1298969616 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f648c100e20 0x7f648c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:42.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.549+0000 7f649144f640 1 -- 172.21.15.64:0/1298969616 >> 172.21.15.64:0/1298969616 conn(0x7f648c0fc7d0 msgr2=0x7f648c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:42.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.549+0000 7f649144f640 1 -- 172.21.15.64:0/1298969616 shutdown_connections 2024-09-12T03:27:42.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:42.549+0000 7f649144f640 1 -- 172.21.15.64:0/1298969616 wait complete. 2024-09-12T03:27:42.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:42 smithi115 ceph-mon[34801]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:27:43.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:43 smithi064 ceph-mon[30952]: pgmap v305: 129 pgs: 129 active+clean; 604 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-09-12T03:27:43.370 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-12T03:27:43.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:43 smithi195 ceph-mon[34786]: pgmap v305: 129 pgs: 129 active+clean; 604 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-09-12T03:27:43.628 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:43.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:43 smithi115 ceph-mon[34801]: pgmap v305: 129 pgs: 129 active+clean; 604 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-09-12T03:27:44.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.114+0000 7ff53da53640 1 -- 172.21.15.64:0/808585996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5381036e0 msgr2=0x7ff53810bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:44.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.114+0000 7ff53da53640 1 --2- 172.21.15.64:0/808585996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5381036e0 0x7ff53810bbd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff52c009900 tx=0x7ff52c02a480 comp rx=0 tx=0).stop 2024-09-12T03:27:44.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.114+0000 7ff53da53640 1 -- 172.21.15.64:0/808585996 shutdown_connections 2024-09-12T03:27:44.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.114+0000 7ff53da53640 1 --2- 172.21.15.64:0/808585996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff53810c1a0 0x7ff53810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.114+0000 7ff53da53640 1 --2- 172.21.15.64:0/808585996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5381036e0 0x7ff53810bbd0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.114+0000 7ff53da53640 1 --2- 172.21.15.64:0/808585996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff538102dc0 0x7ff5381031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.114+0000 7ff53da53640 1 -- 172.21.15.64:0/808585996 >> 172.21.15.64:0/808585996 conn(0x7ff5380fc7a0 msgr2=0x7ff5380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:44.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.114+0000 7ff53da53640 1 -- 172.21.15.64:0/808585996 shutdown_connections 2024-09-12T03:27:44.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.114+0000 7ff53da53640 1 -- 172.21.15.64:0/808585996 wait complete. 2024-09-12T03:27:44.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.115+0000 7ff53da53640 1 Processor -- start 2024-09-12T03:27:44.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.115+0000 7ff53da53640 1 -- start start 2024-09-12T03:27:44.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.115+0000 7ff53da53640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff538102dc0 0x7ff538198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:44.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.115+0000 7ff53da53640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5381036e0 0x7ff538198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:44.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.115+0000 7ff53da53640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff53810c1a0 0x7ff53819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:44.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.116+0000 7ff53da53640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff538111e20 con 0x7ff538102dc0 2024-09-12T03:27:44.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.116+0000 7ff53da53640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff538111ca0 con 0x7ff53810c1a0 2024-09-12T03:27:44.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.116+0000 7ff53da53640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff538111fa0 con 0x7ff5381036e0 2024-09-12T03:27:44.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.116+0000 7ff536ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff538102dc0 0x7ff538198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:44.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.116+0000 7ff536ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff538102dc0 0x7ff538198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49606/0 (socket says 172.21.15.64:49606) 2024-09-12T03:27:44.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.116+0000 7ff536ffd640 1 -- 172.21.15.64:0/2736681286 learned_addr learned my addr 172.21.15.64:0/2736681286 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:27:44.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.116+0000 7ff5377fe640 1 --2- 172.21.15.64:0/2736681286 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff53810c1a0 0x7ff53819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:44.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.116+0000 7ff5367fc640 1 --2- 172.21.15.64:0/2736681286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5381036e0 0x7ff538198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:44.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.116+0000 7ff536ffd640 1 -- 172.21.15.64:0/2736681286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5381036e0 msgr2=0x7ff538198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:44.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.116+0000 7ff536ffd640 1 --2- 172.21.15.64:0/2736681286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5381036e0 0x7ff538198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.116+0000 7ff536ffd640 1 -- 172.21.15.64:0/2736681286 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff53810c1a0 msgr2=0x7ff53819cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:44.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.116+0000 7ff536ffd640 1 --2- 172.21.15.64:0/2736681286 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff53810c1a0 0x7ff53819cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.116+0000 7ff536ffd640 1 -- 172.21.15.64:0/2736681286 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff53819d2c0 con 0x7ff538102dc0 2024-09-12T03:27:44.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.116+0000 7ff536ffd640 1 --2- 172.21.15.64:0/2736681286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff538102dc0 0x7ff538198310 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff52800d770 tx=0x7ff52800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:44.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.116+0000 7ff53ca51640 1 -- 172.21.15.64:0/2736681286 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff528017070 con 0x7ff538102dc0 2024-09-12T03:27:44.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.116+0000 7ff53da53640 1 -- 172.21.15.64:0/2736681286 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff53819d5b0 con 0x7ff538102dc0 2024-09-12T03:27:44.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.116+0000 7ff53ca51640 1 -- 172.21.15.64:0/2736681286 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5280054d0 con 0x7ff538102dc0 2024-09-12T03:27:44.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.117+0000 7ff53da53640 1 -- 172.21.15.64:0/2736681286 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5381a4e90 con 0x7ff538102dc0 2024-09-12T03:27:44.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.117+0000 7ff53ca51640 1 -- 172.21.15.64:0/2736681286 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff528004cf0 con 0x7ff538102dc0 2024-09-12T03:27:44.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.118+0000 7ff53da53640 1 -- 172.21.15.64:0/2736681286 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff524005290 con 0x7ff538102dc0 2024-09-12T03:27:44.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.119+0000 7ff53ca51640 1 -- 172.21.15.64:0/2736681286 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff528009c30 con 0x7ff538102dc0 2024-09-12T03:27:44.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.119+0000 7ff53ca51640 1 --2- 172.21.15.64:0/2736681286 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff5200a2c10 0x7ff5200a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:44.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.120+0000 7ff5367fc640 1 --2- 172.21.15.64:0/2736681286 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff5200a2c10 0x7ff5200a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:44.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.120+0000 7ff53ca51640 1 -- 172.21.15.64:0/2736681286 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7ff5280cec50 con 0x7ff538102dc0 2024-09-12T03:27:44.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.120+0000 7ff5367fc640 1 --2- 172.21.15.64:0/2736681286 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff5200a2c10 0x7ff5200a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff52c031440 tx=0x7ff52c005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:44.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.124+0000 7ff53ca51640 1 -- 172.21.15.64:0/2736681286 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff5280046c0 con 0x7ff538102dc0 2024-09-12T03:27:44.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.326+0000 7ff53da53640 1 -- 172.21.15.64:0/2736681286 --> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7ff524000bc0 con 0x7ff5200a2c10 2024-09-12T03:27:44.327 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:44 smithi064 ceph-mon[30952]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:27:44.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.351+0000 7ff53ca51640 1 -- 172.21.15.64:0/2736681286 <== mgr.14150 v2:172.21.15.64:6800/759931848 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff524000bc0 con 0x7ff5200a2c10 2024-09-12T03:27:44.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.353+0000 7ff53da53640 1 -- 172.21.15.64:0/2736681286 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff5200a2c10 msgr2=0x7ff5200a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:44.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.353+0000 7ff53da53640 1 --2- 172.21.15.64:0/2736681286 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff5200a2c10 0x7ff5200a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff52c031440 tx=0x7ff52c005b50 comp rx=0 tx=0).stop 2024-09-12T03:27:44.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.353+0000 7ff53da53640 1 -- 172.21.15.64:0/2736681286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff538102dc0 msgr2=0x7ff538198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:44.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.353+0000 7ff53da53640 1 --2- 172.21.15.64:0/2736681286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff538102dc0 0x7ff538198310 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff52800d770 tx=0x7ff52800dc30 comp rx=0 tx=0).stop 2024-09-12T03:27:44.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.353+0000 7ff53da53640 1 -- 172.21.15.64:0/2736681286 shutdown_connections 2024-09-12T03:27:44.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.353+0000 7ff53da53640 1 --2- 172.21.15.64:0/2736681286 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff5200a2c10 0x7ff5200a50d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.353+0000 7ff53da53640 1 --2- 172.21.15.64:0/2736681286 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff53810c1a0 0x7ff53819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.353+0000 7ff53da53640 1 --2- 172.21.15.64:0/2736681286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5381036e0 0x7ff538198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.353+0000 7ff53da53640 1 --2- 172.21.15.64:0/2736681286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff538102dc0 0x7ff538198310 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:44.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.353+0000 7ff53da53640 1 -- 172.21.15.64:0/2736681286 >> 172.21.15.64:0/2736681286 conn(0x7ff5380fc7a0 msgr2=0x7ff5380fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:44.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.353+0000 7ff53da53640 1 -- 172.21.15.64:0/2736681286 shutdown_connections 2024-09-12T03:27:44.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:44.354+0000 7ff53da53640 1 -- 172.21.15.64:0/2736681286 wait complete. 2024-09-12T03:27:44.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:44 smithi195 ceph-mon[34786]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:27:44.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:44 smithi115 ceph-mon[34801]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:27:45.103 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-12T03:27:45.360 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:45.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:45 smithi195 ceph-mon[34786]: pgmap v306: 129 pgs: 129 active+clean; 605 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.3 KiB/s wr, 13 op/s 2024-09-12T03:27:45.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:45 smithi195 ceph-mon[34786]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:27:45.482 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:45 smithi064 ceph-mon[30952]: pgmap v306: 129 pgs: 129 active+clean; 605 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.3 KiB/s wr, 13 op/s 2024-09-12T03:27:45.482 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:45 smithi064 ceph-mon[30952]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:27:45.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:45 smithi115 ceph-mon[34801]: pgmap v306: 129 pgs: 129 active+clean; 605 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.3 KiB/s wr, 13 op/s 2024-09-12T03:27:45.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:45 smithi115 ceph-mon[34801]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:27:45.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:45.865+0000 7fd7a756e640 1 -- 172.21.15.64:0/3035881603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7a0100e20 msgr2=0x7fd7a0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:45.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:45.865+0000 7fd7a756e640 1 --2- 172.21.15.64:0/3035881603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7a0100e20 0x7fd7a0101200 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd79c009900 tx=0x7fd79c02a480 comp rx=0 tx=0).stop 2024-09-12T03:27:45.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:45.865+0000 7fd7a756e640 1 -- 172.21.15.64:0/3035881603 shutdown_connections 2024-09-12T03:27:45.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:45.865+0000 7fd7a756e640 1 --2- 172.21.15.64:0/3035881603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7a0102ef0 0x7fd7a010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:45.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:45.865+0000 7fd7a756e640 1 --2- 172.21.15.64:0/3035881603 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7a0102570 0x7fd7a01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:45.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:45.865+0000 7fd7a756e640 1 --2- 172.21.15.64:0/3035881603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7a0100e20 0x7fd7a0101200 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:45.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:45.865+0000 7fd7a756e640 1 -- 172.21.15.64:0/3035881603 >> 172.21.15.64:0/3035881603 conn(0x7fd7a00fc7d0 msgr2=0x7fd7a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:45.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:45.865+0000 7fd7a756e640 1 -- 172.21.15.64:0/3035881603 shutdown_connections 2024-09-12T03:27:45.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:45.865+0000 7fd7a756e640 1 -- 172.21.15.64:0/3035881603 wait complete. 2024-09-12T03:27:45.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:45.865+0000 7fd7a756e640 1 Processor -- start 2024-09-12T03:27:45.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:45.865+0000 7fd7a756e640 1 -- start start 2024-09-12T03:27:45.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:45.866+0000 7fd7a756e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7a0100e20 0x7fd7a0198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:45.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:45.866+0000 7fd7a756e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7a0102570 0x7fd7a0198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:45.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:45.866+0000 7fd7a756e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7a0102ef0 0x7fd7a019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:45.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:45.866+0000 7fd7a756e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a0110e30 con 0x7fd7a0102570 2024-09-12T03:27:45.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:45.866+0000 7fd7a756e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a0110cb0 con 0x7fd7a0102ef0 2024-09-12T03:27:45.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:45.866+0000 7fd7a756e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a0110fb0 con 0x7fd7a0100e20 2024-09-12T03:27:45.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:45.866+0000 7fd7a4b09640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7a0102570 0x7fd7a0198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:45.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:45.866+0000 7fd7a4b09640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7a0102570 0x7fd7a0198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49620/0 (socket says 172.21.15.64:49620) 2024-09-12T03:27:45.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:45.866+0000 7fd7a4b09640 1 -- 172.21.15.64:0/434466957 learned_addr learned my addr 172.21.15.64:0/434466957 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:27:45.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:45.866+0000 7fd7a530a640 1 --2- 172.21.15.64:0/434466957 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7a0100e20 0x7fd7a0198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:45.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:45.866+0000 7fd7a5b0b640 1 --2- 172.21.15.64:0/434466957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7a0102ef0 0x7fd7a019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:45.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:45.866+0000 7fd7a4b09640 1 -- 172.21.15.64:0/434466957 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7a0100e20 msgr2=0x7fd7a0198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:45.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:45.866+0000 7fd7a4b09640 1 --2- 172.21.15.64:0/434466957 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7a0100e20 0x7fd7a0198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:45.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:45.866+0000 7fd7a4b09640 1 -- 172.21.15.64:0/434466957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7a0102ef0 msgr2=0x7fd7a019ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:45.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:45.866+0000 7fd7a4b09640 1 --2- 172.21.15.64:0/434466957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7a0102ef0 0x7fd7a019ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:45.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:45.866+0000 7fd7a4b09640 1 -- 172.21.15.64:0/434466957 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a019d3d0 con 0x7fd7a0102570 2024-09-12T03:27:45.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:45.866+0000 7fd7a4b09640 1 --2- 172.21.15.64:0/434466957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7a0102570 0x7fd7a0198960 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd79000d770 tx=0x7fd79000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:45.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:45.866+0000 7fd7967fc640 1 -- 172.21.15.64:0/434466957 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd790017070 con 0x7fd7a0102570 2024-09-12T03:27:45.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:45.867+0000 7fd7967fc640 1 -- 172.21.15.64:0/434466957 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd7900054d0 con 0x7fd7a0102570 2024-09-12T03:27:45.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:45.867+0000 7fd7a756e640 1 -- 172.21.15.64:0/434466957 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7a019d6c0 con 0x7fd7a0102570 2024-09-12T03:27:45.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:45.867+0000 7fd7967fc640 1 -- 172.21.15.64:0/434466957 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd790004cf0 con 0x7fd7a0102570 2024-09-12T03:27:45.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:45.867+0000 7fd7a756e640 1 -- 172.21.15.64:0/434466957 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7a01a4fa0 con 0x7fd7a0102570 2024-09-12T03:27:45.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:45.868+0000 7fd7a756e640 1 -- 172.21.15.64:0/434466957 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7a0101200 con 0x7fd7a0102570 2024-09-12T03:27:45.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:45.869+0000 7fd7967fc640 1 -- 172.21.15.64:0/434466957 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd790004e90 con 0x7fd7a0102570 2024-09-12T03:27:45.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:45.869+0000 7fd7967fc640 1 --2- 172.21.15.64:0/434466957 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fd78c0a2c90 0x7fd78c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:45.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:45.870+0000 7fd7a530a640 1 --2- 172.21.15.64:0/434466957 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fd78c0a2c90 0x7fd78c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:45.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:45.870+0000 7fd7967fc640 1 -- 172.21.15.64:0/434466957 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fd7900ced50 con 0x7fd7a0102570 2024-09-12T03:27:45.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:45.870+0000 7fd7a530a640 1 --2- 172.21.15.64:0/434466957 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fd78c0a2c90 0x7fd78c0a5150 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd79c009230 tx=0x7fd79c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:45.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:45.874+0000 7fd7967fc640 1 -- 172.21.15.64:0/434466957 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd79009b2c0 con 0x7fd7a0102570 2024-09-12T03:27:46.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:46.079+0000 7fd7a756e640 1 -- 172.21.15.64:0/434466957 --> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fd7a0106f10 con 0x7fd78c0a2c90 2024-09-12T03:27:46.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:46.084+0000 7fd7967fc640 1 -- 172.21.15.64:0/434466957 <== mgr.14150 v2:172.21.15.64:6800/759931848 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fd7a0106f10 con 0x7fd78c0a2c90 2024-09-12T03:27:46.084 INFO:teuthology.orchestra.run.smithi064.stdout:/volumes/qa/sv_1/489b8653-431d-4265-a663-9ca24edeb467 2024-09-12T03:27:46.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:46.087+0000 7fd7a756e640 1 -- 172.21.15.64:0/434466957 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fd78c0a2c90 msgr2=0x7fd78c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:46.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:46.087+0000 7fd7a756e640 1 --2- 172.21.15.64:0/434466957 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fd78c0a2c90 0x7fd78c0a5150 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd79c009230 tx=0x7fd79c0058d0 comp rx=0 tx=0).stop 2024-09-12T03:27:46.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:46.087+0000 7fd7a756e640 1 -- 172.21.15.64:0/434466957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7a0102570 msgr2=0x7fd7a0198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:46.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:46.087+0000 7fd7a756e640 1 --2- 172.21.15.64:0/434466957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7a0102570 0x7fd7a0198960 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd79000d770 tx=0x7fd79000dc30 comp rx=0 tx=0).stop 2024-09-12T03:27:46.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:46.087+0000 7fd7a756e640 1 -- 172.21.15.64:0/434466957 shutdown_connections 2024-09-12T03:27:46.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:46.087+0000 7fd7a756e640 1 --2- 172.21.15.64:0/434466957 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fd78c0a2c90 0x7fd78c0a5150 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:46.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:46.087+0000 7fd7a756e640 1 --2- 172.21.15.64:0/434466957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7a0102ef0 0x7fd7a019ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:46.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:46.087+0000 7fd7a756e640 1 --2- 172.21.15.64:0/434466957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7a0102570 0x7fd7a0198960 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:46.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:46.087+0000 7fd7a756e640 1 --2- 172.21.15.64:0/434466957 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7a0100e20 0x7fd7a0198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:46.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:46.087+0000 7fd7a756e640 1 -- 172.21.15.64:0/434466957 >> 172.21.15.64:0/434466957 conn(0x7fd7a00fc7d0 msgr2=0x7fd7a0069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:46.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:46.087+0000 7fd7a756e640 1 -- 172.21.15.64:0/434466957 shutdown_connections 2024-09-12T03:27:46.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:46.087+0000 7fd7a756e640 1 -- 172.21.15.64:0/434466957 wait complete. 2024-09-12T03:27:46.745 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:27:47.001 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:47.374 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:47 smithi064 ceph-mon[30952]: pgmap v307: 129 pgs: 129 active+clean; 605 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.3 KiB/s wr, 13 op/s 2024-09-12T03:27:47.374 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:47 smithi064 ceph-mon[30952]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:27:47.445 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:47 smithi115 ceph-mon[34801]: pgmap v307: 129 pgs: 129 active+clean; 605 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.3 KiB/s wr, 13 op/s 2024-09-12T03:27:47.445 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:47 smithi115 ceph-mon[34801]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:27:47.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:47 smithi195 ceph-mon[34786]: pgmap v307: 129 pgs: 129 active+clean; 605 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.3 KiB/s wr, 13 op/s 2024-09-12T03:27:47.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:47 smithi195 ceph-mon[34786]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:27:47.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.453+0000 7f8cbd632640 1 -- 172.21.15.64:0/4269895759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cb8101420 msgr2=0x7f8cb810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:47.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.453+0000 7f8cbd632640 1 --2- 172.21.15.64:0/4269895759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cb8101420 0x7f8cb810d950 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0009cd0 tx=0x7f8cb002a8e0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.453+0000 7f8cbd632640 1 -- 172.21.15.64:0/4269895759 shutdown_connections 2024-09-12T03:27:47.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.453+0000 7f8cbd632640 1 --2- 172.21.15.64:0/4269895759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cb8101420 0x7f8cb810d950 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.453+0000 7f8cbd632640 1 --2- 172.21.15.64:0/4269895759 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cb8100a80 0x7f8cb8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.453+0000 7f8cbd632640 1 --2- 172.21.15.64:0/4269895759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8cb8106a80 0x7f8cb8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.453+0000 7f8cbd632640 1 -- 172.21.15.64:0/4269895759 >> 172.21.15.64:0/4269895759 conn(0x7f8cb80fc7a0 msgr2=0x7f8cb80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:47.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.453+0000 7f8cbd632640 1 -- 172.21.15.64:0/4269895759 shutdown_connections 2024-09-12T03:27:47.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.453+0000 7f8cbd632640 1 -- 172.21.15.64:0/4269895759 wait complete. 2024-09-12T03:27:47.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.454+0000 7f8cbd632640 1 Processor -- start 2024-09-12T03:27:47.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.454+0000 7f8cbd632640 1 -- start start 2024-09-12T03:27:47.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.454+0000 7f8cbd632640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cb8100a80 0x7f8cb819a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:47.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.454+0000 7f8cbd632640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cb8101420 0x7f8cb819aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:47.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.454+0000 7f8cbd632640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8cb8106a80 0x7f8cb819ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:47.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.454+0000 7f8cbd632640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb8110d30 con 0x7f8cb8100a80 2024-09-12T03:27:47.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.454+0000 7f8cbd632640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb8110bb0 con 0x7f8cb8101420 2024-09-12T03:27:47.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.454+0000 7f8cbd632640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb8110eb0 con 0x7f8cb8106a80 2024-09-12T03:27:47.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.454+0000 7f8cb6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cb8100a80 0x7f8cb819a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:47.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.454+0000 7f8cb6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cb8100a80 0x7f8cb819a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49642/0 (socket says 172.21.15.64:49642) 2024-09-12T03:27:47.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.454+0000 7f8cb6ffd640 1 -- 172.21.15.64:0/3333641313 learned_addr learned my addr 172.21.15.64:0/3333641313 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:27:47.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.454+0000 7f8caffff640 1 --2- 172.21.15.64:0/3333641313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cb8101420 0x7f8cb819aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:47.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.455+0000 7f8cb6ffd640 1 -- 172.21.15.64:0/3333641313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8cb8106a80 msgr2=0x7f8cb819ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:47.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.455+0000 7f8cb77fe640 1 --2- 172.21.15.64:0/3333641313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8cb8106a80 0x7f8cb819ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:47.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.455+0000 7f8cb6ffd640 1 --2- 172.21.15.64:0/3333641313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8cb8106a80 0x7f8cb819ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.455+0000 7f8cb6ffd640 1 -- 172.21.15.64:0/3333641313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cb8101420 msgr2=0x7f8cb819aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:47.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.455+0000 7f8cb6ffd640 1 --2- 172.21.15.64:0/3333641313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cb8101420 0x7f8cb819aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.455+0000 7f8cb6ffd640 1 -- 172.21.15.64:0/3333641313 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cb819f500 con 0x7f8cb8100a80 2024-09-12T03:27:47.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.455+0000 7f8cb6ffd640 1 --2- 172.21.15.64:0/3333641313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cb8100a80 0x7f8cb819a550 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8ca800c970 tx=0x7f8ca800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:47.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.455+0000 7f8cb4ff9640 1 -- 172.21.15.64:0/3333641313 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca8007bb0 con 0x7f8cb8100a80 2024-09-12T03:27:47.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.455+0000 7f8cbd632640 1 -- 172.21.15.64:0/3333641313 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cb819f7f0 con 0x7f8cb8100a80 2024-09-12T03:27:47.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.455+0000 7f8cb4ff9640 1 -- 172.21.15.64:0/3333641313 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ca8007d50 con 0x7f8cb8100a80 2024-09-12T03:27:47.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.455+0000 7f8cb4ff9640 1 -- 172.21.15.64:0/3333641313 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca800f450 con 0x7f8cb8100a80 2024-09-12T03:27:47.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.455+0000 7f8cbd632640 1 -- 172.21.15.64:0/3333641313 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cb81a70d0 con 0x7f8cb8100a80 2024-09-12T03:27:47.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.456+0000 7f8cbd632640 1 -- 172.21.15.64:0/3333641313 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cb8100ee0 con 0x7f8cb8100a80 2024-09-12T03:27:47.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.457+0000 7f8cb4ff9640 1 -- 172.21.15.64:0/3333641313 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8ca8004f20 con 0x7f8cb8100a80 2024-09-12T03:27:47.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.458+0000 7f8cb4ff9640 1 --2- 172.21.15.64:0/3333641313 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f8ca40a2c10 0x7f8ca40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:47.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.458+0000 7f8caffff640 1 --2- 172.21.15.64:0/3333641313 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f8ca40a2c10 0x7f8ca40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:47.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.458+0000 7f8cb4ff9640 1 -- 172.21.15.64:0/3333641313 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f8ca80cf100 con 0x7f8cb8100a80 2024-09-12T03:27:47.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.458+0000 7f8caffff640 1 --2- 172.21.15.64:0/3333641313 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f8ca40a2c10 0x7f8ca40a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8cb810d760 tx=0x7f8ca000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:47.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.462+0000 7f8cb4ff9640 1 -- 172.21.15.64:0/3333641313 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8ca8014030 con 0x7f8cb8100a80 2024-09-12T03:27:47.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.674+0000 7f8cbd632640 1 -- 172.21.15.64:0/3333641313 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8cb81a7300 con 0x7f8cb8100a80 2024-09-12T03:27:47.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.675+0000 7f8cb4ff9640 1 -- 172.21.15.64:0/3333641313 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11921 (secure 0 0 0) 0x7f8ca809bcc0 con 0x7f8cb8100a80 2024-09-12T03:27:47.675 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:27:47.675 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":24,"btime":"2024-09-12T03:27:37:139728+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2580078738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2580078738},{"type":"v1","addr":"172.21.15.64:6835","nonce":2580078738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2333415004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2333415004},{"type":"v1","addr":"172.21.15.64:6837","nonce":2333415004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/3030618395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":3030618395},{"type":"v1","addr":"172.21.15.64:6839","nonce":3030618395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3333057625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3333057625},{"type":"v1","addr":"172.21.15.64:6841","nonce":3333057625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2991933266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2991933266},{"type":"v1","addr":"172.21.15.195:6833","nonce":2991933266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1138960346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1138960346},{"type":"v1","addr":"172.21.15.115:6833","nonce":1138960346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:26:59.647657+0000","modified":"2024-09-12T03:27:36.865338+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24481,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/332737916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":332737916},{"type":"v1","addr":"172.21.15.115:6835","nonce":332737916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/2531045583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2531045583},{"type":"v1","addr":"172.21.15.195:6835","nonce":2531045583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/71647401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":71647401},{"type":"v1","addr":"172.21.15.195:6837","nonce":71647401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.115:6837/2915155705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2915155705},{"type":"v1","addr":"172.21.15.115:6837","nonce":2915155705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.195:6839/2574695134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":2574695134},{"type":"v1","addr":"172.21.15.195:6839","nonce":2574695134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6839/2238031796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2238031796},{"type":"v1","addr":"172.21.15.115:6839","nonce":2238031796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24470,24481,24500]},"id":1}]} 2024-09-12T03:27:47.675 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 24 2024-09-12T03:27:47.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.677+0000 7f8cbd632640 1 -- 172.21.15.64:0/3333641313 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f8ca40a2c10 msgr2=0x7f8ca40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:47.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.677+0000 7f8cbd632640 1 --2- 172.21.15.64:0/3333641313 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f8ca40a2c10 0x7f8ca40a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8cb810d760 tx=0x7f8ca000b410 comp rx=0 tx=0).stop 2024-09-12T03:27:47.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.677+0000 7f8cbd632640 1 -- 172.21.15.64:0/3333641313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cb8100a80 msgr2=0x7f8cb819a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:47.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.677+0000 7f8cbd632640 1 --2- 172.21.15.64:0/3333641313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cb8100a80 0x7f8cb819a550 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8ca800c970 tx=0x7f8ca800ce30 comp rx=0 tx=0).stop 2024-09-12T03:27:47.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.677+0000 7f8cbd632640 1 -- 172.21.15.64:0/3333641313 shutdown_connections 2024-09-12T03:27:47.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.677+0000 7f8cbd632640 1 --2- 172.21.15.64:0/3333641313 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f8ca40a2c10 0x7f8ca40a50d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.677+0000 7f8cbd632640 1 --2- 172.21.15.64:0/3333641313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8cb8106a80 0x7f8cb819ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.677+0000 7f8cbd632640 1 --2- 172.21.15.64:0/3333641313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cb8101420 0x7f8cb819aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.677+0000 7f8cbd632640 1 --2- 172.21.15.64:0/3333641313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cb8100a80 0x7f8cb819a550 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:47.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.677+0000 7f8cbd632640 1 -- 172.21.15.64:0/3333641313 >> 172.21.15.64:0/3333641313 conn(0x7f8cb80fc7a0 msgr2=0x7f8cb806a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:47.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.677+0000 7f8cbd632640 1 -- 172.21.15.64:0/3333641313 shutdown_connections 2024-09-12T03:27:47.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:47.677+0000 7f8cbd632640 1 -- 172.21.15.64:0/3333641313 wait complete. 2024-09-12T03:27:48.259 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:48 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/3333641313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:27:48.287 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:27:48.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:48 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/3333641313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:27:48.458 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:48 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/3333641313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:27:48.545 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:49.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.076+0000 7f90ad297640 1 -- 172.21.15.64:0/2258252751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90a8101420 msgr2=0x7f90a810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:49.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.076+0000 7f90ad297640 1 --2- 172.21.15.64:0/2258252751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90a8101420 0x7f90a810d950 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009cd0 tx=0x7f90a002a8e0 comp rx=0 tx=0).stop 2024-09-12T03:27:49.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.076+0000 7f90ad297640 1 -- 172.21.15.64:0/2258252751 shutdown_connections 2024-09-12T03:27:49.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.076+0000 7f90ad297640 1 --2- 172.21.15.64:0/2258252751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90a8101420 0x7f90a810d950 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:49.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.076+0000 7f90ad297640 1 --2- 172.21.15.64:0/2258252751 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90a8100a80 0x7f90a8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:49.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.076+0000 7f90ad297640 1 --2- 172.21.15.64:0/2258252751 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90a8106a80 0x7f90a8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:49.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.076+0000 7f90ad297640 1 -- 172.21.15.64:0/2258252751 >> 172.21.15.64:0/2258252751 conn(0x7f90a80fc7a0 msgr2=0x7f90a80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:49.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.076+0000 7f90ad297640 1 -- 172.21.15.64:0/2258252751 shutdown_connections 2024-09-12T03:27:49.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.076+0000 7f90ad297640 1 -- 172.21.15.64:0/2258252751 wait complete. 2024-09-12T03:27:49.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.077+0000 7f90ad297640 1 Processor -- start 2024-09-12T03:27:49.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.077+0000 7f90ad297640 1 -- start start 2024-09-12T03:27:49.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.077+0000 7f90ad297640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90a8100a80 0x7f90a8198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:49.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.077+0000 7f90ad297640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90a8101420 0x7f90a8198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:49.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.077+0000 7f90ad297640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90a8106a80 0x7f90a819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:49.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.077+0000 7f90ad297640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f90a8110d30 con 0x7f90a8101420 2024-09-12T03:27:49.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.077+0000 7f90ad297640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f90a8110bb0 con 0x7f90a8100a80 2024-09-12T03:27:49.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.077+0000 7f90ad297640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f90a8110eb0 con 0x7f90a8106a80 2024-09-12T03:27:49.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.077+0000 7f90a67fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90a8101420 0x7f90a8198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:49.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.077+0000 7f90a67fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90a8101420 0x7f90a8198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51584/0 (socket says 172.21.15.64:51584) 2024-09-12T03:27:49.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.077+0000 7f90a67fc640 1 -- 172.21.15.64:0/2160292327 learned_addr learned my addr 172.21.15.64:0/2160292327 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:27:49.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.077+0000 7f90a6ffd640 1 --2- 172.21.15.64:0/2160292327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90a8100a80 0x7f90a8198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:49.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.077+0000 7f90a77fe640 1 --2- 172.21.15.64:0/2160292327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90a8106a80 0x7f90a819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:49.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.078+0000 7f90a6ffd640 1 -- 172.21.15.64:0/2160292327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90a8106a80 msgr2=0x7f90a819cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:49.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.078+0000 7f90a6ffd640 1 --2- 172.21.15.64:0/2160292327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90a8106a80 0x7f90a819cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:49.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.078+0000 7f90a6ffd640 1 -- 172.21.15.64:0/2160292327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90a8101420 msgr2=0x7f90a8198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:49.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.078+0000 7f90a6ffd640 1 --2- 172.21.15.64:0/2160292327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90a8101420 0x7f90a8198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:49.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.078+0000 7f90a6ffd640 1 -- 172.21.15.64:0/2160292327 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a819d2e0 con 0x7f90a8100a80 2024-09-12T03:27:49.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.078+0000 7f90a67fc640 1 --2- 172.21.15.64:0/2160292327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90a8101420 0x7f90a8198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:27:49.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.078+0000 7f90a6ffd640 1 --2- 172.21.15.64:0/2160292327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90a8100a80 0x7f90a8198330 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f909800d720 tx=0x7f909800dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:49.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.078+0000 7f908ffff640 1 -- 172.21.15.64:0/2160292327 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9098017070 con 0x7f90a8100a80 2024-09-12T03:27:49.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.078+0000 7f90ad297640 1 -- 172.21.15.64:0/2160292327 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90a819d5d0 con 0x7f90a8100a80 2024-09-12T03:27:49.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.078+0000 7f908ffff640 1 -- 172.21.15.64:0/2160292327 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f909800de30 con 0x7f90a8100a80 2024-09-12T03:27:49.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.078+0000 7f908ffff640 1 -- 172.21.15.64:0/2160292327 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90980047a0 con 0x7f90a8100a80 2024-09-12T03:27:49.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.078+0000 7f90ad297640 1 -- 172.21.15.64:0/2160292327 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90a81a4eb0 con 0x7f90a8100a80 2024-09-12T03:27:49.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.079+0000 7f90ad297640 1 -- 172.21.15.64:0/2160292327 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90a8100ee0 con 0x7f90a8100a80 2024-09-12T03:27:49.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.080+0000 7f908ffff640 1 -- 172.21.15.64:0/2160292327 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f90980099d0 con 0x7f90a8100a80 2024-09-12T03:27:49.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.081+0000 7f908ffff640 1 --2- 172.21.15.64:0/2160292327 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f90900a2c10 0x7f90900a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:49.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.081+0000 7f90a67fc640 1 --2- 172.21.15.64:0/2160292327 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f90900a2c10 0x7f90900a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:49.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.081+0000 7f908ffff640 1 -- 172.21.15.64:0/2160292327 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f90980cf260 con 0x7f90a8100a80 2024-09-12T03:27:49.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.081+0000 7f90a67fc640 1 --2- 172.21.15.64:0/2160292327 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f90900a2c10 0x7f90900a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f90a806b7e0 tx=0x7f909c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:49.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.084+0000 7f908ffff640 1 -- 172.21.15.64:0/2160292327 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9098020020 con 0x7f90a8100a80 2024-09-12T03:27:49.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.263+0000 7f90ad297640 1 -- 172.21.15.64:0/2160292327 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f90a81a5080 con 0x7f90a8100a80 2024-09-12T03:27:49.263 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:49 smithi064 ceph-mon[30952]: pgmap v308: 129 pgs: 129 active+clean; 623 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.2 KiB/s wr, 16 op/s 2024-09-12T03:27:49.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.264+0000 7f908ffff640 1 -- 172.21.15.64:0/2160292327 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24209 (secure 0 0 0) 0x7f909809be20 con 0x7f90a8100a80 2024-09-12T03:27:49.265 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:27:49.265 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":88,"fsid":"b0812460-70b5-11ef-bcea-c7b262605968","created":"2024-09-12T03:19:15.900322+0000","modified":"2024-09-12T03:27:08.680818+0000","last_up_change":"2024-09-12T03:26:06.122971+0000","last_in_change":"2024-09-12T03:25:43.130689+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:22:21.915870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:26:55.826348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:26:57.984319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T03:27:04.460292+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"326271c9-1f28-4ce6-b3be-fcc57479d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6803","nonce":1307832590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6805","nonce":1307832590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6809","nonce":1307832590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6807","nonce":1307832590}]},"public_addr":"172.21.15.64:6803/1307832590","cluster_addr":"172.21.15.64:6805/1307832590","heartbeat_back_addr":"172.21.15.64:6809/1307832590","heartbeat_front_addr":"172.21.15.64:6807/1307832590","state":["exists","up"]},{"osd":1,"uuid":"7dca0c28-c234-423c-936e-2351d63f918e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6801","nonce":3123728622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6803","nonce":3123728622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6807","nonce":3123728622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6805","nonce":3123728622}]},"public_addr":"172.21.15.115:6801/3123728622","cluster_addr":"172.21.15.115:6803/3123728622","heartbeat_back_addr":"172.21.15.115:6807/3123728622","heartbeat_front_addr":"172.21.15.115:6805/3123728622","state":["exists","up"]},{"osd":2,"uuid":"4b3a3c7a-aec6-455a-9a9a-1ac9cc900f24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6801","nonce":1664125451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6803","nonce":1664125451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6807","nonce":1664125451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6805","nonce":1664125451}]},"public_addr":"172.21.15.195:6801/1664125451","cluster_addr":"172.21.15.195:6803/1664125451","heartbeat_back_addr":"172.21.15.195:6807/1664125451","heartbeat_front_addr":"172.21.15.195:6805/1664125451","state":["exists","up"]},{"osd":3,"uuid":"8768c339-2c1e-4294-8b78-3c2adab6750f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6811","nonce":3935517185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6813","nonce":3935517185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6817","nonce":3935517185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6815","nonce":3935517185}]},"public_addr":"172.21.15.64:6811/3935517185","cluster_addr":"172.21.15.64:6813/3935517185","heartbeat_back_addr":"172.21.15.64:6817/3935517185","heartbeat_front_addr":"172.21.15.64:6815/3935517185","state":["exists","up"]},{"osd":4,"uuid":"123aac47-a8ae-47c7-93a7-0dd293356322","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6809","nonce":446665227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6811","nonce":446665227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6815","nonce":446665227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6813","nonce":446665227}]},"public_addr":"172.21.15.115:6809/446665227","cluster_addr":"172.21.15.115:6811/446665227","heartbeat_back_addr":"172.21.15.115:6815/446665227","heartbeat_front_addr":"172.21.15.115:6813/446665227","state":["exists","up"]},{"osd":5,"uuid":"c90bcaed-e7a8-4852-8677-b01dc73ea44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6809","nonce":1187293668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6811","nonce":1187293668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6815","nonce":1187293668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6813","nonce":1187293668}]},"public_addr":"172.21.15.195:6809/1187293668","cluster_addr":"172.21.15.195:6811/1187293668","heartbeat_back_addr":"172.21.15.195:6815/1187293668","heartbeat_front_addr":"172.21.15.195:6813/1187293668","state":["exists","up"]},{"osd":6,"uuid":"fd292547-6695-4889-8867-e74318bbb00f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6819","nonce":560343274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6821","nonce":560343274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6825","nonce":560343274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6823","nonce":560343274}]},"public_addr":"172.21.15.64:6819/560343274","cluster_addr":"172.21.15.64:6821/560343274","heartbeat_back_addr":"172.21.15.64:6825/560343274","heartbeat_front_addr":"172.21.15.64:6823/560343274","state":["exists","up"]},{"osd":7,"uuid":"ad7c4151-cd98-4113-879b-ca2b7c83c7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6817","nonce":3075734174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6819","nonce":3075734174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6823","nonce":3075734174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6821","nonce":3075734174}]},"public_addr":"172.21.15.115:6817/3075734174","cluster_addr":"172.21.15.115:6819/3075734174","heartbeat_back_addr":"172.21.15.115:6823/3075734174","heartbeat_front_addr":"172.21.15.115:6821/3075734174","state":["exists","up"]},{"osd":8,"uuid":"6de50049-c28f-4f21-b123-05e55bf59127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6817","nonce":2184742494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6819","nonce":2184742494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6823","nonce":2184742494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6821","nonce":2184742494}]},"public_addr":"172.21.15.195:6817/2184742494","cluster_addr":"172.21.15.195:6819/2184742494","heartbeat_back_addr":"172.21.15.195:6823/2184742494","heartbeat_front_addr":"172.21.15.195:6821/2184742494","state":["exists","up"]},{"osd":9,"uuid":"36ab792c-58f2-4beb-9024-7f3d1e3785dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6827","nonce":561728112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6829","nonce":561728112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6833","nonce":561728112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6831","nonce":561728112}]},"public_addr":"172.21.15.64:6827/561728112","cluster_addr":"172.21.15.64:6829/561728112","heartbeat_back_addr":"172.21.15.64:6833/561728112","heartbeat_front_addr":"172.21.15.64:6831/561728112","state":["exists","up"]},{"osd":10,"uuid":"d1f7881c-ec3f-466e-9cea-1d5cb441d970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6825","nonce":2977887627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6827","nonce":2977887627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6831","nonce":2977887627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6829","nonce":2977887627}]},"public_addr":"172.21.15.115:6825/2977887627","cluster_addr":"172.21.15.115:6827/2977887627","heartbeat_back_addr":"172.21.15.115:6831/2977887627","heartbeat_front_addr":"172.21.15.115:6829/2977887627","state":["exists","up"]},{"osd":11,"uuid":"7c46c028-22c7-450f-9927-2144a7a09e9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6825","nonce":591599352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6827","nonce":591599352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6831","nonce":591599352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6829","nonce":591599352}]},"public_addr":"172.21.15.195:6825/591599352","cluster_addr":"172.21.15.195:6827/591599352","heartbeat_back_addr":"172.21.15.195:6831/591599352","heartbeat_front_addr":"172.21.15.195:6829/591599352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:29.151454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:54.308909+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:18.680995+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:44.687548+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:10.166131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:35.026132+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:24:00.242966+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:24:24.600848+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:24:49.563060+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:25:15.025730+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:25:39.830995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:26:04.394276+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6801/68603379":"2024-09-13T03:20:07.411102+0000","172.21.15.64:0/3827202979":"2024-09-13T03:20:07.411102+0000","172.21.15.64:6800/68603379":"2024-09-13T03:20:07.411102+0000","172.21.15.64:0/1217136221":"2024-09-13T03:20:07.411102+0000","172.21.15.64:0/3079679964":"2024-09-13T03:19:36.839431+0000","172.21.15.64:6801/3186640984":"2024-09-13T03:19:36.839431+0000","172.21.15.64:0/3334629354":"2024-09-13T03:20:07.411102+0000","172.21.15.64:6800/3186640984":"2024-09-13T03:19:36.839431+0000","172.21.15.64:0/3564418115":"2024-09-13T03:19:36.839431+0000","172.21.15.64:0/686137382":"2024-09-13T03:19:36.839431+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:27:49.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.267+0000 7f90ad297640 1 -- 172.21.15.64:0/2160292327 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f90900a2c10 msgr2=0x7f90900a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:49.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.267+0000 7f90ad297640 1 --2- 172.21.15.64:0/2160292327 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f90900a2c10 0x7f90900a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f90a806b7e0 tx=0x7f909c005f90 comp rx=0 tx=0).stop 2024-09-12T03:27:49.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.267+0000 7f90ad297640 1 -- 172.21.15.64:0/2160292327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90a8100a80 msgr2=0x7f90a8198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:49.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.267+0000 7f90ad297640 1 --2- 172.21.15.64:0/2160292327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90a8100a80 0x7f90a8198330 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f909800d720 tx=0x7f909800dbe0 comp rx=0 tx=0).stop 2024-09-12T03:27:49.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.267+0000 7f90ad297640 1 -- 172.21.15.64:0/2160292327 shutdown_connections 2024-09-12T03:27:49.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.267+0000 7f90ad297640 1 --2- 172.21.15.64:0/2160292327 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f90900a2c10 0x7f90900a50d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:49.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.267+0000 7f90ad297640 1 --2- 172.21.15.64:0/2160292327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90a8106a80 0x7f90a819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:49.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.267+0000 7f90ad297640 1 --2- 172.21.15.64:0/2160292327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90a8101420 0x7f90a8198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:49.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.267+0000 7f90ad297640 1 --2- 172.21.15.64:0/2160292327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90a8100a80 0x7f90a8198330 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:49.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.267+0000 7f90ad297640 1 -- 172.21.15.64:0/2160292327 >> 172.21.15.64:0/2160292327 conn(0x7f90a80fc7a0 msgr2=0x7f90a8104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:49.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.267+0000 7f90ad297640 1 -- 172.21.15.64:0/2160292327 shutdown_connections 2024-09-12T03:27:49.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:49.267+0000 7f90ad297640 1 -- 172.21.15.64:0/2160292327 wait complete. 2024-09-12T03:27:49.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:49 smithi195 ceph-mon[34786]: pgmap v308: 129 pgs: 129 active+clean; 623 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.2 KiB/s wr, 16 op/s 2024-09-12T03:27:49.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:49 smithi115 ceph-mon[34801]: pgmap v308: 129 pgs: 129 active+clean; 623 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.2 KiB/s wr, 16 op/s 2024-09-12T03:27:49.902 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-12T03:27:49.902 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T03:27:50.162 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:50.448 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:50 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/2160292327' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:27:50.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:50 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/2160292327' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:27:50.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:50 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/2160292327' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:27:50.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.670+0000 7fefdd497640 1 -- 172.21.15.64:0/1317795241 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefd8101de0 msgr2=0x7fefd81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:50.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.670+0000 7fefdd497640 1 --2- 172.21.15.64:0/1317795241 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefd8101de0 0x7fefd81021c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fefc0009900 tx=0x7fefc002a410 comp rx=0 tx=0).stop 2024-09-12T03:27:50.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.670+0000 7fefdd497640 1 -- 172.21.15.64:0/1317795241 shutdown_connections 2024-09-12T03:27:50.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.670+0000 7fefdd497640 1 --2- 172.21.15.64:0/1317795241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefd8106f10 0x7fefd810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:50.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.670+0000 7fefdd497640 1 --2- 172.21.15.64:0/1317795241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fefd8102790 0x7fefd81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:50.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.670+0000 7fefdd497640 1 --2- 172.21.15.64:0/1317795241 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefd8101de0 0x7fefd81021c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:50.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.670+0000 7fefdd497640 1 -- 172.21.15.64:0/1317795241 >> 172.21.15.64:0/1317795241 conn(0x7fefd80fd8c0 msgr2=0x7fefd80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:50.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.670+0000 7fefdd497640 1 -- 172.21.15.64:0/1317795241 shutdown_connections 2024-09-12T03:27:50.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.670+0000 7fefdd497640 1 -- 172.21.15.64:0/1317795241 wait complete. 2024-09-12T03:27:50.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.670+0000 7fefdd497640 1 Processor -- start 2024-09-12T03:27:50.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.671+0000 7fefdd497640 1 -- start start 2024-09-12T03:27:50.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.671+0000 7fefdd497640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefd8101de0 0x7fefd8198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:50.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.671+0000 7fefdd497640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fefd8102790 0x7fefd81988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:50.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.671+0000 7fefdd497640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefd8106f10 0x7fefd819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:50.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.671+0000 7fefdd497640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fefd8111e70 con 0x7fefd8106f10 2024-09-12T03:27:50.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.671+0000 7fefdd497640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fefd8111cf0 con 0x7fefd8102790 2024-09-12T03:27:50.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.671+0000 7fefdd497640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fefd8111ff0 con 0x7fefd8101de0 2024-09-12T03:27:50.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.671+0000 7fefd77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefd8106f10 0x7fefd819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:50.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.671+0000 7fefd77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefd8106f10 0x7fefd819cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51596/0 (socket says 172.21.15.64:51596) 2024-09-12T03:27:50.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.671+0000 7fefd77fe640 1 -- 172.21.15.64:0/2901739856 learned_addr learned my addr 172.21.15.64:0/2901739856 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:27:50.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.671+0000 7fefd6ffd640 1 --2- 172.21.15.64:0/2901739856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefd8101de0 0x7fefd8198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:50.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.671+0000 7fefd67fc640 1 --2- 172.21.15.64:0/2901739856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fefd8102790 0x7fefd81988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:50.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.671+0000 7fefd77fe640 1 -- 172.21.15.64:0/2901739856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefd8101de0 msgr2=0x7fefd8198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:50.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.671+0000 7fefd77fe640 1 --2- 172.21.15.64:0/2901739856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefd8101de0 0x7fefd8198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:50.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.671+0000 7fefd77fe640 1 -- 172.21.15.64:0/2901739856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fefd8102790 msgr2=0x7fefd81988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:50.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.671+0000 7fefd77fe640 1 --2- 172.21.15.64:0/2901739856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fefd8102790 0x7fefd81988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:50.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.671+0000 7fefd77fe640 1 -- 172.21.15.64:0/2901739856 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd819d310 con 0x7fefd8106f10 2024-09-12T03:27:50.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.671+0000 7fefd6ffd640 1 --2- 172.21.15.64:0/2901739856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefd8101de0 0x7fefd8198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:27:50.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.671+0000 7fefd77fe640 1 --2- 172.21.15.64:0/2901739856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefd8106f10 0x7fefd819cc30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fefd000eaa0 tx=0x7fefd000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:50.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.672+0000 7fefcffff640 1 -- 172.21.15.64:0/2901739856 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd000cdb0 con 0x7fefd8106f10 2024-09-12T03:27:50.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.672+0000 7fefcffff640 1 -- 172.21.15.64:0/2901739856 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fefd0005020 con 0x7fefd8106f10 2024-09-12T03:27:50.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.672+0000 7fefdd497640 1 -- 172.21.15.64:0/2901739856 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefd819d600 con 0x7fefd8106f10 2024-09-12T03:27:50.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.672+0000 7fefcffff640 1 -- 172.21.15.64:0/2901739856 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd00116d0 con 0x7fefd8106f10 2024-09-12T03:27:50.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.672+0000 7fefdd497640 1 -- 172.21.15.64:0/2901739856 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefd81a4ee0 con 0x7fefd8106f10 2024-09-12T03:27:50.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.673+0000 7fefdd497640 1 -- 172.21.15.64:0/2901739856 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefd81021c0 con 0x7fefd8106f10 2024-09-12T03:27:50.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.674+0000 7fefcffff640 1 -- 172.21.15.64:0/2901739856 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fefd00051d0 con 0x7fefd8106f10 2024-09-12T03:27:50.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.674+0000 7fefcffff640 1 --2- 172.21.15.64:0/2901739856 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fefc40a2c90 0x7fefc40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:50.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.675+0000 7fefd6ffd640 1 --2- 172.21.15.64:0/2901739856 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fefc40a2c90 0x7fefc40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:50.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.675+0000 7fefcffff640 1 -- 172.21.15.64:0/2901739856 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fefd00cf580 con 0x7fefd8106f10 2024-09-12T03:27:50.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.675+0000 7fefd6ffd640 1 --2- 172.21.15.64:0/2901739856 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fefc40a2c90 0x7fefc40a5150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fefc0000c00 tx=0x7fefc0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:50.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.679+0000 7fefcffff640 1 -- 172.21.15.64:0/2901739856 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fefd0016030 con 0x7fefd8106f10 2024-09-12T03:27:50.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.852+0000 7fefdd497640 1 -- 172.21.15.64:0/2901739856 --> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fefd8062630 con 0x7fefc40a2c90 2024-09-12T03:27:50.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.861+0000 7fefcffff640 1 -- 172.21.15.64:0/2901739856 <== mgr.14150 v2:172.21.15.64:6800/759931848 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340591 (secure 0 0 0) 0x7fefd8062630 con 0x7fefc40a2c90 2024-09-12T03:27:50.867 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:27:50.869 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-09-12T03:27:50.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.864+0000 7fefdd497640 1 -- 172.21.15.64:0/2901739856 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fefc40a2c90 msgr2=0x7fefc40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:50.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.864+0000 7fefdd497640 1 --2- 172.21.15.64:0/2901739856 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fefc40a2c90 0x7fefc40a5150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fefc0000c00 tx=0x7fefc0005a20 comp rx=0 tx=0).stop 2024-09-12T03:27:50.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.864+0000 7fefdd497640 1 -- 172.21.15.64:0/2901739856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefd8106f10 msgr2=0x7fefd819cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:50.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.864+0000 7fefdd497640 1 --2- 172.21.15.64:0/2901739856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefd8106f10 0x7fefd819cc30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fefd000eaa0 tx=0x7fefd000ef60 comp rx=0 tx=0).stop 2024-09-12T03:27:50.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.864+0000 7fefdd497640 1 -- 172.21.15.64:0/2901739856 shutdown_connections 2024-09-12T03:27:50.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.864+0000 7fefdd497640 1 --2- 172.21.15.64:0/2901739856 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fefc40a2c90 0x7fefc40a5150 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:50.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.864+0000 7fefdd497640 1 --2- 172.21.15.64:0/2901739856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefd8106f10 0x7fefd819cc30 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:50.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.864+0000 7fefdd497640 1 --2- 172.21.15.64:0/2901739856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fefd8102790 0x7fefd81988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:50.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.864+0000 7fefdd497640 1 --2- 172.21.15.64:0/2901739856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefd8101de0 0x7fefd8198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:50.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.865+0000 7fefdd497640 1 -- 172.21.15.64:0/2901739856 >> 172.21.15.64:0/2901739856 conn(0x7fefd80fd8c0 msgr2=0x7fefd80fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:50.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.865+0000 7fefdd497640 1 -- 172.21.15.64:0/2901739856 shutdown_connections 2024-09-12T03:27:50.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:50.865+0000 7fefdd497640 1 -- 172.21.15.64:0/2901739856 wait complete. 2024-09-12T03:27:51.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:51 smithi195 ceph-mon[34786]: pgmap v309: 129 pgs: 129 active+clean; 623 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.0 KiB/s wr, 9 op/s 2024-09-12T03:27:51.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:51 smithi064 ceph-mon[30952]: pgmap v309: 129 pgs: 129 active+clean; 623 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.0 KiB/s wr, 9 op/s 2024-09-12T03:27:51.589 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":309,"stamp":"2024-09-12T03:27:49.503601+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":637950,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":235,"num_read_kb":220,"num_write":341,"num_write_kb":3033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":274,"ondisk_log_size":274,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365176,"kb_used_data":11768,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576648,"statfs":{"total":1151940427776,"available":1151566487552,"internally_reserved":0,"allocated":12050432,"data_stored":7762760,"data_compressed":318972,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":40840,"num_objects":11,"num_object_clones":0,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":77,"num_read_kb":80,"num_write":42,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003326"},"pg_stats":[{"pgid":"2.1f","version":"88'9","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:34.143755+0000","last_change":"2024-09-12T03:26:56.956415+0000","last_active":"2024-09-12T03:27:34.143755+0000","last_peered":"2024-09-12T03:27:34.143755+0000","last_clean":"2024-09-12T03:27:34.143755+0000","last_became_active":"2024-09-12T03:26:56.956089+0000","last_became_peered":"2024-09-12T03:26:56.956089+0000","last_unstale":"2024-09-12T03:27:34.143755+0000","last_undegraded":"2024-09-12T03:27:34.143755+0000","last_fullsized":"2024-09-12T03:27:34.143755+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:32:22.089322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059577+0000","last_change":"2024-09-12T03:26:59.024102+0000","last_active":"2024-09-12T03:27:10.059577+0000","last_peered":"2024-09-12T03:27:10.059577+0000","last_clean":"2024-09-12T03:27:10.059577+0000","last_became_active":"2024-09-12T03:26:59.023795+0000","last_became_peered":"2024-09-12T03:26:59.023795+0000","last_unstale":"2024-09-12T03:27:10.059577+0000","last_undegraded":"2024-09-12T03:27:10.059577+0000","last_fullsized":"2024-09-12T03:27:10.059577+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:25:55.392622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059419+0000","last_change":"2024-09-12T03:27:06.052565+0000","last_active":"2024-09-12T03:27:10.059419+0000","last_peered":"2024-09-12T03:27:10.059419+0000","last_clean":"2024-09-12T03:27:10.059419+0000","last_became_active":"2024-09-12T03:27:06.052209+0000","last_became_peered":"2024-09-12T03:27:06.052209+0000","last_unstale":"2024-09-12T03:27:10.059419+0000","last_undegraded":"2024-09-12T03:27:10.059419+0000","last_fullsized":"2024-09-12T03:27:10.059419+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:24:28.153740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059776+0000","last_change":"2024-09-12T03:26:57.018418+0000","last_active":"2024-09-12T03:27:10.059776+0000","last_peered":"2024-09-12T03:27:10.059776+0000","last_clean":"2024-09-12T03:27:10.059776+0000","last_became_active":"2024-09-12T03:26:57.018081+0000","last_became_peered":"2024-09-12T03:26:57.018081+0000","last_unstale":"2024-09-12T03:27:10.059776+0000","last_undegraded":"2024-09-12T03:27:10.059776+0000","last_fullsized":"2024-09-12T03:27:10.059776+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:19.954332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059140+0000","last_change":"2024-09-12T03:26:59.019632+0000","last_active":"2024-09-12T03:27:10.059140+0000","last_peered":"2024-09-12T03:27:10.059140+0000","last_clean":"2024-09-12T03:27:10.059140+0000","last_became_active":"2024-09-12T03:26:59.016847+0000","last_became_peered":"2024-09-12T03:26:59.016847+0000","last_unstale":"2024-09-12T03:27:10.059140+0000","last_undegraded":"2024-09-12T03:27:10.059140+0000","last_fullsized":"2024-09-12T03:27:10.059140+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:32:28.261954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060327+0000","last_change":"2024-09-12T03:27:06.045628+0000","last_active":"2024-09-12T03:27:10.060327+0000","last_peered":"2024-09-12T03:27:10.060327+0000","last_clean":"2024-09-12T03:27:10.060327+0000","last_became_active":"2024-09-12T03:27:06.045141+0000","last_became_peered":"2024-09-12T03:27:06.045141+0000","last_unstale":"2024-09-12T03:27:10.060327+0000","last_undegraded":"2024-09-12T03:27:10.060327+0000","last_fullsized":"2024-09-12T03:27:10.060327+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:10:26.154446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:34.141754+0000","last_change":"2024-09-12T03:26:56.958615+0000","last_active":"2024-09-12T03:27:34.141754+0000","last_peered":"2024-09-12T03:27:34.141754+0000","last_clean":"2024-09-12T03:27:34.141754+0000","last_became_active":"2024-09-12T03:26:56.958295+0000","last_became_peered":"2024-09-12T03:26:56.958295+0000","last_unstale":"2024-09-12T03:27:34.141754+0000","last_undegraded":"2024-09-12T03:27:34.141754+0000","last_fullsized":"2024-09-12T03:27:34.141754+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:13:09.031983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059797+0000","last_change":"2024-09-12T03:26:59.015376+0000","last_active":"2024-09-12T03:27:10.059797+0000","last_peered":"2024-09-12T03:27:10.059797+0000","last_clean":"2024-09-12T03:27:10.059797+0000","last_became_active":"2024-09-12T03:26:59.015055+0000","last_became_peered":"2024-09-12T03:26:59.015055+0000","last_unstale":"2024-09-12T03:27:10.059797+0000","last_undegraded":"2024-09-12T03:27:10.059797+0000","last_fullsized":"2024-09-12T03:27:10.059797+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:44:38.077558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.061132+0000","last_change":"2024-09-12T03:27:06.042785+0000","last_active":"2024-09-12T03:27:10.061132+0000","last_peered":"2024-09-12T03:27:10.061132+0000","last_clean":"2024-09-12T03:27:10.061132+0000","last_became_active":"2024-09-12T03:27:06.042379+0000","last_became_peered":"2024-09-12T03:27:06.042379+0000","last_unstale":"2024-09-12T03:27:10.061132+0000","last_undegraded":"2024-09-12T03:27:10.061132+0000","last_fullsized":"2024-09-12T03:27:10.061132+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:08:51.683470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.958667+0000","last_change":"2024-09-12T03:26:56.955846+0000","last_active":"2024-09-12T03:27:09.958667+0000","last_peered":"2024-09-12T03:27:09.958667+0000","last_clean":"2024-09-12T03:27:09.958667+0000","last_became_active":"2024-09-12T03:26:56.955476+0000","last_became_peered":"2024-09-12T03:26:56.955476+0000","last_unstale":"2024-09-12T03:27:09.958667+0000","last_undegraded":"2024-09-12T03:27:09.958667+0000","last_fullsized":"2024-09-12T03:27:09.958667+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:10:46.811039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060357+0000","last_change":"2024-09-12T03:26:59.021498+0000","last_active":"2024-09-12T03:27:10.060357+0000","last_peered":"2024-09-12T03:27:10.060357+0000","last_clean":"2024-09-12T03:27:10.060357+0000","last_became_active":"2024-09-12T03:26:59.021196+0000","last_became_peered":"2024-09-12T03:26:59.021196+0000","last_unstale":"2024-09-12T03:27:10.060357+0000","last_undegraded":"2024-09-12T03:27:10.060357+0000","last_fullsized":"2024-09-12T03:27:10.060357+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:13:17.537648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060494+0000","last_change":"2024-09-12T03:27:06.040597+0000","last_active":"2024-09-12T03:27:10.060494+0000","last_peered":"2024-09-12T03:27:10.060494+0000","last_clean":"2024-09-12T03:27:10.060494+0000","last_became_active":"2024-09-12T03:27:06.040149+0000","last_became_peered":"2024-09-12T03:27:06.040149+0000","last_unstale":"2024-09-12T03:27:10.060494+0000","last_undegraded":"2024-09-12T03:27:10.060494+0000","last_fullsized":"2024-09-12T03:27:10.060494+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:27.192367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060247+0000","last_change":"2024-09-12T03:26:59.020542+0000","last_active":"2024-09-12T03:27:10.060247+0000","last_peered":"2024-09-12T03:27:10.060247+0000","last_clean":"2024-09-12T03:27:10.060247+0000","last_became_active":"2024-09-12T03:26:59.020189+0000","last_became_peered":"2024-09-12T03:26:59.020189+0000","last_unstale":"2024-09-12T03:27:10.060247+0000","last_undegraded":"2024-09-12T03:27:10.060247+0000","last_fullsized":"2024-09-12T03:27:10.060247+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:06:12.761331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:33.133816+0000","last_change":"2024-09-12T03:26:56.958320+0000","last_active":"2024-09-12T03:27:33.133816+0000","last_peered":"2024-09-12T03:27:33.133816+0000","last_clean":"2024-09-12T03:27:33.133816+0000","last_became_active":"2024-09-12T03:26:56.956965+0000","last_became_peered":"2024-09-12T03:26:56.956965+0000","last_unstale":"2024-09-12T03:27:33.133816+0000","last_undegraded":"2024-09-12T03:27:33.133816+0000","last_fullsized":"2024-09-12T03:27:33.133816+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:46.685994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060669+0000","last_change":"2024-09-12T03:27:06.189973+0000","last_active":"2024-09-12T03:27:10.060669+0000","last_peered":"2024-09-12T03:27:10.060669+0000","last_clean":"2024-09-12T03:27:10.060669+0000","last_became_active":"2024-09-12T03:27:06.189454+0000","last_became_peered":"2024-09-12T03:27:06.189454+0000","last_unstale":"2024-09-12T03:27:10.060669+0000","last_undegraded":"2024-09-12T03:27:10.060669+0000","last_fullsized":"2024-09-12T03:27:10.060669+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:25.130566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059324+0000","last_change":"2024-09-12T03:26:59.014586+0000","last_active":"2024-09-12T03:27:10.059324+0000","last_peered":"2024-09-12T03:27:10.059324+0000","last_clean":"2024-09-12T03:27:10.059324+0000","last_became_active":"2024-09-12T03:26:59.013764+0000","last_became_peered":"2024-09-12T03:26:59.013764+0000","last_unstale":"2024-09-12T03:27:10.059324+0000","last_undegraded":"2024-09-12T03:27:10.059324+0000","last_fullsized":"2024-09-12T03:27:10.059324+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:55:34.136770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059248+0000","last_change":"2024-09-12T03:26:56.960415+0000","last_active":"2024-09-12T03:27:10.059248+0000","last_peered":"2024-09-12T03:27:10.059248+0000","last_clean":"2024-09-12T03:27:10.059248+0000","last_became_active":"2024-09-12T03:26:56.960043+0000","last_became_peered":"2024-09-12T03:26:56.960043+0000","last_unstale":"2024-09-12T03:27:10.059248+0000","last_undegraded":"2024-09-12T03:27:10.059248+0000","last_fullsized":"2024-09-12T03:27:10.059248+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:02:12.209929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060777+0000","last_change":"2024-09-12T03:27:06.047002+0000","last_active":"2024-09-12T03:27:10.060777+0000","last_peered":"2024-09-12T03:27:10.060777+0000","last_clean":"2024-09-12T03:27:10.060777+0000","last_became_active":"2024-09-12T03:27:06.046520+0000","last_became_peered":"2024-09-12T03:27:06.046520+0000","last_unstale":"2024-09-12T03:27:10.060777+0000","last_undegraded":"2024-09-12T03:27:10.060777+0000","last_fullsized":"2024-09-12T03:27:10.060777+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:30.332831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.054604+0000","last_change":"2024-09-12T03:26:59.016811+0000","last_active":"2024-09-12T03:27:09.054604+0000","last_peered":"2024-09-12T03:27:09.054604+0000","last_clean":"2024-09-12T03:27:09.054604+0000","last_became_active":"2024-09-12T03:26:59.014667+0000","last_became_peered":"2024-09-12T03:26:59.014667+0000","last_unstale":"2024-09-12T03:27:09.054604+0000","last_undegraded":"2024-09-12T03:27:09.054604+0000","last_fullsized":"2024-09-12T03:27:09.054604+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:28:51.128895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:33.132610+0000","last_change":"2024-09-12T03:26:56.956177+0000","last_active":"2024-09-12T03:27:33.132610+0000","last_peered":"2024-09-12T03:27:33.132610+0000","last_clean":"2024-09-12T03:27:33.132610+0000","last_became_active":"2024-09-12T03:26:56.955730+0000","last_became_peered":"2024-09-12T03:26:56.955730+0000","last_unstale":"2024-09-12T03:27:33.132610+0000","last_undegraded":"2024-09-12T03:27:33.132610+0000","last_fullsized":"2024-09-12T03:27:33.132610+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:05:18.890917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.053901+0000","last_change":"2024-09-12T03:27:06.050894+0000","last_active":"2024-09-12T03:27:09.053901+0000","last_peered":"2024-09-12T03:27:09.053901+0000","last_clean":"2024-09-12T03:27:09.053901+0000","last_became_active":"2024-09-12T03:27:06.050440+0000","last_became_peered":"2024-09-12T03:27:06.050440+0000","last_unstale":"2024-09-12T03:27:09.053901+0000","last_undegraded":"2024-09-12T03:27:09.053901+0000","last_fullsized":"2024-09-12T03:27:09.053901+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:31:25.559665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.958146+0000","last_change":"2024-09-12T03:26:59.018167+0000","last_active":"2024-09-12T03:27:09.958146+0000","last_peered":"2024-09-12T03:27:09.958146+0000","last_clean":"2024-09-12T03:27:09.958146+0000","last_became_active":"2024-09-12T03:26:59.017718+0000","last_became_peered":"2024-09-12T03:26:59.017718+0000","last_unstale":"2024-09-12T03:27:09.958146+0000","last_undegraded":"2024-09-12T03:27:09.958146+0000","last_fullsized":"2024-09-12T03:27:09.958146+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:36:12.742570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:33.133529+0000","last_change":"2024-09-12T03:26:57.018829+0000","last_active":"2024-09-12T03:27:33.133529+0000","last_peered":"2024-09-12T03:27:33.133529+0000","last_clean":"2024-09-12T03:27:33.133529+0000","last_became_active":"2024-09-12T03:26:57.018467+0000","last_became_peered":"2024-09-12T03:26:57.018467+0000","last_unstale":"2024-09-12T03:27:33.133529+0000","last_undegraded":"2024-09-12T03:27:33.133529+0000","last_fullsized":"2024-09-12T03:27:33.133529+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:33:37.318703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060987+0000","last_change":"2024-09-12T03:27:06.190560+0000","last_active":"2024-09-12T03:27:10.060987+0000","last_peered":"2024-09-12T03:27:10.060987+0000","last_clean":"2024-09-12T03:27:10.060987+0000","last_became_active":"2024-09-12T03:27:06.190008+0000","last_became_peered":"2024-09-12T03:27:06.190008+0000","last_unstale":"2024-09-12T03:27:10.060987+0000","last_undegraded":"2024-09-12T03:27:10.060987+0000","last_fullsized":"2024-09-12T03:27:10.060987+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:14.423372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059131+0000","last_change":"2024-09-12T03:26:59.022149+0000","last_active":"2024-09-12T03:27:10.059131+0000","last_peered":"2024-09-12T03:27:10.059131+0000","last_clean":"2024-09-12T03:27:10.059131+0000","last_became_active":"2024-09-12T03:26:59.021830+0000","last_became_peered":"2024-09-12T03:26:59.021830+0000","last_unstale":"2024-09-12T03:27:10.059131+0000","last_undegraded":"2024-09-12T03:27:10.059131+0000","last_fullsized":"2024-09-12T03:27:10.059131+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:07.096908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:33.130934+0000","last_change":"2024-09-12T03:26:57.018335+0000","last_active":"2024-09-12T03:27:33.130934+0000","last_peered":"2024-09-12T03:27:33.130934+0000","last_clean":"2024-09-12T03:27:33.130934+0000","last_became_active":"2024-09-12T03:26:57.017993+0000","last_became_peered":"2024-09-12T03:26:57.017993+0000","last_unstale":"2024-09-12T03:27:33.130934+0000","last_undegraded":"2024-09-12T03:27:33.130934+0000","last_fullsized":"2024-09-12T03:27:33.130934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:27:52.828836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059108+0000","last_change":"2024-09-12T03:27:06.051146+0000","last_active":"2024-09-12T03:27:10.059108+0000","last_peered":"2024-09-12T03:27:10.059108+0000","last_clean":"2024-09-12T03:27:10.059108+0000","last_became_active":"2024-09-12T03:27:06.050599+0000","last_became_peered":"2024-09-12T03:27:06.050599+0000","last_unstale":"2024-09-12T03:27:10.059108+0000","last_undegraded":"2024-09-12T03:27:10.059108+0000","last_fullsized":"2024-09-12T03:27:10.059108+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:17:57.543994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059857+0000","last_change":"2024-09-12T03:26:59.024832+0000","last_active":"2024-09-12T03:27:10.059857+0000","last_peered":"2024-09-12T03:27:10.059857+0000","last_clean":"2024-09-12T03:27:10.059857+0000","last_became_active":"2024-09-12T03:26:59.024473+0000","last_became_peered":"2024-09-12T03:26:59.024473+0000","last_unstale":"2024-09-12T03:27:10.059857+0000","last_undegraded":"2024-09-12T03:27:10.059857+0000","last_fullsized":"2024-09-12T03:27:10.059857+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:05:54.568496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'10","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:45.013576+0000","last_change":"2024-09-12T03:26:56.955213+0000","last_active":"2024-09-12T03:27:45.013576+0000","last_peered":"2024-09-12T03:27:45.013576+0000","last_clean":"2024-09-12T03:27:45.013576+0000","last_became_active":"2024-09-12T03:26:56.954746+0000","last_became_peered":"2024-09-12T03:26:56.954746+0000","last_unstale":"2024-09-12T03:27:45.013576+0000","last_undegraded":"2024-09-12T03:27:45.013576+0000","last_fullsized":"2024-09-12T03:27:45.013576+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:03.303931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.058609+0000","last_change":"2024-09-12T03:27:06.190398+0000","last_active":"2024-09-12T03:27:10.058609+0000","last_peered":"2024-09-12T03:27:10.058609+0000","last_clean":"2024-09-12T03:27:10.058609+0000","last_became_active":"2024-09-12T03:27:06.189861+0000","last_became_peered":"2024-09-12T03:27:06.189861+0000","last_unstale":"2024-09-12T03:27:10.058609+0000","last_undegraded":"2024-09-12T03:27:10.058609+0000","last_fullsized":"2024-09-12T03:27:10.058609+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:36:57.366931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.015434+0000","last_change":"2024-09-12T03:26:59.012702+0000","last_active":"2024-09-12T03:27:10.015434+0000","last_peered":"2024-09-12T03:27:10.015434+0000","last_clean":"2024-09-12T03:27:10.015434+0000","last_became_active":"2024-09-12T03:26:59.012421+0000","last_became_peered":"2024-09-12T03:26:59.012421+0000","last_unstale":"2024-09-12T03:27:10.015434+0000","last_undegraded":"2024-09-12T03:27:10.015434+0000","last_fullsized":"2024-09-12T03:27:10.015434+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:21:23.866131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:33.132212+0000","last_change":"2024-09-12T03:26:57.019744+0000","last_active":"2024-09-12T03:27:33.132212+0000","last_peered":"2024-09-12T03:27:33.132212+0000","last_clean":"2024-09-12T03:27:33.132212+0000","last_became_active":"2024-09-12T03:26:57.019397+0000","last_became_peered":"2024-09-12T03:26:57.019397+0000","last_unstale":"2024-09-12T03:27:33.132212+0000","last_undegraded":"2024-09-12T03:27:33.132212+0000","last_fullsized":"2024-09-12T03:27:33.132212+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:44.891829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.054722+0000","last_change":"2024-09-12T03:27:06.048452+0000","last_active":"2024-09-12T03:27:09.054722+0000","last_peered":"2024-09-12T03:27:09.054722+0000","last_clean":"2024-09-12T03:27:09.054722+0000","last_became_active":"2024-09-12T03:27:06.048011+0000","last_became_peered":"2024-09-12T03:27:06.048011+0000","last_unstale":"2024-09-12T03:27:09.054722+0000","last_undegraded":"2024-09-12T03:27:09.054722+0000","last_fullsized":"2024-09-12T03:27:09.054722+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:20:38.360428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.055374+0000","last_change":"2024-09-12T03:26:59.017212+0000","last_active":"2024-09-12T03:27:09.055374+0000","last_peered":"2024-09-12T03:27:09.055374+0000","last_clean":"2024-09-12T03:27:09.055374+0000","last_became_active":"2024-09-12T03:26:59.016605+0000","last_became_peered":"2024-09-12T03:26:59.016605+0000","last_unstale":"2024-09-12T03:27:09.055374+0000","last_undegraded":"2024-09-12T03:27:09.055374+0000","last_fullsized":"2024-09-12T03:27:09.055374+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:34.639464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.054076+0000","last_change":"2024-09-12T03:26:57.020236+0000","last_active":"2024-09-12T03:27:09.054076+0000","last_peered":"2024-09-12T03:27:09.054076+0000","last_clean":"2024-09-12T03:27:09.054076+0000","last_became_active":"2024-09-12T03:26:57.019899+0000","last_became_peered":"2024-09-12T03:26:57.019899+0000","last_unstale":"2024-09-12T03:27:09.054076+0000","last_undegraded":"2024-09-12T03:27:09.054076+0000","last_fullsized":"2024-09-12T03:27:09.054076+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:03:16.184419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.958270+0000","last_change":"2024-09-12T03:27:06.190268+0000","last_active":"2024-09-12T03:27:09.958270+0000","last_peered":"2024-09-12T03:27:09.958270+0000","last_clean":"2024-09-12T03:27:09.958270+0000","last_became_active":"2024-09-12T03:27:06.189680+0000","last_became_peered":"2024-09-12T03:27:06.189680+0000","last_unstale":"2024-09-12T03:27:09.958270+0000","last_undegraded":"2024-09-12T03:27:09.958270+0000","last_fullsized":"2024-09-12T03:27:09.958270+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:03:00.098788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.055207+0000","last_change":"2024-09-12T03:26:59.014332+0000","last_active":"2024-09-12T03:27:09.055207+0000","last_peered":"2024-09-12T03:27:09.055207+0000","last_clean":"2024-09-12T03:27:09.055207+0000","last_became_active":"2024-09-12T03:26:59.014019+0000","last_became_peered":"2024-09-12T03:26:59.014019+0000","last_unstale":"2024-09-12T03:27:09.055207+0000","last_undegraded":"2024-09-12T03:27:09.055207+0000","last_fullsized":"2024-09-12T03:27:09.055207+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:03:18.020391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060457+0000","last_change":"2024-09-12T03:26:56.956591+0000","last_active":"2024-09-12T03:27:10.060457+0000","last_peered":"2024-09-12T03:27:10.060457+0000","last_clean":"2024-09-12T03:27:10.060457+0000","last_became_active":"2024-09-12T03:26:56.956028+0000","last_became_peered":"2024-09-12T03:26:56.956028+0000","last_unstale":"2024-09-12T03:27:10.060457+0000","last_undegraded":"2024-09-12T03:27:10.060457+0000","last_fullsized":"2024-09-12T03:27:10.060457+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:55:20.500536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.015329+0000","last_change":"2024-09-12T03:27:06.052630+0000","last_active":"2024-09-12T03:27:10.015329+0000","last_peered":"2024-09-12T03:27:10.015329+0000","last_clean":"2024-09-12T03:27:10.015329+0000","last_became_active":"2024-09-12T03:27:06.052240+0000","last_became_peered":"2024-09-12T03:27:06.052240+0000","last_unstale":"2024-09-12T03:27:10.015329+0000","last_undegraded":"2024-09-12T03:27:10.015329+0000","last_fullsized":"2024-09-12T03:27:10.015329+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:26:49.356330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059858+0000","last_change":"2024-09-12T03:26:59.022054+0000","last_active":"2024-09-12T03:27:10.059858+0000","last_peered":"2024-09-12T03:27:10.059858+0000","last_clean":"2024-09-12T03:27:10.059858+0000","last_became_active":"2024-09-12T03:26:59.021673+0000","last_became_peered":"2024-09-12T03:26:59.021673+0000","last_unstale":"2024-09-12T03:27:10.059858+0000","last_undegraded":"2024-09-12T03:27:10.059858+0000","last_fullsized":"2024-09-12T03:27:10.059858+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:07:07.581766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:46.161308+0000","last_change":"2024-09-12T03:26:57.018607+0000","last_active":"2024-09-12T03:27:46.161308+0000","last_peered":"2024-09-12T03:27:46.161308+0000","last_clean":"2024-09-12T03:27:46.161308+0000","last_became_active":"2024-09-12T03:26:57.018293+0000","last_became_peered":"2024-09-12T03:26:57.018293+0000","last_unstale":"2024-09-12T03:27:46.161308+0000","last_undegraded":"2024-09-12T03:27:46.161308+0000","last_fullsized":"2024-09-12T03:27:46.161308+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:23:57.245446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.015517+0000","last_change":"2024-09-12T03:27:06.190452+0000","last_active":"2024-09-12T03:27:10.015517+0000","last_peered":"2024-09-12T03:27:10.015517+0000","last_clean":"2024-09-12T03:27:10.015517+0000","last_became_active":"2024-09-12T03:27:06.189940+0000","last_became_peered":"2024-09-12T03:27:06.189940+0000","last_unstale":"2024-09-12T03:27:10.015517+0000","last_undegraded":"2024-09-12T03:27:10.015517+0000","last_fullsized":"2024-09-12T03:27:10.015517+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:25:23.114124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.014969+0000","last_change":"2024-09-12T03:26:59.019294+0000","last_active":"2024-09-12T03:27:10.014969+0000","last_peered":"2024-09-12T03:27:10.014969+0000","last_clean":"2024-09-12T03:27:10.014969+0000","last_became_active":"2024-09-12T03:26:59.018976+0000","last_became_peered":"2024-09-12T03:26:59.018976+0000","last_unstale":"2024-09-12T03:27:10.014969+0000","last_undegraded":"2024-09-12T03:27:10.014969+0000","last_fullsized":"2024-09-12T03:27:10.014969+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:06:38.571171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:34.142372+0000","last_change":"2024-09-12T03:26:56.960077+0000","last_active":"2024-09-12T03:27:34.142372+0000","last_peered":"2024-09-12T03:27:34.142372+0000","last_clean":"2024-09-12T03:27:34.142372+0000","last_became_active":"2024-09-12T03:26:56.958892+0000","last_became_peered":"2024-09-12T03:26:56.958892+0000","last_unstale":"2024-09-12T03:27:34.142372+0000","last_undegraded":"2024-09-12T03:27:34.142372+0000","last_fullsized":"2024-09-12T03:27:34.142372+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:27:04.779026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060256+0000","last_change":"2024-09-12T03:27:06.190439+0000","last_active":"2024-09-12T03:27:10.060256+0000","last_peered":"2024-09-12T03:27:10.060256+0000","last_clean":"2024-09-12T03:27:10.060256+0000","last_became_active":"2024-09-12T03:27:06.190010+0000","last_became_peered":"2024-09-12T03:27:06.190010+0000","last_unstale":"2024-09-12T03:27:10.060256+0000","last_undegraded":"2024-09-12T03:27:10.060256+0000","last_fullsized":"2024-09-12T03:27:10.060256+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:54:17.362910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059758+0000","last_change":"2024-09-12T03:26:59.024060+0000","last_active":"2024-09-12T03:27:10.059758+0000","last_peered":"2024-09-12T03:27:10.059758+0000","last_clean":"2024-09-12T03:27:10.059758+0000","last_became_active":"2024-09-12T03:26:59.023726+0000","last_became_peered":"2024-09-12T03:26:59.023726+0000","last_unstale":"2024-09-12T03:27:10.059758+0000","last_undegraded":"2024-09-12T03:27:10.059758+0000","last_fullsized":"2024-09-12T03:27:10.059758+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:49:44.984188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:35.142702+0000","last_change":"2024-09-12T03:26:56.957847+0000","last_active":"2024-09-12T03:27:35.142702+0000","last_peered":"2024-09-12T03:27:35.142702+0000","last_clean":"2024-09-12T03:27:35.142702+0000","last_became_active":"2024-09-12T03:26:56.957491+0000","last_became_peered":"2024-09-12T03:26:56.957491+0000","last_unstale":"2024-09-12T03:27:35.142702+0000","last_undegraded":"2024-09-12T03:27:35.142702+0000","last_fullsized":"2024-09-12T03:27:35.142702+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:06:23.028881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":358,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059971+0000","last_change":"2024-09-12T03:25:45.211570+0000","last_active":"2024-09-12T03:27:10.059971+0000","last_peered":"2024-09-12T03:27:10.059971+0000","last_clean":"2024-09-12T03:27:10.059971+0000","last_became_active":"2024-09-12T03:25:45.211052+0000","last_became_peered":"2024-09-12T03:25:45.211052+0000","last_unstale":"2024-09-12T03:27:10.059971+0000","last_undegraded":"2024-09-12T03:27:10.059971+0000","last_fullsized":"2024-09-12T03:27:10.059971+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:21.915950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:21.915950+0000","last_clean_scrub_stamp":"2024-09-12T03:22:21.915950+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:03:06.560617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059967+0000","last_change":"2024-09-12T03:27:06.046319+0000","last_active":"2024-09-12T03:27:10.059967+0000","last_peered":"2024-09-12T03:27:10.059967+0000","last_clean":"2024-09-12T03:27:10.059967+0000","last_became_active":"2024-09-12T03:27:06.045864+0000","last_became_peered":"2024-09-12T03:27:06.045864+0000","last_unstale":"2024-09-12T03:27:10.059967+0000","last_undegraded":"2024-09-12T03:27:10.059967+0000","last_fullsized":"2024-09-12T03:27:10.059967+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:45:25.224775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060194+0000","last_change":"2024-09-12T03:26:59.019197+0000","last_active":"2024-09-12T03:27:10.060194+0000","last_peered":"2024-09-12T03:27:10.060194+0000","last_clean":"2024-09-12T03:27:10.060194+0000","last_became_active":"2024-09-12T03:26:59.018868+0000","last_became_peered":"2024-09-12T03:26:59.018868+0000","last_unstale":"2024-09-12T03:27:10.060194+0000","last_undegraded":"2024-09-12T03:27:10.060194+0000","last_fullsized":"2024-09-12T03:27:10.060194+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:50:51.707556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:34.143077+0000","last_change":"2024-09-12T03:26:57.019888+0000","last_active":"2024-09-12T03:27:34.143077+0000","last_peered":"2024-09-12T03:27:34.143077+0000","last_clean":"2024-09-12T03:27:34.143077+0000","last_became_active":"2024-09-12T03:26:57.019378+0000","last_became_peered":"2024-09-12T03:26:57.019378+0000","last_unstale":"2024-09-12T03:27:34.143077+0000","last_undegraded":"2024-09-12T03:27:34.143077+0000","last_fullsized":"2024-09-12T03:27:34.143077+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:44:02.298726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059309+0000","last_change":"2024-09-12T03:27:06.043780+0000","last_active":"2024-09-12T03:27:10.059309+0000","last_peered":"2024-09-12T03:27:10.059309+0000","last_clean":"2024-09-12T03:27:10.059309+0000","last_became_active":"2024-09-12T03:27:06.042770+0000","last_became_peered":"2024-09-12T03:27:06.042770+0000","last_unstale":"2024-09-12T03:27:10.059309+0000","last_undegraded":"2024-09-12T03:27:10.059309+0000","last_fullsized":"2024-09-12T03:27:10.059309+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:32:40.801908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060195+0000","last_change":"2024-09-12T03:26:59.022996+0000","last_active":"2024-09-12T03:27:10.060195+0000","last_peered":"2024-09-12T03:27:10.060195+0000","last_clean":"2024-09-12T03:27:10.060195+0000","last_became_active":"2024-09-12T03:26:59.022675+0000","last_became_peered":"2024-09-12T03:26:59.022675+0000","last_unstale":"2024-09-12T03:27:10.060195+0000","last_undegraded":"2024-09-12T03:27:10.060195+0000","last_fullsized":"2024-09-12T03:27:10.060195+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:09.674971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060495+0000","last_change":"2024-09-12T03:26:56.957101+0000","last_active":"2024-09-12T03:27:10.060495+0000","last_peered":"2024-09-12T03:27:10.060495+0000","last_clean":"2024-09-12T03:27:10.060495+0000","last_became_active":"2024-09-12T03:26:56.956694+0000","last_became_peered":"2024-09-12T03:26:56.956694+0000","last_unstale":"2024-09-12T03:27:10.060495+0000","last_undegraded":"2024-09-12T03:27:10.060495+0000","last_fullsized":"2024-09-12T03:27:10.060495+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:29.793229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059123+0000","last_change":"2024-09-12T03:27:06.049382+0000","last_active":"2024-09-12T03:27:10.059123+0000","last_peered":"2024-09-12T03:27:10.059123+0000","last_clean":"2024-09-12T03:27:10.059123+0000","last_became_active":"2024-09-12T03:27:06.048961+0000","last_became_peered":"2024-09-12T03:27:06.048961+0000","last_unstale":"2024-09-12T03:27:10.059123+0000","last_undegraded":"2024-09-12T03:27:10.059123+0000","last_fullsized":"2024-09-12T03:27:10.059123+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:32.400328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060586+0000","last_change":"2024-09-12T03:26:59.020146+0000","last_active":"2024-09-12T03:27:10.060586+0000","last_peered":"2024-09-12T03:27:10.060586+0000","last_clean":"2024-09-12T03:27:10.060586+0000","last_became_active":"2024-09-12T03:26:59.019854+0000","last_became_peered":"2024-09-12T03:26:59.019854+0000","last_unstale":"2024-09-12T03:27:10.060586+0000","last_undegraded":"2024-09-12T03:27:10.060586+0000","last_fullsized":"2024-09-12T03:27:10.060586+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:38:47.476070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.015505+0000","last_change":"2024-09-12T03:26:56.956237+0000","last_active":"2024-09-12T03:27:10.015505+0000","last_peered":"2024-09-12T03:27:10.015505+0000","last_clean":"2024-09-12T03:27:10.015505+0000","last_became_active":"2024-09-12T03:26:56.955941+0000","last_became_peered":"2024-09-12T03:26:56.955941+0000","last_unstale":"2024-09-12T03:27:10.015505+0000","last_undegraded":"2024-09-12T03:27:10.015505+0000","last_fullsized":"2024-09-12T03:27:10.015505+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:08:53.422429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060001+0000","last_change":"2024-09-12T03:27:06.050916+0000","last_active":"2024-09-12T03:27:10.060001+0000","last_peered":"2024-09-12T03:27:10.060001+0000","last_clean":"2024-09-12T03:27:10.060001+0000","last_became_active":"2024-09-12T03:27:06.050494+0000","last_became_peered":"2024-09-12T03:27:06.050494+0000","last_unstale":"2024-09-12T03:27:10.060001+0000","last_undegraded":"2024-09-12T03:27:10.060001+0000","last_fullsized":"2024-09-12T03:27:10.060001+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:58:40.830354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.055016+0000","last_change":"2024-09-12T03:26:59.025310+0000","last_active":"2024-09-12T03:27:09.055016+0000","last_peered":"2024-09-12T03:27:09.055016+0000","last_clean":"2024-09-12T03:27:09.055016+0000","last_became_active":"2024-09-12T03:26:59.024917+0000","last_became_peered":"2024-09-12T03:26:59.024917+0000","last_unstale":"2024-09-12T03:27:09.055016+0000","last_undegraded":"2024-09-12T03:27:09.055016+0000","last_fullsized":"2024-09-12T03:27:09.055016+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:14:19.786496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:35.138137+0000","last_change":"2024-09-12T03:26:56.959434+0000","last_active":"2024-09-12T03:27:35.138137+0000","last_peered":"2024-09-12T03:27:35.138137+0000","last_clean":"2024-09-12T03:27:35.138137+0000","last_became_active":"2024-09-12T03:26:56.959095+0000","last_became_peered":"2024-09-12T03:26:56.959095+0000","last_unstale":"2024-09-12T03:27:35.138137+0000","last_undegraded":"2024-09-12T03:27:35.138137+0000","last_fullsized":"2024-09-12T03:27:35.138137+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:43:39.291022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060059+0000","last_change":"2024-09-12T03:27:06.054028+0000","last_active":"2024-09-12T03:27:10.060059+0000","last_peered":"2024-09-12T03:27:10.060059+0000","last_clean":"2024-09-12T03:27:10.060059+0000","last_became_active":"2024-09-12T03:27:06.053573+0000","last_became_peered":"2024-09-12T03:27:06.053573+0000","last_unstale":"2024-09-12T03:27:10.060059+0000","last_undegraded":"2024-09-12T03:27:10.060059+0000","last_fullsized":"2024-09-12T03:27:10.060059+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:05.977501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060148+0000","last_change":"2024-09-12T03:26:59.022978+0000","last_active":"2024-09-12T03:27:10.060148+0000","last_peered":"2024-09-12T03:27:10.060148+0000","last_clean":"2024-09-12T03:27:10.060148+0000","last_became_active":"2024-09-12T03:26:59.022555+0000","last_became_peered":"2024-09-12T03:26:59.022555+0000","last_unstale":"2024-09-12T03:27:10.060148+0000","last_undegraded":"2024-09-12T03:27:10.060148+0000","last_fullsized":"2024-09-12T03:27:10.060148+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:51.903788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:34.143883+0000","last_change":"2024-09-12T03:26:57.018735+0000","last_active":"2024-09-12T03:27:34.143883+0000","last_peered":"2024-09-12T03:27:34.143883+0000","last_clean":"2024-09-12T03:27:34.143883+0000","last_became_active":"2024-09-12T03:26:57.018367+0000","last_became_peered":"2024-09-12T03:26:57.018367+0000","last_unstale":"2024-09-12T03:27:34.143883+0000","last_undegraded":"2024-09-12T03:27:34.143883+0000","last_fullsized":"2024-09-12T03:27:34.143883+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:25.420324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.015527+0000","last_change":"2024-09-12T03:27:06.046626+0000","last_active":"2024-09-12T03:27:10.015527+0000","last_peered":"2024-09-12T03:27:10.015527+0000","last_clean":"2024-09-12T03:27:10.015527+0000","last_became_active":"2024-09-12T03:27:06.046234+0000","last_became_peered":"2024-09-12T03:27:06.046234+0000","last_unstale":"2024-09-12T03:27:10.015527+0000","last_undegraded":"2024-09-12T03:27:10.015527+0000","last_fullsized":"2024-09-12T03:27:10.015527+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:05:57.353206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059282+0000","last_change":"2024-09-12T03:26:59.023541+0000","last_active":"2024-09-12T03:27:10.059282+0000","last_peered":"2024-09-12T03:27:10.059282+0000","last_clean":"2024-09-12T03:27:10.059282+0000","last_became_active":"2024-09-12T03:26:59.023229+0000","last_became_peered":"2024-09-12T03:26:59.023229+0000","last_unstale":"2024-09-12T03:27:10.059282+0000","last_undegraded":"2024-09-12T03:27:10.059282+0000","last_fullsized":"2024-09-12T03:27:10.059282+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:27:58.514616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'4","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:46.161938+0000","last_change":"2024-09-12T03:26:56.962703+0000","last_active":"2024-09-12T03:27:46.161938+0000","last_peered":"2024-09-12T03:27:46.161938+0000","last_clean":"2024-09-12T03:27:46.161938+0000","last_became_active":"2024-09-12T03:26:56.962280+0000","last_became_peered":"2024-09-12T03:26:56.962280+0000","last_unstale":"2024-09-12T03:27:46.161938+0000","last_undegraded":"2024-09-12T03:27:46.161938+0000","last_fullsized":"2024-09-12T03:27:46.161938+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:01.045200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.053992+0000","last_change":"2024-09-12T03:27:06.054791+0000","last_active":"2024-09-12T03:27:09.053992+0000","last_peered":"2024-09-12T03:27:09.053992+0000","last_clean":"2024-09-12T03:27:09.053992+0000","last_became_active":"2024-09-12T03:27:06.054311+0000","last_became_peered":"2024-09-12T03:27:06.054311+0000","last_unstale":"2024-09-12T03:27:09.053992+0000","last_undegraded":"2024-09-12T03:27:09.053992+0000","last_fullsized":"2024-09-12T03:27:09.053992+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:21:59.115200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059750+0000","last_change":"2024-09-12T03:26:59.023971+0000","last_active":"2024-09-12T03:27:10.059750+0000","last_peered":"2024-09-12T03:27:10.059750+0000","last_clean":"2024-09-12T03:27:10.059750+0000","last_became_active":"2024-09-12T03:26:59.023653+0000","last_became_peered":"2024-09-12T03:26:59.023653+0000","last_unstale":"2024-09-12T03:27:10.059750+0000","last_undegraded":"2024-09-12T03:27:10.059750+0000","last_fullsized":"2024-09-12T03:27:10.059750+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:10:08.571177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:33.131896+0000","last_change":"2024-09-12T03:26:56.962459+0000","last_active":"2024-09-12T03:27:33.131896+0000","last_peered":"2024-09-12T03:27:33.131896+0000","last_clean":"2024-09-12T03:27:33.131896+0000","last_became_active":"2024-09-12T03:26:56.962039+0000","last_became_peered":"2024-09-12T03:26:56.962039+0000","last_unstale":"2024-09-12T03:27:33.131896+0000","last_undegraded":"2024-09-12T03:27:33.131896+0000","last_fullsized":"2024-09-12T03:27:33.131896+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:08:44.214788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.054787+0000","last_change":"2024-09-12T03:27:06.049058+0000","last_active":"2024-09-12T03:27:09.054787+0000","last_peered":"2024-09-12T03:27:09.054787+0000","last_clean":"2024-09-12T03:27:09.054787+0000","last_became_active":"2024-09-12T03:27:06.048606+0000","last_became_peered":"2024-09-12T03:27:06.048606+0000","last_unstale":"2024-09-12T03:27:09.054787+0000","last_undegraded":"2024-09-12T03:27:09.054787+0000","last_fullsized":"2024-09-12T03:27:09.054787+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:23:14.571184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.015146+0000","last_change":"2024-09-12T03:26:59.017832+0000","last_active":"2024-09-12T03:27:10.015146+0000","last_peered":"2024-09-12T03:27:10.015146+0000","last_clean":"2024-09-12T03:27:10.015146+0000","last_became_active":"2024-09-12T03:26:59.017497+0000","last_became_peered":"2024-09-12T03:26:59.017497+0000","last_unstale":"2024-09-12T03:27:10.015146+0000","last_undegraded":"2024-09-12T03:27:10.015146+0000","last_fullsized":"2024-09-12T03:27:10.015146+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:49.753556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:34.142779+0000","last_change":"2024-09-12T03:26:56.957877+0000","last_active":"2024-09-12T03:27:34.142779+0000","last_peered":"2024-09-12T03:27:34.142779+0000","last_clean":"2024-09-12T03:27:34.142779+0000","last_became_active":"2024-09-12T03:26:56.957502+0000","last_became_peered":"2024-09-12T03:26:56.957502+0000","last_unstale":"2024-09-12T03:27:34.142779+0000","last_undegraded":"2024-09-12T03:27:34.142779+0000","last_fullsized":"2024-09-12T03:27:34.142779+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:35.851957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060295+0000","last_change":"2024-09-12T03:27:06.047070+0000","last_active":"2024-09-12T03:27:10.060295+0000","last_peered":"2024-09-12T03:27:10.060295+0000","last_clean":"2024-09-12T03:27:10.060295+0000","last_became_active":"2024-09-12T03:27:06.046612+0000","last_became_peered":"2024-09-12T03:27:06.046612+0000","last_unstale":"2024-09-12T03:27:10.060295+0000","last_undegraded":"2024-09-12T03:27:10.060295+0000","last_fullsized":"2024-09-12T03:27:10.060295+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:36:53.960456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.958597+0000","last_change":"2024-09-12T03:26:59.019856+0000","last_active":"2024-09-12T03:27:09.958597+0000","last_peered":"2024-09-12T03:27:09.958597+0000","last_clean":"2024-09-12T03:27:09.958597+0000","last_became_active":"2024-09-12T03:26:59.019498+0000","last_became_peered":"2024-09-12T03:26:59.019498+0000","last_unstale":"2024-09-12T03:27:09.958597+0000","last_undegraded":"2024-09-12T03:27:09.958597+0000","last_fullsized":"2024-09-12T03:27:09.958597+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:02:18.000549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'26","reported_seq":86,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:46.163740+0000","last_change":"2024-09-12T03:26:56.958919+0000","last_active":"2024-09-12T03:27:46.163740+0000","last_peered":"2024-09-12T03:27:46.163740+0000","last_clean":"2024-09-12T03:27:46.163740+0000","last_became_active":"2024-09-12T03:26:56.958455+0000","last_became_peered":"2024-09-12T03:26:56.958455+0000","last_unstale":"2024-09-12T03:27:46.163740+0000","last_undegraded":"2024-09-12T03:27:46.163740+0000","last_fullsized":"2024-09-12T03:27:46.163740+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:05:19.318922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43345,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":57,"num_write":27,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.053719+0000","last_change":"2024-09-12T03:27:06.047094+0000","last_active":"2024-09-12T03:27:09.053719+0000","last_peered":"2024-09-12T03:27:09.053719+0000","last_clean":"2024-09-12T03:27:09.053719+0000","last_became_active":"2024-09-12T03:27:06.046702+0000","last_became_peered":"2024-09-12T03:27:06.046702+0000","last_unstale":"2024-09-12T03:27:09.053719+0000","last_undegraded":"2024-09-12T03:27:09.053719+0000","last_fullsized":"2024-09-12T03:27:09.053719+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:46:57.557056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059911+0000","last_change":"2024-09-12T03:26:59.018905+0000","last_active":"2024-09-12T03:27:10.059911+0000","last_peered":"2024-09-12T03:27:10.059911+0000","last_clean":"2024-09-12T03:27:10.059911+0000","last_became_active":"2024-09-12T03:26:59.018492+0000","last_became_peered":"2024-09-12T03:26:59.018492+0000","last_unstale":"2024-09-12T03:27:10.059911+0000","last_undegraded":"2024-09-12T03:27:10.059911+0000","last_fullsized":"2024-09-12T03:27:10.059911+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:11.953999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:33.132376+0000","last_change":"2024-09-12T03:26:57.018607+0000","last_active":"2024-09-12T03:27:33.132376+0000","last_peered":"2024-09-12T03:27:33.132376+0000","last_clean":"2024-09-12T03:27:33.132376+0000","last_became_active":"2024-09-12T03:26:57.018255+0000","last_became_peered":"2024-09-12T03:26:57.018255+0000","last_unstale":"2024-09-12T03:27:33.132376+0000","last_undegraded":"2024-09-12T03:27:33.132376+0000","last_fullsized":"2024-09-12T03:27:33.132376+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:39.029308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.061212+0000","last_change":"2024-09-12T03:27:06.051078+0000","last_active":"2024-09-12T03:27:10.061212+0000","last_peered":"2024-09-12T03:27:10.061212+0000","last_clean":"2024-09-12T03:27:10.061212+0000","last_became_active":"2024-09-12T03:27:06.050257+0000","last_became_peered":"2024-09-12T03:27:06.050257+0000","last_unstale":"2024-09-12T03:27:10.061212+0000","last_undegraded":"2024-09-12T03:27:10.061212+0000","last_fullsized":"2024-09-12T03:27:10.061212+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:56.315922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059234+0000","last_change":"2024-09-12T03:26:59.025406+0000","last_active":"2024-09-12T03:27:10.059234+0000","last_peered":"2024-09-12T03:27:10.059234+0000","last_clean":"2024-09-12T03:27:10.059234+0000","last_became_active":"2024-09-12T03:26:59.025091+0000","last_became_peered":"2024-09-12T03:26:59.025091+0000","last_unstale":"2024-09-12T03:27:10.059234+0000","last_undegraded":"2024-09-12T03:27:10.059234+0000","last_fullsized":"2024-09-12T03:27:10.059234+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:39:08.297785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:33.132791+0000","last_change":"2024-09-12T03:26:56.961325+0000","last_active":"2024-09-12T03:27:33.132791+0000","last_peered":"2024-09-12T03:27:33.132791+0000","last_clean":"2024-09-12T03:27:33.132791+0000","last_became_active":"2024-09-12T03:26:56.960806+0000","last_became_peered":"2024-09-12T03:26:56.960806+0000","last_unstale":"2024-09-12T03:27:33.132791+0000","last_undegraded":"2024-09-12T03:27:33.132791+0000","last_fullsized":"2024-09-12T03:27:33.132791+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:44:11.209394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.053588+0000","last_change":"2024-09-12T03:27:06.055331+0000","last_active":"2024-09-12T03:27:09.053588+0000","last_peered":"2024-09-12T03:27:09.053588+0000","last_clean":"2024-09-12T03:27:09.053588+0000","last_became_active":"2024-09-12T03:27:06.054227+0000","last_became_peered":"2024-09-12T03:27:06.054227+0000","last_unstale":"2024-09-12T03:27:09.053588+0000","last_undegraded":"2024-09-12T03:27:09.053588+0000","last_fullsized":"2024-09-12T03:27:09.053588+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:35:10.446231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060255+0000","last_change":"2024-09-12T03:26:59.012095+0000","last_active":"2024-09-12T03:27:10.060255+0000","last_peered":"2024-09-12T03:27:10.060255+0000","last_clean":"2024-09-12T03:27:10.060255+0000","last_became_active":"2024-09-12T03:26:59.011592+0000","last_became_peered":"2024-09-12T03:26:59.011592+0000","last_unstale":"2024-09-12T03:27:10.060255+0000","last_undegraded":"2024-09-12T03:27:10.060255+0000","last_fullsized":"2024-09-12T03:27:10.060255+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:49:33.026996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:45.160629+0000","last_change":"2024-09-12T03:26:56.959353+0000","last_active":"2024-09-12T03:27:45.160629+0000","last_peered":"2024-09-12T03:27:45.160629+0000","last_clean":"2024-09-12T03:27:45.160629+0000","last_became_active":"2024-09-12T03:26:56.958767+0000","last_became_peered":"2024-09-12T03:26:56.958767+0000","last_unstale":"2024-09-12T03:27:45.160629+0000","last_undegraded":"2024-09-12T03:27:45.160629+0000","last_fullsized":"2024-09-12T03:27:45.160629+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:37:51.518907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059468+0000","last_change":"2024-09-12T03:27:06.043745+0000","last_active":"2024-09-12T03:27:10.059468+0000","last_peered":"2024-09-12T03:27:10.059468+0000","last_clean":"2024-09-12T03:27:10.059468+0000","last_became_active":"2024-09-12T03:27:06.042855+0000","last_became_peered":"2024-09-12T03:27:06.042855+0000","last_unstale":"2024-09-12T03:27:10.059468+0000","last_undegraded":"2024-09-12T03:27:10.059468+0000","last_fullsized":"2024-09-12T03:27:10.059468+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:14:34.939593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059029+0000","last_change":"2024-09-12T03:27:06.052561+0000","last_active":"2024-09-12T03:27:10.059029+0000","last_peered":"2024-09-12T03:27:10.059029+0000","last_clean":"2024-09-12T03:27:10.059029+0000","last_became_active":"2024-09-12T03:27:06.052192+0000","last_became_peered":"2024-09-12T03:27:06.052192+0000","last_unstale":"2024-09-12T03:27:10.059029+0000","last_undegraded":"2024-09-12T03:27:10.059029+0000","last_fullsized":"2024-09-12T03:27:10.059029+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:31:31.800188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.053794+0000","last_change":"2024-09-12T03:26:59.012762+0000","last_active":"2024-09-12T03:27:09.053794+0000","last_peered":"2024-09-12T03:27:09.053794+0000","last_clean":"2024-09-12T03:27:09.053794+0000","last_became_active":"2024-09-12T03:26:59.012400+0000","last_became_peered":"2024-09-12T03:26:59.012400+0000","last_unstale":"2024-09-12T03:27:09.053794+0000","last_undegraded":"2024-09-12T03:27:09.053794+0000","last_fullsized":"2024-09-12T03:27:09.053794+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:35:13.573284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060076+0000","last_change":"2024-09-12T03:26:56.962454+0000","last_active":"2024-09-12T03:27:10.060076+0000","last_peered":"2024-09-12T03:27:10.060076+0000","last_clean":"2024-09-12T03:27:10.060076+0000","last_became_active":"2024-09-12T03:26:56.962026+0000","last_became_peered":"2024-09-12T03:26:56.962026+0000","last_unstale":"2024-09-12T03:27:10.060076+0000","last_undegraded":"2024-09-12T03:27:10.060076+0000","last_fullsized":"2024-09-12T03:27:10.060076+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:00:23.501942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.055112+0000","last_change":"2024-09-12T03:27:06.053946+0000","last_active":"2024-09-12T03:27:09.055112+0000","last_peered":"2024-09-12T03:27:09.055112+0000","last_clean":"2024-09-12T03:27:09.055112+0000","last_became_active":"2024-09-12T03:27:06.053494+0000","last_became_peered":"2024-09-12T03:27:06.053494+0000","last_unstale":"2024-09-12T03:27:09.055112+0000","last_undegraded":"2024-09-12T03:27:09.055112+0000","last_fullsized":"2024-09-12T03:27:09.055112+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:13:11.537918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.058720+0000","last_change":"2024-09-12T03:26:59.021472+0000","last_active":"2024-09-12T03:27:10.058720+0000","last_peered":"2024-09-12T03:27:10.058720+0000","last_clean":"2024-09-12T03:27:10.058720+0000","last_became_active":"2024-09-12T03:26:59.021160+0000","last_became_peered":"2024-09-12T03:26:59.021160+0000","last_unstale":"2024-09-12T03:27:10.058720+0000","last_undegraded":"2024-09-12T03:27:10.058720+0000","last_fullsized":"2024-09-12T03:27:10.058720+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:47.146617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:34.142514+0000","last_change":"2024-09-12T03:26:56.958944+0000","last_active":"2024-09-12T03:27:34.142514+0000","last_peered":"2024-09-12T03:27:34.142514+0000","last_clean":"2024-09-12T03:27:34.142514+0000","last_became_active":"2024-09-12T03:26:56.958511+0000","last_became_peered":"2024-09-12T03:26:56.958511+0000","last_unstale":"2024-09-12T03:27:34.142514+0000","last_undegraded":"2024-09-12T03:27:34.142514+0000","last_fullsized":"2024-09-12T03:27:34.142514+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:52:37.871482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:34.141804+0000","last_change":"2024-09-12T03:26:56.961442+0000","last_active":"2024-09-12T03:27:34.141804+0000","last_peered":"2024-09-12T03:27:34.141804+0000","last_clean":"2024-09-12T03:27:34.141804+0000","last_became_active":"2024-09-12T03:26:56.961078+0000","last_became_peered":"2024-09-12T03:26:56.961078+0000","last_unstale":"2024-09-12T03:27:34.141804+0000","last_undegraded":"2024-09-12T03:27:34.141804+0000","last_fullsized":"2024-09-12T03:27:34.141804+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:17.185990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060209+0000","last_change":"2024-09-12T03:26:59.022873+0000","last_active":"2024-09-12T03:27:10.060209+0000","last_peered":"2024-09-12T03:27:10.060209+0000","last_clean":"2024-09-12T03:27:10.060209+0000","last_became_active":"2024-09-12T03:26:59.022391+0000","last_became_peered":"2024-09-12T03:26:59.022391+0000","last_unstale":"2024-09-12T03:27:10.060209+0000","last_undegraded":"2024-09-12T03:27:10.060209+0000","last_fullsized":"2024-09-12T03:27:10.060209+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:05:08.138480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059355+0000","last_change":"2024-09-12T03:27:06.049829+0000","last_active":"2024-09-12T03:27:10.059355+0000","last_peered":"2024-09-12T03:27:10.059355+0000","last_clean":"2024-09-12T03:27:10.059355+0000","last_became_active":"2024-09-12T03:27:06.049444+0000","last_became_peered":"2024-09-12T03:27:06.049444+0000","last_unstale":"2024-09-12T03:27:10.059355+0000","last_undegraded":"2024-09-12T03:27:10.059355+0000","last_fullsized":"2024-09-12T03:27:10.059355+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:57:07.617935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:45.161421+0000","last_change":"2024-09-12T03:26:56.962474+0000","last_active":"2024-09-12T03:27:45.161421+0000","last_peered":"2024-09-12T03:27:45.161421+0000","last_clean":"2024-09-12T03:27:45.161421+0000","last_became_active":"2024-09-12T03:26:56.962105+0000","last_became_peered":"2024-09-12T03:26:56.962105+0000","last_unstale":"2024-09-12T03:27:45.161421+0000","last_undegraded":"2024-09-12T03:27:45.161421+0000","last_fullsized":"2024-09-12T03:27:45.161421+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:40:41.905790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.958250+0000","last_change":"2024-09-12T03:26:59.011561+0000","last_active":"2024-09-12T03:27:09.958250+0000","last_peered":"2024-09-12T03:27:09.958250+0000","last_clean":"2024-09-12T03:27:09.958250+0000","last_became_active":"2024-09-12T03:26:59.011006+0000","last_became_peered":"2024-09-12T03:26:59.011006+0000","last_unstale":"2024-09-12T03:27:09.958250+0000","last_undegraded":"2024-09-12T03:27:09.958250+0000","last_fullsized":"2024-09-12T03:27:09.958250+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:09:20.221637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059427+0000","last_change":"2024-09-12T03:27:06.050816+0000","last_active":"2024-09-12T03:27:10.059427+0000","last_peered":"2024-09-12T03:27:10.059427+0000","last_clean":"2024-09-12T03:27:10.059427+0000","last_became_active":"2024-09-12T03:27:06.050403+0000","last_became_peered":"2024-09-12T03:27:06.050403+0000","last_unstale":"2024-09-12T03:27:10.059427+0000","last_undegraded":"2024-09-12T03:27:10.059427+0000","last_fullsized":"2024-09-12T03:27:10.059427+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:02:24.285363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.054522+0000","last_change":"2024-09-12T03:26:59.013900+0000","last_active":"2024-09-12T03:27:09.054522+0000","last_peered":"2024-09-12T03:27:09.054522+0000","last_clean":"2024-09-12T03:27:09.054522+0000","last_became_active":"2024-09-12T03:26:59.013308+0000","last_became_peered":"2024-09-12T03:26:59.013308+0000","last_unstale":"2024-09-12T03:27:09.054522+0000","last_undegraded":"2024-09-12T03:27:09.054522+0000","last_fullsized":"2024-09-12T03:27:09.054522+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:50.943991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059748+0000","last_change":"2024-09-12T03:26:59.022861+0000","last_active":"2024-09-12T03:27:10.059748+0000","last_peered":"2024-09-12T03:27:10.059748+0000","last_clean":"2024-09-12T03:27:10.059748+0000","last_became_active":"2024-09-12T03:26:59.022485+0000","last_became_peered":"2024-09-12T03:26:59.022485+0000","last_unstale":"2024-09-12T03:27:10.059748+0000","last_undegraded":"2024-09-12T03:27:10.059748+0000","last_fullsized":"2024-09-12T03:27:10.059748+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:46.961156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059176+0000","last_change":"2024-09-12T03:26:59.017779+0000","last_active":"2024-09-12T03:27:10.059176+0000","last_peered":"2024-09-12T03:27:10.059176+0000","last_clean":"2024-09-12T03:27:10.059176+0000","last_became_active":"2024-09-12T03:26:59.017438+0000","last_became_peered":"2024-09-12T03:26:59.017438+0000","last_unstale":"2024-09-12T03:27:10.059176+0000","last_undegraded":"2024-09-12T03:27:10.059176+0000","last_fullsized":"2024-09-12T03:27:10.059176+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:29.224776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059138+0000","last_change":"2024-09-12T03:26:59.017542+0000","last_active":"2024-09-12T03:27:10.059138+0000","last_peered":"2024-09-12T03:27:10.059138+0000","last_clean":"2024-09-12T03:27:10.059138+0000","last_became_active":"2024-09-12T03:26:59.017253+0000","last_became_peered":"2024-09-12T03:26:59.017253+0000","last_unstale":"2024-09-12T03:27:10.059138+0000","last_undegraded":"2024-09-12T03:27:10.059138+0000","last_fullsized":"2024-09-12T03:27:10.059138+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:06:51.389627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.015306+0000","last_change":"2024-09-12T03:26:59.013574+0000","last_active":"2024-09-12T03:27:10.015306+0000","last_peered":"2024-09-12T03:27:10.015306+0000","last_clean":"2024-09-12T03:27:10.015306+0000","last_became_active":"2024-09-12T03:26:59.013253+0000","last_became_peered":"2024-09-12T03:26:59.013253+0000","last_unstale":"2024-09-12T03:27:10.015306+0000","last_undegraded":"2024-09-12T03:27:10.015306+0000","last_fullsized":"2024-09-12T03:27:10.015306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:37.610909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.055055+0000","last_change":"2024-09-12T03:26:59.012430+0000","last_active":"2024-09-12T03:27:09.055055+0000","last_peered":"2024-09-12T03:27:09.055055+0000","last_clean":"2024-09-12T03:27:09.055055+0000","last_became_active":"2024-09-12T03:26:59.011672+0000","last_became_peered":"2024-09-12T03:26:59.011672+0000","last_unstale":"2024-09-12T03:27:09.055055+0000","last_undegraded":"2024-09-12T03:27:09.055055+0000","last_fullsized":"2024-09-12T03:27:09.055055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:50:44.888834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059475+0000","last_change":"2024-09-12T03:26:59.015704+0000","last_active":"2024-09-12T03:27:10.059475+0000","last_peered":"2024-09-12T03:27:10.059475+0000","last_clean":"2024-09-12T03:27:10.059475+0000","last_became_active":"2024-09-12T03:26:59.015389+0000","last_became_peered":"2024-09-12T03:26:59.015389+0000","last_unstale":"2024-09-12T03:27:10.059475+0000","last_undegraded":"2024-09-12T03:27:10.059475+0000","last_fullsized":"2024-09-12T03:27:10.059475+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:19:52.917902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059839+0000","last_change":"2024-09-12T03:26:59.026325+0000","last_active":"2024-09-12T03:27:10.059839+0000","last_peered":"2024-09-12T03:27:10.059839+0000","last_clean":"2024-09-12T03:27:10.059839+0000","last_became_active":"2024-09-12T03:26:59.026014+0000","last_became_peered":"2024-09-12T03:26:59.026014+0000","last_unstale":"2024-09-12T03:27:10.059839+0000","last_undegraded":"2024-09-12T03:27:10.059839+0000","last_fullsized":"2024-09-12T03:27:10.059839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:32:50.778286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059760+0000","last_change":"2024-09-12T03:26:59.020405+0000","last_active":"2024-09-12T03:27:10.059760+0000","last_peered":"2024-09-12T03:27:10.059760+0000","last_clean":"2024-09-12T03:27:10.059760+0000","last_became_active":"2024-09-12T03:26:59.020047+0000","last_became_peered":"2024-09-12T03:26:59.020047+0000","last_unstale":"2024-09-12T03:27:10.059760+0000","last_undegraded":"2024-09-12T03:27:10.059760+0000","last_fullsized":"2024-09-12T03:27:10.059760+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:03:47.024894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059626+0000","last_change":"2024-09-12T03:26:59.023143+0000","last_active":"2024-09-12T03:27:10.059626+0000","last_peered":"2024-09-12T03:27:10.059626+0000","last_clean":"2024-09-12T03:27:10.059626+0000","last_became_active":"2024-09-12T03:26:59.022282+0000","last_became_peered":"2024-09-12T03:26:59.022282+0000","last_unstale":"2024-09-12T03:27:10.059626+0000","last_undegraded":"2024-09-12T03:27:10.059626+0000","last_fullsized":"2024-09-12T03:27:10.059626+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:59.864192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059511+0000","last_change":"2024-09-12T03:26:59.025505+0000","last_active":"2024-09-12T03:27:10.059511+0000","last_peered":"2024-09-12T03:27:10.059511+0000","last_clean":"2024-09-12T03:27:10.059511+0000","last_became_active":"2024-09-12T03:26:59.025081+0000","last_became_peered":"2024-09-12T03:26:59.025081+0000","last_unstale":"2024-09-12T03:27:10.059511+0000","last_undegraded":"2024-09-12T03:27:10.059511+0000","last_fullsized":"2024-09-12T03:27:10.059511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:42:18.012217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.015426+0000","last_change":"2024-09-12T03:26:59.011315+0000","last_active":"2024-09-12T03:27:10.015426+0000","last_peered":"2024-09-12T03:27:10.015426+0000","last_clean":"2024-09-12T03:27:10.015426+0000","last_became_active":"2024-09-12T03:26:59.011028+0000","last_became_peered":"2024-09-12T03:26:59.011028+0000","last_unstale":"2024-09-12T03:27:10.015426+0000","last_undegraded":"2024-09-12T03:27:10.015426+0000","last_fullsized":"2024-09-12T03:27:10.015426+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:18:10.541578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.054746+0000","last_change":"2024-09-12T03:26:59.024518+0000","last_active":"2024-09-12T03:27:09.054746+0000","last_peered":"2024-09-12T03:27:09.054746+0000","last_clean":"2024-09-12T03:27:09.054746+0000","last_became_active":"2024-09-12T03:26:59.024197+0000","last_became_peered":"2024-09-12T03:26:59.024197+0000","last_unstale":"2024-09-12T03:27:09.054746+0000","last_undegraded":"2024-09-12T03:27:09.054746+0000","last_fullsized":"2024-09-12T03:27:09.054746+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:52.113096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059606+0000","last_change":"2024-09-12T03:26:59.018707+0000","last_active":"2024-09-12T03:27:10.059606+0000","last_peered":"2024-09-12T03:27:10.059606+0000","last_clean":"2024-09-12T03:27:10.059606+0000","last_became_active":"2024-09-12T03:26:59.017997+0000","last_became_peered":"2024-09-12T03:26:59.017997+0000","last_unstale":"2024-09-12T03:27:10.059606+0000","last_undegraded":"2024-09-12T03:27:10.059606+0000","last_fullsized":"2024-09-12T03:27:10.059606+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:37:58.896056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.054272+0000","last_change":"2024-09-12T03:26:59.013523+0000","last_active":"2024-09-12T03:27:09.054272+0000","last_peered":"2024-09-12T03:27:09.054272+0000","last_clean":"2024-09-12T03:27:09.054272+0000","last_became_active":"2024-09-12T03:26:59.013210+0000","last_became_peered":"2024-09-12T03:26:59.013210+0000","last_unstale":"2024-09-12T03:27:09.054272+0000","last_undegraded":"2024-09-12T03:27:09.054272+0000","last_fullsized":"2024-09-12T03:27:09.054272+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:03:42.390871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.055270+0000","last_change":"2024-09-12T03:26:59.016719+0000","last_active":"2024-09-12T03:27:09.055270+0000","last_peered":"2024-09-12T03:27:09.055270+0000","last_clean":"2024-09-12T03:27:09.055270+0000","last_became_active":"2024-09-12T03:26:59.015245+0000","last_became_peered":"2024-09-12T03:26:59.015245+0000","last_unstale":"2024-09-12T03:27:09.055270+0000","last_undegraded":"2024-09-12T03:27:09.055270+0000","last_fullsized":"2024-09-12T03:27:09.055270+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:12.163847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.058982+0000","last_change":"2024-09-12T03:26:59.024317+0000","last_active":"2024-09-12T03:27:10.058982+0000","last_peered":"2024-09-12T03:27:10.058982+0000","last_clean":"2024-09-12T03:27:10.058982+0000","last_became_active":"2024-09-12T03:26:59.023998+0000","last_became_peered":"2024-09-12T03:26:59.023998+0000","last_unstale":"2024-09-12T03:27:10.058982+0000","last_undegraded":"2024-09-12T03:27:10.058982+0000","last_fullsized":"2024-09-12T03:27:10.058982+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:29:11.713342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.058724+0000","last_change":"2024-09-12T03:26:59.022114+0000","last_active":"2024-09-12T03:27:10.058724+0000","last_peered":"2024-09-12T03:27:10.058724+0000","last_clean":"2024-09-12T03:27:10.058724+0000","last_became_active":"2024-09-12T03:26:59.021806+0000","last_became_peered":"2024-09-12T03:26:59.021806+0000","last_unstale":"2024-09-12T03:27:10.058724+0000","last_undegraded":"2024-09-12T03:27:10.058724+0000","last_fullsized":"2024-09-12T03:27:10.058724+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:57:36.655297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.058697+0000","last_change":"2024-09-12T03:26:59.011340+0000","last_active":"2024-09-12T03:27:10.058697+0000","last_peered":"2024-09-12T03:27:10.058697+0000","last_clean":"2024-09-12T03:27:10.058697+0000","last_became_active":"2024-09-12T03:26:59.011042+0000","last_became_peered":"2024-09-12T03:26:59.011042+0000","last_unstale":"2024-09-12T03:27:10.058697+0000","last_undegraded":"2024-09-12T03:27:10.058697+0000","last_fullsized":"2024-09-12T03:27:10.058697+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:49.568438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.015311+0000","last_change":"2024-09-12T03:26:59.016449+0000","last_active":"2024-09-12T03:27:10.015311+0000","last_peered":"2024-09-12T03:27:10.015311+0000","last_clean":"2024-09-12T03:27:10.015311+0000","last_became_active":"2024-09-12T03:26:59.016087+0000","last_became_peered":"2024-09-12T03:26:59.016087+0000","last_unstale":"2024-09-12T03:27:10.015311+0000","last_undegraded":"2024-09-12T03:27:10.015311+0000","last_fullsized":"2024-09-12T03:27:10.015311+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:13:40.683471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060849+0000","last_change":"2024-09-12T03:26:59.011272+0000","last_active":"2024-09-12T03:27:10.060849+0000","last_peered":"2024-09-12T03:27:10.060849+0000","last_clean":"2024-09-12T03:27:10.060849+0000","last_became_active":"2024-09-12T03:26:59.010937+0000","last_became_peered":"2024-09-12T03:26:59.010937+0000","last_unstale":"2024-09-12T03:27:10.060849+0000","last_undegraded":"2024-09-12T03:27:10.060849+0000","last_fullsized":"2024-09-12T03:27:10.060849+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:56:00.600771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060097+0000","last_change":"2024-09-12T03:26:59.021434+0000","last_active":"2024-09-12T03:27:10.060097+0000","last_peered":"2024-09-12T03:27:10.060097+0000","last_clean":"2024-09-12T03:27:10.060097+0000","last_became_active":"2024-09-12T03:26:59.021071+0000","last_became_peered":"2024-09-12T03:26:59.021071+0000","last_unstale":"2024-09-12T03:27:10.060097+0000","last_undegraded":"2024-09-12T03:27:10.060097+0000","last_fullsized":"2024-09-12T03:27:10.060097+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:14:46.774992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.055595+0000","last_change":"2024-09-12T03:26:59.019310+0000","last_active":"2024-09-12T03:27:09.055595+0000","last_peered":"2024-09-12T03:27:09.055595+0000","last_clean":"2024-09-12T03:27:09.055595+0000","last_became_active":"2024-09-12T03:26:59.018967+0000","last_became_peered":"2024-09-12T03:26:59.018967+0000","last_unstale":"2024-09-12T03:27:09.055595+0000","last_undegraded":"2024-09-12T03:27:09.055595+0000","last_fullsized":"2024-09-12T03:27:09.055595+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:37:10.830500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059970+0000","last_change":"2024-09-12T03:26:59.022088+0000","last_active":"2024-09-12T03:27:10.059970+0000","last_peered":"2024-09-12T03:27:10.059970+0000","last_clean":"2024-09-12T03:27:10.059970+0000","last_became_active":"2024-09-12T03:26:59.021655+0000","last_became_peered":"2024-09-12T03:26:59.021655+0000","last_unstale":"2024-09-12T03:27:10.059970+0000","last_undegraded":"2024-09-12T03:27:10.059970+0000","last_fullsized":"2024-09-12T03:27:10.059970+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:39.490597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.058671+0000","last_change":"2024-09-12T03:26:59.024898+0000","last_active":"2024-09-12T03:27:10.058671+0000","last_peered":"2024-09-12T03:27:10.058671+0000","last_clean":"2024-09-12T03:27:10.058671+0000","last_became_active":"2024-09-12T03:26:59.024511+0000","last_became_peered":"2024-09-12T03:26:59.024511+0000","last_unstale":"2024-09-12T03:27:10.058671+0000","last_undegraded":"2024-09-12T03:27:10.058671+0000","last_fullsized":"2024-09-12T03:27:10.058671+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:24.191365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059386+0000","last_change":"2024-09-12T03:26:59.024829+0000","last_active":"2024-09-12T03:27:10.059386+0000","last_peered":"2024-09-12T03:27:10.059386+0000","last_clean":"2024-09-12T03:27:10.059386+0000","last_became_active":"2024-09-12T03:26:59.024493+0000","last_became_peered":"2024-09-12T03:26:59.024493+0000","last_unstale":"2024-09-12T03:27:10.059386+0000","last_undegraded":"2024-09-12T03:27:10.059386+0000","last_fullsized":"2024-09-12T03:27:10.059386+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:28:35.322783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.055164+0000","last_change":"2024-09-12T03:26:59.014536+0000","last_active":"2024-09-12T03:27:09.055164+0000","last_peered":"2024-09-12T03:27:09.055164+0000","last_clean":"2024-09-12T03:27:09.055164+0000","last_became_active":"2024-09-12T03:26:59.013910+0000","last_became_peered":"2024-09-12T03:26:59.013910+0000","last_unstale":"2024-09-12T03:27:09.055164+0000","last_undegraded":"2024-09-12T03:27:09.055164+0000","last_fullsized":"2024-09-12T03:27:09.055164+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:27.519367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.958268+0000","last_change":"2024-09-12T03:26:59.016656+0000","last_active":"2024-09-12T03:27:09.958268+0000","last_peered":"2024-09-12T03:27:09.958268+0000","last_clean":"2024-09-12T03:27:09.958268+0000","last_became_active":"2024-09-12T03:26:59.016217+0000","last_became_peered":"2024-09-12T03:26:59.016217+0000","last_unstale":"2024-09-12T03:27:09.958268+0000","last_undegraded":"2024-09-12T03:27:09.958268+0000","last_fullsized":"2024-09-12T03:27:09.958268+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:07:42.718394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060816+0000","last_change":"2024-09-12T03:26:59.015940+0000","last_active":"2024-09-12T03:27:10.060816+0000","last_peered":"2024-09-12T03:27:10.060816+0000","last_clean":"2024-09-12T03:27:10.060816+0000","last_became_active":"2024-09-12T03:26:59.015578+0000","last_became_peered":"2024-09-12T03:26:59.015578+0000","last_unstale":"2024-09-12T03:27:10.060816+0000","last_undegraded":"2024-09-12T03:27:10.060816+0000","last_fullsized":"2024-09-12T03:27:10.060816+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:07:46.559935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.058974+0000","last_change":"2024-09-12T03:26:59.016283+0000","last_active":"2024-09-12T03:27:10.058974+0000","last_peered":"2024-09-12T03:27:10.058974+0000","last_clean":"2024-09-12T03:27:10.058974+0000","last_became_active":"2024-09-12T03:26:59.015109+0000","last_became_peered":"2024-09-12T03:26:59.015109+0000","last_unstale":"2024-09-12T03:27:10.058974+0000","last_undegraded":"2024-09-12T03:27:10.058974+0000","last_fullsized":"2024-09-12T03:27:10.058974+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:02:44.493393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059084+0000","last_change":"2024-09-12T03:26:59.020161+0000","last_active":"2024-09-12T03:27:10.059084+0000","last_peered":"2024-09-12T03:27:10.059084+0000","last_clean":"2024-09-12T03:27:10.059084+0000","last_became_active":"2024-09-12T03:26:59.019796+0000","last_became_peered":"2024-09-12T03:26:59.019796+0000","last_unstale":"2024-09-12T03:27:10.059084+0000","last_undegraded":"2024-09-12T03:27:10.059084+0000","last_fullsized":"2024-09-12T03:27:10.059084+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:04:02.817005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.055202+0000","last_change":"2024-09-12T03:26:59.015763+0000","last_active":"2024-09-12T03:27:09.055202+0000","last_peered":"2024-09-12T03:27:09.055202+0000","last_clean":"2024-09-12T03:27:09.055202+0000","last_became_active":"2024-09-12T03:26:59.014979+0000","last_became_peered":"2024-09-12T03:26:59.014979+0000","last_unstale":"2024-09-12T03:27:09.055202+0000","last_undegraded":"2024-09-12T03:27:09.055202+0000","last_fullsized":"2024-09-12T03:27:09.055202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:49:26.959019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":47348,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":99,"num_read_kb":101,"num_write":158,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":368640,"data_stored":142044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":247982,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514519,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":489415,"data_compressed":17549,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776155,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1093632,"data_stored":1120421,"data_compressed":53677,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939681,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":493747,"data_compressed":17549,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201317,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":489571,"data_compressed":17549,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430250,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":1085482,"data_compressed":53677,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593776,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":484827,"data_compressed":17549,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822708,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":1080677,"data_compressed":53677,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953529,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":489870,"data_compressed":17549,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215166,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":485021,"data_compressed":17549,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1036288,"data_stored":529432,"data_compressed":17549,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":528082,"data_compressed":17549,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738446,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":987136,"data_stored":486215,"data_compressed":17549,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34490,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34490,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34490,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36128,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36128,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36128,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36128,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":43345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":44695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":45316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:27:51.597 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T03:27:51.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:51 smithi115 ceph-mon[34801]: pgmap v309: 129 pgs: 129 active+clean; 623 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.0 KiB/s wr, 9 op/s 2024-09-12T03:27:51.857 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:52.271 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:52 smithi064 ceph-mon[30952]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:27:52.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.380+0000 7ff538b5f640 1 -- 172.21.15.64:0/3674467010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff534102790 msgr2=0x7ff5341067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:52.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.380+0000 7ff538b5f640 1 --2- 172.21.15.64:0/3674467010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff534102790 0x7ff5341067e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff51c009900 tx=0x7ff51c02a410 comp rx=0 tx=0).stop 2024-09-12T03:27:52.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.380+0000 7ff538b5f640 1 -- 172.21.15.64:0/3674467010 shutdown_connections 2024-09-12T03:27:52.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.380+0000 7ff538b5f640 1 --2- 172.21.15.64:0/3674467010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff534106f10 0x7ff53410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:52.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.380+0000 7ff538b5f640 1 --2- 172.21.15.64:0/3674467010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff534102790 0x7ff5341067e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:52.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.380+0000 7ff538b5f640 1 --2- 172.21.15.64:0/3674467010 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff534101de0 0x7ff5341021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:52.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.380+0000 7ff538b5f640 1 -- 172.21.15.64:0/3674467010 >> 172.21.15.64:0/3674467010 conn(0x7ff5340fd8c0 msgr2=0x7ff5340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:52.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.380+0000 7ff538b5f640 1 -- 172.21.15.64:0/3674467010 shutdown_connections 2024-09-12T03:27:52.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.380+0000 7ff538b5f640 1 -- 172.21.15.64:0/3674467010 wait complete. 2024-09-12T03:27:52.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.381+0000 7ff538b5f640 1 Processor -- start 2024-09-12T03:27:52.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.381+0000 7ff538b5f640 1 -- start start 2024-09-12T03:27:52.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.381+0000 7ff538b5f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff534101de0 0x7ff53419a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:52.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.381+0000 7ff538b5f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff534102790 0x7ff53419aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:52.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.381+0000 7ff538b5f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff534106f10 0x7ff53419ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:52.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.381+0000 7ff538b5f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff534111e20 con 0x7ff534102790 2024-09-12T03:27:52.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.381+0000 7ff538b5f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff534111ca0 con 0x7ff534101de0 2024-09-12T03:27:52.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.381+0000 7ff538b5f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff534111fa0 con 0x7ff534106f10 2024-09-12T03:27:52.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.381+0000 7ff531d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff534102790 0x7ff53419aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:52.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.381+0000 7ff531d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff534102790 0x7ff53419aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51604/0 (socket says 172.21.15.64:51604) 2024-09-12T03:27:52.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.381+0000 7ff531d9b640 1 -- 172.21.15.64:0/971604169 learned_addr learned my addr 172.21.15.64:0/971604169 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:27:52.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.381+0000 7ff53259c640 1 --2- 172.21.15.64:0/971604169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff534101de0 0x7ff53419a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:52.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.381+0000 7ff532d9d640 1 --2- 172.21.15.64:0/971604169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff534106f10 0x7ff53419ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:52.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.381+0000 7ff531d9b640 1 -- 172.21.15.64:0/971604169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff534106f10 msgr2=0x7ff53419ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:52.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.381+0000 7ff531d9b640 1 --2- 172.21.15.64:0/971604169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff534106f10 0x7ff53419ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:52.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.381+0000 7ff531d9b640 1 -- 172.21.15.64:0/971604169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff534101de0 msgr2=0x7ff53419a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:52.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.381+0000 7ff531d9b640 1 --2- 172.21.15.64:0/971604169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff534101de0 0x7ff53419a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:52.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.381+0000 7ff531d9b640 1 -- 172.21.15.64:0/971604169 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff53419f550 con 0x7ff534102790 2024-09-12T03:27:52.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.381+0000 7ff532d9d640 1 --2- 172.21.15.64:0/971604169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff534106f10 0x7ff53419ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:27:52.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.382+0000 7ff531d9b640 1 --2- 172.21.15.64:0/971604169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff534102790 0x7ff53419aae0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff51c005940 tx=0x7ff51c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:52.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.382+0000 7ff52b7fe640 1 -- 172.21.15.64:0/971604169 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51c03f030 con 0x7ff534102790 2024-09-12T03:27:52.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.382+0000 7ff52b7fe640 1 -- 172.21.15.64:0/971604169 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff51c003aa0 con 0x7ff534102790 2024-09-12T03:27:52.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.382+0000 7ff538b5f640 1 -- 172.21.15.64:0/971604169 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff53419f7e0 con 0x7ff534102790 2024-09-12T03:27:52.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.382+0000 7ff52b7fe640 1 -- 172.21.15.64:0/971604169 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51c035070 con 0x7ff534102790 2024-09-12T03:27:52.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.382+0000 7ff538b5f640 1 -- 172.21.15.64:0/971604169 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5341a7120 con 0x7ff534102790 2024-09-12T03:27:52.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.383+0000 7ff538b5f640 1 -- 172.21.15.64:0/971604169 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5341021c0 con 0x7ff534102790 2024-09-12T03:27:52.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.384+0000 7ff52b7fe640 1 -- 172.21.15.64:0/971604169 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff51c042030 con 0x7ff534102790 2024-09-12T03:27:52.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.385+0000 7ff52b7fe640 1 --2- 172.21.15.64:0/971604169 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff5200a2c10 0x7ff5200a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:52.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.385+0000 7ff53259c640 1 --2- 172.21.15.64:0/971604169 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff5200a2c10 0x7ff5200a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:52.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.385+0000 7ff52b7fe640 1 -- 172.21.15.64:0/971604169 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7ff51c0ef660 con 0x7ff534102790 2024-09-12T03:27:52.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.385+0000 7ff53259c640 1 --2- 172.21.15.64:0/971604169 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff5200a2c10 0x7ff5200a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff524005fd0 tx=0x7ff52400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:52.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.389+0000 7ff52b7fe640 1 -- 172.21.15.64:0/971604169 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff51c0bc220 con 0x7ff534102790 2024-09-12T03:27:52.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:52 smithi195 ceph-mon[34786]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:27:52.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.565+0000 7ff538b5f640 1 -- 172.21.15.64:0/971604169 --> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff534075430 con 0x7ff5200a2c10 2024-09-12T03:27:52.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.573+0000 7ff52b7fe640 1 -- 172.21.15.64:0/971604169 <== mgr.14150 v2:172.21.15.64:6800/759931848 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340591 (secure 0 0 0) 0x7ff534075430 con 0x7ff5200a2c10 2024-09-12T03:27:52.576 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:27:52.578 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-09-12T03:27:52.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.576+0000 7ff538b5f640 1 -- 172.21.15.64:0/971604169 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff5200a2c10 msgr2=0x7ff5200a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:52.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.576+0000 7ff538b5f640 1 --2- 172.21.15.64:0/971604169 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff5200a2c10 0x7ff5200a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff524005fd0 tx=0x7ff52400b390 comp rx=0 tx=0).stop 2024-09-12T03:27:52.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.576+0000 7ff538b5f640 1 -- 172.21.15.64:0/971604169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff534102790 msgr2=0x7ff53419aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:52.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.576+0000 7ff538b5f640 1 --2- 172.21.15.64:0/971604169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff534102790 0x7ff53419aae0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff51c005940 tx=0x7ff51c004270 comp rx=0 tx=0).stop 2024-09-12T03:27:52.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.576+0000 7ff538b5f640 1 -- 172.21.15.64:0/971604169 shutdown_connections 2024-09-12T03:27:52.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.576+0000 7ff538b5f640 1 --2- 172.21.15.64:0/971604169 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff5200a2c10 0x7ff5200a50d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:52.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.576+0000 7ff538b5f640 1 --2- 172.21.15.64:0/971604169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff534106f10 0x7ff53419ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:52.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.576+0000 7ff538b5f640 1 --2- 172.21.15.64:0/971604169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff534102790 0x7ff53419aae0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:52.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.576+0000 7ff538b5f640 1 --2- 172.21.15.64:0/971604169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff534101de0 0x7ff53419a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:52.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.576+0000 7ff538b5f640 1 -- 172.21.15.64:0/971604169 >> 172.21.15.64:0/971604169 conn(0x7ff5340fd8c0 msgr2=0x7ff5340fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:52.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.577+0000 7ff538b5f640 1 -- 172.21.15.64:0/971604169 shutdown_connections 2024-09-12T03:27:52.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:52.577+0000 7ff538b5f640 1 -- 172.21.15.64:0/971604169 wait complete. 2024-09-12T03:27:52.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:52 smithi115 ceph-mon[34801]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:27:53.269 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-09-12T03:27:51.504175+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651278,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":256,"num_read_kb":244,"num_write":345,"num_write_kb":3048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":278,"ondisk_log_size":278,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365200,"kb_used_data":11792,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576624,"statfs":{"total":1151940427776,"available":1151566462976,"internally_reserved":0,"allocated":12075008,"data_stored":7789416,"data_compressed":318972,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":53456,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":95,"num_write":30,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003254"},"pg_stats":[{"pgid":"2.1f","version":"88'9","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:34.143755+0000","last_change":"2024-09-12T03:26:56.956415+0000","last_active":"2024-09-12T03:27:34.143755+0000","last_peered":"2024-09-12T03:27:34.143755+0000","last_clean":"2024-09-12T03:27:34.143755+0000","last_became_active":"2024-09-12T03:26:56.956089+0000","last_became_peered":"2024-09-12T03:26:56.956089+0000","last_unstale":"2024-09-12T03:27:34.143755+0000","last_undegraded":"2024-09-12T03:27:34.143755+0000","last_fullsized":"2024-09-12T03:27:34.143755+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:32:22.089322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059577+0000","last_change":"2024-09-12T03:26:59.024102+0000","last_active":"2024-09-12T03:27:10.059577+0000","last_peered":"2024-09-12T03:27:10.059577+0000","last_clean":"2024-09-12T03:27:10.059577+0000","last_became_active":"2024-09-12T03:26:59.023795+0000","last_became_peered":"2024-09-12T03:26:59.023795+0000","last_unstale":"2024-09-12T03:27:10.059577+0000","last_undegraded":"2024-09-12T03:27:10.059577+0000","last_fullsized":"2024-09-12T03:27:10.059577+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:25:55.392622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059419+0000","last_change":"2024-09-12T03:27:06.052565+0000","last_active":"2024-09-12T03:27:10.059419+0000","last_peered":"2024-09-12T03:27:10.059419+0000","last_clean":"2024-09-12T03:27:10.059419+0000","last_became_active":"2024-09-12T03:27:06.052209+0000","last_became_peered":"2024-09-12T03:27:06.052209+0000","last_unstale":"2024-09-12T03:27:10.059419+0000","last_undegraded":"2024-09-12T03:27:10.059419+0000","last_fullsized":"2024-09-12T03:27:10.059419+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:24:28.153740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059776+0000","last_change":"2024-09-12T03:26:57.018418+0000","last_active":"2024-09-12T03:27:10.059776+0000","last_peered":"2024-09-12T03:27:10.059776+0000","last_clean":"2024-09-12T03:27:10.059776+0000","last_became_active":"2024-09-12T03:26:57.018081+0000","last_became_peered":"2024-09-12T03:26:57.018081+0000","last_unstale":"2024-09-12T03:27:10.059776+0000","last_undegraded":"2024-09-12T03:27:10.059776+0000","last_fullsized":"2024-09-12T03:27:10.059776+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:19.954332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059140+0000","last_change":"2024-09-12T03:26:59.019632+0000","last_active":"2024-09-12T03:27:10.059140+0000","last_peered":"2024-09-12T03:27:10.059140+0000","last_clean":"2024-09-12T03:27:10.059140+0000","last_became_active":"2024-09-12T03:26:59.016847+0000","last_became_peered":"2024-09-12T03:26:59.016847+0000","last_unstale":"2024-09-12T03:27:10.059140+0000","last_undegraded":"2024-09-12T03:27:10.059140+0000","last_fullsized":"2024-09-12T03:27:10.059140+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:32:28.261954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060327+0000","last_change":"2024-09-12T03:27:06.045628+0000","last_active":"2024-09-12T03:27:10.060327+0000","last_peered":"2024-09-12T03:27:10.060327+0000","last_clean":"2024-09-12T03:27:10.060327+0000","last_became_active":"2024-09-12T03:27:06.045141+0000","last_became_peered":"2024-09-12T03:27:06.045141+0000","last_unstale":"2024-09-12T03:27:10.060327+0000","last_undegraded":"2024-09-12T03:27:10.060327+0000","last_fullsized":"2024-09-12T03:27:10.060327+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:10:26.154446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:34.141754+0000","last_change":"2024-09-12T03:26:56.958615+0000","last_active":"2024-09-12T03:27:34.141754+0000","last_peered":"2024-09-12T03:27:34.141754+0000","last_clean":"2024-09-12T03:27:34.141754+0000","last_became_active":"2024-09-12T03:26:56.958295+0000","last_became_peered":"2024-09-12T03:26:56.958295+0000","last_unstale":"2024-09-12T03:27:34.141754+0000","last_undegraded":"2024-09-12T03:27:34.141754+0000","last_fullsized":"2024-09-12T03:27:34.141754+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:13:09.031983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059797+0000","last_change":"2024-09-12T03:26:59.015376+0000","last_active":"2024-09-12T03:27:10.059797+0000","last_peered":"2024-09-12T03:27:10.059797+0000","last_clean":"2024-09-12T03:27:10.059797+0000","last_became_active":"2024-09-12T03:26:59.015055+0000","last_became_peered":"2024-09-12T03:26:59.015055+0000","last_unstale":"2024-09-12T03:27:10.059797+0000","last_undegraded":"2024-09-12T03:27:10.059797+0000","last_fullsized":"2024-09-12T03:27:10.059797+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:44:38.077558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.061132+0000","last_change":"2024-09-12T03:27:06.042785+0000","last_active":"2024-09-12T03:27:10.061132+0000","last_peered":"2024-09-12T03:27:10.061132+0000","last_clean":"2024-09-12T03:27:10.061132+0000","last_became_active":"2024-09-12T03:27:06.042379+0000","last_became_peered":"2024-09-12T03:27:06.042379+0000","last_unstale":"2024-09-12T03:27:10.061132+0000","last_undegraded":"2024-09-12T03:27:10.061132+0000","last_fullsized":"2024-09-12T03:27:10.061132+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:08:51.683470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.958667+0000","last_change":"2024-09-12T03:26:56.955846+0000","last_active":"2024-09-12T03:27:09.958667+0000","last_peered":"2024-09-12T03:27:09.958667+0000","last_clean":"2024-09-12T03:27:09.958667+0000","last_became_active":"2024-09-12T03:26:56.955476+0000","last_became_peered":"2024-09-12T03:26:56.955476+0000","last_unstale":"2024-09-12T03:27:09.958667+0000","last_undegraded":"2024-09-12T03:27:09.958667+0000","last_fullsized":"2024-09-12T03:27:09.958667+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:10:46.811039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060357+0000","last_change":"2024-09-12T03:26:59.021498+0000","last_active":"2024-09-12T03:27:10.060357+0000","last_peered":"2024-09-12T03:27:10.060357+0000","last_clean":"2024-09-12T03:27:10.060357+0000","last_became_active":"2024-09-12T03:26:59.021196+0000","last_became_peered":"2024-09-12T03:26:59.021196+0000","last_unstale":"2024-09-12T03:27:10.060357+0000","last_undegraded":"2024-09-12T03:27:10.060357+0000","last_fullsized":"2024-09-12T03:27:10.060357+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:13:17.537648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060494+0000","last_change":"2024-09-12T03:27:06.040597+0000","last_active":"2024-09-12T03:27:10.060494+0000","last_peered":"2024-09-12T03:27:10.060494+0000","last_clean":"2024-09-12T03:27:10.060494+0000","last_became_active":"2024-09-12T03:27:06.040149+0000","last_became_peered":"2024-09-12T03:27:06.040149+0000","last_unstale":"2024-09-12T03:27:10.060494+0000","last_undegraded":"2024-09-12T03:27:10.060494+0000","last_fullsized":"2024-09-12T03:27:10.060494+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:27.192367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060247+0000","last_change":"2024-09-12T03:26:59.020542+0000","last_active":"2024-09-12T03:27:10.060247+0000","last_peered":"2024-09-12T03:27:10.060247+0000","last_clean":"2024-09-12T03:27:10.060247+0000","last_became_active":"2024-09-12T03:26:59.020189+0000","last_became_peered":"2024-09-12T03:26:59.020189+0000","last_unstale":"2024-09-12T03:27:10.060247+0000","last_undegraded":"2024-09-12T03:27:10.060247+0000","last_fullsized":"2024-09-12T03:27:10.060247+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:06:12.761331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:33.133816+0000","last_change":"2024-09-12T03:26:56.958320+0000","last_active":"2024-09-12T03:27:33.133816+0000","last_peered":"2024-09-12T03:27:33.133816+0000","last_clean":"2024-09-12T03:27:33.133816+0000","last_became_active":"2024-09-12T03:26:56.956965+0000","last_became_peered":"2024-09-12T03:26:56.956965+0000","last_unstale":"2024-09-12T03:27:33.133816+0000","last_undegraded":"2024-09-12T03:27:33.133816+0000","last_fullsized":"2024-09-12T03:27:33.133816+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:46.685994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060669+0000","last_change":"2024-09-12T03:27:06.189973+0000","last_active":"2024-09-12T03:27:10.060669+0000","last_peered":"2024-09-12T03:27:10.060669+0000","last_clean":"2024-09-12T03:27:10.060669+0000","last_became_active":"2024-09-12T03:27:06.189454+0000","last_became_peered":"2024-09-12T03:27:06.189454+0000","last_unstale":"2024-09-12T03:27:10.060669+0000","last_undegraded":"2024-09-12T03:27:10.060669+0000","last_fullsized":"2024-09-12T03:27:10.060669+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:25.130566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059324+0000","last_change":"2024-09-12T03:26:59.014586+0000","last_active":"2024-09-12T03:27:10.059324+0000","last_peered":"2024-09-12T03:27:10.059324+0000","last_clean":"2024-09-12T03:27:10.059324+0000","last_became_active":"2024-09-12T03:26:59.013764+0000","last_became_peered":"2024-09-12T03:26:59.013764+0000","last_unstale":"2024-09-12T03:27:10.059324+0000","last_undegraded":"2024-09-12T03:27:10.059324+0000","last_fullsized":"2024-09-12T03:27:10.059324+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:55:34.136770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059248+0000","last_change":"2024-09-12T03:26:56.960415+0000","last_active":"2024-09-12T03:27:10.059248+0000","last_peered":"2024-09-12T03:27:10.059248+0000","last_clean":"2024-09-12T03:27:10.059248+0000","last_became_active":"2024-09-12T03:26:56.960043+0000","last_became_peered":"2024-09-12T03:26:56.960043+0000","last_unstale":"2024-09-12T03:27:10.059248+0000","last_undegraded":"2024-09-12T03:27:10.059248+0000","last_fullsized":"2024-09-12T03:27:10.059248+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:02:12.209929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060777+0000","last_change":"2024-09-12T03:27:06.047002+0000","last_active":"2024-09-12T03:27:10.060777+0000","last_peered":"2024-09-12T03:27:10.060777+0000","last_clean":"2024-09-12T03:27:10.060777+0000","last_became_active":"2024-09-12T03:27:06.046520+0000","last_became_peered":"2024-09-12T03:27:06.046520+0000","last_unstale":"2024-09-12T03:27:10.060777+0000","last_undegraded":"2024-09-12T03:27:10.060777+0000","last_fullsized":"2024-09-12T03:27:10.060777+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:30.332831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.054604+0000","last_change":"2024-09-12T03:26:59.016811+0000","last_active":"2024-09-12T03:27:09.054604+0000","last_peered":"2024-09-12T03:27:09.054604+0000","last_clean":"2024-09-12T03:27:09.054604+0000","last_became_active":"2024-09-12T03:26:59.014667+0000","last_became_peered":"2024-09-12T03:26:59.014667+0000","last_unstale":"2024-09-12T03:27:09.054604+0000","last_undegraded":"2024-09-12T03:27:09.054604+0000","last_fullsized":"2024-09-12T03:27:09.054604+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:28:51.128895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:33.132610+0000","last_change":"2024-09-12T03:26:56.956177+0000","last_active":"2024-09-12T03:27:33.132610+0000","last_peered":"2024-09-12T03:27:33.132610+0000","last_clean":"2024-09-12T03:27:33.132610+0000","last_became_active":"2024-09-12T03:26:56.955730+0000","last_became_peered":"2024-09-12T03:26:56.955730+0000","last_unstale":"2024-09-12T03:27:33.132610+0000","last_undegraded":"2024-09-12T03:27:33.132610+0000","last_fullsized":"2024-09-12T03:27:33.132610+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:05:18.890917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.053901+0000","last_change":"2024-09-12T03:27:06.050894+0000","last_active":"2024-09-12T03:27:09.053901+0000","last_peered":"2024-09-12T03:27:09.053901+0000","last_clean":"2024-09-12T03:27:09.053901+0000","last_became_active":"2024-09-12T03:27:06.050440+0000","last_became_peered":"2024-09-12T03:27:06.050440+0000","last_unstale":"2024-09-12T03:27:09.053901+0000","last_undegraded":"2024-09-12T03:27:09.053901+0000","last_fullsized":"2024-09-12T03:27:09.053901+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:31:25.559665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.958146+0000","last_change":"2024-09-12T03:26:59.018167+0000","last_active":"2024-09-12T03:27:09.958146+0000","last_peered":"2024-09-12T03:27:09.958146+0000","last_clean":"2024-09-12T03:27:09.958146+0000","last_became_active":"2024-09-12T03:26:59.017718+0000","last_became_peered":"2024-09-12T03:26:59.017718+0000","last_unstale":"2024-09-12T03:27:09.958146+0000","last_undegraded":"2024-09-12T03:27:09.958146+0000","last_fullsized":"2024-09-12T03:27:09.958146+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:36:12.742570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:33.133529+0000","last_change":"2024-09-12T03:26:57.018829+0000","last_active":"2024-09-12T03:27:33.133529+0000","last_peered":"2024-09-12T03:27:33.133529+0000","last_clean":"2024-09-12T03:27:33.133529+0000","last_became_active":"2024-09-12T03:26:57.018467+0000","last_became_peered":"2024-09-12T03:26:57.018467+0000","last_unstale":"2024-09-12T03:27:33.133529+0000","last_undegraded":"2024-09-12T03:27:33.133529+0000","last_fullsized":"2024-09-12T03:27:33.133529+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:33:37.318703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060987+0000","last_change":"2024-09-12T03:27:06.190560+0000","last_active":"2024-09-12T03:27:10.060987+0000","last_peered":"2024-09-12T03:27:10.060987+0000","last_clean":"2024-09-12T03:27:10.060987+0000","last_became_active":"2024-09-12T03:27:06.190008+0000","last_became_peered":"2024-09-12T03:27:06.190008+0000","last_unstale":"2024-09-12T03:27:10.060987+0000","last_undegraded":"2024-09-12T03:27:10.060987+0000","last_fullsized":"2024-09-12T03:27:10.060987+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:14.423372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059131+0000","last_change":"2024-09-12T03:26:59.022149+0000","last_active":"2024-09-12T03:27:10.059131+0000","last_peered":"2024-09-12T03:27:10.059131+0000","last_clean":"2024-09-12T03:27:10.059131+0000","last_became_active":"2024-09-12T03:26:59.021830+0000","last_became_peered":"2024-09-12T03:26:59.021830+0000","last_unstale":"2024-09-12T03:27:10.059131+0000","last_undegraded":"2024-09-12T03:27:10.059131+0000","last_fullsized":"2024-09-12T03:27:10.059131+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:07.096908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:33.130934+0000","last_change":"2024-09-12T03:26:57.018335+0000","last_active":"2024-09-12T03:27:33.130934+0000","last_peered":"2024-09-12T03:27:33.130934+0000","last_clean":"2024-09-12T03:27:33.130934+0000","last_became_active":"2024-09-12T03:26:57.017993+0000","last_became_peered":"2024-09-12T03:26:57.017993+0000","last_unstale":"2024-09-12T03:27:33.130934+0000","last_undegraded":"2024-09-12T03:27:33.130934+0000","last_fullsized":"2024-09-12T03:27:33.130934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:27:52.828836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059108+0000","last_change":"2024-09-12T03:27:06.051146+0000","last_active":"2024-09-12T03:27:10.059108+0000","last_peered":"2024-09-12T03:27:10.059108+0000","last_clean":"2024-09-12T03:27:10.059108+0000","last_became_active":"2024-09-12T03:27:06.050599+0000","last_became_peered":"2024-09-12T03:27:06.050599+0000","last_unstale":"2024-09-12T03:27:10.059108+0000","last_undegraded":"2024-09-12T03:27:10.059108+0000","last_fullsized":"2024-09-12T03:27:10.059108+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:17:57.543994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059857+0000","last_change":"2024-09-12T03:26:59.024832+0000","last_active":"2024-09-12T03:27:10.059857+0000","last_peered":"2024-09-12T03:27:10.059857+0000","last_clean":"2024-09-12T03:27:10.059857+0000","last_became_active":"2024-09-12T03:26:59.024473+0000","last_became_peered":"2024-09-12T03:26:59.024473+0000","last_unstale":"2024-09-12T03:27:10.059857+0000","last_undegraded":"2024-09-12T03:27:10.059857+0000","last_fullsized":"2024-09-12T03:27:10.059857+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:05:54.568496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'10","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:45.013576+0000","last_change":"2024-09-12T03:26:56.955213+0000","last_active":"2024-09-12T03:27:45.013576+0000","last_peered":"2024-09-12T03:27:45.013576+0000","last_clean":"2024-09-12T03:27:45.013576+0000","last_became_active":"2024-09-12T03:26:56.954746+0000","last_became_peered":"2024-09-12T03:26:56.954746+0000","last_unstale":"2024-09-12T03:27:45.013576+0000","last_undegraded":"2024-09-12T03:27:45.013576+0000","last_fullsized":"2024-09-12T03:27:45.013576+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:03.303931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.058609+0000","last_change":"2024-09-12T03:27:06.190398+0000","last_active":"2024-09-12T03:27:10.058609+0000","last_peered":"2024-09-12T03:27:10.058609+0000","last_clean":"2024-09-12T03:27:10.058609+0000","last_became_active":"2024-09-12T03:27:06.189861+0000","last_became_peered":"2024-09-12T03:27:06.189861+0000","last_unstale":"2024-09-12T03:27:10.058609+0000","last_undegraded":"2024-09-12T03:27:10.058609+0000","last_fullsized":"2024-09-12T03:27:10.058609+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:36:57.366931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.015434+0000","last_change":"2024-09-12T03:26:59.012702+0000","last_active":"2024-09-12T03:27:10.015434+0000","last_peered":"2024-09-12T03:27:10.015434+0000","last_clean":"2024-09-12T03:27:10.015434+0000","last_became_active":"2024-09-12T03:26:59.012421+0000","last_became_peered":"2024-09-12T03:26:59.012421+0000","last_unstale":"2024-09-12T03:27:10.015434+0000","last_undegraded":"2024-09-12T03:27:10.015434+0000","last_fullsized":"2024-09-12T03:27:10.015434+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:21:23.866131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:33.132212+0000","last_change":"2024-09-12T03:26:57.019744+0000","last_active":"2024-09-12T03:27:33.132212+0000","last_peered":"2024-09-12T03:27:33.132212+0000","last_clean":"2024-09-12T03:27:33.132212+0000","last_became_active":"2024-09-12T03:26:57.019397+0000","last_became_peered":"2024-09-12T03:26:57.019397+0000","last_unstale":"2024-09-12T03:27:33.132212+0000","last_undegraded":"2024-09-12T03:27:33.132212+0000","last_fullsized":"2024-09-12T03:27:33.132212+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:44.891829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.054722+0000","last_change":"2024-09-12T03:27:06.048452+0000","last_active":"2024-09-12T03:27:09.054722+0000","last_peered":"2024-09-12T03:27:09.054722+0000","last_clean":"2024-09-12T03:27:09.054722+0000","last_became_active":"2024-09-12T03:27:06.048011+0000","last_became_peered":"2024-09-12T03:27:06.048011+0000","last_unstale":"2024-09-12T03:27:09.054722+0000","last_undegraded":"2024-09-12T03:27:09.054722+0000","last_fullsized":"2024-09-12T03:27:09.054722+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:20:38.360428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.055374+0000","last_change":"2024-09-12T03:26:59.017212+0000","last_active":"2024-09-12T03:27:09.055374+0000","last_peered":"2024-09-12T03:27:09.055374+0000","last_clean":"2024-09-12T03:27:09.055374+0000","last_became_active":"2024-09-12T03:26:59.016605+0000","last_became_peered":"2024-09-12T03:26:59.016605+0000","last_unstale":"2024-09-12T03:27:09.055374+0000","last_undegraded":"2024-09-12T03:27:09.055374+0000","last_fullsized":"2024-09-12T03:27:09.055374+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:34.639464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.054076+0000","last_change":"2024-09-12T03:26:57.020236+0000","last_active":"2024-09-12T03:27:09.054076+0000","last_peered":"2024-09-12T03:27:09.054076+0000","last_clean":"2024-09-12T03:27:09.054076+0000","last_became_active":"2024-09-12T03:26:57.019899+0000","last_became_peered":"2024-09-12T03:26:57.019899+0000","last_unstale":"2024-09-12T03:27:09.054076+0000","last_undegraded":"2024-09-12T03:27:09.054076+0000","last_fullsized":"2024-09-12T03:27:09.054076+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:03:16.184419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.958270+0000","last_change":"2024-09-12T03:27:06.190268+0000","last_active":"2024-09-12T03:27:09.958270+0000","last_peered":"2024-09-12T03:27:09.958270+0000","last_clean":"2024-09-12T03:27:09.958270+0000","last_became_active":"2024-09-12T03:27:06.189680+0000","last_became_peered":"2024-09-12T03:27:06.189680+0000","last_unstale":"2024-09-12T03:27:09.958270+0000","last_undegraded":"2024-09-12T03:27:09.958270+0000","last_fullsized":"2024-09-12T03:27:09.958270+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:03:00.098788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.055207+0000","last_change":"2024-09-12T03:26:59.014332+0000","last_active":"2024-09-12T03:27:09.055207+0000","last_peered":"2024-09-12T03:27:09.055207+0000","last_clean":"2024-09-12T03:27:09.055207+0000","last_became_active":"2024-09-12T03:26:59.014019+0000","last_became_peered":"2024-09-12T03:26:59.014019+0000","last_unstale":"2024-09-12T03:27:09.055207+0000","last_undegraded":"2024-09-12T03:27:09.055207+0000","last_fullsized":"2024-09-12T03:27:09.055207+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:03:18.020391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060457+0000","last_change":"2024-09-12T03:26:56.956591+0000","last_active":"2024-09-12T03:27:10.060457+0000","last_peered":"2024-09-12T03:27:10.060457+0000","last_clean":"2024-09-12T03:27:10.060457+0000","last_became_active":"2024-09-12T03:26:56.956028+0000","last_became_peered":"2024-09-12T03:26:56.956028+0000","last_unstale":"2024-09-12T03:27:10.060457+0000","last_undegraded":"2024-09-12T03:27:10.060457+0000","last_fullsized":"2024-09-12T03:27:10.060457+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:55:20.500536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.015329+0000","last_change":"2024-09-12T03:27:06.052630+0000","last_active":"2024-09-12T03:27:10.015329+0000","last_peered":"2024-09-12T03:27:10.015329+0000","last_clean":"2024-09-12T03:27:10.015329+0000","last_became_active":"2024-09-12T03:27:06.052240+0000","last_became_peered":"2024-09-12T03:27:06.052240+0000","last_unstale":"2024-09-12T03:27:10.015329+0000","last_undegraded":"2024-09-12T03:27:10.015329+0000","last_fullsized":"2024-09-12T03:27:10.015329+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:26:49.356330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059858+0000","last_change":"2024-09-12T03:26:59.022054+0000","last_active":"2024-09-12T03:27:10.059858+0000","last_peered":"2024-09-12T03:27:10.059858+0000","last_clean":"2024-09-12T03:27:10.059858+0000","last_became_active":"2024-09-12T03:26:59.021673+0000","last_became_peered":"2024-09-12T03:26:59.021673+0000","last_unstale":"2024-09-12T03:27:10.059858+0000","last_undegraded":"2024-09-12T03:27:10.059858+0000","last_fullsized":"2024-09-12T03:27:10.059858+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:07:07.581766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:46.161308+0000","last_change":"2024-09-12T03:26:57.018607+0000","last_active":"2024-09-12T03:27:46.161308+0000","last_peered":"2024-09-12T03:27:46.161308+0000","last_clean":"2024-09-12T03:27:46.161308+0000","last_became_active":"2024-09-12T03:26:57.018293+0000","last_became_peered":"2024-09-12T03:26:57.018293+0000","last_unstale":"2024-09-12T03:27:46.161308+0000","last_undegraded":"2024-09-12T03:27:46.161308+0000","last_fullsized":"2024-09-12T03:27:46.161308+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:23:57.245446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.015517+0000","last_change":"2024-09-12T03:27:06.190452+0000","last_active":"2024-09-12T03:27:10.015517+0000","last_peered":"2024-09-12T03:27:10.015517+0000","last_clean":"2024-09-12T03:27:10.015517+0000","last_became_active":"2024-09-12T03:27:06.189940+0000","last_became_peered":"2024-09-12T03:27:06.189940+0000","last_unstale":"2024-09-12T03:27:10.015517+0000","last_undegraded":"2024-09-12T03:27:10.015517+0000","last_fullsized":"2024-09-12T03:27:10.015517+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:25:23.114124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.014969+0000","last_change":"2024-09-12T03:26:59.019294+0000","last_active":"2024-09-12T03:27:10.014969+0000","last_peered":"2024-09-12T03:27:10.014969+0000","last_clean":"2024-09-12T03:27:10.014969+0000","last_became_active":"2024-09-12T03:26:59.018976+0000","last_became_peered":"2024-09-12T03:26:59.018976+0000","last_unstale":"2024-09-12T03:27:10.014969+0000","last_undegraded":"2024-09-12T03:27:10.014969+0000","last_fullsized":"2024-09-12T03:27:10.014969+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:06:38.571171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:34.142372+0000","last_change":"2024-09-12T03:26:56.960077+0000","last_active":"2024-09-12T03:27:34.142372+0000","last_peered":"2024-09-12T03:27:34.142372+0000","last_clean":"2024-09-12T03:27:34.142372+0000","last_became_active":"2024-09-12T03:26:56.958892+0000","last_became_peered":"2024-09-12T03:26:56.958892+0000","last_unstale":"2024-09-12T03:27:34.142372+0000","last_undegraded":"2024-09-12T03:27:34.142372+0000","last_fullsized":"2024-09-12T03:27:34.142372+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:27:04.779026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060256+0000","last_change":"2024-09-12T03:27:06.190439+0000","last_active":"2024-09-12T03:27:10.060256+0000","last_peered":"2024-09-12T03:27:10.060256+0000","last_clean":"2024-09-12T03:27:10.060256+0000","last_became_active":"2024-09-12T03:27:06.190010+0000","last_became_peered":"2024-09-12T03:27:06.190010+0000","last_unstale":"2024-09-12T03:27:10.060256+0000","last_undegraded":"2024-09-12T03:27:10.060256+0000","last_fullsized":"2024-09-12T03:27:10.060256+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:54:17.362910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059758+0000","last_change":"2024-09-12T03:26:59.024060+0000","last_active":"2024-09-12T03:27:10.059758+0000","last_peered":"2024-09-12T03:27:10.059758+0000","last_clean":"2024-09-12T03:27:10.059758+0000","last_became_active":"2024-09-12T03:26:59.023726+0000","last_became_peered":"2024-09-12T03:26:59.023726+0000","last_unstale":"2024-09-12T03:27:10.059758+0000","last_undegraded":"2024-09-12T03:27:10.059758+0000","last_fullsized":"2024-09-12T03:27:10.059758+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:49:44.984188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:35.142702+0000","last_change":"2024-09-12T03:26:56.957847+0000","last_active":"2024-09-12T03:27:35.142702+0000","last_peered":"2024-09-12T03:27:35.142702+0000","last_clean":"2024-09-12T03:27:35.142702+0000","last_became_active":"2024-09-12T03:26:56.957491+0000","last_became_peered":"2024-09-12T03:26:56.957491+0000","last_unstale":"2024-09-12T03:27:35.142702+0000","last_undegraded":"2024-09-12T03:27:35.142702+0000","last_fullsized":"2024-09-12T03:27:35.142702+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:06:23.028881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":358,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059971+0000","last_change":"2024-09-12T03:25:45.211570+0000","last_active":"2024-09-12T03:27:10.059971+0000","last_peered":"2024-09-12T03:27:10.059971+0000","last_clean":"2024-09-12T03:27:10.059971+0000","last_became_active":"2024-09-12T03:25:45.211052+0000","last_became_peered":"2024-09-12T03:25:45.211052+0000","last_unstale":"2024-09-12T03:27:10.059971+0000","last_undegraded":"2024-09-12T03:27:10.059971+0000","last_fullsized":"2024-09-12T03:27:10.059971+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:21.915950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:21.915950+0000","last_clean_scrub_stamp":"2024-09-12T03:22:21.915950+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:03:06.560617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059967+0000","last_change":"2024-09-12T03:27:06.046319+0000","last_active":"2024-09-12T03:27:10.059967+0000","last_peered":"2024-09-12T03:27:10.059967+0000","last_clean":"2024-09-12T03:27:10.059967+0000","last_became_active":"2024-09-12T03:27:06.045864+0000","last_became_peered":"2024-09-12T03:27:06.045864+0000","last_unstale":"2024-09-12T03:27:10.059967+0000","last_undegraded":"2024-09-12T03:27:10.059967+0000","last_fullsized":"2024-09-12T03:27:10.059967+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:45:25.224775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060194+0000","last_change":"2024-09-12T03:26:59.019197+0000","last_active":"2024-09-12T03:27:10.060194+0000","last_peered":"2024-09-12T03:27:10.060194+0000","last_clean":"2024-09-12T03:27:10.060194+0000","last_became_active":"2024-09-12T03:26:59.018868+0000","last_became_peered":"2024-09-12T03:26:59.018868+0000","last_unstale":"2024-09-12T03:27:10.060194+0000","last_undegraded":"2024-09-12T03:27:10.060194+0000","last_fullsized":"2024-09-12T03:27:10.060194+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:50:51.707556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:34.143077+0000","last_change":"2024-09-12T03:26:57.019888+0000","last_active":"2024-09-12T03:27:34.143077+0000","last_peered":"2024-09-12T03:27:34.143077+0000","last_clean":"2024-09-12T03:27:34.143077+0000","last_became_active":"2024-09-12T03:26:57.019378+0000","last_became_peered":"2024-09-12T03:26:57.019378+0000","last_unstale":"2024-09-12T03:27:34.143077+0000","last_undegraded":"2024-09-12T03:27:34.143077+0000","last_fullsized":"2024-09-12T03:27:34.143077+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:44:02.298726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059309+0000","last_change":"2024-09-12T03:27:06.043780+0000","last_active":"2024-09-12T03:27:10.059309+0000","last_peered":"2024-09-12T03:27:10.059309+0000","last_clean":"2024-09-12T03:27:10.059309+0000","last_became_active":"2024-09-12T03:27:06.042770+0000","last_became_peered":"2024-09-12T03:27:06.042770+0000","last_unstale":"2024-09-12T03:27:10.059309+0000","last_undegraded":"2024-09-12T03:27:10.059309+0000","last_fullsized":"2024-09-12T03:27:10.059309+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:32:40.801908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060195+0000","last_change":"2024-09-12T03:26:59.022996+0000","last_active":"2024-09-12T03:27:10.060195+0000","last_peered":"2024-09-12T03:27:10.060195+0000","last_clean":"2024-09-12T03:27:10.060195+0000","last_became_active":"2024-09-12T03:26:59.022675+0000","last_became_peered":"2024-09-12T03:26:59.022675+0000","last_unstale":"2024-09-12T03:27:10.060195+0000","last_undegraded":"2024-09-12T03:27:10.060195+0000","last_fullsized":"2024-09-12T03:27:10.060195+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:09.674971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060495+0000","last_change":"2024-09-12T03:26:56.957101+0000","last_active":"2024-09-12T03:27:10.060495+0000","last_peered":"2024-09-12T03:27:10.060495+0000","last_clean":"2024-09-12T03:27:10.060495+0000","last_became_active":"2024-09-12T03:26:56.956694+0000","last_became_peered":"2024-09-12T03:26:56.956694+0000","last_unstale":"2024-09-12T03:27:10.060495+0000","last_undegraded":"2024-09-12T03:27:10.060495+0000","last_fullsized":"2024-09-12T03:27:10.060495+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:29.793229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059123+0000","last_change":"2024-09-12T03:27:06.049382+0000","last_active":"2024-09-12T03:27:10.059123+0000","last_peered":"2024-09-12T03:27:10.059123+0000","last_clean":"2024-09-12T03:27:10.059123+0000","last_became_active":"2024-09-12T03:27:06.048961+0000","last_became_peered":"2024-09-12T03:27:06.048961+0000","last_unstale":"2024-09-12T03:27:10.059123+0000","last_undegraded":"2024-09-12T03:27:10.059123+0000","last_fullsized":"2024-09-12T03:27:10.059123+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:32.400328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060586+0000","last_change":"2024-09-12T03:26:59.020146+0000","last_active":"2024-09-12T03:27:10.060586+0000","last_peered":"2024-09-12T03:27:10.060586+0000","last_clean":"2024-09-12T03:27:10.060586+0000","last_became_active":"2024-09-12T03:26:59.019854+0000","last_became_peered":"2024-09-12T03:26:59.019854+0000","last_unstale":"2024-09-12T03:27:10.060586+0000","last_undegraded":"2024-09-12T03:27:10.060586+0000","last_fullsized":"2024-09-12T03:27:10.060586+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:38:47.476070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.015505+0000","last_change":"2024-09-12T03:26:56.956237+0000","last_active":"2024-09-12T03:27:10.015505+0000","last_peered":"2024-09-12T03:27:10.015505+0000","last_clean":"2024-09-12T03:27:10.015505+0000","last_became_active":"2024-09-12T03:26:56.955941+0000","last_became_peered":"2024-09-12T03:26:56.955941+0000","last_unstale":"2024-09-12T03:27:10.015505+0000","last_undegraded":"2024-09-12T03:27:10.015505+0000","last_fullsized":"2024-09-12T03:27:10.015505+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:08:53.422429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060001+0000","last_change":"2024-09-12T03:27:06.050916+0000","last_active":"2024-09-12T03:27:10.060001+0000","last_peered":"2024-09-12T03:27:10.060001+0000","last_clean":"2024-09-12T03:27:10.060001+0000","last_became_active":"2024-09-12T03:27:06.050494+0000","last_became_peered":"2024-09-12T03:27:06.050494+0000","last_unstale":"2024-09-12T03:27:10.060001+0000","last_undegraded":"2024-09-12T03:27:10.060001+0000","last_fullsized":"2024-09-12T03:27:10.060001+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:58:40.830354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.055016+0000","last_change":"2024-09-12T03:26:59.025310+0000","last_active":"2024-09-12T03:27:09.055016+0000","last_peered":"2024-09-12T03:27:09.055016+0000","last_clean":"2024-09-12T03:27:09.055016+0000","last_became_active":"2024-09-12T03:26:59.024917+0000","last_became_peered":"2024-09-12T03:26:59.024917+0000","last_unstale":"2024-09-12T03:27:09.055016+0000","last_undegraded":"2024-09-12T03:27:09.055016+0000","last_fullsized":"2024-09-12T03:27:09.055016+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:14:19.786496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:35.138137+0000","last_change":"2024-09-12T03:26:56.959434+0000","last_active":"2024-09-12T03:27:35.138137+0000","last_peered":"2024-09-12T03:27:35.138137+0000","last_clean":"2024-09-12T03:27:35.138137+0000","last_became_active":"2024-09-12T03:26:56.959095+0000","last_became_peered":"2024-09-12T03:26:56.959095+0000","last_unstale":"2024-09-12T03:27:35.138137+0000","last_undegraded":"2024-09-12T03:27:35.138137+0000","last_fullsized":"2024-09-12T03:27:35.138137+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:43:39.291022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060059+0000","last_change":"2024-09-12T03:27:06.054028+0000","last_active":"2024-09-12T03:27:10.060059+0000","last_peered":"2024-09-12T03:27:10.060059+0000","last_clean":"2024-09-12T03:27:10.060059+0000","last_became_active":"2024-09-12T03:27:06.053573+0000","last_became_peered":"2024-09-12T03:27:06.053573+0000","last_unstale":"2024-09-12T03:27:10.060059+0000","last_undegraded":"2024-09-12T03:27:10.060059+0000","last_fullsized":"2024-09-12T03:27:10.060059+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:05.977501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060148+0000","last_change":"2024-09-12T03:26:59.022978+0000","last_active":"2024-09-12T03:27:10.060148+0000","last_peered":"2024-09-12T03:27:10.060148+0000","last_clean":"2024-09-12T03:27:10.060148+0000","last_became_active":"2024-09-12T03:26:59.022555+0000","last_became_peered":"2024-09-12T03:26:59.022555+0000","last_unstale":"2024-09-12T03:27:10.060148+0000","last_undegraded":"2024-09-12T03:27:10.060148+0000","last_fullsized":"2024-09-12T03:27:10.060148+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:51.903788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:34.143883+0000","last_change":"2024-09-12T03:26:57.018735+0000","last_active":"2024-09-12T03:27:34.143883+0000","last_peered":"2024-09-12T03:27:34.143883+0000","last_clean":"2024-09-12T03:27:34.143883+0000","last_became_active":"2024-09-12T03:26:57.018367+0000","last_became_peered":"2024-09-12T03:26:57.018367+0000","last_unstale":"2024-09-12T03:27:34.143883+0000","last_undegraded":"2024-09-12T03:27:34.143883+0000","last_fullsized":"2024-09-12T03:27:34.143883+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:25.420324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.015527+0000","last_change":"2024-09-12T03:27:06.046626+0000","last_active":"2024-09-12T03:27:10.015527+0000","last_peered":"2024-09-12T03:27:10.015527+0000","last_clean":"2024-09-12T03:27:10.015527+0000","last_became_active":"2024-09-12T03:27:06.046234+0000","last_became_peered":"2024-09-12T03:27:06.046234+0000","last_unstale":"2024-09-12T03:27:10.015527+0000","last_undegraded":"2024-09-12T03:27:10.015527+0000","last_fullsized":"2024-09-12T03:27:10.015527+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:05:57.353206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059282+0000","last_change":"2024-09-12T03:26:59.023541+0000","last_active":"2024-09-12T03:27:10.059282+0000","last_peered":"2024-09-12T03:27:10.059282+0000","last_clean":"2024-09-12T03:27:10.059282+0000","last_became_active":"2024-09-12T03:26:59.023229+0000","last_became_peered":"2024-09-12T03:26:59.023229+0000","last_unstale":"2024-09-12T03:27:10.059282+0000","last_undegraded":"2024-09-12T03:27:10.059282+0000","last_fullsized":"2024-09-12T03:27:10.059282+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:27:58.514616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'4","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:46.161938+0000","last_change":"2024-09-12T03:26:56.962703+0000","last_active":"2024-09-12T03:27:46.161938+0000","last_peered":"2024-09-12T03:27:46.161938+0000","last_clean":"2024-09-12T03:27:46.161938+0000","last_became_active":"2024-09-12T03:26:56.962280+0000","last_became_peered":"2024-09-12T03:26:56.962280+0000","last_unstale":"2024-09-12T03:27:46.161938+0000","last_undegraded":"2024-09-12T03:27:46.161938+0000","last_fullsized":"2024-09-12T03:27:46.161938+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:01.045200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.053992+0000","last_change":"2024-09-12T03:27:06.054791+0000","last_active":"2024-09-12T03:27:09.053992+0000","last_peered":"2024-09-12T03:27:09.053992+0000","last_clean":"2024-09-12T03:27:09.053992+0000","last_became_active":"2024-09-12T03:27:06.054311+0000","last_became_peered":"2024-09-12T03:27:06.054311+0000","last_unstale":"2024-09-12T03:27:09.053992+0000","last_undegraded":"2024-09-12T03:27:09.053992+0000","last_fullsized":"2024-09-12T03:27:09.053992+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:21:59.115200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059750+0000","last_change":"2024-09-12T03:26:59.023971+0000","last_active":"2024-09-12T03:27:10.059750+0000","last_peered":"2024-09-12T03:27:10.059750+0000","last_clean":"2024-09-12T03:27:10.059750+0000","last_became_active":"2024-09-12T03:26:59.023653+0000","last_became_peered":"2024-09-12T03:26:59.023653+0000","last_unstale":"2024-09-12T03:27:10.059750+0000","last_undegraded":"2024-09-12T03:27:10.059750+0000","last_fullsized":"2024-09-12T03:27:10.059750+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:10:08.571177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:33.131896+0000","last_change":"2024-09-12T03:26:56.962459+0000","last_active":"2024-09-12T03:27:33.131896+0000","last_peered":"2024-09-12T03:27:33.131896+0000","last_clean":"2024-09-12T03:27:33.131896+0000","last_became_active":"2024-09-12T03:26:56.962039+0000","last_became_peered":"2024-09-12T03:26:56.962039+0000","last_unstale":"2024-09-12T03:27:33.131896+0000","last_undegraded":"2024-09-12T03:27:33.131896+0000","last_fullsized":"2024-09-12T03:27:33.131896+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:08:44.214788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.054787+0000","last_change":"2024-09-12T03:27:06.049058+0000","last_active":"2024-09-12T03:27:09.054787+0000","last_peered":"2024-09-12T03:27:09.054787+0000","last_clean":"2024-09-12T03:27:09.054787+0000","last_became_active":"2024-09-12T03:27:06.048606+0000","last_became_peered":"2024-09-12T03:27:06.048606+0000","last_unstale":"2024-09-12T03:27:09.054787+0000","last_undegraded":"2024-09-12T03:27:09.054787+0000","last_fullsized":"2024-09-12T03:27:09.054787+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:23:14.571184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.015146+0000","last_change":"2024-09-12T03:26:59.017832+0000","last_active":"2024-09-12T03:27:10.015146+0000","last_peered":"2024-09-12T03:27:10.015146+0000","last_clean":"2024-09-12T03:27:10.015146+0000","last_became_active":"2024-09-12T03:26:59.017497+0000","last_became_peered":"2024-09-12T03:26:59.017497+0000","last_unstale":"2024-09-12T03:27:10.015146+0000","last_undegraded":"2024-09-12T03:27:10.015146+0000","last_fullsized":"2024-09-12T03:27:10.015146+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:49.753556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:34.142779+0000","last_change":"2024-09-12T03:26:56.957877+0000","last_active":"2024-09-12T03:27:34.142779+0000","last_peered":"2024-09-12T03:27:34.142779+0000","last_clean":"2024-09-12T03:27:34.142779+0000","last_became_active":"2024-09-12T03:26:56.957502+0000","last_became_peered":"2024-09-12T03:26:56.957502+0000","last_unstale":"2024-09-12T03:27:34.142779+0000","last_undegraded":"2024-09-12T03:27:34.142779+0000","last_fullsized":"2024-09-12T03:27:34.142779+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:35.851957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060295+0000","last_change":"2024-09-12T03:27:06.047070+0000","last_active":"2024-09-12T03:27:10.060295+0000","last_peered":"2024-09-12T03:27:10.060295+0000","last_clean":"2024-09-12T03:27:10.060295+0000","last_became_active":"2024-09-12T03:27:06.046612+0000","last_became_peered":"2024-09-12T03:27:06.046612+0000","last_unstale":"2024-09-12T03:27:10.060295+0000","last_undegraded":"2024-09-12T03:27:10.060295+0000","last_fullsized":"2024-09-12T03:27:10.060295+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:36:53.960456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.958597+0000","last_change":"2024-09-12T03:26:59.019856+0000","last_active":"2024-09-12T03:27:09.958597+0000","last_peered":"2024-09-12T03:27:09.958597+0000","last_clean":"2024-09-12T03:27:09.958597+0000","last_became_active":"2024-09-12T03:26:59.019498+0000","last_became_peered":"2024-09-12T03:26:59.019498+0000","last_unstale":"2024-09-12T03:27:09.958597+0000","last_undegraded":"2024-09-12T03:27:09.958597+0000","last_fullsized":"2024-09-12T03:27:09.958597+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:02:18.000549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'30","reported_seq":101,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:51.170106+0000","last_change":"2024-09-12T03:26:56.958919+0000","last_active":"2024-09-12T03:27:51.170106+0000","last_peered":"2024-09-12T03:27:51.170106+0000","last_clean":"2024-09-12T03:27:51.170106+0000","last_became_active":"2024-09-12T03:26:56.958455+0000","last_became_peered":"2024-09-12T03:26:56.958455+0000","last_unstale":"2024-09-12T03:27:51.170106+0000","last_undegraded":"2024-09-12T03:27:51.170106+0000","last_fullsized":"2024-09-12T03:27:51.170106+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:05:19.318922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56673,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":41,"num_read_kb":76,"num_write":31,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.053719+0000","last_change":"2024-09-12T03:27:06.047094+0000","last_active":"2024-09-12T03:27:09.053719+0000","last_peered":"2024-09-12T03:27:09.053719+0000","last_clean":"2024-09-12T03:27:09.053719+0000","last_became_active":"2024-09-12T03:27:06.046702+0000","last_became_peered":"2024-09-12T03:27:06.046702+0000","last_unstale":"2024-09-12T03:27:09.053719+0000","last_undegraded":"2024-09-12T03:27:09.053719+0000","last_fullsized":"2024-09-12T03:27:09.053719+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:46:57.557056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059911+0000","last_change":"2024-09-12T03:26:59.018905+0000","last_active":"2024-09-12T03:27:10.059911+0000","last_peered":"2024-09-12T03:27:10.059911+0000","last_clean":"2024-09-12T03:27:10.059911+0000","last_became_active":"2024-09-12T03:26:59.018492+0000","last_became_peered":"2024-09-12T03:26:59.018492+0000","last_unstale":"2024-09-12T03:27:10.059911+0000","last_undegraded":"2024-09-12T03:27:10.059911+0000","last_fullsized":"2024-09-12T03:27:10.059911+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:11.953999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:33.132376+0000","last_change":"2024-09-12T03:26:57.018607+0000","last_active":"2024-09-12T03:27:33.132376+0000","last_peered":"2024-09-12T03:27:33.132376+0000","last_clean":"2024-09-12T03:27:33.132376+0000","last_became_active":"2024-09-12T03:26:57.018255+0000","last_became_peered":"2024-09-12T03:26:57.018255+0000","last_unstale":"2024-09-12T03:27:33.132376+0000","last_undegraded":"2024-09-12T03:27:33.132376+0000","last_fullsized":"2024-09-12T03:27:33.132376+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:39.029308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.061212+0000","last_change":"2024-09-12T03:27:06.051078+0000","last_active":"2024-09-12T03:27:10.061212+0000","last_peered":"2024-09-12T03:27:10.061212+0000","last_clean":"2024-09-12T03:27:10.061212+0000","last_became_active":"2024-09-12T03:27:06.050257+0000","last_became_peered":"2024-09-12T03:27:06.050257+0000","last_unstale":"2024-09-12T03:27:10.061212+0000","last_undegraded":"2024-09-12T03:27:10.061212+0000","last_fullsized":"2024-09-12T03:27:10.061212+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:56.315922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059234+0000","last_change":"2024-09-12T03:26:59.025406+0000","last_active":"2024-09-12T03:27:10.059234+0000","last_peered":"2024-09-12T03:27:10.059234+0000","last_clean":"2024-09-12T03:27:10.059234+0000","last_became_active":"2024-09-12T03:26:59.025091+0000","last_became_peered":"2024-09-12T03:26:59.025091+0000","last_unstale":"2024-09-12T03:27:10.059234+0000","last_undegraded":"2024-09-12T03:27:10.059234+0000","last_fullsized":"2024-09-12T03:27:10.059234+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:39:08.297785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:33.132791+0000","last_change":"2024-09-12T03:26:56.961325+0000","last_active":"2024-09-12T03:27:33.132791+0000","last_peered":"2024-09-12T03:27:33.132791+0000","last_clean":"2024-09-12T03:27:33.132791+0000","last_became_active":"2024-09-12T03:26:56.960806+0000","last_became_peered":"2024-09-12T03:26:56.960806+0000","last_unstale":"2024-09-12T03:27:33.132791+0000","last_undegraded":"2024-09-12T03:27:33.132791+0000","last_fullsized":"2024-09-12T03:27:33.132791+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:44:11.209394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.053588+0000","last_change":"2024-09-12T03:27:06.055331+0000","last_active":"2024-09-12T03:27:09.053588+0000","last_peered":"2024-09-12T03:27:09.053588+0000","last_clean":"2024-09-12T03:27:09.053588+0000","last_became_active":"2024-09-12T03:27:06.054227+0000","last_became_peered":"2024-09-12T03:27:06.054227+0000","last_unstale":"2024-09-12T03:27:09.053588+0000","last_undegraded":"2024-09-12T03:27:09.053588+0000","last_fullsized":"2024-09-12T03:27:09.053588+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:35:10.446231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060255+0000","last_change":"2024-09-12T03:26:59.012095+0000","last_active":"2024-09-12T03:27:10.060255+0000","last_peered":"2024-09-12T03:27:10.060255+0000","last_clean":"2024-09-12T03:27:10.060255+0000","last_became_active":"2024-09-12T03:26:59.011592+0000","last_became_peered":"2024-09-12T03:26:59.011592+0000","last_unstale":"2024-09-12T03:27:10.060255+0000","last_undegraded":"2024-09-12T03:27:10.060255+0000","last_fullsized":"2024-09-12T03:27:10.060255+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:49:33.026996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:50.167226+0000","last_change":"2024-09-12T03:26:56.959353+0000","last_active":"2024-09-12T03:27:50.167226+0000","last_peered":"2024-09-12T03:27:50.167226+0000","last_clean":"2024-09-12T03:27:50.167226+0000","last_became_active":"2024-09-12T03:26:56.958767+0000","last_became_peered":"2024-09-12T03:26:56.958767+0000","last_unstale":"2024-09-12T03:27:50.167226+0000","last_undegraded":"2024-09-12T03:27:50.167226+0000","last_fullsized":"2024-09-12T03:27:50.167226+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:37:51.518907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059468+0000","last_change":"2024-09-12T03:27:06.043745+0000","last_active":"2024-09-12T03:27:10.059468+0000","last_peered":"2024-09-12T03:27:10.059468+0000","last_clean":"2024-09-12T03:27:10.059468+0000","last_became_active":"2024-09-12T03:27:06.042855+0000","last_became_peered":"2024-09-12T03:27:06.042855+0000","last_unstale":"2024-09-12T03:27:10.059468+0000","last_undegraded":"2024-09-12T03:27:10.059468+0000","last_fullsized":"2024-09-12T03:27:10.059468+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:14:34.939593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059029+0000","last_change":"2024-09-12T03:27:06.052561+0000","last_active":"2024-09-12T03:27:10.059029+0000","last_peered":"2024-09-12T03:27:10.059029+0000","last_clean":"2024-09-12T03:27:10.059029+0000","last_became_active":"2024-09-12T03:27:06.052192+0000","last_became_peered":"2024-09-12T03:27:06.052192+0000","last_unstale":"2024-09-12T03:27:10.059029+0000","last_undegraded":"2024-09-12T03:27:10.059029+0000","last_fullsized":"2024-09-12T03:27:10.059029+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:31:31.800188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.053794+0000","last_change":"2024-09-12T03:26:59.012762+0000","last_active":"2024-09-12T03:27:09.053794+0000","last_peered":"2024-09-12T03:27:09.053794+0000","last_clean":"2024-09-12T03:27:09.053794+0000","last_became_active":"2024-09-12T03:26:59.012400+0000","last_became_peered":"2024-09-12T03:26:59.012400+0000","last_unstale":"2024-09-12T03:27:09.053794+0000","last_undegraded":"2024-09-12T03:27:09.053794+0000","last_fullsized":"2024-09-12T03:27:09.053794+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:35:13.573284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060076+0000","last_change":"2024-09-12T03:26:56.962454+0000","last_active":"2024-09-12T03:27:10.060076+0000","last_peered":"2024-09-12T03:27:10.060076+0000","last_clean":"2024-09-12T03:27:10.060076+0000","last_became_active":"2024-09-12T03:26:56.962026+0000","last_became_peered":"2024-09-12T03:26:56.962026+0000","last_unstale":"2024-09-12T03:27:10.060076+0000","last_undegraded":"2024-09-12T03:27:10.060076+0000","last_fullsized":"2024-09-12T03:27:10.060076+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:00:23.501942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.055112+0000","last_change":"2024-09-12T03:27:06.053946+0000","last_active":"2024-09-12T03:27:09.055112+0000","last_peered":"2024-09-12T03:27:09.055112+0000","last_clean":"2024-09-12T03:27:09.055112+0000","last_became_active":"2024-09-12T03:27:06.053494+0000","last_became_peered":"2024-09-12T03:27:06.053494+0000","last_unstale":"2024-09-12T03:27:09.055112+0000","last_undegraded":"2024-09-12T03:27:09.055112+0000","last_fullsized":"2024-09-12T03:27:09.055112+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:13:11.537918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.058720+0000","last_change":"2024-09-12T03:26:59.021472+0000","last_active":"2024-09-12T03:27:10.058720+0000","last_peered":"2024-09-12T03:27:10.058720+0000","last_clean":"2024-09-12T03:27:10.058720+0000","last_became_active":"2024-09-12T03:26:59.021160+0000","last_became_peered":"2024-09-12T03:26:59.021160+0000","last_unstale":"2024-09-12T03:27:10.058720+0000","last_undegraded":"2024-09-12T03:27:10.058720+0000","last_fullsized":"2024-09-12T03:27:10.058720+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:47.146617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:34.142514+0000","last_change":"2024-09-12T03:26:56.958944+0000","last_active":"2024-09-12T03:27:34.142514+0000","last_peered":"2024-09-12T03:27:34.142514+0000","last_clean":"2024-09-12T03:27:34.142514+0000","last_became_active":"2024-09-12T03:26:56.958511+0000","last_became_peered":"2024-09-12T03:26:56.958511+0000","last_unstale":"2024-09-12T03:27:34.142514+0000","last_undegraded":"2024-09-12T03:27:34.142514+0000","last_fullsized":"2024-09-12T03:27:34.142514+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:52:37.871482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:34.141804+0000","last_change":"2024-09-12T03:26:56.961442+0000","last_active":"2024-09-12T03:27:34.141804+0000","last_peered":"2024-09-12T03:27:34.141804+0000","last_clean":"2024-09-12T03:27:34.141804+0000","last_became_active":"2024-09-12T03:26:56.961078+0000","last_became_peered":"2024-09-12T03:26:56.961078+0000","last_unstale":"2024-09-12T03:27:34.141804+0000","last_undegraded":"2024-09-12T03:27:34.141804+0000","last_fullsized":"2024-09-12T03:27:34.141804+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:17.185990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060209+0000","last_change":"2024-09-12T03:26:59.022873+0000","last_active":"2024-09-12T03:27:10.060209+0000","last_peered":"2024-09-12T03:27:10.060209+0000","last_clean":"2024-09-12T03:27:10.060209+0000","last_became_active":"2024-09-12T03:26:59.022391+0000","last_became_peered":"2024-09-12T03:26:59.022391+0000","last_unstale":"2024-09-12T03:27:10.060209+0000","last_undegraded":"2024-09-12T03:27:10.060209+0000","last_fullsized":"2024-09-12T03:27:10.060209+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:05:08.138480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059355+0000","last_change":"2024-09-12T03:27:06.049829+0000","last_active":"2024-09-12T03:27:10.059355+0000","last_peered":"2024-09-12T03:27:10.059355+0000","last_clean":"2024-09-12T03:27:10.059355+0000","last_became_active":"2024-09-12T03:27:06.049444+0000","last_became_peered":"2024-09-12T03:27:06.049444+0000","last_unstale":"2024-09-12T03:27:10.059355+0000","last_undegraded":"2024-09-12T03:27:10.059355+0000","last_fullsized":"2024-09-12T03:27:10.059355+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:57:07.617935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":54,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:50.167751+0000","last_change":"2024-09-12T03:26:56.962474+0000","last_active":"2024-09-12T03:27:50.167751+0000","last_peered":"2024-09-12T03:27:50.167751+0000","last_clean":"2024-09-12T03:27:50.167751+0000","last_became_active":"2024-09-12T03:26:56.962105+0000","last_became_peered":"2024-09-12T03:26:56.962105+0000","last_unstale":"2024-09-12T03:27:50.167751+0000","last_undegraded":"2024-09-12T03:27:50.167751+0000","last_fullsized":"2024-09-12T03:27:50.167751+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:40:41.905790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.958250+0000","last_change":"2024-09-12T03:26:59.011561+0000","last_active":"2024-09-12T03:27:09.958250+0000","last_peered":"2024-09-12T03:27:09.958250+0000","last_clean":"2024-09-12T03:27:09.958250+0000","last_became_active":"2024-09-12T03:26:59.011006+0000","last_became_peered":"2024-09-12T03:26:59.011006+0000","last_unstale":"2024-09-12T03:27:09.958250+0000","last_undegraded":"2024-09-12T03:27:09.958250+0000","last_fullsized":"2024-09-12T03:27:09.958250+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:09:20.221637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059427+0000","last_change":"2024-09-12T03:27:06.050816+0000","last_active":"2024-09-12T03:27:10.059427+0000","last_peered":"2024-09-12T03:27:10.059427+0000","last_clean":"2024-09-12T03:27:10.059427+0000","last_became_active":"2024-09-12T03:27:06.050403+0000","last_became_peered":"2024-09-12T03:27:06.050403+0000","last_unstale":"2024-09-12T03:27:10.059427+0000","last_undegraded":"2024-09-12T03:27:10.059427+0000","last_fullsized":"2024-09-12T03:27:10.059427+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:02:24.285363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.054522+0000","last_change":"2024-09-12T03:26:59.013900+0000","last_active":"2024-09-12T03:27:09.054522+0000","last_peered":"2024-09-12T03:27:09.054522+0000","last_clean":"2024-09-12T03:27:09.054522+0000","last_became_active":"2024-09-12T03:26:59.013308+0000","last_became_peered":"2024-09-12T03:26:59.013308+0000","last_unstale":"2024-09-12T03:27:09.054522+0000","last_undegraded":"2024-09-12T03:27:09.054522+0000","last_fullsized":"2024-09-12T03:27:09.054522+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:50.943991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059748+0000","last_change":"2024-09-12T03:26:59.022861+0000","last_active":"2024-09-12T03:27:10.059748+0000","last_peered":"2024-09-12T03:27:10.059748+0000","last_clean":"2024-09-12T03:27:10.059748+0000","last_became_active":"2024-09-12T03:26:59.022485+0000","last_became_peered":"2024-09-12T03:26:59.022485+0000","last_unstale":"2024-09-12T03:27:10.059748+0000","last_undegraded":"2024-09-12T03:27:10.059748+0000","last_fullsized":"2024-09-12T03:27:10.059748+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:46.961156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059176+0000","last_change":"2024-09-12T03:26:59.017779+0000","last_active":"2024-09-12T03:27:10.059176+0000","last_peered":"2024-09-12T03:27:10.059176+0000","last_clean":"2024-09-12T03:27:10.059176+0000","last_became_active":"2024-09-12T03:26:59.017438+0000","last_became_peered":"2024-09-12T03:26:59.017438+0000","last_unstale":"2024-09-12T03:27:10.059176+0000","last_undegraded":"2024-09-12T03:27:10.059176+0000","last_fullsized":"2024-09-12T03:27:10.059176+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:29.224776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059138+0000","last_change":"2024-09-12T03:26:59.017542+0000","last_active":"2024-09-12T03:27:10.059138+0000","last_peered":"2024-09-12T03:27:10.059138+0000","last_clean":"2024-09-12T03:27:10.059138+0000","last_became_active":"2024-09-12T03:26:59.017253+0000","last_became_peered":"2024-09-12T03:26:59.017253+0000","last_unstale":"2024-09-12T03:27:10.059138+0000","last_undegraded":"2024-09-12T03:27:10.059138+0000","last_fullsized":"2024-09-12T03:27:10.059138+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:06:51.389627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.015306+0000","last_change":"2024-09-12T03:26:59.013574+0000","last_active":"2024-09-12T03:27:10.015306+0000","last_peered":"2024-09-12T03:27:10.015306+0000","last_clean":"2024-09-12T03:27:10.015306+0000","last_became_active":"2024-09-12T03:26:59.013253+0000","last_became_peered":"2024-09-12T03:26:59.013253+0000","last_unstale":"2024-09-12T03:27:10.015306+0000","last_undegraded":"2024-09-12T03:27:10.015306+0000","last_fullsized":"2024-09-12T03:27:10.015306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:37.610909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.055055+0000","last_change":"2024-09-12T03:26:59.012430+0000","last_active":"2024-09-12T03:27:09.055055+0000","last_peered":"2024-09-12T03:27:09.055055+0000","last_clean":"2024-09-12T03:27:09.055055+0000","last_became_active":"2024-09-12T03:26:59.011672+0000","last_became_peered":"2024-09-12T03:26:59.011672+0000","last_unstale":"2024-09-12T03:27:09.055055+0000","last_undegraded":"2024-09-12T03:27:09.055055+0000","last_fullsized":"2024-09-12T03:27:09.055055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:50:44.888834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059475+0000","last_change":"2024-09-12T03:26:59.015704+0000","last_active":"2024-09-12T03:27:10.059475+0000","last_peered":"2024-09-12T03:27:10.059475+0000","last_clean":"2024-09-12T03:27:10.059475+0000","last_became_active":"2024-09-12T03:26:59.015389+0000","last_became_peered":"2024-09-12T03:26:59.015389+0000","last_unstale":"2024-09-12T03:27:10.059475+0000","last_undegraded":"2024-09-12T03:27:10.059475+0000","last_fullsized":"2024-09-12T03:27:10.059475+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:19:52.917902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059839+0000","last_change":"2024-09-12T03:26:59.026325+0000","last_active":"2024-09-12T03:27:10.059839+0000","last_peered":"2024-09-12T03:27:10.059839+0000","last_clean":"2024-09-12T03:27:10.059839+0000","last_became_active":"2024-09-12T03:26:59.026014+0000","last_became_peered":"2024-09-12T03:26:59.026014+0000","last_unstale":"2024-09-12T03:27:10.059839+0000","last_undegraded":"2024-09-12T03:27:10.059839+0000","last_fullsized":"2024-09-12T03:27:10.059839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:32:50.778286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059760+0000","last_change":"2024-09-12T03:26:59.020405+0000","last_active":"2024-09-12T03:27:10.059760+0000","last_peered":"2024-09-12T03:27:10.059760+0000","last_clean":"2024-09-12T03:27:10.059760+0000","last_became_active":"2024-09-12T03:26:59.020047+0000","last_became_peered":"2024-09-12T03:26:59.020047+0000","last_unstale":"2024-09-12T03:27:10.059760+0000","last_undegraded":"2024-09-12T03:27:10.059760+0000","last_fullsized":"2024-09-12T03:27:10.059760+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:03:47.024894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059626+0000","last_change":"2024-09-12T03:26:59.023143+0000","last_active":"2024-09-12T03:27:10.059626+0000","last_peered":"2024-09-12T03:27:10.059626+0000","last_clean":"2024-09-12T03:27:10.059626+0000","last_became_active":"2024-09-12T03:26:59.022282+0000","last_became_peered":"2024-09-12T03:26:59.022282+0000","last_unstale":"2024-09-12T03:27:10.059626+0000","last_undegraded":"2024-09-12T03:27:10.059626+0000","last_fullsized":"2024-09-12T03:27:10.059626+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:59.864192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059511+0000","last_change":"2024-09-12T03:26:59.025505+0000","last_active":"2024-09-12T03:27:10.059511+0000","last_peered":"2024-09-12T03:27:10.059511+0000","last_clean":"2024-09-12T03:27:10.059511+0000","last_became_active":"2024-09-12T03:26:59.025081+0000","last_became_peered":"2024-09-12T03:26:59.025081+0000","last_unstale":"2024-09-12T03:27:10.059511+0000","last_undegraded":"2024-09-12T03:27:10.059511+0000","last_fullsized":"2024-09-12T03:27:10.059511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:42:18.012217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.015426+0000","last_change":"2024-09-12T03:26:59.011315+0000","last_active":"2024-09-12T03:27:10.015426+0000","last_peered":"2024-09-12T03:27:10.015426+0000","last_clean":"2024-09-12T03:27:10.015426+0000","last_became_active":"2024-09-12T03:26:59.011028+0000","last_became_peered":"2024-09-12T03:26:59.011028+0000","last_unstale":"2024-09-12T03:27:10.015426+0000","last_undegraded":"2024-09-12T03:27:10.015426+0000","last_fullsized":"2024-09-12T03:27:10.015426+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:18:10.541578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.054746+0000","last_change":"2024-09-12T03:26:59.024518+0000","last_active":"2024-09-12T03:27:09.054746+0000","last_peered":"2024-09-12T03:27:09.054746+0000","last_clean":"2024-09-12T03:27:09.054746+0000","last_became_active":"2024-09-12T03:26:59.024197+0000","last_became_peered":"2024-09-12T03:26:59.024197+0000","last_unstale":"2024-09-12T03:27:09.054746+0000","last_undegraded":"2024-09-12T03:27:09.054746+0000","last_fullsized":"2024-09-12T03:27:09.054746+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:52.113096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059606+0000","last_change":"2024-09-12T03:26:59.018707+0000","last_active":"2024-09-12T03:27:10.059606+0000","last_peered":"2024-09-12T03:27:10.059606+0000","last_clean":"2024-09-12T03:27:10.059606+0000","last_became_active":"2024-09-12T03:26:59.017997+0000","last_became_peered":"2024-09-12T03:26:59.017997+0000","last_unstale":"2024-09-12T03:27:10.059606+0000","last_undegraded":"2024-09-12T03:27:10.059606+0000","last_fullsized":"2024-09-12T03:27:10.059606+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:37:58.896056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.054272+0000","last_change":"2024-09-12T03:26:59.013523+0000","last_active":"2024-09-12T03:27:09.054272+0000","last_peered":"2024-09-12T03:27:09.054272+0000","last_clean":"2024-09-12T03:27:09.054272+0000","last_became_active":"2024-09-12T03:26:59.013210+0000","last_became_peered":"2024-09-12T03:26:59.013210+0000","last_unstale":"2024-09-12T03:27:09.054272+0000","last_undegraded":"2024-09-12T03:27:09.054272+0000","last_fullsized":"2024-09-12T03:27:09.054272+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:03:42.390871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.055270+0000","last_change":"2024-09-12T03:26:59.016719+0000","last_active":"2024-09-12T03:27:09.055270+0000","last_peered":"2024-09-12T03:27:09.055270+0000","last_clean":"2024-09-12T03:27:09.055270+0000","last_became_active":"2024-09-12T03:26:59.015245+0000","last_became_peered":"2024-09-12T03:26:59.015245+0000","last_unstale":"2024-09-12T03:27:09.055270+0000","last_undegraded":"2024-09-12T03:27:09.055270+0000","last_fullsized":"2024-09-12T03:27:09.055270+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:12.163847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.058982+0000","last_change":"2024-09-12T03:26:59.024317+0000","last_active":"2024-09-12T03:27:10.058982+0000","last_peered":"2024-09-12T03:27:10.058982+0000","last_clean":"2024-09-12T03:27:10.058982+0000","last_became_active":"2024-09-12T03:26:59.023998+0000","last_became_peered":"2024-09-12T03:26:59.023998+0000","last_unstale":"2024-09-12T03:27:10.058982+0000","last_undegraded":"2024-09-12T03:27:10.058982+0000","last_fullsized":"2024-09-12T03:27:10.058982+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:29:11.713342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.058724+0000","last_change":"2024-09-12T03:26:59.022114+0000","last_active":"2024-09-12T03:27:10.058724+0000","last_peered":"2024-09-12T03:27:10.058724+0000","last_clean":"2024-09-12T03:27:10.058724+0000","last_became_active":"2024-09-12T03:26:59.021806+0000","last_became_peered":"2024-09-12T03:26:59.021806+0000","last_unstale":"2024-09-12T03:27:10.058724+0000","last_undegraded":"2024-09-12T03:27:10.058724+0000","last_fullsized":"2024-09-12T03:27:10.058724+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:57:36.655297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.058697+0000","last_change":"2024-09-12T03:26:59.011340+0000","last_active":"2024-09-12T03:27:10.058697+0000","last_peered":"2024-09-12T03:27:10.058697+0000","last_clean":"2024-09-12T03:27:10.058697+0000","last_became_active":"2024-09-12T03:26:59.011042+0000","last_became_peered":"2024-09-12T03:26:59.011042+0000","last_unstale":"2024-09-12T03:27:10.058697+0000","last_undegraded":"2024-09-12T03:27:10.058697+0000","last_fullsized":"2024-09-12T03:27:10.058697+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:49.568438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.015311+0000","last_change":"2024-09-12T03:26:59.016449+0000","last_active":"2024-09-12T03:27:10.015311+0000","last_peered":"2024-09-12T03:27:10.015311+0000","last_clean":"2024-09-12T03:27:10.015311+0000","last_became_active":"2024-09-12T03:26:59.016087+0000","last_became_peered":"2024-09-12T03:26:59.016087+0000","last_unstale":"2024-09-12T03:27:10.015311+0000","last_undegraded":"2024-09-12T03:27:10.015311+0000","last_fullsized":"2024-09-12T03:27:10.015311+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:13:40.683471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060849+0000","last_change":"2024-09-12T03:26:59.011272+0000","last_active":"2024-09-12T03:27:10.060849+0000","last_peered":"2024-09-12T03:27:10.060849+0000","last_clean":"2024-09-12T03:27:10.060849+0000","last_became_active":"2024-09-12T03:26:59.010937+0000","last_became_peered":"2024-09-12T03:26:59.010937+0000","last_unstale":"2024-09-12T03:27:10.060849+0000","last_undegraded":"2024-09-12T03:27:10.060849+0000","last_fullsized":"2024-09-12T03:27:10.060849+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:56:00.600771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060097+0000","last_change":"2024-09-12T03:26:59.021434+0000","last_active":"2024-09-12T03:27:10.060097+0000","last_peered":"2024-09-12T03:27:10.060097+0000","last_clean":"2024-09-12T03:27:10.060097+0000","last_became_active":"2024-09-12T03:26:59.021071+0000","last_became_peered":"2024-09-12T03:26:59.021071+0000","last_unstale":"2024-09-12T03:27:10.060097+0000","last_undegraded":"2024-09-12T03:27:10.060097+0000","last_fullsized":"2024-09-12T03:27:10.060097+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:14:46.774992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.055595+0000","last_change":"2024-09-12T03:26:59.019310+0000","last_active":"2024-09-12T03:27:09.055595+0000","last_peered":"2024-09-12T03:27:09.055595+0000","last_clean":"2024-09-12T03:27:09.055595+0000","last_became_active":"2024-09-12T03:26:59.018967+0000","last_became_peered":"2024-09-12T03:26:59.018967+0000","last_unstale":"2024-09-12T03:27:09.055595+0000","last_undegraded":"2024-09-12T03:27:09.055595+0000","last_fullsized":"2024-09-12T03:27:09.055595+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:37:10.830500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059970+0000","last_change":"2024-09-12T03:26:59.022088+0000","last_active":"2024-09-12T03:27:10.059970+0000","last_peered":"2024-09-12T03:27:10.059970+0000","last_clean":"2024-09-12T03:27:10.059970+0000","last_became_active":"2024-09-12T03:26:59.021655+0000","last_became_peered":"2024-09-12T03:26:59.021655+0000","last_unstale":"2024-09-12T03:27:10.059970+0000","last_undegraded":"2024-09-12T03:27:10.059970+0000","last_fullsized":"2024-09-12T03:27:10.059970+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:39.490597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.058671+0000","last_change":"2024-09-12T03:26:59.024898+0000","last_active":"2024-09-12T03:27:10.058671+0000","last_peered":"2024-09-12T03:27:10.058671+0000","last_clean":"2024-09-12T03:27:10.058671+0000","last_became_active":"2024-09-12T03:26:59.024511+0000","last_became_peered":"2024-09-12T03:26:59.024511+0000","last_unstale":"2024-09-12T03:27:10.058671+0000","last_undegraded":"2024-09-12T03:27:10.058671+0000","last_fullsized":"2024-09-12T03:27:10.058671+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:24.191365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059386+0000","last_change":"2024-09-12T03:26:59.024829+0000","last_active":"2024-09-12T03:27:10.059386+0000","last_peered":"2024-09-12T03:27:10.059386+0000","last_clean":"2024-09-12T03:27:10.059386+0000","last_became_active":"2024-09-12T03:26:59.024493+0000","last_became_peered":"2024-09-12T03:26:59.024493+0000","last_unstale":"2024-09-12T03:27:10.059386+0000","last_undegraded":"2024-09-12T03:27:10.059386+0000","last_fullsized":"2024-09-12T03:27:10.059386+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:28:35.322783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.055164+0000","last_change":"2024-09-12T03:26:59.014536+0000","last_active":"2024-09-12T03:27:09.055164+0000","last_peered":"2024-09-12T03:27:09.055164+0000","last_clean":"2024-09-12T03:27:09.055164+0000","last_became_active":"2024-09-12T03:26:59.013910+0000","last_became_peered":"2024-09-12T03:26:59.013910+0000","last_unstale":"2024-09-12T03:27:09.055164+0000","last_undegraded":"2024-09-12T03:27:09.055164+0000","last_fullsized":"2024-09-12T03:27:09.055164+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:27.519367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.958268+0000","last_change":"2024-09-12T03:26:59.016656+0000","last_active":"2024-09-12T03:27:09.958268+0000","last_peered":"2024-09-12T03:27:09.958268+0000","last_clean":"2024-09-12T03:27:09.958268+0000","last_became_active":"2024-09-12T03:26:59.016217+0000","last_became_peered":"2024-09-12T03:26:59.016217+0000","last_unstale":"2024-09-12T03:27:09.958268+0000","last_undegraded":"2024-09-12T03:27:09.958268+0000","last_fullsized":"2024-09-12T03:27:09.958268+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:07:42.718394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.060816+0000","last_change":"2024-09-12T03:26:59.015940+0000","last_active":"2024-09-12T03:27:10.060816+0000","last_peered":"2024-09-12T03:27:10.060816+0000","last_clean":"2024-09-12T03:27:10.060816+0000","last_became_active":"2024-09-12T03:26:59.015578+0000","last_became_peered":"2024-09-12T03:26:59.015578+0000","last_unstale":"2024-09-12T03:27:10.060816+0000","last_undegraded":"2024-09-12T03:27:10.060816+0000","last_fullsized":"2024-09-12T03:27:10.060816+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:07:46.559935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.058974+0000","last_change":"2024-09-12T03:26:59.016283+0000","last_active":"2024-09-12T03:27:10.058974+0000","last_peered":"2024-09-12T03:27:10.058974+0000","last_clean":"2024-09-12T03:27:10.058974+0000","last_became_active":"2024-09-12T03:26:59.015109+0000","last_became_peered":"2024-09-12T03:26:59.015109+0000","last_unstale":"2024-09-12T03:27:10.058974+0000","last_undegraded":"2024-09-12T03:27:10.058974+0000","last_fullsized":"2024-09-12T03:27:10.058974+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:02:44.493393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:10.059084+0000","last_change":"2024-09-12T03:26:59.020161+0000","last_active":"2024-09-12T03:27:10.059084+0000","last_peered":"2024-09-12T03:27:10.059084+0000","last_clean":"2024-09-12T03:27:10.059084+0000","last_became_active":"2024-09-12T03:26:59.019796+0000","last_became_peered":"2024-09-12T03:26:59.019796+0000","last_unstale":"2024-09-12T03:27:10.059084+0000","last_undegraded":"2024-09-12T03:27:10.059084+0000","last_fullsized":"2024-09-12T03:27:10.059084+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:04:02.817005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T03:27:09.055202+0000","last_change":"2024-09-12T03:26:59.015763+0000","last_active":"2024-09-12T03:27:09.055202+0000","last_peered":"2024-09-12T03:27:09.055202+0000","last_clean":"2024-09-12T03:27:09.055202+0000","last_became_active":"2024-09-12T03:26:59.014979+0000","last_became_peered":"2024-09-12T03:26:59.014979+0000","last_unstale":"2024-09-12T03:27:09.055202+0000","last_undegraded":"2024-09-12T03:27:09.055202+0000","last_fullsized":"2024-09-12T03:27:09.055202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:49:26.959019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":60676,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":120,"num_read_kb":125,"num_write":162,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":168700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":247982,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514519,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":489415,"data_compressed":17549,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776156,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1105920,"data_stored":1133749,"data_compressed":53677,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939681,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":493747,"data_compressed":17549,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201318,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":489571,"data_compressed":17549,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430251,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":1085482,"data_compressed":53677,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593776,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":484827,"data_compressed":17549,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822709,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":1080677,"data_compressed":53677,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953529,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":489870,"data_compressed":17549,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215167,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":485021,"data_compressed":17549,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411397,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1048576,"data_stored":542760,"data_compressed":17549,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":528082,"data_compressed":17549,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738446,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":987136,"data_stored":486215,"data_compressed":17549,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34490,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34490,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34490,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36128,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36128,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36128,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36128,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":43345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":58023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:27:53.274 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-12T03:27:53.275 INFO:tasks.cephadm:Setting up client nodes... 2024-09-12T03:27:53.275 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-12T03:27:53.298 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:53 smithi064 ceph-mon[30952]: pgmap v310: 129 pgs: 129 active+clean; 636 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.5 KiB/s wr, 9 op/s 2024-09-12T03:27:53.298 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:53 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:27:53.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:53 smithi195 ceph-mon[34786]: pgmap v310: 129 pgs: 129 active+clean; 636 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.5 KiB/s wr, 9 op/s 2024-09-12T03:27:53.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:53 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:27:53.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:53 smithi115 ceph-mon[34801]: pgmap v310: 129 pgs: 129 active+clean; 636 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.5 KiB/s wr, 9 op/s 2024-09-12T03:27:53.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:53 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:27:54.204 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:54.271 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:54 smithi064 ceph-mon[30952]: from='client.14841 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:27:54.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:54 smithi115 ceph-mon[34801]: from='client.14841 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:27:54.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:54 smithi195 ceph-mon[34786]: from='client.14841 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:27:54.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.705+0000 7fdeaa60c640 1 -- 172.21.15.64:0/1454753370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdea4102e00 msgr2=0x7fdea4103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:54.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.705+0000 7fdeaa60c640 1 --2- 172.21.15.64:0/1454753370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdea4102e00 0x7fdea4103280 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fde98009900 tx=0x7fde9802a410 comp rx=0 tx=0).stop 2024-09-12T03:27:54.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.705+0000 7fdeaa60c640 1 -- 172.21.15.64:0/1454753370 shutdown_connections 2024-09-12T03:27:54.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.705+0000 7fdeaa60c640 1 --2- 172.21.15.64:0/1454753370 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdea41037c0 0x7fdea410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:54.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.705+0000 7fdeaa60c640 1 --2- 172.21.15.64:0/1454753370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdea4102e00 0x7fdea4103280 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:54.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.705+0000 7fdeaa60c640 1 --2- 172.21.15.64:0/1454753370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdea4101c00 0x7fdea4102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:54.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.705+0000 7fdeaa60c640 1 -- 172.21.15.64:0/1454753370 >> 172.21.15.64:0/1454753370 conn(0x7fdea40fd390 msgr2=0x7fdea40ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:54.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.705+0000 7fdeaa60c640 1 -- 172.21.15.64:0/1454753370 shutdown_connections 2024-09-12T03:27:54.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.705+0000 7fdeaa60c640 1 -- 172.21.15.64:0/1454753370 wait complete. 2024-09-12T03:27:54.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.706+0000 7fdeaa60c640 1 Processor -- start 2024-09-12T03:27:54.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.706+0000 7fdeaa60c640 1 -- start start 2024-09-12T03:27:54.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.706+0000 7fdeaa60c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdea4101c00 0x7fdea4197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:54.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.706+0000 7fdeaa60c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdea4102e00 0x7fdea4198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:54.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.706+0000 7fdeaa60c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdea41037c0 0x7fdea419f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:54.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.706+0000 7fdeaa60c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdea410d710 con 0x7fdea41037c0 2024-09-12T03:27:54.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.706+0000 7fdeaa60c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdea410d590 con 0x7fdea4101c00 2024-09-12T03:27:54.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.706+0000 7fdeaa60c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdea410d890 con 0x7fdea4102e00 2024-09-12T03:27:54.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.706+0000 7fdea8ba9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdea41037c0 0x7fdea419f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:54.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.706+0000 7fdea8ba9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdea41037c0 0x7fdea419f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51636/0 (socket says 172.21.15.64:51636) 2024-09-12T03:27:54.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.706+0000 7fdea3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdea4101c00 0x7fdea4197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:54.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.706+0000 7fdea8ba9640 1 -- 172.21.15.64:0/2356981558 learned_addr learned my addr 172.21.15.64:0/2356981558 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:27:54.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.706+0000 7fdea37fe640 1 --2- 172.21.15.64:0/2356981558 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdea4102e00 0x7fdea4198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:54.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.707+0000 7fdea8ba9640 1 -- 172.21.15.64:0/2356981558 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdea4102e00 msgr2=0x7fdea4198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:54.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.707+0000 7fdea8ba9640 1 --2- 172.21.15.64:0/2356981558 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdea4102e00 0x7fdea4198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:54.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.707+0000 7fdea8ba9640 1 -- 172.21.15.64:0/2356981558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdea4101c00 msgr2=0x7fdea4197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:54.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.707+0000 7fdea8ba9640 1 --2- 172.21.15.64:0/2356981558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdea4101c00 0x7fdea4197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:54.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.707+0000 7fdea8ba9640 1 -- 172.21.15.64:0/2356981558 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea419f980 con 0x7fdea41037c0 2024-09-12T03:27:54.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.707+0000 7fdea3fff640 1 --2- 172.21.15.64:0/2356981558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdea4101c00 0x7fdea4197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:27:54.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.707+0000 7fdea8ba9640 1 --2- 172.21.15.64:0/2356981558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdea41037c0 0x7fdea419f310 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fde9c00eaa0 tx=0x7fde9c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:54.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.707+0000 7fdea17fa640 1 -- 172.21.15.64:0/2356981558 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9c00cdb0 con 0x7fdea41037c0 2024-09-12T03:27:54.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.707+0000 7fdea17fa640 1 -- 172.21.15.64:0/2356981558 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde9c005020 con 0x7fdea41037c0 2024-09-12T03:27:54.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.707+0000 7fdeaa60c640 1 -- 172.21.15.64:0/2356981558 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdea419fc70 con 0x7fdea41037c0 2024-09-12T03:27:54.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.707+0000 7fdea17fa640 1 -- 172.21.15.64:0/2356981558 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9c0116d0 con 0x7fdea41037c0 2024-09-12T03:27:54.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.707+0000 7fdeaa60c640 1 -- 172.21.15.64:0/2356981558 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdea41a0130 con 0x7fdea41037c0 2024-09-12T03:27:54.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.708+0000 7fdeaa60c640 1 -- 172.21.15.64:0/2356981558 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdea4108fa0 con 0x7fdea41037c0 2024-09-12T03:27:54.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.709+0000 7fdea17fa640 1 -- 172.21.15.64:0/2356981558 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fde9c0051d0 con 0x7fdea41037c0 2024-09-12T03:27:54.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.710+0000 7fdea17fa640 1 --2- 172.21.15.64:0/2356981558 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fde8c0a2c40 0x7fde8c0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:54.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.710+0000 7fdea3fff640 1 --2- 172.21.15.64:0/2356981558 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fde8c0a2c40 0x7fde8c0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:54.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.710+0000 7fdea17fa640 1 -- 172.21.15.64:0/2356981558 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fde9c0cf530 con 0x7fdea41037c0 2024-09-12T03:27:54.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.711+0000 7fdea3fff640 1 --2- 172.21.15.64:0/2356981558 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fde8c0a2c40 0x7fde8c0a5100 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fdea4102c60 tx=0x7fde94005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:54.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.714+0000 7fdea17fa640 1 -- 172.21.15.64:0/2356981558 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fde9c016030 con 0x7fdea41037c0 2024-09-12T03:27:54.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.977+0000 7fdeaa60c640 1 -- 172.21.15.64:0/2356981558 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fdea4062e90 con 0x7fdea41037c0 2024-09-12T03:27:54.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.980+0000 7fdea17fa640 1 -- 172.21.15.64:0/2356981558 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fde9c09baa0 con 0x7fdea41037c0 2024-09-12T03:27:54.980 INFO:teuthology.orchestra.run.smithi064.stdout:[client.0] 2024-09-12T03:27:54.981 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQC6X+JmywZTOhAA5NrKwHrXjeFGpZYdaB8zwg== 2024-09-12T03:27:54.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.983+0000 7fdeaa60c640 1 -- 172.21.15.64:0/2356981558 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fde8c0a2c40 msgr2=0x7fde8c0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:54.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.983+0000 7fdeaa60c640 1 --2- 172.21.15.64:0/2356981558 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fde8c0a2c40 0x7fde8c0a5100 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fdea4102c60 tx=0x7fde94005f90 comp rx=0 tx=0).stop 2024-09-12T03:27:54.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.983+0000 7fdeaa60c640 1 -- 172.21.15.64:0/2356981558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdea41037c0 msgr2=0x7fdea419f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:54.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.983+0000 7fdeaa60c640 1 --2- 172.21.15.64:0/2356981558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdea41037c0 0x7fdea419f310 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fde9c00eaa0 tx=0x7fde9c00ef60 comp rx=0 tx=0).stop 2024-09-12T03:27:54.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.983+0000 7fdeaa60c640 1 -- 172.21.15.64:0/2356981558 shutdown_connections 2024-09-12T03:27:54.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.983+0000 7fdeaa60c640 1 --2- 172.21.15.64:0/2356981558 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fde8c0a2c40 0x7fde8c0a5100 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:54.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.983+0000 7fdeaa60c640 1 --2- 172.21.15.64:0/2356981558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdea41037c0 0x7fdea419f310 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:54.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.983+0000 7fdeaa60c640 1 --2- 172.21.15.64:0/2356981558 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdea4102e00 0x7fdea4198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:54.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.983+0000 7fdeaa60c640 1 --2- 172.21.15.64:0/2356981558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdea4101c00 0x7fdea4197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:54.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.983+0000 7fdeaa60c640 1 -- 172.21.15.64:0/2356981558 >> 172.21.15.64:0/2356981558 conn(0x7fdea40fd390 msgr2=0x7fdea4106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:54.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.983+0000 7fdeaa60c640 1 -- 172.21.15.64:0/2356981558 shutdown_connections 2024-09-12T03:27:54.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:54.983+0000 7fdeaa60c640 1 -- 172.21.15.64:0/2356981558 wait complete. 2024-09-12T03:27:55.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:55 smithi064 ceph-mon[30952]: pgmap v311: 129 pgs: 129 active+clean; 636 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2024-09-12T03:27:55.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:55 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:27:55.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:55 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:27:55.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:55 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:27:55.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:55 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:27:55.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:55 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/2356981558' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T03:27:55.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:55 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/2356981558' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T03:27:55.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:55 smithi115 ceph-mon[34801]: pgmap v311: 129 pgs: 129 active+clean; 636 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2024-09-12T03:27:55.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:55 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:27:55.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:55 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:27:55.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:55 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:27:55.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:55 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:27:55.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:55 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/2356981558' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T03:27:55.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:55 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/2356981558' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T03:27:55.657 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-12T03:27:55.657 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-12T03:27:55.657 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-12T03:27:55.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:55 smithi195 ceph-mon[34786]: pgmap v311: 129 pgs: 129 active+clean; 636 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2024-09-12T03:27:55.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:55 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:27:55.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:55 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:27:55.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:55 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:27:55.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:55 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:27:55.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:55 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/2356981558' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T03:27:55.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:55 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/2356981558' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T03:27:55.738 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-12T03:27:55.739 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-12T03:27:55.739 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-12T03:27:56.046 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:56.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.548+0000 7fb014884640 1 -- 172.21.15.64:0/725529625 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb010101420 msgr2=0x7fb01010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:56.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.548+0000 7fb014884640 1 --2- 172.21.15.64:0/725529625 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb010101420 0x7fb01010d950 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb008009cd0 tx=0x7fb00802a8e0 comp rx=0 tx=0).stop 2024-09-12T03:27:56.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.548+0000 7fb014884640 1 -- 172.21.15.64:0/725529625 shutdown_connections 2024-09-12T03:27:56.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.548+0000 7fb014884640 1 --2- 172.21.15.64:0/725529625 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb010101420 0x7fb01010d950 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:56.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.548+0000 7fb014884640 1 --2- 172.21.15.64:0/725529625 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb010100a80 0x7fb010100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:56.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.548+0000 7fb014884640 1 --2- 172.21.15.64:0/725529625 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb010106a80 0x7fb010106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:56.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.548+0000 7fb014884640 1 -- 172.21.15.64:0/725529625 >> 172.21.15.64:0/725529625 conn(0x7fb0100fc7a0 msgr2=0x7fb0100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:56.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.548+0000 7fb014884640 1 -- 172.21.15.64:0/725529625 shutdown_connections 2024-09-12T03:27:56.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.548+0000 7fb014884640 1 -- 172.21.15.64:0/725529625 wait complete. 2024-09-12T03:27:56.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.548+0000 7fb014884640 1 Processor -- start 2024-09-12T03:27:56.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.549+0000 7fb014884640 1 -- start start 2024-09-12T03:27:56.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.549+0000 7fb014884640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb010100a80 0x7fb010102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:56.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.549+0000 7fb014884640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb010101420 0x7fb010102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:56.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.549+0000 7fb014884640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb010106a80 0x7fb0101030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:56.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.549+0000 7fb014884640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb010110d30 con 0x7fb010100a80 2024-09-12T03:27:56.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.549+0000 7fb014884640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb010110bb0 con 0x7fb010106a80 2024-09-12T03:27:56.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.549+0000 7fb014884640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb010110eb0 con 0x7fb010101420 2024-09-12T03:27:56.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.549+0000 7fb00e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb010100a80 0x7fb010102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:56.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.549+0000 7fb00e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb010100a80 0x7fb010102650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51660/0 (socket says 172.21.15.64:51660) 2024-09-12T03:27:56.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.549+0000 7fb00e59c640 1 -- 172.21.15.64:0/3326357665 learned_addr learned my addr 172.21.15.64:0/3326357665 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:27:56.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.549+0000 7fb00dd9b640 1 --2- 172.21.15.64:0/3326357665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb010101420 0x7fb010102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:56.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.549+0000 7fb00ed9d640 1 --2- 172.21.15.64:0/3326357665 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb010106a80 0x7fb0101030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:56.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.549+0000 7fb00e59c640 1 -- 172.21.15.64:0/3326357665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb010101420 msgr2=0x7fb010102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:56.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.549+0000 7fb00e59c640 1 --2- 172.21.15.64:0/3326357665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb010101420 0x7fb010102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:56.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.549+0000 7fb00e59c640 1 -- 172.21.15.64:0/3326357665 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb010106a80 msgr2=0x7fb0101030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:56.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.549+0000 7fb00e59c640 1 --2- 172.21.15.64:0/3326357665 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb010106a80 0x7fb0101030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:56.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.549+0000 7fb00e59c640 1 -- 172.21.15.64:0/3326357665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0101a6810 con 0x7fb010100a80 2024-09-12T03:27:56.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.549+0000 7fb00e59c640 1 --2- 172.21.15.64:0/3326357665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb010100a80 0x7fb010102650 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb00000d950 tx=0x7fb00000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:56.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.550+0000 7fafff7fe640 1 -- 172.21.15.64:0/3326357665 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb000017070 con 0x7fb010100a80 2024-09-12T03:27:56.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.550+0000 7fafff7fe640 1 -- 172.21.15.64:0/3326357665 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb000009c30 con 0x7fb010100a80 2024-09-12T03:27:56.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.550+0000 7fb014884640 1 -- 172.21.15.64:0/3326357665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0101a6b00 con 0x7fb010100a80 2024-09-12T03:27:56.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.550+0000 7fafff7fe640 1 -- 172.21.15.64:0/3326357665 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00000f620 con 0x7fb010100a80 2024-09-12T03:27:56.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.550+0000 7fb014884640 1 -- 172.21.15.64:0/3326357665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0101a7040 con 0x7fb010100a80 2024-09-12T03:27:56.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.551+0000 7fb014884640 1 -- 172.21.15.64:0/3326357665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb010100ee0 con 0x7fb010100a80 2024-09-12T03:27:56.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.552+0000 7fafff7fe640 1 -- 172.21.15.64:0/3326357665 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb0000109a0 con 0x7fb010100a80 2024-09-12T03:27:56.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.553+0000 7fafff7fe640 1 --2- 172.21.15.64:0/3326357665 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7faff80a2c10 0x7faff80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:56.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.553+0000 7fb00dd9b640 1 --2- 172.21.15.64:0/3326357665 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7faff80a2c10 0x7faff80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:56.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.553+0000 7fafff7fe640 1 -- 172.21.15.64:0/3326357665 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fb0000ce910 con 0x7fb010100a80 2024-09-12T03:27:56.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.553+0000 7fb00dd9b640 1 --2- 172.21.15.64:0/3326357665 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7faff80a2c10 0x7faff80a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb0101040e0 tx=0x7fb00400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:56.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.557+0000 7fafff7fe640 1 -- 172.21.15.64:0/3326357665 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb000009de0 con 0x7fb010100a80 2024-09-12T03:27:56.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.780+0000 7fb014884640 1 -- 172.21.15.64:0/3326357665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fb0101a7320 con 0x7fb010100a80 2024-09-12T03:27:56.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.785+0000 7fafff7fe640 1 -- 172.21.15.64:0/3326357665 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263198 (secure 0 0 0) 0x7fb00009b4d0 con 0x7fb010100a80 2024-09-12T03:27:56.788 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:27:56.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.789+0000 7fb014884640 1 -- 172.21.15.64:0/3326357665 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7faff80a2c10 msgr2=0x7faff80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:56.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.789+0000 7fb014884640 1 --2- 172.21.15.64:0/3326357665 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7faff80a2c10 0x7faff80a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb0101040e0 tx=0x7fb00400a2d0 comp rx=0 tx=0).stop 2024-09-12T03:27:56.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.789+0000 7fb014884640 1 -- 172.21.15.64:0/3326357665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb010100a80 msgr2=0x7fb010102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:56.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.789+0000 7fb014884640 1 --2- 172.21.15.64:0/3326357665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb010100a80 0x7fb010102650 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb00000d950 tx=0x7fb00000de10 comp rx=0 tx=0).stop 2024-09-12T03:27:56.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.789+0000 7fb014884640 1 -- 172.21.15.64:0/3326357665 shutdown_connections 2024-09-12T03:27:56.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.789+0000 7fb014884640 1 --2- 172.21.15.64:0/3326357665 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7faff80a2c10 0x7faff80a50d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:56.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.789+0000 7fb014884640 1 --2- 172.21.15.64:0/3326357665 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb010106a80 0x7fb0101030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:56.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.789+0000 7fb014884640 1 --2- 172.21.15.64:0/3326357665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb010101420 0x7fb010102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:56.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.789+0000 7fb014884640 1 --2- 172.21.15.64:0/3326357665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb010100a80 0x7fb010102650 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:56.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.789+0000 7fb014884640 1 -- 172.21.15.64:0/3326357665 >> 172.21.15.64:0/3326357665 conn(0x7fb0100fc7a0 msgr2=0x7fb0101048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:56.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.789+0000 7fb014884640 1 -- 172.21.15.64:0/3326357665 shutdown_connections 2024-09-12T03:27:56.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:56.789+0000 7fb014884640 1 -- 172.21.15.64:0/3326357665 wait complete. 2024-09-12T03:27:57.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:56 smithi115 ceph-mon[34801]: pgmap v312: 129 pgs: 129 active+clean; 636 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-09-12T03:27:57.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:56 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:27:57.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:56 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:27:57.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:56 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:27:57.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:56 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:27:57.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:56 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:27:57.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:56 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/3326357665' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T03:27:57.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:56 smithi195 ceph-mon[34786]: pgmap v312: 129 pgs: 129 active+clean; 636 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-09-12T03:27:57.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:56 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:27:57.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:56 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:27:57.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:56 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:27:57.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:56 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:27:57.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:56 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:27:57.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:56 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/3326357665' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T03:27:57.207 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:56 smithi064 ceph-mon[30952]: pgmap v312: 129 pgs: 129 active+clean; 636 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-09-12T03:27:57.207 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:56 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:27:57.207 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:56 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:27:57.207 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:56 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:27:57.207 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:56 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:27:57.207 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:56 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:27:57.208 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:56 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/3326357665' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T03:27:57.567 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":759931848},{"type":"v1","addr":"172.21.15.64:6801","nonce":759931848}]},"active_addr":"172.21.15.64:6801/759931848","active_change":"2024-09-12T03:20:07.411206+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.64:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":3764771090}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":1757703005}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":1537191801}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":3775732799}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":4216725923}]}]} 2024-09-12T03:27:57.576 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-12T03:27:57.576 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-12T03:27:57.577 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:27:57.836 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:58.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.291+0000 7f39f2bb4640 1 -- 172.21.15.64:0/2072982980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39ec102ef0 msgr2=0x7f39ec10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:58.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.291+0000 7f39f2bb4640 1 --2- 172.21.15.64:0/2072982980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39ec102ef0 0x7f39ec10da40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f39e4009ca0 tx=0x7f39e402a950 comp rx=0 tx=0).stop 2024-09-12T03:27:58.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.292+0000 7f39f2bb4640 1 -- 172.21.15.64:0/2072982980 shutdown_connections 2024-09-12T03:27:58.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.292+0000 7f39f2bb4640 1 --2- 172.21.15.64:0/2072982980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39ec102ef0 0x7f39ec10da40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:58.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.292+0000 7f39f2bb4640 1 --2- 172.21.15.64:0/2072982980 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39ec102570 0x7f39ec1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:58.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.292+0000 7f39f2bb4640 1 --2- 172.21.15.64:0/2072982980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39ec100e20 0x7f39ec101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:58.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.292+0000 7f39f2bb4640 1 -- 172.21.15.64:0/2072982980 >> 172.21.15.64:0/2072982980 conn(0x7f39ec0fc7d0 msgr2=0x7f39ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:58.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.292+0000 7f39f2bb4640 1 -- 172.21.15.64:0/2072982980 shutdown_connections 2024-09-12T03:27:58.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.292+0000 7f39f2bb4640 1 -- 172.21.15.64:0/2072982980 wait complete. 2024-09-12T03:27:58.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.292+0000 7f39f2bb4640 1 Processor -- start 2024-09-12T03:27:58.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.292+0000 7f39f2bb4640 1 -- start start 2024-09-12T03:27:58.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.292+0000 7f39f2bb4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39ec100e20 0x7f39ec1972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:58.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.292+0000 7f39f2bb4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39ec102570 0x7f39ec197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:58.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.292+0000 7f39f2bb4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39ec102ef0 0x7f39ec19bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:58.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.292+0000 7f39f2bb4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f39ec110e50 con 0x7f39ec102ef0 2024-09-12T03:27:58.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.292+0000 7f39f2bb4640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f39ec110cd0 con 0x7f39ec100e20 2024-09-12T03:27:58.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.292+0000 7f39f2bb4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f39ec110fd0 con 0x7f39ec102570 2024-09-12T03:27:58.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.293+0000 7f39f1151640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39ec102ef0 0x7f39ec19bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:58.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.293+0000 7f39f1151640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39ec102ef0 0x7f39ec19bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54838/0 (socket says 172.21.15.64:54838) 2024-09-12T03:27:58.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.293+0000 7f39f1151640 1 -- 172.21.15.64:0/1995539753 learned_addr learned my addr 172.21.15.64:0/1995539753 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:27:58.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.293+0000 7f39f0950640 1 --2- 172.21.15.64:0/1995539753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39ec100e20 0x7f39ec1972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:58.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.293+0000 7f39f1151640 1 -- 172.21.15.64:0/1995539753 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39ec102570 msgr2=0x7f39ec197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:58.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.293+0000 7f39f1151640 1 --2- 172.21.15.64:0/1995539753 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39ec102570 0x7f39ec197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:58.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.293+0000 7f39f1151640 1 -- 172.21.15.64:0/1995539753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39ec100e20 msgr2=0x7f39ec1972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.293+0000 7f39f1151640 1 --2- 172.21.15.64:0/1995539753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39ec100e20 0x7f39ec1972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.293+0000 7f39f1151640 1 -- 172.21.15.64:0/1995539753 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39ec19c2a0 con 0x7f39ec102ef0 2024-09-12T03:27:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.293+0000 7f39f1151640 1 --2- 172.21.15.64:0/1995539753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39ec102ef0 0x7f39ec19bbc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f39e402ae70 tx=0x7f39e402ff60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.293+0000 7f39e9ffb640 1 -- 172.21.15.64:0/1995539753 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e4034040 con 0x7f39ec102ef0 2024-09-12T03:27:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.293+0000 7f39e9ffb640 1 -- 172.21.15.64:0/1995539753 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39e402f6d0 con 0x7f39ec102ef0 2024-09-12T03:27:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.293+0000 7f39f2bb4640 1 -- 172.21.15.64:0/1995539753 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39ec19c530 con 0x7f39ec102ef0 2024-09-12T03:27:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.293+0000 7f39e9ffb640 1 -- 172.21.15.64:0/1995539753 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e403cc80 con 0x7f39ec102ef0 2024-09-12T03:27:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.293+0000 7f39f2bb4640 1 -- 172.21.15.64:0/1995539753 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39ec069f70 con 0x7f39ec102ef0 2024-09-12T03:27:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.295+0000 7f39f2bb4640 1 -- 172.21.15.64:0/1995539753 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39ec101200 con 0x7f39ec102ef0 2024-09-12T03:27:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.296+0000 7f39e9ffb640 1 -- 172.21.15.64:0/1995539753 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f39e4037020 con 0x7f39ec102ef0 2024-09-12T03:27:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.296+0000 7f39e9ffb640 1 --2- 172.21.15.64:0/1995539753 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f39dc0a2c90 0x7f39dc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.297+0000 7f39f0950640 1 --2- 172.21.15.64:0/1995539753 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f39dc0a2c90 0x7f39dc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.297+0000 7f39e9ffb640 1 -- 172.21.15.64:0/1995539753 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f39e40f2430 con 0x7f39ec102ef0 2024-09-12T03:27:58.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.297+0000 7f39f0950640 1 --2- 172.21.15.64:0/1995539753 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f39dc0a2c90 0x7f39dc0a5150 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f39e0009990 tx=0x7f39e0009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:58.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.301+0000 7f39e9ffb640 1 -- 172.21.15.64:0/1995539753 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f39e40bf100 con 0x7f39ec102ef0 2024-09-12T03:27:58.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.477+0000 7f39f2bb4640 1 -- 172.21.15.64:0/1995539753 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f39ec06a990 con 0x7f39ec102ef0 2024-09-12T03:27:58.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.478+0000 7f39e9ffb640 1 -- 172.21.15.64:0/1995539753 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24209 (secure 0 0 0) 0x7f39e403ce20 con 0x7f39ec102ef0 2024-09-12T03:27:58.478 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:27:58.478 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":88,"fsid":"b0812460-70b5-11ef-bcea-c7b262605968","created":"2024-09-12T03:19:15.900322+0000","modified":"2024-09-12T03:27:08.680818+0000","last_up_change":"2024-09-12T03:26:06.122971+0000","last_in_change":"2024-09-12T03:25:43.130689+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:22:21.915870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:26:55.826348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:26:57.984319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T03:27:04.460292+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"326271c9-1f28-4ce6-b3be-fcc57479d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6803","nonce":1307832590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6805","nonce":1307832590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6809","nonce":1307832590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6807","nonce":1307832590}]},"public_addr":"172.21.15.64:6803/1307832590","cluster_addr":"172.21.15.64:6805/1307832590","heartbeat_back_addr":"172.21.15.64:6809/1307832590","heartbeat_front_addr":"172.21.15.64:6807/1307832590","state":["exists","up"]},{"osd":1,"uuid":"7dca0c28-c234-423c-936e-2351d63f918e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6801","nonce":3123728622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6803","nonce":3123728622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6807","nonce":3123728622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6805","nonce":3123728622}]},"public_addr":"172.21.15.115:6801/3123728622","cluster_addr":"172.21.15.115:6803/3123728622","heartbeat_back_addr":"172.21.15.115:6807/3123728622","heartbeat_front_addr":"172.21.15.115:6805/3123728622","state":["exists","up"]},{"osd":2,"uuid":"4b3a3c7a-aec6-455a-9a9a-1ac9cc900f24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6801","nonce":1664125451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6803","nonce":1664125451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6807","nonce":1664125451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6805","nonce":1664125451}]},"public_addr":"172.21.15.195:6801/1664125451","cluster_addr":"172.21.15.195:6803/1664125451","heartbeat_back_addr":"172.21.15.195:6807/1664125451","heartbeat_front_addr":"172.21.15.195:6805/1664125451","state":["exists","up"]},{"osd":3,"uuid":"8768c339-2c1e-4294-8b78-3c2adab6750f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6811","nonce":3935517185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6813","nonce":3935517185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6817","nonce":3935517185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6815","nonce":3935517185}]},"public_addr":"172.21.15.64:6811/3935517185","cluster_addr":"172.21.15.64:6813/3935517185","heartbeat_back_addr":"172.21.15.64:6817/3935517185","heartbeat_front_addr":"172.21.15.64:6815/3935517185","state":["exists","up"]},{"osd":4,"uuid":"123aac47-a8ae-47c7-93a7-0dd293356322","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6809","nonce":446665227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6811","nonce":446665227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6815","nonce":446665227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6813","nonce":446665227}]},"public_addr":"172.21.15.115:6809/446665227","cluster_addr":"172.21.15.115:6811/446665227","heartbeat_back_addr":"172.21.15.115:6815/446665227","heartbeat_front_addr":"172.21.15.115:6813/446665227","state":["exists","up"]},{"osd":5,"uuid":"c90bcaed-e7a8-4852-8677-b01dc73ea44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6809","nonce":1187293668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6811","nonce":1187293668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6815","nonce":1187293668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6813","nonce":1187293668}]},"public_addr":"172.21.15.195:6809/1187293668","cluster_addr":"172.21.15.195:6811/1187293668","heartbeat_back_addr":"172.21.15.195:6815/1187293668","heartbeat_front_addr":"172.21.15.195:6813/1187293668","state":["exists","up"]},{"osd":6,"uuid":"fd292547-6695-4889-8867-e74318bbb00f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6819","nonce":560343274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6821","nonce":560343274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6825","nonce":560343274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6823","nonce":560343274}]},"public_addr":"172.21.15.64:6819/560343274","cluster_addr":"172.21.15.64:6821/560343274","heartbeat_back_addr":"172.21.15.64:6825/560343274","heartbeat_front_addr":"172.21.15.64:6823/560343274","state":["exists","up"]},{"osd":7,"uuid":"ad7c4151-cd98-4113-879b-ca2b7c83c7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6817","nonce":3075734174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6819","nonce":3075734174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6823","nonce":3075734174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6821","nonce":3075734174}]},"public_addr":"172.21.15.115:6817/3075734174","cluster_addr":"172.21.15.115:6819/3075734174","heartbeat_back_addr":"172.21.15.115:6823/3075734174","heartbeat_front_addr":"172.21.15.115:6821/3075734174","state":["exists","up"]},{"osd":8,"uuid":"6de50049-c28f-4f21-b123-05e55bf59127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6817","nonce":2184742494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6819","nonce":2184742494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6823","nonce":2184742494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6821","nonce":2184742494}]},"public_addr":"172.21.15.195:6817/2184742494","cluster_addr":"172.21.15.195:6819/2184742494","heartbeat_back_addr":"172.21.15.195:6823/2184742494","heartbeat_front_addr":"172.21.15.195:6821/2184742494","state":["exists","up"]},{"osd":9,"uuid":"36ab792c-58f2-4beb-9024-7f3d1e3785dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6827","nonce":561728112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6829","nonce":561728112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6833","nonce":561728112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6831","nonce":561728112}]},"public_addr":"172.21.15.64:6827/561728112","cluster_addr":"172.21.15.64:6829/561728112","heartbeat_back_addr":"172.21.15.64:6833/561728112","heartbeat_front_addr":"172.21.15.64:6831/561728112","state":["exists","up"]},{"osd":10,"uuid":"d1f7881c-ec3f-466e-9cea-1d5cb441d970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6825","nonce":2977887627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6827","nonce":2977887627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6831","nonce":2977887627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6829","nonce":2977887627}]},"public_addr":"172.21.15.115:6825/2977887627","cluster_addr":"172.21.15.115:6827/2977887627","heartbeat_back_addr":"172.21.15.115:6831/2977887627","heartbeat_front_addr":"172.21.15.115:6829/2977887627","state":["exists","up"]},{"osd":11,"uuid":"7c46c028-22c7-450f-9927-2144a7a09e9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6825","nonce":591599352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6827","nonce":591599352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6831","nonce":591599352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6829","nonce":591599352}]},"public_addr":"172.21.15.195:6825/591599352","cluster_addr":"172.21.15.195:6827/591599352","heartbeat_back_addr":"172.21.15.195:6831/591599352","heartbeat_front_addr":"172.21.15.195:6829/591599352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:29.151454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:54.308909+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:18.680995+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:44.687548+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:10.166131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:35.026132+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:24:00.242966+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:24:24.600848+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:24:49.563060+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:25:15.025730+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:25:39.830995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:26:04.394276+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6801/68603379":"2024-09-13T03:20:07.411102+0000","172.21.15.64:0/3827202979":"2024-09-13T03:20:07.411102+0000","172.21.15.64:6800/68603379":"2024-09-13T03:20:07.411102+0000","172.21.15.64:0/1217136221":"2024-09-13T03:20:07.411102+0000","172.21.15.64:0/3079679964":"2024-09-13T03:19:36.839431+0000","172.21.15.64:6801/3186640984":"2024-09-13T03:19:36.839431+0000","172.21.15.64:0/3334629354":"2024-09-13T03:20:07.411102+0000","172.21.15.64:6800/3186640984":"2024-09-13T03:19:36.839431+0000","172.21.15.64:0/3564418115":"2024-09-13T03:19:36.839431+0000","172.21.15.64:0/686137382":"2024-09-13T03:19:36.839431+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:27:58.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.480+0000 7f39f2bb4640 1 -- 172.21.15.64:0/1995539753 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f39dc0a2c90 msgr2=0x7f39dc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:58.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.480+0000 7f39f2bb4640 1 --2- 172.21.15.64:0/1995539753 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f39dc0a2c90 0x7f39dc0a5150 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f39e0009990 tx=0x7f39e0009110 comp rx=0 tx=0).stop 2024-09-12T03:27:58.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.480+0000 7f39f2bb4640 1 -- 172.21.15.64:0/1995539753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39ec102ef0 msgr2=0x7f39ec19bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:58.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.480+0000 7f39f2bb4640 1 --2- 172.21.15.64:0/1995539753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39ec102ef0 0x7f39ec19bbc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f39e402ae70 tx=0x7f39e402ff60 comp rx=0 tx=0).stop 2024-09-12T03:27:58.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.480+0000 7f39f2bb4640 1 -- 172.21.15.64:0/1995539753 shutdown_connections 2024-09-12T03:27:58.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.480+0000 7f39f2bb4640 1 --2- 172.21.15.64:0/1995539753 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f39dc0a2c90 0x7f39dc0a5150 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:58.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.480+0000 7f39f2bb4640 1 --2- 172.21.15.64:0/1995539753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39ec102ef0 0x7f39ec19bbc0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:58.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.480+0000 7f39f2bb4640 1 --2- 172.21.15.64:0/1995539753 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39ec102570 0x7f39ec197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:58.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.480+0000 7f39f2bb4640 1 --2- 172.21.15.64:0/1995539753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39ec100e20 0x7f39ec1972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:58.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.480+0000 7f39f2bb4640 1 -- 172.21.15.64:0/1995539753 >> 172.21.15.64:0/1995539753 conn(0x7f39ec0fc7d0 msgr2=0x7f39ec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:58.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.480+0000 7f39f2bb4640 1 -- 172.21.15.64:0/1995539753 shutdown_connections 2024-09-12T03:27:58.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:58.480+0000 7f39f2bb4640 1 -- 172.21.15.64:0/1995539753 wait complete. 2024-09-12T03:27:59.113 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:58 smithi064 ceph-mon[30952]: pgmap v313: 129 pgs: 129 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2024-09-12T03:27:59.113 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:27:58 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/1995539753' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:27:59.142 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-12T03:27:59.143 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:27:59.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:58 smithi115 ceph-mon[34801]: pgmap v313: 129 pgs: 129 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2024-09-12T03:27:59.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:27:58 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/1995539753' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:27:59.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:58 smithi195 ceph-mon[34786]: pgmap v313: 129 pgs: 129 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2024-09-12T03:27:59.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:27:58 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/1995539753' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:27:59.401 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:59.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:59.899+0000 7efea1029640 1 -- 172.21.15.64:0/2439004740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe9c102790 msgr2=0x7efe9c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:59.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:59.899+0000 7efea1029640 1 --2- 172.21.15.64:0/2439004740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe9c102790 0x7efe9c1067e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efe90009900 tx=0x7efe9002a410 comp rx=0 tx=0).stop 2024-09-12T03:27:59.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:59.899+0000 7efea1029640 1 -- 172.21.15.64:0/2439004740 shutdown_connections 2024-09-12T03:27:59.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:59.899+0000 7efea1029640 1 --2- 172.21.15.64:0/2439004740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe9c106f10 0x7efe9c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:59.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:59.899+0000 7efea1029640 1 --2- 172.21.15.64:0/2439004740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe9c102790 0x7efe9c1067e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:59.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:59.899+0000 7efea1029640 1 --2- 172.21.15.64:0/2439004740 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe9c101de0 0x7efe9c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:59.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:59.899+0000 7efea1029640 1 -- 172.21.15.64:0/2439004740 >> 172.21.15.64:0/2439004740 conn(0x7efe9c0fd8c0 msgr2=0x7efe9c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:59.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:59.899+0000 7efea1029640 1 -- 172.21.15.64:0/2439004740 shutdown_connections 2024-09-12T03:27:59.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:59.899+0000 7efea1029640 1 -- 172.21.15.64:0/2439004740 wait complete. 2024-09-12T03:27:59.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:59.900+0000 7efea1029640 1 Processor -- start 2024-09-12T03:27:59.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:59.900+0000 7efea1029640 1 -- start start 2024-09-12T03:27:59.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:59.900+0000 7efea1029640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe9c101de0 0x7efe9c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:59.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:59.900+0000 7efea1029640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe9c102790 0x7efe9c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:59.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:59.900+0000 7efea1029640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe9c106f10 0x7efe9c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:59.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:59.900+0000 7efea1029640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efe9c111e20 con 0x7efe9c106f10 2024-09-12T03:27:59.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:59.900+0000 7efea1029640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efe9c111ca0 con 0x7efe9c101de0 2024-09-12T03:27:59.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:59.901+0000 7efea1029640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efe9c111fa0 con 0x7efe9c102790 2024-09-12T03:27:59.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:59.901+0000 7efe9b59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe9c106f10 0x7efe9c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:59.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:59.901+0000 7efe9b59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe9c106f10 0x7efe9c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54856/0 (socket says 172.21.15.64:54856) 2024-09-12T03:27:59.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:59.901+0000 7efe9b59e640 1 -- 172.21.15.64:0/1770468715 learned_addr learned my addr 172.21.15.64:0/1770468715 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:27:59.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:59.901+0000 7efe9ad9d640 1 --2- 172.21.15.64:0/1770468715 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe9c101de0 0x7efe9c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:59.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:59.901+0000 7efe9a59c640 1 --2- 172.21.15.64:0/1770468715 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe9c102790 0x7efe9c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:59.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:59.901+0000 7efe9b59e640 1 -- 172.21.15.64:0/1770468715 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe9c102790 msgr2=0x7efe9c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:59.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:59.901+0000 7efe9b59e640 1 --2- 172.21.15.64:0/1770468715 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe9c102790 0x7efe9c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:59.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:59.901+0000 7efe9b59e640 1 -- 172.21.15.64:0/1770468715 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe9c101de0 msgr2=0x7efe9c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:59.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:59.901+0000 7efe9b59e640 1 --2- 172.21.15.64:0/1770468715 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe9c101de0 0x7efe9c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:59.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:59.901+0000 7efe9b59e640 1 -- 172.21.15.64:0/1770468715 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe9c1a6840 con 0x7efe9c106f10 2024-09-12T03:27:59.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:59.901+0000 7efe9ad9d640 1 --2- 172.21.15.64:0/1770468715 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe9c101de0 0x7efe9c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:27:59.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:59.901+0000 7efe9b59e640 1 --2- 172.21.15.64:0/1770468715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe9c106f10 0x7efe9c074f70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7efe9400eb10 tx=0x7efe9400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:59.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:59.901+0000 7efe83fff640 1 -- 172.21.15.64:0/1770468715 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe94004b20 con 0x7efe9c106f10 2024-09-12T03:27:59.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:59.901+0000 7efea1029640 1 -- 172.21.15.64:0/1770468715 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe9c1a6b30 con 0x7efe9c106f10 2024-09-12T03:27:59.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:59.901+0000 7efe83fff640 1 -- 172.21.15.64:0/1770468715 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe94004cc0 con 0x7efe9c106f10 2024-09-12T03:27:59.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:59.901+0000 7efe83fff640 1 -- 172.21.15.64:0/1770468715 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe94010690 con 0x7efe9c106f10 2024-09-12T03:27:59.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:59.901+0000 7efea1029640 1 -- 172.21.15.64:0/1770468715 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe9c1a7040 con 0x7efe9c106f10 2024-09-12T03:27:59.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:59.903+0000 7efea1029640 1 -- 172.21.15.64:0/1770468715 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe9c1021c0 con 0x7efe9c106f10 2024-09-12T03:27:59.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:59.904+0000 7efe83fff640 1 -- 172.21.15.64:0/1770468715 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7efe94005020 con 0x7efe9c106f10 2024-09-12T03:27:59.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:59.904+0000 7efe83fff640 1 --2- 172.21.15.64:0/1770468715 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7efe840a2c10 0x7efe840a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:59.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:59.905+0000 7efe9ad9d640 1 --2- 172.21.15.64:0/1770468715 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7efe840a2c10 0x7efe840a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:59.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:59.905+0000 7efe83fff640 1 -- 172.21.15.64:0/1770468715 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7efe940cf920 con 0x7efe9c106f10 2024-09-12T03:27:59.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:59.905+0000 7efe9ad9d640 1 --2- 172.21.15.64:0/1770468715 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7efe840a2c10 0x7efe840a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efe8c005fd0 tx=0x7efe8c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:59.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:27:59.909+0000 7efe83fff640 1 -- 172.21.15.64:0/1770468715 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efe9409c4e0 con 0x7efe9c106f10 2024-09-12T03:28:00.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:00.086+0000 7efea1029640 1 -- 172.21.15.64:0/1770468715 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efe9c10a0f0 con 0x7efe9c106f10 2024-09-12T03:28:00.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:00.087+0000 7efe83fff640 1 -- 172.21.15.64:0/1770468715 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24209 (secure 0 0 0) 0x7efe9409bde0 con 0x7efe9c106f10 2024-09-12T03:28:00.087 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:28:00.088 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":88,"fsid":"b0812460-70b5-11ef-bcea-c7b262605968","created":"2024-09-12T03:19:15.900322+0000","modified":"2024-09-12T03:27:08.680818+0000","last_up_change":"2024-09-12T03:26:06.122971+0000","last_in_change":"2024-09-12T03:25:43.130689+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:22:21.915870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:26:55.826348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:26:57.984319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T03:27:04.460292+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"326271c9-1f28-4ce6-b3be-fcc57479d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6803","nonce":1307832590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6805","nonce":1307832590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6809","nonce":1307832590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6807","nonce":1307832590}]},"public_addr":"172.21.15.64:6803/1307832590","cluster_addr":"172.21.15.64:6805/1307832590","heartbeat_back_addr":"172.21.15.64:6809/1307832590","heartbeat_front_addr":"172.21.15.64:6807/1307832590","state":["exists","up"]},{"osd":1,"uuid":"7dca0c28-c234-423c-936e-2351d63f918e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6801","nonce":3123728622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6803","nonce":3123728622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6807","nonce":3123728622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6805","nonce":3123728622}]},"public_addr":"172.21.15.115:6801/3123728622","cluster_addr":"172.21.15.115:6803/3123728622","heartbeat_back_addr":"172.21.15.115:6807/3123728622","heartbeat_front_addr":"172.21.15.115:6805/3123728622","state":["exists","up"]},{"osd":2,"uuid":"4b3a3c7a-aec6-455a-9a9a-1ac9cc900f24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6801","nonce":1664125451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6803","nonce":1664125451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6807","nonce":1664125451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6805","nonce":1664125451}]},"public_addr":"172.21.15.195:6801/1664125451","cluster_addr":"172.21.15.195:6803/1664125451","heartbeat_back_addr":"172.21.15.195:6807/1664125451","heartbeat_front_addr":"172.21.15.195:6805/1664125451","state":["exists","up"]},{"osd":3,"uuid":"8768c339-2c1e-4294-8b78-3c2adab6750f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6811","nonce":3935517185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6813","nonce":3935517185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6817","nonce":3935517185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6815","nonce":3935517185}]},"public_addr":"172.21.15.64:6811/3935517185","cluster_addr":"172.21.15.64:6813/3935517185","heartbeat_back_addr":"172.21.15.64:6817/3935517185","heartbeat_front_addr":"172.21.15.64:6815/3935517185","state":["exists","up"]},{"osd":4,"uuid":"123aac47-a8ae-47c7-93a7-0dd293356322","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6809","nonce":446665227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6811","nonce":446665227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6815","nonce":446665227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6813","nonce":446665227}]},"public_addr":"172.21.15.115:6809/446665227","cluster_addr":"172.21.15.115:6811/446665227","heartbeat_back_addr":"172.21.15.115:6815/446665227","heartbeat_front_addr":"172.21.15.115:6813/446665227","state":["exists","up"]},{"osd":5,"uuid":"c90bcaed-e7a8-4852-8677-b01dc73ea44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6809","nonce":1187293668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6811","nonce":1187293668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6815","nonce":1187293668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6813","nonce":1187293668}]},"public_addr":"172.21.15.195:6809/1187293668","cluster_addr":"172.21.15.195:6811/1187293668","heartbeat_back_addr":"172.21.15.195:6815/1187293668","heartbeat_front_addr":"172.21.15.195:6813/1187293668","state":["exists","up"]},{"osd":6,"uuid":"fd292547-6695-4889-8867-e74318bbb00f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6819","nonce":560343274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6821","nonce":560343274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6825","nonce":560343274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6823","nonce":560343274}]},"public_addr":"172.21.15.64:6819/560343274","cluster_addr":"172.21.15.64:6821/560343274","heartbeat_back_addr":"172.21.15.64:6825/560343274","heartbeat_front_addr":"172.21.15.64:6823/560343274","state":["exists","up"]},{"osd":7,"uuid":"ad7c4151-cd98-4113-879b-ca2b7c83c7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6817","nonce":3075734174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6819","nonce":3075734174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6823","nonce":3075734174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6821","nonce":3075734174}]},"public_addr":"172.21.15.115:6817/3075734174","cluster_addr":"172.21.15.115:6819/3075734174","heartbeat_back_addr":"172.21.15.115:6823/3075734174","heartbeat_front_addr":"172.21.15.115:6821/3075734174","state":["exists","up"]},{"osd":8,"uuid":"6de50049-c28f-4f21-b123-05e55bf59127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6817","nonce":2184742494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6819","nonce":2184742494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6823","nonce":2184742494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6821","nonce":2184742494}]},"public_addr":"172.21.15.195:6817/2184742494","cluster_addr":"172.21.15.195:6819/2184742494","heartbeat_back_addr":"172.21.15.195:6823/2184742494","heartbeat_front_addr":"172.21.15.195:6821/2184742494","state":["exists","up"]},{"osd":9,"uuid":"36ab792c-58f2-4beb-9024-7f3d1e3785dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6827","nonce":561728112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6829","nonce":561728112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6833","nonce":561728112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6831","nonce":561728112}]},"public_addr":"172.21.15.64:6827/561728112","cluster_addr":"172.21.15.64:6829/561728112","heartbeat_back_addr":"172.21.15.64:6833/561728112","heartbeat_front_addr":"172.21.15.64:6831/561728112","state":["exists","up"]},{"osd":10,"uuid":"d1f7881c-ec3f-466e-9cea-1d5cb441d970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6825","nonce":2977887627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6827","nonce":2977887627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6831","nonce":2977887627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6829","nonce":2977887627}]},"public_addr":"172.21.15.115:6825/2977887627","cluster_addr":"172.21.15.115:6827/2977887627","heartbeat_back_addr":"172.21.15.115:6831/2977887627","heartbeat_front_addr":"172.21.15.115:6829/2977887627","state":["exists","up"]},{"osd":11,"uuid":"7c46c028-22c7-450f-9927-2144a7a09e9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6825","nonce":591599352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6827","nonce":591599352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6831","nonce":591599352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6829","nonce":591599352}]},"public_addr":"172.21.15.195:6825/591599352","cluster_addr":"172.21.15.195:6827/591599352","heartbeat_back_addr":"172.21.15.195:6831/591599352","heartbeat_front_addr":"172.21.15.195:6829/591599352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:29.151454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:54.308909+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:18.680995+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:44.687548+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:10.166131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:35.026132+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:24:00.242966+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:24:24.600848+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:24:49.563060+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:25:15.025730+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:25:39.830995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:26:04.394276+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6801/68603379":"2024-09-13T03:20:07.411102+0000","172.21.15.64:0/3827202979":"2024-09-13T03:20:07.411102+0000","172.21.15.64:6800/68603379":"2024-09-13T03:20:07.411102+0000","172.21.15.64:0/1217136221":"2024-09-13T03:20:07.411102+0000","172.21.15.64:0/3079679964":"2024-09-13T03:19:36.839431+0000","172.21.15.64:6801/3186640984":"2024-09-13T03:19:36.839431+0000","172.21.15.64:0/3334629354":"2024-09-13T03:20:07.411102+0000","172.21.15.64:6800/3186640984":"2024-09-13T03:19:36.839431+0000","172.21.15.64:0/3564418115":"2024-09-13T03:19:36.839431+0000","172.21.15.64:0/686137382":"2024-09-13T03:19:36.839431+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:28:00.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:00.089+0000 7efea1029640 1 -- 172.21.15.64:0/1770468715 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7efe840a2c10 msgr2=0x7efe840a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:00.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:00.090+0000 7efea1029640 1 --2- 172.21.15.64:0/1770468715 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7efe840a2c10 0x7efe840a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efe8c005fd0 tx=0x7efe8c00b390 comp rx=0 tx=0).stop 2024-09-12T03:28:00.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:00.090+0000 7efea1029640 1 -- 172.21.15.64:0/1770468715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe9c106f10 msgr2=0x7efe9c074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:00.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:00.090+0000 7efea1029640 1 --2- 172.21.15.64:0/1770468715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe9c106f10 0x7efe9c074f70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7efe9400eb10 tx=0x7efe9400c2d0 comp rx=0 tx=0).stop 2024-09-12T03:28:00.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:00.090+0000 7efea1029640 1 -- 172.21.15.64:0/1770468715 shutdown_connections 2024-09-12T03:28:00.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:00.090+0000 7efea1029640 1 --2- 172.21.15.64:0/1770468715 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7efe840a2c10 0x7efe840a50d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:00.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:00.090+0000 7efea1029640 1 --2- 172.21.15.64:0/1770468715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe9c106f10 0x7efe9c074f70 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:00.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:00.090+0000 7efea1029640 1 --2- 172.21.15.64:0/1770468715 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe9c102790 0x7efe9c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:00.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:00.090+0000 7efea1029640 1 --2- 172.21.15.64:0/1770468715 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe9c101de0 0x7efe9c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:00.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:00.090+0000 7efea1029640 1 -- 172.21.15.64:0/1770468715 >> 172.21.15.64:0/1770468715 conn(0x7efe9c0fd8c0 msgr2=0x7efe9c0fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:00.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:00.090+0000 7efea1029640 1 -- 172.21.15.64:0/1770468715 shutdown_connections 2024-09-12T03:28:00.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:00.090+0000 7efea1029640 1 -- 172.21.15.64:0/1770468715 wait complete. 2024-09-12T03:28:00.785 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-12T03:28:00.786 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-12T03:28:00.786 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-12T03:28:00.787 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-12T03:28:00.787 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-12T03:28:00.787 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-12T03:28:00.788 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-12T03:28:00.788 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-12T03:28:00.789 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-12T03:28:00.789 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-12T03:28:00.790 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-12T03:28:00.790 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-12T03:28:00.828 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:00 smithi064 ceph-mon[30952]: pgmap v314: 129 pgs: 129 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-09-12T03:28:00.828 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:00 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/1770468715' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:28:00.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:00 smithi195 ceph-mon[34786]: pgmap v314: 129 pgs: 129 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-09-12T03:28:00.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:00 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/1770468715' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:28:01.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:00 smithi115 ceph-mon[34801]: pgmap v314: 129 pgs: 129 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-09-12T03:28:01.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:00 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/1770468715' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:28:01.619 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:01.622 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:01.622 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:01.626 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:01.626 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:01.627 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:01.627 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:01.631 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:01.634 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:01.638 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:01.641 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:01.727 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:02.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.358+0000 7f47caf65640 1 -- 172.21.15.64:0/2826653620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47c4106f10 msgr2=0x7f47c410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:02.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.358+0000 7f47caf65640 1 --2- 172.21.15.64:0/2826653620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47c4106f10 0x7f47c410ea90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f47bc009ca0 tx=0x7f47bc02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:28:02.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.359+0000 7f47caf65640 1 -- 172.21.15.64:0/2826653620 shutdown_connections 2024-09-12T03:28:02.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.359+0000 7f47caf65640 1 --2- 172.21.15.64:0/2826653620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47c4106f10 0x7f47c410ea90 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:02.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.359+0000 7f47caf65640 1 --2- 172.21.15.64:0/2826653620 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47c4102790 0x7f47c41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:02.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.359+0000 7f47caf65640 1 --2- 172.21.15.64:0/2826653620 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47c4101de0 0x7f47c41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:02.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.359+0000 7f47caf65640 1 -- 172.21.15.64:0/2826653620 >> 172.21.15.64:0/2826653620 conn(0x7f47c40fd8c0 msgr2=0x7f47c40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:02.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.359+0000 7f47caf65640 1 -- 172.21.15.64:0/2826653620 shutdown_connections 2024-09-12T03:28:02.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.359+0000 7f47caf65640 1 -- 172.21.15.64:0/2826653620 wait complete. 2024-09-12T03:28:02.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.359+0000 7f47caf65640 1 Processor -- start 2024-09-12T03:28:02.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.359+0000 7f47caf65640 1 -- start start 2024-09-12T03:28:02.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.359+0000 7f47caf65640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47c4101de0 0x7f47c41983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:02.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.359+0000 7f47caf65640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47c4102790 0x7f47c4198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:02.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.359+0000 7f47caf65640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47c4106f10 0x7f47c419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:02.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.359+0000 7f47caf65640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f47c4111e40 con 0x7f47c4106f10 2024-09-12T03:28:02.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.359+0000 7f47caf65640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f47c4111cc0 con 0x7f47c4101de0 2024-09-12T03:28:02.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.359+0000 7f47caf65640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f47c4111fc0 con 0x7f47c4102790 2024-09-12T03:28:02.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.360+0000 7f47c9502640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47c4106f10 0x7f47c419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:02.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.360+0000 7f47c9502640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47c4106f10 0x7f47c419ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54866/0 (socket says 172.21.15.64:54866) 2024-09-12T03:28:02.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.360+0000 7f47c9502640 1 -- 172.21.15.64:0/1541471919 learned_addr learned my addr 172.21.15.64:0/1541471919 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:02.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.360+0000 7f47c8d01640 1 --2- 172.21.15.64:0/1541471919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47c4101de0 0x7f47c41983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:02.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.360+0000 7f47c3fff640 1 --2- 172.21.15.64:0/1541471919 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47c4102790 0x7f47c4198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:02.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.360+0000 7f47c9502640 1 -- 172.21.15.64:0/1541471919 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47c4102790 msgr2=0x7f47c4198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:02.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.360+0000 7f47c9502640 1 --2- 172.21.15.64:0/1541471919 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47c4102790 0x7f47c4198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:02.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.360+0000 7f47c9502640 1 -- 172.21.15.64:0/1541471919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47c4101de0 msgr2=0x7f47c41983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:02.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.360+0000 7f47c9502640 1 --2- 172.21.15.64:0/1541471919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47c4101de0 0x7f47c41983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:02.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.360+0000 7f47c9502640 1 -- 172.21.15.64:0/1541471919 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c419d390 con 0x7f47c4106f10 2024-09-12T03:28:02.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.360+0000 7f47c3fff640 1 --2- 172.21.15.64:0/1541471919 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47c4102790 0x7f47c4198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:28:02.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.360+0000 7f47c9502640 1 --2- 172.21.15.64:0/1541471919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47c4106f10 0x7f47c419ccb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f47bc009dc0 tx=0x7f47bc032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:02.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.360+0000 7f47c1ffb640 1 -- 172.21.15.64:0/1541471919 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc003cf0 con 0x7f47c4106f10 2024-09-12T03:28:02.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.360+0000 7f47c1ffb640 1 -- 172.21.15.64:0/1541471919 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47bc0044f0 con 0x7f47c4106f10 2024-09-12T03:28:02.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.360+0000 7f47caf65640 1 -- 172.21.15.64:0/1541471919 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47c419d620 con 0x7f47c4106f10 2024-09-12T03:28:02.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.360+0000 7f47c1ffb640 1 -- 172.21.15.64:0/1541471919 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc030bf0 con 0x7f47c4106f10 2024-09-12T03:28:02.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.361+0000 7f47caf65640 1 -- 172.21.15.64:0/1541471919 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47c41a4f60 con 0x7f47c4106f10 2024-09-12T03:28:02.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.361+0000 7f47caf65640 1 -- 172.21.15.64:0/1541471919 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f47c4062900 con 0x7f47c4106f10 2024-09-12T03:28:02.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.363+0000 7f47c1ffb640 1 -- 172.21.15.64:0/1541471919 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f47bc004040 con 0x7f47c4106f10 2024-09-12T03:28:02.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.364+0000 7f47c1ffb640 1 --2- 172.21.15.64:0/1541471919 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f47ac0a2c90 0x7f47ac0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:02.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.364+0000 7f47c8d01640 1 --2- 172.21.15.64:0/1541471919 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f47ac0a2c90 0x7f47ac0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:02.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.364+0000 7f47c1ffb640 1 -- 172.21.15.64:0/1541471919 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f47bc0f0b80 con 0x7f47c4106f10 2024-09-12T03:28:02.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.364+0000 7f47c1ffb640 1 --2- 172.21.15.64:0/1541471919 >> [v2:172.21.15.195:6800/1664125451,v1:172.21.15.195:6801/1664125451] conn(0x7f47ac0ad9c0 0x7f47ac0afe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:02.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.364+0000 7f47c1ffb640 1 -- 172.21.15.64:0/1541471919 --> [v2:172.21.15.195:6800/1664125451,v1:172.21.15.195:6801/1664125451] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f47bc00b150 con 0x7f47ac0ad9c0 2024-09-12T03:28:02.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.364+0000 7f47c1ffb640 1 -- 172.21.15.64:0/1541471919 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f47bc0f0fd0 con 0x7f47c4106f10 2024-09-12T03:28:02.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.364+0000 7f47c8d01640 1 --2- 172.21.15.64:0/1541471919 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f47ac0a2c90 0x7f47ac0a5150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f47b400bbe0 tx=0x7f47b400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:02.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.364+0000 7f47c3fff640 1 --2- 172.21.15.64:0/1541471919 >> [v2:172.21.15.195:6800/1664125451,v1:172.21.15.195:6801/1664125451] conn(0x7f47ac0ad9c0 0x7f47ac0afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:02.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.365+0000 7f47c3fff640 1 --2- 172.21.15.64:0/1541471919 >> [v2:172.21.15.195:6800/1664125451,v1:172.21.15.195:6801/1664125451] conn(0x7f47ac0ad9c0 0x7f47ac0afe00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:02.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.366+0000 7f47c1ffb640 1 -- 172.21.15.64:0/1541471919 <== osd.2 v2:172.21.15.195:6800/1664125451 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f47bc00b150 con 0x7f47ac0ad9c0 2024-09-12T03:28:02.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.379+0000 7f47caf65640 1 -- 172.21.15.64:0/1541471919 --> [v2:172.21.15.195:6800/1664125451,v1:172.21.15.195:6801/1664125451] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f47c4062d00 con 0x7f47ac0ad9c0 2024-09-12T03:28:02.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.380+0000 7f47c1ffb640 1 -- 172.21.15.64:0/1541471919 <== osd.2 v2:172.21.15.195:6800/1664125451 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f47c4062d00 con 0x7f47ac0ad9c0 2024-09-12T03:28:02.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.381+0000 7f47caf65640 1 -- 172.21.15.64:0/1541471919 >> [v2:172.21.15.195:6800/1664125451,v1:172.21.15.195:6801/1664125451] conn(0x7f47ac0ad9c0 msgr2=0x7f47ac0afe00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:02.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.381+0000 7f47caf65640 1 --2- 172.21.15.64:0/1541471919 >> [v2:172.21.15.195:6800/1664125451,v1:172.21.15.195:6801/1664125451] conn(0x7f47ac0ad9c0 0x7f47ac0afe00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:02.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.381+0000 7f47caf65640 1 -- 172.21.15.64:0/1541471919 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f47ac0a2c90 msgr2=0x7f47ac0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:02.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.381+0000 7f47caf65640 1 --2- 172.21.15.64:0/1541471919 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f47ac0a2c90 0x7f47ac0a5150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f47b400bbe0 tx=0x7f47b400b390 comp rx=0 tx=0).stop 2024-09-12T03:28:02.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.381+0000 7f47caf65640 1 -- 172.21.15.64:0/1541471919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47c4106f10 msgr2=0x7f47c419ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:02.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.381+0000 7f47caf65640 1 --2- 172.21.15.64:0/1541471919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47c4106f10 0x7f47c419ccb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f47bc009dc0 tx=0x7f47bc032ff0 comp rx=0 tx=0).stop 2024-09-12T03:28:02.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.381+0000 7f47c9502640 1 -- 172.21.15.64:0/1541471919 reap_dead start 2024-09-12T03:28:02.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.381+0000 7f47caf65640 1 -- 172.21.15.64:0/1541471919 shutdown_connections 2024-09-12T03:28:02.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.381+0000 7f47caf65640 1 -- 172.21.15.64:0/1541471919 >> 172.21.15.64:0/1541471919 conn(0x7f47c40fd8c0 msgr2=0x7f47c40fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:02.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.381+0000 7f47caf65640 1 -- 172.21.15.64:0/1541471919 shutdown_connections 2024-09-12T03:28:02.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:02.381+0000 7f47caf65640 1 -- 172.21.15.64:0/1541471919 wait complete. 2024-09-12T03:28:03.009 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:02 smithi064 ceph-mon[30952]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-12T03:28:03.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:02 smithi115 ceph-mon[34801]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-12T03:28:03.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:02 smithi195 ceph-mon[34786]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-12T03:28:03.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.650+0000 7f887c771640 1 -- 172.21.15.64:0/279989398 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8874106a80 msgr2=0x7f8874106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:03.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.650+0000 7f887c771640 1 --2- 172.21.15.64:0/279989398 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8874106a80 0x7f8874106e60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8870009930 tx=0x7f887002a420 comp rx=0 tx=0).stop 2024-09-12T03:28:03.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.651+0000 7f887c771640 1 -- 172.21.15.64:0/279989398 shutdown_connections 2024-09-12T03:28:03.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.651+0000 7f887c771640 1 --2- 172.21.15.64:0/279989398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8874101420 0x7f887410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:03.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.651+0000 7f887c771640 1 --2- 172.21.15.64:0/279989398 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8874100a80 0x7f8874100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:03.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.651+0000 7f887c771640 1 --2- 172.21.15.64:0/279989398 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8874106a80 0x7f8874106e60 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:03.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.651+0000 7f887c771640 1 -- 172.21.15.64:0/279989398 >> 172.21.15.64:0/279989398 conn(0x7f88740fc7a0 msgr2=0x7f88740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:03.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.651+0000 7f887c771640 1 -- 172.21.15.64:0/279989398 shutdown_connections 2024-09-12T03:28:03.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.651+0000 7f887c771640 1 -- 172.21.15.64:0/279989398 wait complete. 2024-09-12T03:28:03.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.651+0000 7f887c771640 1 Processor -- start 2024-09-12T03:28:03.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.651+0000 7f887c771640 1 -- start start 2024-09-12T03:28:03.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.652+0000 7f887c771640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8874100a80 0x7f88741983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:03.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.652+0000 7f887c771640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8874101420 0x7f88741988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:03.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.652+0000 7f887c771640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8874106a80 0x7f887419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:03.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.652+0000 7f887c771640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8874111e50 con 0x7f8874106a80 2024-09-12T03:28:03.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.652+0000 7f887c771640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8874111cd0 con 0x7f8874101420 2024-09-12T03:28:03.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.652+0000 7f887c771640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8874111fd0 con 0x7f8874100a80 2024-09-12T03:28:03.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.652+0000 7f887ad0e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8874106a80 0x7f887419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:03.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.652+0000 7f887ad0e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8874106a80 0x7f887419cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54884/0 (socket says 172.21.15.64:54884) 2024-09-12T03:28:03.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.652+0000 7f887ad0e640 1 -- 172.21.15.64:0/3144161325 learned_addr learned my addr 172.21.15.64:0/3144161325 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:03.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.652+0000 7f8879d0c640 1 --2- 172.21.15.64:0/3144161325 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8874101420 0x7f88741988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:03.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.652+0000 7f887a50d640 1 --2- 172.21.15.64:0/3144161325 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8874100a80 0x7f88741983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:03.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.652+0000 7f887ad0e640 1 -- 172.21.15.64:0/3144161325 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8874100a80 msgr2=0x7f88741983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:03.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.652+0000 7f887ad0e640 1 --2- 172.21.15.64:0/3144161325 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8874100a80 0x7f88741983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:03.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.652+0000 7f887ad0e640 1 -- 172.21.15.64:0/3144161325 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8874101420 msgr2=0x7f88741988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:03.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.652+0000 7f887ad0e640 1 --2- 172.21.15.64:0/3144161325 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8874101420 0x7f88741988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:03.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.652+0000 7f887ad0e640 1 -- 172.21.15.64:0/3144161325 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f887419d360 con 0x7f8874106a80 2024-09-12T03:28:03.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.652+0000 7f8879d0c640 1 --2- 172.21.15.64:0/3144161325 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8874101420 0x7f88741988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:03.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.652+0000 7f887ad0e640 1 --2- 172.21.15.64:0/3144161325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8874106a80 0x7f887419cc80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f886c00eaa0 tx=0x7f886c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:03.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.653+0000 7f88677fe640 1 -- 172.21.15.64:0/3144161325 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886c00cdb0 con 0x7f8874106a80 2024-09-12T03:28:03.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.653+0000 7f887c771640 1 -- 172.21.15.64:0/3144161325 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f887419d650 con 0x7f8874106a80 2024-09-12T03:28:03.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.653+0000 7f887c771640 1 -- 172.21.15.64:0/3144161325 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88741a4f30 con 0x7f8874106a80 2024-09-12T03:28:03.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.653+0000 7f88677fe640 1 -- 172.21.15.64:0/3144161325 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f886c005020 con 0x7f8874106a80 2024-09-12T03:28:03.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.653+0000 7f88677fe640 1 -- 172.21.15.64:0/3144161325 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886c01f060 con 0x7f8874106a80 2024-09-12T03:28:03.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.654+0000 7f887c771640 1 -- 172.21.15.64:0/3144161325 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8874062980 con 0x7f8874106a80 2024-09-12T03:28:03.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.655+0000 7f88677fe640 1 -- 172.21.15.64:0/3144161325 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f886c0051d0 con 0x7f8874106a80 2024-09-12T03:28:03.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.656+0000 7f88677fe640 1 --2- 172.21.15.64:0/3144161325 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f88600a2c10 0x7f88600a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:03.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.656+0000 7f887a50d640 1 --2- 172.21.15.64:0/3144161325 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f88600a2c10 0x7f88600a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:03.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.656+0000 7f88677fe640 1 -- 172.21.15.64:0/3144161325 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f886c0cf8d0 con 0x7f8874106a80 2024-09-12T03:28:03.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.656+0000 7f88677fe640 1 --2- 172.21.15.64:0/3144161325 >> [v2:172.21.15.195:6824/591599352,v1:172.21.15.195:6825/591599352] conn(0x7f88600ad940 0x7f88600afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:03.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.656+0000 7f88677fe640 1 -- 172.21.15.64:0/3144161325 --> [v2:172.21.15.195:6824/591599352,v1:172.21.15.195:6825/591599352] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f886c014ec0 con 0x7f88600ad940 2024-09-12T03:28:03.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.656+0000 7f88677fe640 1 -- 172.21.15.64:0/3144161325 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f886c0cfd20 con 0x7f8874106a80 2024-09-12T03:28:03.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.656+0000 7f887a50d640 1 --2- 172.21.15.64:0/3144161325 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f88600a2c10 0x7f88600a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8870009900 tx=0x7f8870000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:03.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.656+0000 7f8879d0c640 1 --2- 172.21.15.64:0/3144161325 >> [v2:172.21.15.195:6824/591599352,v1:172.21.15.195:6825/591599352] conn(0x7f88600ad940 0x7f88600afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:03.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.657+0000 7f8879d0c640 1 --2- 172.21.15.64:0/3144161325 >> [v2:172.21.15.195:6824/591599352,v1:172.21.15.195:6825/591599352] conn(0x7f88600ad940 0x7f88600afd80 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:03.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.658+0000 7f88677fe640 1 -- 172.21.15.64:0/3144161325 <== osd.11 v2:172.21.15.195:6824/591599352 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f886c014ec0 con 0x7f88600ad940 2024-09-12T03:28:03.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.672+0000 7f887c771640 1 -- 172.21.15.64:0/3144161325 --> [v2:172.21.15.195:6824/591599352,v1:172.21.15.195:6825/591599352] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8874062e00 con 0x7f88600ad940 2024-09-12T03:28:03.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.672+0000 7f88677fe640 1 -- 172.21.15.64:0/3144161325 <== osd.11 v2:172.21.15.195:6824/591599352 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8874062e00 con 0x7f88600ad940 2024-09-12T03:28:03.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.672+0000 7f887c771640 1 -- 172.21.15.64:0/3144161325 >> [v2:172.21.15.195:6824/591599352,v1:172.21.15.195:6825/591599352] conn(0x7f88600ad940 msgr2=0x7f88600afd80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:03.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.672+0000 7f887c771640 1 --2- 172.21.15.64:0/3144161325 >> [v2:172.21.15.195:6824/591599352,v1:172.21.15.195:6825/591599352] conn(0x7f88600ad940 0x7f88600afd80 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:03.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.673+0000 7f887c771640 1 -- 172.21.15.64:0/3144161325 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f88600a2c10 msgr2=0x7f88600a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:03.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.673+0000 7f887c771640 1 --2- 172.21.15.64:0/3144161325 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f88600a2c10 0x7f88600a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8870009900 tx=0x7f8870000bc0 comp rx=0 tx=0).stop 2024-09-12T03:28:03.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.673+0000 7f887c771640 1 -- 172.21.15.64:0/3144161325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8874106a80 msgr2=0x7f887419cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:03.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.673+0000 7f887c771640 1 --2- 172.21.15.64:0/3144161325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8874106a80 0x7f887419cc80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f886c00eaa0 tx=0x7f886c00ef60 comp rx=0 tx=0).stop 2024-09-12T03:28:03.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.673+0000 7f887ad0e640 1 -- 172.21.15.64:0/3144161325 reap_dead start 2024-09-12T03:28:03.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.673+0000 7f887c771640 1 -- 172.21.15.64:0/3144161325 shutdown_connections 2024-09-12T03:28:03.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.673+0000 7f887c771640 1 -- 172.21.15.64:0/3144161325 >> 172.21.15.64:0/3144161325 conn(0x7f88740fc7a0 msgr2=0x7f88740fd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:03.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.673+0000 7f887c771640 1 -- 172.21.15.64:0/3144161325 shutdown_connections 2024-09-12T03:28:03.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.673+0000 7f887c771640 1 -- 172.21.15.64:0/3144161325 wait complete. 2024-09-12T03:28:04.895 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:04 smithi064 ceph-mon[30952]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-12T03:28:05.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:04 smithi115 ceph-mon[34801]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-12T03:28:05.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.175+0000 7eff74c61640 1 -- 172.21.15.64:0/3409848953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff70100e20 msgr2=0x7eff70101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:05.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.175+0000 7eff74c61640 1 --2- 172.21.15.64:0/3409848953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff70100e20 0x7eff70101200 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7eff60009900 tx=0x7eff6002a480 comp rx=0 tx=0).stop 2024-09-12T03:28:05.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.175+0000 7eff74c61640 1 -- 172.21.15.64:0/3409848953 shutdown_connections 2024-09-12T03:28:05.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.175+0000 7eff74c61640 1 --2- 172.21.15.64:0/3409848953 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff70102ef0 0x7eff7010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:05.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.175+0000 7eff74c61640 1 --2- 172.21.15.64:0/3409848953 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff70102570 0x7eff701029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:05.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.175+0000 7eff74c61640 1 --2- 172.21.15.64:0/3409848953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff70100e20 0x7eff70101200 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:05.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.175+0000 7eff74c61640 1 -- 172.21.15.64:0/3409848953 >> 172.21.15.64:0/3409848953 conn(0x7eff700fc7d0 msgr2=0x7eff700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:05.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.175+0000 7eff74c61640 1 -- 172.21.15.64:0/3409848953 shutdown_connections 2024-09-12T03:28:05.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.175+0000 7eff74c61640 1 -- 172.21.15.64:0/3409848953 wait complete. 2024-09-12T03:28:05.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.176+0000 7eff74c61640 1 Processor -- start 2024-09-12T03:28:05.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.176+0000 7eff74c61640 1 -- start start 2024-09-12T03:28:05.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.176+0000 7eff74c61640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff70100e20 0x7eff701992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:05.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.176+0000 7eff74c61640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff70102570 0x7eff701997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:05.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.176+0000 7eff74c61640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff70102ef0 0x7eff701a4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:05.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.176+0000 7eff74c61640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7eff70110e30 con 0x7eff70102ef0 2024-09-12T03:28:05.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.176+0000 7eff74c61640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7eff70110cb0 con 0x7eff70100e20 2024-09-12T03:28:05.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.176+0000 7eff74c61640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7eff70110fb0 con 0x7eff70102570 2024-09-12T03:28:05.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.176+0000 7eff6ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff70102ef0 0x7eff701a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:05.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.176+0000 7eff6ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff70102ef0 0x7eff701a4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54940/0 (socket says 172.21.15.64:54940) 2024-09-12T03:28:05.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.176+0000 7eff6ed9d640 1 -- 172.21.15.64:0/2554404323 learned_addr learned my addr 172.21.15.64:0/2554404323 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:05.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.177+0000 7eff6e59c640 1 --2- 172.21.15.64:0/2554404323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff70100e20 0x7eff701992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:05.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.177+0000 7eff6dd9b640 1 --2- 172.21.15.64:0/2554404323 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff70102570 0x7eff701997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:05.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.177+0000 7eff6ed9d640 1 -- 172.21.15.64:0/2554404323 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff70102570 msgr2=0x7eff701997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:05.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.177+0000 7eff6ed9d640 1 --2- 172.21.15.64:0/2554404323 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff70102570 0x7eff701997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:05.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.177+0000 7eff6ed9d640 1 -- 172.21.15.64:0/2554404323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff70100e20 msgr2=0x7eff701992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:05.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.177+0000 7eff6ed9d640 1 --2- 172.21.15.64:0/2554404323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff70100e20 0x7eff701992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:05.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.177+0000 7eff6ed9d640 1 -- 172.21.15.64:0/2554404323 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff701a4860 con 0x7eff70102ef0 2024-09-12T03:28:05.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.177+0000 7eff6ed9d640 1 --2- 172.21.15.64:0/2554404323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff70102ef0 0x7eff701a4320 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7eff6800eaa0 tx=0x7eff6800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:05.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.177+0000 7eff677fe640 1 -- 172.21.15.64:0/2554404323 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6800cdb0 con 0x7eff70102ef0 2024-09-12T03:28:05.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.177+0000 7eff74c61640 1 -- 172.21.15.64:0/2554404323 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff701a4a30 con 0x7eff70102ef0 2024-09-12T03:28:05.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.177+0000 7eff677fe640 1 -- 172.21.15.64:0/2554404323 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff68005020 con 0x7eff70102ef0 2024-09-12T03:28:05.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.177+0000 7eff677fe640 1 -- 172.21.15.64:0/2554404323 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff680116d0 con 0x7eff70102ef0 2024-09-12T03:28:05.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.177+0000 7eff74c61640 1 -- 172.21.15.64:0/2554404323 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff701a4ef0 con 0x7eff70102ef0 2024-09-12T03:28:05.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.178+0000 7eff74c61640 1 -- 172.21.15.64:0/2554404323 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7eff70062980 con 0x7eff70102ef0 2024-09-12T03:28:05.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.180+0000 7eff677fe640 1 -- 172.21.15.64:0/2554404323 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7eff680051d0 con 0x7eff70102ef0 2024-09-12T03:28:05.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.181+0000 7eff677fe640 1 --2- 172.21.15.64:0/2554404323 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7eff5c0a2c10 0x7eff5c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:05.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.181+0000 7eff677fe640 1 -- 172.21.15.64:0/2554404323 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7eff680cf5a0 con 0x7eff70102ef0 2024-09-12T03:28:05.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.181+0000 7eff6e59c640 1 --2- 172.21.15.64:0/2554404323 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7eff5c0a2c10 0x7eff5c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:05.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.181+0000 7eff677fe640 1 --2- 172.21.15.64:0/2554404323 >> [v2:172.21.15.64:6810/3935517185,v1:172.21.15.64:6811/3935517185] conn(0x7eff5c0ad940 0x7eff5c0afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:05.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.181+0000 7eff677fe640 1 -- 172.21.15.64:0/2554404323 --> [v2:172.21.15.64:6810/3935517185,v1:172.21.15.64:6811/3935517185] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7eff68018ec0 con 0x7eff5c0ad940 2024-09-12T03:28:05.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.181+0000 7eff677fe640 1 -- 172.21.15.64:0/2554404323 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7eff680d6050 con 0x7eff70102ef0 2024-09-12T03:28:05.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.181+0000 7eff6dd9b640 1 --2- 172.21.15.64:0/2554404323 >> [v2:172.21.15.64:6810/3935517185,v1:172.21.15.64:6811/3935517185] conn(0x7eff5c0ad940 0x7eff5c0afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:05.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.181+0000 7eff6e59c640 1 --2- 172.21.15.64:0/2554404323 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7eff5c0a2c10 0x7eff5c0a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7eff60009230 tx=0x7eff600058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:05.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.182+0000 7eff6dd9b640 1 --2- 172.21.15.64:0/2554404323 >> [v2:172.21.15.64:6810/3935517185,v1:172.21.15.64:6811/3935517185] conn(0x7eff5c0ad940 0x7eff5c0afd80 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:05.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.182+0000 7eff677fe640 1 -- 172.21.15.64:0/2554404323 <== osd.3 v2:172.21.15.64:6810/3935517185 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7eff68018ec0 con 0x7eff5c0ad940 2024-09-12T03:28:05.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.199+0000 7eff74c61640 1 -- 172.21.15.64:0/2554404323 --> [v2:172.21.15.64:6810/3935517185,v1:172.21.15.64:6811/3935517185] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7eff70062e30 con 0x7eff5c0ad940 2024-09-12T03:28:05.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.200+0000 7eff677fe640 1 -- 172.21.15.64:0/2554404323 <== osd.3 v2:172.21.15.64:6810/3935517185 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7eff70062e30 con 0x7eff5c0ad940 2024-09-12T03:28:05.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.200+0000 7eff74c61640 1 -- 172.21.15.64:0/2554404323 >> [v2:172.21.15.64:6810/3935517185,v1:172.21.15.64:6811/3935517185] conn(0x7eff5c0ad940 msgr2=0x7eff5c0afd80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:05.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.200+0000 7eff74c61640 1 --2- 172.21.15.64:0/2554404323 >> [v2:172.21.15.64:6810/3935517185,v1:172.21.15.64:6811/3935517185] conn(0x7eff5c0ad940 0x7eff5c0afd80 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:05.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.201+0000 7eff74c61640 1 -- 172.21.15.64:0/2554404323 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7eff5c0a2c10 msgr2=0x7eff5c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:05.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.201+0000 7eff74c61640 1 --2- 172.21.15.64:0/2554404323 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7eff5c0a2c10 0x7eff5c0a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7eff60009230 tx=0x7eff600058d0 comp rx=0 tx=0).stop 2024-09-12T03:28:05.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.201+0000 7eff74c61640 1 -- 172.21.15.64:0/2554404323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff70102ef0 msgr2=0x7eff701a4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:05.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.201+0000 7eff74c61640 1 --2- 172.21.15.64:0/2554404323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff70102ef0 0x7eff701a4320 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7eff6800eaa0 tx=0x7eff6800ef60 comp rx=0 tx=0).stop 2024-09-12T03:28:05.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.201+0000 7eff6ed9d640 1 -- 172.21.15.64:0/2554404323 reap_dead start 2024-09-12T03:28:05.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.201+0000 7eff74c61640 1 -- 172.21.15.64:0/2554404323 shutdown_connections 2024-09-12T03:28:05.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.201+0000 7eff74c61640 1 -- 172.21.15.64:0/2554404323 >> 172.21.15.64:0/2554404323 conn(0x7eff700fc7d0 msgr2=0x7eff700fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:05.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.201+0000 7eff74c61640 1 -- 172.21.15.64:0/2554404323 shutdown_connections 2024-09-12T03:28:05.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.201+0000 7eff74c61640 1 -- 172.21.15.64:0/2554404323 wait complete. 2024-09-12T03:28:05.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:04 smithi195 ceph-mon[34786]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-12T03:28:05.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.549+0000 7fddad54e640 1 -- 172.21.15.64:0/2596943385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdda81036e0 msgr2=0x7fdda8107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:05.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.549+0000 7fddad54e640 1 --2- 172.21.15.64:0/2596943385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdda81036e0 0x7fdda8107b90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fdd90009930 tx=0x7fdd9002a420 comp rx=0 tx=0).stop 2024-09-12T03:28:05.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.549+0000 7fddad54e640 1 -- 172.21.15.64:0/2596943385 shutdown_connections 2024-09-12T03:28:05.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.549+0000 7fddad54e640 1 --2- 172.21.15.64:0/2596943385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdda81082a0 0x7fdda810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:05.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.549+0000 7fddad54e640 1 --2- 172.21.15.64:0/2596943385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdda81036e0 0x7fdda8107b90 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:05.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.549+0000 7fddad54e640 1 --2- 172.21.15.64:0/2596943385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdda8102d30 0x7fdda8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:05.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.550+0000 7fddad54e640 1 -- 172.21.15.64:0/2596943385 >> 172.21.15.64:0/2596943385 conn(0x7fdda80fc7d0 msgr2=0x7fdda80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:05.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.550+0000 7fddad54e640 1 -- 172.21.15.64:0/2596943385 shutdown_connections 2024-09-12T03:28:05.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.550+0000 7fddad54e640 1 -- 172.21.15.64:0/2596943385 wait complete. 2024-09-12T03:28:05.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.550+0000 7fddad54e640 1 Processor -- start 2024-09-12T03:28:05.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.550+0000 7fddad54e640 1 -- start start 2024-09-12T03:28:05.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.551+0000 7fddad54e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdda8102d30 0x7fdda8100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:05.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.551+0000 7fddad54e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdda81036e0 0x7fdda8100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:05.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.551+0000 7fddad54e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdda81082a0 0x7fdda8101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:05.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.551+0000 7fddad54e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdda8111f10 con 0x7fdda81082a0 2024-09-12T03:28:05.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.551+0000 7fddad54e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdda8111d90 con 0x7fdda81036e0 2024-09-12T03:28:05.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.551+0000 7fddad54e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdda8112090 con 0x7fdda8102d30 2024-09-12T03:28:05.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.551+0000 7fdda77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdda81082a0 0x7fdda8101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:05.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.551+0000 7fdda77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdda81082a0 0x7fdda8101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54870/0 (socket says 172.21.15.64:54870) 2024-09-12T03:28:05.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.551+0000 7fdda77fe640 1 -- 172.21.15.64:0/1217001985 learned_addr learned my addr 172.21.15.64:0/1217001985 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:05.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.551+0000 7fdda67fc640 1 --2- 172.21.15.64:0/1217001985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdda81036e0 0x7fdda8100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:05.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.551+0000 7fdda6ffd640 1 --2- 172.21.15.64:0/1217001985 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdda8102d30 0x7fdda8100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:05.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.551+0000 7fdda77fe640 1 -- 172.21.15.64:0/1217001985 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdda8102d30 msgr2=0x7fdda8100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:05.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.551+0000 7fdda77fe640 1 --2- 172.21.15.64:0/1217001985 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdda8102d30 0x7fdda8100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:05.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.551+0000 7fdda77fe640 1 -- 172.21.15.64:0/1217001985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdda81036e0 msgr2=0x7fdda8100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:05.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.551+0000 7fdda77fe640 1 --2- 172.21.15.64:0/1217001985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdda81036e0 0x7fdda8100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:05.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.551+0000 7fdda77fe640 1 -- 172.21.15.64:0/1217001985 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda81a69b0 con 0x7fdda81082a0 2024-09-12T03:28:05.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.551+0000 7fdda6ffd640 1 --2- 172.21.15.64:0/1217001985 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdda8102d30 0x7fdda8100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:28:05.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.551+0000 7fdda77fe640 1 --2- 172.21.15.64:0/1217001985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdda81082a0 0x7fdda8101410 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdd9800d950 tx=0x7fdd9800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:05.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.551+0000 7fdd9ffff640 1 -- 172.21.15.64:0/1217001985 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd98017030 con 0x7fdda81082a0 2024-09-12T03:28:05.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.551+0000 7fdd9ffff640 1 -- 172.21.15.64:0/1217001985 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd98004020 con 0x7fdda81082a0 2024-09-12T03:28:05.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.551+0000 7fddad54e640 1 -- 172.21.15.64:0/1217001985 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdda81a6ca0 con 0x7fdda81082a0 2024-09-12T03:28:05.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.551+0000 7fdd9ffff640 1 -- 172.21.15.64:0/1217001985 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd98003b20 con 0x7fdda81082a0 2024-09-12T03:28:05.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.552+0000 7fddad54e640 1 -- 172.21.15.64:0/1217001985 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdda81a7160 con 0x7fdda81082a0 2024-09-12T03:28:05.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.553+0000 7fddad54e640 1 -- 172.21.15.64:0/1217001985 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdda80628c0 con 0x7fdda81082a0 2024-09-12T03:28:05.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.554+0000 7fdd9ffff640 1 -- 172.21.15.64:0/1217001985 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fdd9801a070 con 0x7fdda81082a0 2024-09-12T03:28:05.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.554+0000 7fdd9ffff640 1 --2- 172.21.15.64:0/1217001985 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fdd940a2c10 0x7fdd940a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:05.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.554+0000 7fdda6ffd640 1 --2- 172.21.15.64:0/1217001985 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fdd940a2c10 0x7fdd940a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:05.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.554+0000 7fdd9ffff640 1 -- 172.21.15.64:0/1217001985 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fdd980cf230 con 0x7fdda81082a0 2024-09-12T03:28:05.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.555+0000 7fdd9ffff640 1 --2- 172.21.15.64:0/1217001985 >> [v2:172.21.15.195:6808/1187293668,v1:172.21.15.195:6809/1187293668] conn(0x7fdd940ad940 0x7fdd940afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:05.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.555+0000 7fdd9ffff640 1 -- 172.21.15.64:0/1217001985 --> [v2:172.21.15.195:6808/1187293668,v1:172.21.15.195:6809/1187293668] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdd98004370 con 0x7fdd940ad940 2024-09-12T03:28:05.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.555+0000 7fdd9ffff640 1 -- 172.21.15.64:0/1217001985 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fdd980cf710 con 0x7fdda81082a0 2024-09-12T03:28:05.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.555+0000 7fdda6ffd640 1 --2- 172.21.15.64:0/1217001985 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fdd940a2c10 0x7fdd940a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdda0007e70 tx=0x7fdda0007850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:05.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.555+0000 7fdda67fc640 1 --2- 172.21.15.64:0/1217001985 >> [v2:172.21.15.195:6808/1187293668,v1:172.21.15.195:6809/1187293668] conn(0x7fdd940ad940 0x7fdd940afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:05.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.555+0000 7fdda67fc640 1 --2- 172.21.15.64:0/1217001985 >> [v2:172.21.15.195:6808/1187293668,v1:172.21.15.195:6809/1187293668] conn(0x7fdd940ad940 0x7fdd940afd80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:05.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.556+0000 7fdd9ffff640 1 -- 172.21.15.64:0/1217001985 <== osd.5 v2:172.21.15.195:6808/1187293668 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdd98004370 con 0x7fdd940ad940 2024-09-12T03:28:05.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.570+0000 7fddad54e640 1 -- 172.21.15.64:0/1217001985 --> [v2:172.21.15.195:6808/1187293668,v1:172.21.15.195:6809/1187293668] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdda8062cc0 con 0x7fdd940ad940 2024-09-12T03:28:05.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.570+0000 7fdd9ffff640 1 -- 172.21.15.64:0/1217001985 <== osd.5 v2:172.21.15.195:6808/1187293668 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdda8062cc0 con 0x7fdd940ad940 2024-09-12T03:28:05.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.571+0000 7fddad54e640 1 -- 172.21.15.64:0/1217001985 >> [v2:172.21.15.195:6808/1187293668,v1:172.21.15.195:6809/1187293668] conn(0x7fdd940ad940 msgr2=0x7fdd940afd80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:05.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.571+0000 7fddad54e640 1 --2- 172.21.15.64:0/1217001985 >> [v2:172.21.15.195:6808/1187293668,v1:172.21.15.195:6809/1187293668] conn(0x7fdd940ad940 0x7fdd940afd80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:05.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.571+0000 7fddad54e640 1 -- 172.21.15.64:0/1217001985 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fdd940a2c10 msgr2=0x7fdd940a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:05.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.571+0000 7fddad54e640 1 --2- 172.21.15.64:0/1217001985 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fdd940a2c10 0x7fdd940a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdda0007e70 tx=0x7fdda0007850 comp rx=0 tx=0).stop 2024-09-12T03:28:05.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.571+0000 7fddad54e640 1 -- 172.21.15.64:0/1217001985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdda81082a0 msgr2=0x7fdda8101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:05.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.571+0000 7fddad54e640 1 --2- 172.21.15.64:0/1217001985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdda81082a0 0x7fdda8101410 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdd9800d950 tx=0x7fdd9800de10 comp rx=0 tx=0).stop 2024-09-12T03:28:05.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.571+0000 7fdda77fe640 1 -- 172.21.15.64:0/1217001985 reap_dead start 2024-09-12T03:28:05.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.571+0000 7fddad54e640 1 -- 172.21.15.64:0/1217001985 shutdown_connections 2024-09-12T03:28:05.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.571+0000 7fddad54e640 1 -- 172.21.15.64:0/1217001985 >> 172.21.15.64:0/1217001985 conn(0x7fdda80fc7d0 msgr2=0x7fdda8105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:05.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.571+0000 7fddad54e640 1 -- 172.21.15.64:0/1217001985 shutdown_connections 2024-09-12T03:28:05.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:03.571+0000 7fddad54e640 1 -- 172.21.15.64:0/1217001985 wait complete. 2024-09-12T03:28:06.285 INFO:teuthology.orchestra.run.smithi064.stdout:326417514523 2024-09-12T03:28:06.286 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-12T03:28:06.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.061+0000 7f16a47fb640 1 -- 172.21.15.64:0/2858256675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f169c100a80 msgr2=0x7f169c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:06.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.061+0000 7f16a47fb640 1 --2- 172.21.15.64:0/2858256675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f169c100a80 0x7f169c100ee0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f168c009930 tx=0x7f168c02a420 comp rx=0 tx=0).stop 2024-09-12T03:28:06.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.061+0000 7f16a47fb640 1 -- 172.21.15.64:0/2858256675 shutdown_connections 2024-09-12T03:28:06.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.061+0000 7f16a47fb640 1 --2- 172.21.15.64:0/2858256675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f169c101420 0x7f169c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.061+0000 7f16a47fb640 1 --2- 172.21.15.64:0/2858256675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f169c100a80 0x7f169c100ee0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.061+0000 7f16a47fb640 1 --2- 172.21.15.64:0/2858256675 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f169c106a80 0x7f169c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.061+0000 7f16a47fb640 1 -- 172.21.15.64:0/2858256675 >> 172.21.15.64:0/2858256675 conn(0x7f169c0fc7a0 msgr2=0x7f169c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:06.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.061+0000 7f16a47fb640 1 -- 172.21.15.64:0/2858256675 shutdown_connections 2024-09-12T03:28:06.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.061+0000 7f16a47fb640 1 -- 172.21.15.64:0/2858256675 wait complete. 2024-09-12T03:28:06.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.062+0000 7f16a47fb640 1 Processor -- start 2024-09-12T03:28:06.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.062+0000 7f16a47fb640 1 -- start start 2024-09-12T03:28:06.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.062+0000 7f16a47fb640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f169c100a80 0x7f169c19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:06.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.062+0000 7f16a47fb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f169c101420 0x7f169c19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:06.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.062+0000 7f16a47fb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f169c106a80 0x7f169c19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:06.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.062+0000 7f16a47fb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f169c111e50 con 0x7f169c101420 2024-09-12T03:28:06.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.062+0000 7f16a47fb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f169c111cd0 con 0x7f169c106a80 2024-09-12T03:28:06.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.062+0000 7f16a47fb640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f169c111fd0 con 0x7f169c100a80 2024-09-12T03:28:06.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.062+0000 7f16a1d96640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f169c101420 0x7f169c19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:06.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.062+0000 7f16a1d96640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f169c101420 0x7f169c19ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54912/0 (socket says 172.21.15.64:54912) 2024-09-12T03:28:06.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.062+0000 7f16a1d96640 1 -- 172.21.15.64:0/1209320714 learned_addr learned my addr 172.21.15.64:0/1209320714 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:06.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.062+0000 7f16a2597640 1 --2- 172.21.15.64:0/1209320714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f169c100a80 0x7f169c19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:06.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.062+0000 7f16a1d96640 1 -- 172.21.15.64:0/1209320714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f169c100a80 msgr2=0x7f169c19a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:06.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.062+0000 7f16a2d98640 1 --2- 172.21.15.64:0/1209320714 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f169c106a80 0x7f169c19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:06.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.062+0000 7f16a1d96640 1 --2- 172.21.15.64:0/1209320714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f169c100a80 0x7f169c19a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.062+0000 7f16a1d96640 1 -- 172.21.15.64:0/1209320714 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f169c106a80 msgr2=0x7f169c19eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:06.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.062+0000 7f16a1d96640 1 --2- 172.21.15.64:0/1209320714 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f169c106a80 0x7f169c19eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.062+0000 7f16a1d96640 1 -- 172.21.15.64:0/1209320714 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f169c19f5c0 con 0x7f169c101420 2024-09-12T03:28:06.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.062+0000 7f16a2597640 1 --2- 172.21.15.64:0/1209320714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f169c100a80 0x7f169c19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:28:06.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.062+0000 7f16a2d98640 1 --2- 172.21.15.64:0/1209320714 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f169c106a80 0x7f169c19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:28:06.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.063+0000 7f16a1d96640 1 --2- 172.21.15.64:0/1209320714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f169c101420 0x7f169c19ab50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f168c004770 tx=0x7f168c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:06.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.063+0000 7f16937fe640 1 -- 172.21.15.64:0/1209320714 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c03f070 con 0x7f169c101420 2024-09-12T03:28:06.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.063+0000 7f16937fe640 1 -- 172.21.15.64:0/1209320714 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f168c03b460 con 0x7f169c101420 2024-09-12T03:28:06.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.063+0000 7f16a47fb640 1 -- 172.21.15.64:0/1209320714 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f169c19f850 con 0x7f169c101420 2024-09-12T03:28:06.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.063+0000 7f16937fe640 1 -- 172.21.15.64:0/1209320714 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c035070 con 0x7f169c101420 2024-09-12T03:28:06.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.063+0000 7f16a47fb640 1 -- 172.21.15.64:0/1209320714 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f169c1a7190 con 0x7f169c101420 2024-09-12T03:28:06.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.063+0000 7f16a47fb640 1 -- 172.21.15.64:0/1209320714 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f169c0fe170 con 0x7f169c101420 2024-09-12T03:28:06.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.065+0000 7f16937fe640 1 -- 172.21.15.64:0/1209320714 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f168c03b610 con 0x7f169c101420 2024-09-12T03:28:06.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.065+0000 7f16937fe640 1 --2- 172.21.15.64:0/1209320714 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f16880a2c90 0x7f16880a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:06.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.066+0000 7f16a2597640 1 --2- 172.21.15.64:0/1209320714 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f16880a2c90 0x7f16880a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:06.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.066+0000 7f16937fe640 1 -- 172.21.15.64:0/1209320714 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f168c0efe60 con 0x7f169c101420 2024-09-12T03:28:06.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.066+0000 7f16937fe640 1 --2- 172.21.15.64:0/1209320714 >> [v2:172.21.15.195:6816/2184742494,v1:172.21.15.195:6817/2184742494] conn(0x7f16880ad9c0 0x7f16880afe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:06.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.066+0000 7f16937fe640 1 -- 172.21.15.64:0/1209320714 --> [v2:172.21.15.195:6816/2184742494,v1:172.21.15.195:6817/2184742494] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f168c04be50 con 0x7f16880ad9c0 2024-09-12T03:28:06.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.066+0000 7f16937fe640 1 -- 172.21.15.64:0/1209320714 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f168c0f02b0 con 0x7f169c101420 2024-09-12T03:28:06.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.066+0000 7f16a2597640 1 --2- 172.21.15.64:0/1209320714 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f16880a2c90 0x7f16880a5150 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1698005fd0 tx=0x7f16980094d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:06.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.066+0000 7f16a2d98640 1 --2- 172.21.15.64:0/1209320714 >> [v2:172.21.15.195:6816/2184742494,v1:172.21.15.195:6817/2184742494] conn(0x7f16880ad9c0 0x7f16880afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:06.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.067+0000 7f16a2d98640 1 --2- 172.21.15.64:0/1209320714 >> [v2:172.21.15.195:6816/2184742494,v1:172.21.15.195:6817/2184742494] conn(0x7f16880ad9c0 0x7f16880afe00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:06.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.067+0000 7f16937fe640 1 -- 172.21.15.64:0/1209320714 <== osd.8 v2:172.21.15.195:6816/2184742494 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f168c04be50 con 0x7f16880ad9c0 2024-09-12T03:28:06.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.081+0000 7f16a47fb640 1 -- 172.21.15.64:0/1209320714 --> [v2:172.21.15.195:6816/2184742494,v1:172.21.15.195:6817/2184742494] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f169c1021c0 con 0x7f16880ad9c0 2024-09-12T03:28:06.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.082+0000 7f16937fe640 1 -- 172.21.15.64:0/1209320714 <== osd.8 v2:172.21.15.195:6816/2184742494 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f169c1021c0 con 0x7f16880ad9c0 2024-09-12T03:28:06.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.082+0000 7f16a47fb640 1 -- 172.21.15.64:0/1209320714 >> [v2:172.21.15.195:6816/2184742494,v1:172.21.15.195:6817/2184742494] conn(0x7f16880ad9c0 msgr2=0x7f16880afe00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:06.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.082+0000 7f16a47fb640 1 --2- 172.21.15.64:0/1209320714 >> [v2:172.21.15.195:6816/2184742494,v1:172.21.15.195:6817/2184742494] conn(0x7f16880ad9c0 0x7f16880afe00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.082+0000 7f16a47fb640 1 -- 172.21.15.64:0/1209320714 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f16880a2c90 msgr2=0x7f16880a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:06.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.082+0000 7f16a47fb640 1 --2- 172.21.15.64:0/1209320714 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f16880a2c90 0x7f16880a5150 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1698005fd0 tx=0x7f16980094d0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.082+0000 7f16a47fb640 1 -- 172.21.15.64:0/1209320714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f169c101420 msgr2=0x7f169c19ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:06.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.082+0000 7f16a47fb640 1 --2- 172.21.15.64:0/1209320714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f169c101420 0x7f169c19ab50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f168c004770 tx=0x7f168c0042f0 comp rx=0 tx=0).stop 2024-09-12T03:28:06.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.082+0000 7f16a2d98640 1 -- 172.21.15.64:0/1209320714 reap_dead start 2024-09-12T03:28:06.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.082+0000 7f16a47fb640 1 -- 172.21.15.64:0/1209320714 shutdown_connections 2024-09-12T03:28:06.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.082+0000 7f16a47fb640 1 -- 172.21.15.64:0/1209320714 >> 172.21.15.64:0/1209320714 conn(0x7f169c0fc7a0 msgr2=0x7f169c10c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:06.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.082+0000 7f16a47fb640 1 -- 172.21.15.64:0/1209320714 shutdown_connections 2024-09-12T03:28:06.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:05.082+0000 7f16a47fb640 1 -- 172.21.15.64:0/1209320714 wait complete. 2024-09-12T03:28:07.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:06 smithi115 ceph-mon[34801]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-12T03:28:07.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:06 smithi195 ceph-mon[34786]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-12T03:28:07.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:06 smithi064 ceph-mon[30952]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-12T03:28:07.337 INFO:teuthology.orchestra.run.smithi064.stdout:236223201321 2024-09-12T03:28:07.338 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-12T03:28:07.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.505+0000 7f6911007640 1 -- 172.21.15.64:0/2498563674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f690c101dd0 msgr2=0x7f690c1021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:07.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.505+0000 7f6911007640 1 --2- 172.21.15.64:0/2498563674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f690c101dd0 0x7f690c1021b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f68fc009900 tx=0x7f68fc02a410 comp rx=0 tx=0).stop 2024-09-12T03:28:07.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.505+0000 7f6911007640 1 -- 172.21.15.64:0/2498563674 shutdown_connections 2024-09-12T03:28:07.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.505+0000 7f6911007640 1 --2- 172.21.15.64:0/2498563674 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f690c106f00 0x7f690c10ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:07.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.505+0000 7f6911007640 1 --2- 172.21.15.64:0/2498563674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f690c102780 0x7f690c1067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:07.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.505+0000 7f6911007640 1 --2- 172.21.15.64:0/2498563674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f690c101dd0 0x7f690c1021b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:07.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.505+0000 7f6911007640 1 -- 172.21.15.64:0/2498563674 >> 172.21.15.64:0/2498563674 conn(0x7f690c0fd8b0 msgr2=0x7f690c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:07.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.506+0000 7f6911007640 1 -- 172.21.15.64:0/2498563674 shutdown_connections 2024-09-12T03:28:07.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.506+0000 7f6911007640 1 -- 172.21.15.64:0/2498563674 wait complete. 2024-09-12T03:28:07.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.506+0000 7f6911007640 1 Processor -- start 2024-09-12T03:28:07.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.507+0000 7f6911007640 1 -- start start 2024-09-12T03:28:07.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.507+0000 7f6911007640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f690c101dd0 0x7f690c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:07.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.507+0000 7f6911007640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f690c102780 0x7f690c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:07.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.507+0000 7f6911007640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f690c106f00 0x7f690c19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:07.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.507+0000 7f6911007640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f690c111e10 con 0x7f690c102780 2024-09-12T03:28:07.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.507+0000 7f6911007640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f690c111c90 con 0x7f690c101dd0 2024-09-12T03:28:07.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.507+0000 7f6911007640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f690c111f90 con 0x7f690c106f00 2024-09-12T03:28:07.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.507+0000 7f690a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f690c102780 0x7f690c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:07.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.507+0000 7f690a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f690c102780 0x7f690c1988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54962/0 (socket says 172.21.15.64:54962) 2024-09-12T03:28:07.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.507+0000 7f690a59c640 1 -- 172.21.15.64:0/1648639102 learned_addr learned my addr 172.21.15.64:0/1648639102 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:07.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.507+0000 7f690b59e640 1 --2- 172.21.15.64:0/1648639102 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f690c106f00 0x7f690c19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:07.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.507+0000 7f690ad9d640 1 --2- 172.21.15.64:0/1648639102 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f690c101dd0 0x7f690c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:07.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.507+0000 7f690a59c640 1 -- 172.21.15.64:0/1648639102 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f690c106f00 msgr2=0x7f690c19cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:07.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.507+0000 7f690a59c640 1 --2- 172.21.15.64:0/1648639102 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f690c106f00 0x7f690c19cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:07.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.507+0000 7f690a59c640 1 -- 172.21.15.64:0/1648639102 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f690c101dd0 msgr2=0x7f690c198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:07.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.507+0000 7f690a59c640 1 --2- 172.21.15.64:0/1648639102 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f690c101dd0 0x7f690c198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:07.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.507+0000 7f690a59c640 1 -- 172.21.15.64:0/1648639102 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f690c19d340 con 0x7f690c102780 2024-09-12T03:28:07.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.507+0000 7f690ad9d640 1 --2- 172.21.15.64:0/1648639102 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f690c101dd0 0x7f690c198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:28:07.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.508+0000 7f690a59c640 1 --2- 172.21.15.64:0/1648639102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f690c102780 0x7f690c1988d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f690000d950 tx=0x7f690000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:07.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.508+0000 7f68f3fff640 1 -- 172.21.15.64:0/1648639102 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6900017030 con 0x7f690c102780 2024-09-12T03:28:07.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.508+0000 7f6911007640 1 -- 172.21.15.64:0/1648639102 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f690c19d630 con 0x7f690c102780 2024-09-12T03:28:07.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.508+0000 7f68f3fff640 1 -- 172.21.15.64:0/1648639102 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6900004020 con 0x7f690c102780 2024-09-12T03:28:07.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.508+0000 7f68f3fff640 1 -- 172.21.15.64:0/1648639102 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69000047a0 con 0x7f690c102780 2024-09-12T03:28:07.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.508+0000 7f6911007640 1 -- 172.21.15.64:0/1648639102 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f690c1a4f10 con 0x7f690c102780 2024-09-12T03:28:07.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.509+0000 7f6911007640 1 -- 172.21.15.64:0/1648639102 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f690c062940 con 0x7f690c102780 2024-09-12T03:28:07.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.510+0000 7f68f3fff640 1 -- 172.21.15.64:0/1648639102 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f69000041d0 con 0x7f690c102780 2024-09-12T03:28:07.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.511+0000 7f68f3fff640 1 --2- 172.21.15.64:0/1648639102 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f68f40a2c10 0x7f68f40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:07.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.511+0000 7f690ad9d640 1 --2- 172.21.15.64:0/1648639102 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f68f40a2c10 0x7f68f40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:07.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.511+0000 7f68f3fff640 1 -- 172.21.15.64:0/1648639102 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f69000cf590 con 0x7f690c102780 2024-09-12T03:28:07.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.511+0000 7f68f3fff640 1 --2- 172.21.15.64:0/1648639102 >> [v2:172.21.15.64:6818/560343274,v1:172.21.15.64:6819/560343274] conn(0x7f68f40ad940 0x7f68f40afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:07.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.511+0000 7f68f3fff640 1 -- 172.21.15.64:0/1648639102 --> [v2:172.21.15.64:6818/560343274,v1:172.21.15.64:6819/560343274] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f690001de50 con 0x7f68f40ad940 2024-09-12T03:28:07.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.511+0000 7f68f3fff640 1 -- 172.21.15.64:0/1648639102 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f69000cf9e0 con 0x7f690c102780 2024-09-12T03:28:07.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.511+0000 7f690b59e640 1 --2- 172.21.15.64:0/1648639102 >> [v2:172.21.15.64:6818/560343274,v1:172.21.15.64:6819/560343274] conn(0x7f68f40ad940 0x7f68f40afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:07.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.512+0000 7f690ad9d640 1 --2- 172.21.15.64:0/1648639102 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f68f40a2c10 0x7f68f40a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f68fc031440 tx=0x7f68fc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:07.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.512+0000 7f690b59e640 1 --2- 172.21.15.64:0/1648639102 >> [v2:172.21.15.64:6818/560343274,v1:172.21.15.64:6819/560343274] conn(0x7f68f40ad940 0x7f68f40afd80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:07.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.513+0000 7f68f3fff640 1 -- 172.21.15.64:0/1648639102 <== osd.6 v2:172.21.15.64:6818/560343274 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f690001de50 con 0x7f68f40ad940 2024-09-12T03:28:07.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.527+0000 7f6911007640 1 -- 172.21.15.64:0/1648639102 --> [v2:172.21.15.64:6818/560343274,v1:172.21.15.64:6819/560343274] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f690c062d40 con 0x7f68f40ad940 2024-09-12T03:28:07.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.527+0000 7f68f3fff640 1 -- 172.21.15.64:0/1648639102 <== osd.6 v2:172.21.15.64:6818/560343274 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f690c062d40 con 0x7f68f40ad940 2024-09-12T03:28:07.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.528+0000 7f6911007640 1 -- 172.21.15.64:0/1648639102 >> [v2:172.21.15.64:6818/560343274,v1:172.21.15.64:6819/560343274] conn(0x7f68f40ad940 msgr2=0x7f68f40afd80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:07.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.528+0000 7f6911007640 1 --2- 172.21.15.64:0/1648639102 >> [v2:172.21.15.64:6818/560343274,v1:172.21.15.64:6819/560343274] conn(0x7f68f40ad940 0x7f68f40afd80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:07.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.528+0000 7f6911007640 1 -- 172.21.15.64:0/1648639102 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f68f40a2c10 msgr2=0x7f68f40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:07.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.528+0000 7f6911007640 1 --2- 172.21.15.64:0/1648639102 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f68f40a2c10 0x7f68f40a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f68fc031440 tx=0x7f68fc005a20 comp rx=0 tx=0).stop 2024-09-12T03:28:07.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.528+0000 7f6911007640 1 -- 172.21.15.64:0/1648639102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f690c102780 msgr2=0x7f690c1988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:07.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.528+0000 7f6911007640 1 --2- 172.21.15.64:0/1648639102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f690c102780 0x7f690c1988d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f690000d950 tx=0x7f690000de10 comp rx=0 tx=0).stop 2024-09-12T03:28:07.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.528+0000 7f690b59e640 1 -- 172.21.15.64:0/1648639102 reap_dead start 2024-09-12T03:28:07.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.528+0000 7f6911007640 1 -- 172.21.15.64:0/1648639102 shutdown_connections 2024-09-12T03:28:07.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.528+0000 7f6911007640 1 -- 172.21.15.64:0/1648639102 >> 172.21.15.64:0/1648639102 conn(0x7f690c0fd8b0 msgr2=0x7f690c0fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:07.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.528+0000 7f6911007640 1 -- 172.21.15.64:0/1648639102 shutdown_connections 2024-09-12T03:28:07.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:07.528+0000 7f6911007640 1 -- 172.21.15.64:0/1648639102 wait complete. 2024-09-12T03:28:08.005 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:07 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2024-09-12T03:28:08.006 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:07 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-09-12T03:28:08.006 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:07 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]: dispatch 2024-09-12T03:28:08.006 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:07 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]: dispatch 2024-09-12T03:28:08.006 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:07 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-09-12T03:28:08.006 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:07 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]: dispatch 2024-09-12T03:28:08.006 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:07 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-09-12T03:28:08.006 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:07 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]: dispatch 2024-09-12T03:28:08.006 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:07 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]: dispatch 2024-09-12T03:28:08.006 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:07 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-09-12T03:28:08.033 INFO:teuthology.orchestra.run.smithi064.stdout:77309411400 2024-09-12T03:28:08.034 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-12T03:28:08.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:07 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2024-09-12T03:28:08.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:07 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-09-12T03:28:08.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:07 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]: dispatch 2024-09-12T03:28:08.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:07 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]: dispatch 2024-09-12T03:28:08.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:07 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-09-12T03:28:08.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:07 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]: dispatch 2024-09-12T03:28:08.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:07 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-09-12T03:28:08.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:07 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]: dispatch 2024-09-12T03:28:08.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:07 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]: dispatch 2024-09-12T03:28:08.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:07 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-09-12T03:28:08.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:07 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2024-09-12T03:28:08.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:07 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-09-12T03:28:08.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:07 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]: dispatch 2024-09-12T03:28:08.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:07 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]: dispatch 2024-09-12T03:28:08.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:07 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-09-12T03:28:08.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:07 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]: dispatch 2024-09-12T03:28:08.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:07 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-09-12T03:28:08.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:07 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]: dispatch 2024-09-12T03:28:08.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:07 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]: dispatch 2024-09-12T03:28:08.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:07 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-09-12T03:28:08.900 INFO:teuthology.orchestra.run.smithi064.stdout:154618822712 2024-09-12T03:28:08.901 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-12T03:28:08.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.826+0000 7f1f6035f640 1 -- 172.21.15.64:0/597756011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f58102570 msgr2=0x7f1f581029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:08.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.826+0000 7f1f6035f640 1 --2- 172.21.15.64:0/597756011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f58102570 0x7f1f581029b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c009900 tx=0x7f1f4c02a480 comp rx=0 tx=0).stop 2024-09-12T03:28:08.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.826+0000 7f1f6035f640 1 -- 172.21.15.64:0/597756011 shutdown_connections 2024-09-12T03:28:08.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.826+0000 7f1f6035f640 1 --2- 172.21.15.64:0/597756011 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f58102ef0 0x7f1f5810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:08.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.826+0000 7f1f6035f640 1 --2- 172.21.15.64:0/597756011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f58102570 0x7f1f581029b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:08.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.826+0000 7f1f6035f640 1 --2- 172.21.15.64:0/597756011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f58100e20 0x7f1f58101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:08.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.826+0000 7f1f6035f640 1 -- 172.21.15.64:0/597756011 >> 172.21.15.64:0/597756011 conn(0x7f1f580fc7d0 msgr2=0x7f1f580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:08.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.826+0000 7f1f6035f640 1 -- 172.21.15.64:0/597756011 shutdown_connections 2024-09-12T03:28:08.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.826+0000 7f1f6035f640 1 -- 172.21.15.64:0/597756011 wait complete. 2024-09-12T03:28:08.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.827+0000 7f1f6035f640 1 Processor -- start 2024-09-12T03:28:08.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.827+0000 7f1f6035f640 1 -- start start 2024-09-12T03:28:08.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.827+0000 7f1f6035f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f58100e20 0x7f1f580fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:08.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.827+0000 7f1f6035f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f58102570 0x7f1f580fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:08.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.827+0000 7f1f6035f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f58102ef0 0x7f1f580fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:08.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.827+0000 7f1f6035f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1f58110e30 con 0x7f1f58102ef0 2024-09-12T03:28:08.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.827+0000 7f1f6035f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1f58110cb0 con 0x7f1f58100e20 2024-09-12T03:28:08.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.827+0000 7f1f6035f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1f58110fb0 con 0x7f1f58102570 2024-09-12T03:28:08.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.827+0000 7f1f5e8fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f58102ef0 0x7f1f580fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:08.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.827+0000 7f1f5e0fb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f58100e20 0x7f1f580fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:08.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.827+0000 7f1f5e8fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f58102ef0 0x7f1f580fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54890/0 (socket says 172.21.15.64:54890) 2024-09-12T03:28:08.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.827+0000 7f1f5e8fc640 1 -- 172.21.15.64:0/3189496252 learned_addr learned my addr 172.21.15.64:0/3189496252 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:08.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.827+0000 7f1f5d8fa640 1 --2- 172.21.15.64:0/3189496252 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f58102570 0x7f1f580fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:08.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.828+0000 7f1f5e8fc640 1 -- 172.21.15.64:0/3189496252 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f58102570 msgr2=0x7f1f580fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:08.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.828+0000 7f1f5e8fc640 1 --2- 172.21.15.64:0/3189496252 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f58102570 0x7f1f580fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:08.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.828+0000 7f1f5e8fc640 1 -- 172.21.15.64:0/3189496252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f58100e20 msgr2=0x7f1f580fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:08.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.828+0000 7f1f5e8fc640 1 --2- 172.21.15.64:0/3189496252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f58100e20 0x7f1f580fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:08.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.828+0000 7f1f5e8fc640 1 -- 172.21.15.64:0/3189496252 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f581a68e0 con 0x7f1f58102ef0 2024-09-12T03:28:08.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.828+0000 7f1f5e8fc640 1 --2- 172.21.15.64:0/3189496252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f58102ef0 0x7f1f580fd700 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1f4800eaa0 tx=0x7f1f4800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:08.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.830+0000 7f1f577fe640 1 -- 172.21.15.64:0/3189496252 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4800cdb0 con 0x7f1f58102ef0 2024-09-12T03:28:08.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.831+0000 7f1f6035f640 1 -- 172.21.15.64:0/3189496252 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f581a6bd0 con 0x7f1f58102ef0 2024-09-12T03:28:08.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.831+0000 7f1f577fe640 1 -- 172.21.15.64:0/3189496252 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f48005020 con 0x7f1f58102ef0 2024-09-12T03:28:08.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.831+0000 7f1f577fe640 1 -- 172.21.15.64:0/3189496252 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f480116d0 con 0x7f1f58102ef0 2024-09-12T03:28:08.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.831+0000 7f1f6035f640 1 -- 172.21.15.64:0/3189496252 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f581a7130 con 0x7f1f58102ef0 2024-09-12T03:28:08.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.831+0000 7f1f6035f640 1 -- 172.21.15.64:0/3189496252 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1f58062940 con 0x7f1f58102ef0 2024-09-12T03:28:08.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.833+0000 7f1f577fe640 1 -- 172.21.15.64:0/3189496252 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1f480051d0 con 0x7f1f58102ef0 2024-09-12T03:28:08.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.834+0000 7f1f577fe640 1 --2- 172.21.15.64:0/3189496252 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f1f400a2c90 0x7f1f400a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:08.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.834+0000 7f1f577fe640 1 -- 172.21.15.64:0/3189496252 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f1f480cf820 con 0x7f1f58102ef0 2024-09-12T03:28:08.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.834+0000 7f1f5e0fb640 1 --2- 172.21.15.64:0/3189496252 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f1f400a2c90 0x7f1f400a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:08.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.834+0000 7f1f577fe640 1 --2- 172.21.15.64:0/3189496252 >> [v2:172.21.15.64:6802/1307832590,v1:172.21.15.64:6803/1307832590] conn(0x7f1f400ad9c0 0x7f1f400afe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:08.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.834+0000 7f1f577fe640 1 -- 172.21.15.64:0/3189496252 --> [v2:172.21.15.64:6802/1307832590,v1:172.21.15.64:6803/1307832590] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1f4801be50 con 0x7f1f400ad9c0 2024-09-12T03:28:08.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.834+0000 7f1f577fe640 1 -- 172.21.15.64:0/3189496252 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f1f480cfc70 con 0x7f1f58102ef0 2024-09-12T03:28:08.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.834+0000 7f1f5d8fa640 1 --2- 172.21.15.64:0/3189496252 >> [v2:172.21.15.64:6802/1307832590,v1:172.21.15.64:6803/1307832590] conn(0x7f1f400ad9c0 0x7f1f400afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:08.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.834+0000 7f1f5e0fb640 1 --2- 172.21.15.64:0/3189496252 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f1f400a2c90 0x7f1f400a5150 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1f5000bbe0 tx=0x7f1f5000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:08.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.834+0000 7f1f5d8fa640 1 --2- 172.21.15.64:0/3189496252 >> [v2:172.21.15.64:6802/1307832590,v1:172.21.15.64:6803/1307832590] conn(0x7f1f400ad9c0 0x7f1f400afe00 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:08.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.835+0000 7f1f577fe640 1 -- 172.21.15.64:0/3189496252 <== osd.0 v2:172.21.15.64:6802/1307832590 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1f4801be50 con 0x7f1f400ad9c0 2024-09-12T03:28:08.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.849+0000 7f1f6035f640 1 -- 172.21.15.64:0/3189496252 --> [v2:172.21.15.64:6802/1307832590,v1:172.21.15.64:6803/1307832590] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1f58062dc0 con 0x7f1f400ad9c0 2024-09-12T03:28:08.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.849+0000 7f1f577fe640 1 -- 172.21.15.64:0/3189496252 <== osd.0 v2:172.21.15.64:6802/1307832590 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1f58062dc0 con 0x7f1f400ad9c0 2024-09-12T03:28:08.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.850+0000 7f1f6035f640 1 -- 172.21.15.64:0/3189496252 >> [v2:172.21.15.64:6802/1307832590,v1:172.21.15.64:6803/1307832590] conn(0x7f1f400ad9c0 msgr2=0x7f1f400afe00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:08.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.850+0000 7f1f6035f640 1 --2- 172.21.15.64:0/3189496252 >> [v2:172.21.15.64:6802/1307832590,v1:172.21.15.64:6803/1307832590] conn(0x7f1f400ad9c0 0x7f1f400afe00 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:08.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.850+0000 7f1f6035f640 1 -- 172.21.15.64:0/3189496252 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f1f400a2c90 msgr2=0x7f1f400a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:08.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.850+0000 7f1f6035f640 1 --2- 172.21.15.64:0/3189496252 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f1f400a2c90 0x7f1f400a5150 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1f5000bbe0 tx=0x7f1f5000b390 comp rx=0 tx=0).stop 2024-09-12T03:28:08.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.850+0000 7f1f6035f640 1 -- 172.21.15.64:0/3189496252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f58102ef0 msgr2=0x7f1f580fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:08.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.850+0000 7f1f6035f640 1 --2- 172.21.15.64:0/3189496252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f58102ef0 0x7f1f580fd700 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1f4800eaa0 tx=0x7f1f4800ef60 comp rx=0 tx=0).stop 2024-09-12T03:28:08.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.850+0000 7f1f5e8fc640 1 -- 172.21.15.64:0/3189496252 reap_dead start 2024-09-12T03:28:08.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.850+0000 7f1f6035f640 1 -- 172.21.15.64:0/3189496252 shutdown_connections 2024-09-12T03:28:08.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.850+0000 7f1f6035f640 1 -- 172.21.15.64:0/3189496252 >> 172.21.15.64:0/3189496252 conn(0x7f1f580fc7d0 msgr2=0x7f1f5806a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:08.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.850+0000 7f1f6035f640 1 -- 172.21.15.64:0/3189496252 shutdown_connections 2024-09-12T03:28:08.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:04.850+0000 7f1f6035f640 1 -- 172.21.15.64:0/3189496252 wait complete. 2024-09-12T03:28:09.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:08 smithi115 ceph-mon[34801]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-12T03:28:09.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:08 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2024-09-12T03:28:09.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:08 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-09-12T03:28:09.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:08 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2024-09-12T03:28:09.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:08 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]': finished 2024-09-12T03:28:09.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:08 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-12T03:28:09.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:08 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-09-12T03:28:09.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:08 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-09-12T03:28:09.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:08 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2024-09-12T03:28:09.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:08 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]': finished 2024-09-12T03:28:09.144 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:08 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-09-12T03:28:09.144 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:08 smithi115 ceph-mon[34801]: osdmap e89: 12 total, 12 up, 12 in 2024-09-12T03:28:09.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:08 smithi195 ceph-mon[34786]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-12T03:28:09.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:08 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2024-09-12T03:28:09.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:08 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-09-12T03:28:09.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:08 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2024-09-12T03:28:09.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:08 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]': finished 2024-09-12T03:28:09.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:08 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-12T03:28:09.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:08 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-09-12T03:28:09.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:08 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-09-12T03:28:09.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:08 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2024-09-12T03:28:09.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:08 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]': finished 2024-09-12T03:28:09.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:08 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-09-12T03:28:09.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:08 smithi195 ceph-mon[34786]: osdmap e89: 12 total, 12 up, 12 in 2024-09-12T03:28:09.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:08 smithi064 ceph-mon[30952]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-12T03:28:09.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:08 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2024-09-12T03:28:09.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:08 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-09-12T03:28:09.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:08 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2024-09-12T03:28:09.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:08 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]': finished 2024-09-12T03:28:09.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:08 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-12T03:28:09.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:08 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-09-12T03:28:09.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:08 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-09-12T03:28:09.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:08 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2024-09-12T03:28:09.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:08 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]': finished 2024-09-12T03:28:09.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:08 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-09-12T03:28:09.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:08 smithi064 ceph-mon[30952]: osdmap e89: 12 total, 12 up, 12 in 2024-09-12T03:28:09.611 INFO:teuthology.orchestra.run.smithi064.stdout:184683593780 2024-09-12T03:28:09.611 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-12T03:28:09.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.769+0000 7efe7a62d640 1 -- 172.21.15.64:0/2897156154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe74102ef0 msgr2=0x7efe7410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:09.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.769+0000 7efe7a62d640 1 --2- 172.21.15.64:0/2897156154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe74102ef0 0x7efe7410da40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efe6c009ca0 tx=0x7efe6c02a950 comp rx=0 tx=0).stop 2024-09-12T03:28:09.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.770+0000 7efe7a62d640 1 -- 172.21.15.64:0/2897156154 shutdown_connections 2024-09-12T03:28:09.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.770+0000 7efe7a62d640 1 --2- 172.21.15.64:0/2897156154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe74102ef0 0x7efe7410da40 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:09.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.770+0000 7efe7a62d640 1 --2- 172.21.15.64:0/2897156154 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe74102570 0x7efe741029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:09.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.770+0000 7efe7a62d640 1 --2- 172.21.15.64:0/2897156154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe74100e20 0x7efe74101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:09.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.770+0000 7efe7a62d640 1 -- 172.21.15.64:0/2897156154 >> 172.21.15.64:0/2897156154 conn(0x7efe740fc7d0 msgr2=0x7efe740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:09.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.770+0000 7efe7a62d640 1 -- 172.21.15.64:0/2897156154 shutdown_connections 2024-09-12T03:28:09.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.770+0000 7efe7a62d640 1 -- 172.21.15.64:0/2897156154 wait complete. 2024-09-12T03:28:09.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.770+0000 7efe7a62d640 1 Processor -- start 2024-09-12T03:28:09.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.770+0000 7efe7a62d640 1 -- start start 2024-09-12T03:28:09.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.771+0000 7efe7a62d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe74100e20 0x7efe74197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:09.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.771+0000 7efe7a62d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe74102570 0x7efe741979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:09.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.771+0000 7efe7a62d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe74102ef0 0x7efe7419bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:09.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.771+0000 7efe7a62d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efe74110f20 con 0x7efe74100e20 2024-09-12T03:28:09.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.771+0000 7efe7a62d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efe74110da0 con 0x7efe74102ef0 2024-09-12T03:28:09.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.771+0000 7efe7a62d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efe741110a0 con 0x7efe74102570 2024-09-12T03:28:09.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.771+0000 7efe73fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe74100e20 0x7efe74197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:09.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.771+0000 7efe73fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe74100e20 0x7efe74197460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41900/0 (socket says 172.21.15.64:41900) 2024-09-12T03:28:09.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.771+0000 7efe73fff640 1 -- 172.21.15.64:0/4087211248 learned_addr learned my addr 172.21.15.64:0/4087211248 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:09.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.771+0000 7efe737fe640 1 --2- 172.21.15.64:0/4087211248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe74102570 0x7efe741979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:09.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.771+0000 7efe78bca640 1 --2- 172.21.15.64:0/4087211248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe74102ef0 0x7efe7419bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:09.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.771+0000 7efe73fff640 1 -- 172.21.15.64:0/4087211248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe74102570 msgr2=0x7efe741979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:09.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.771+0000 7efe73fff640 1 --2- 172.21.15.64:0/4087211248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe74102570 0x7efe741979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:09.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.771+0000 7efe73fff640 1 -- 172.21.15.64:0/4087211248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe74102ef0 msgr2=0x7efe7419bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:09.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.771+0000 7efe73fff640 1 --2- 172.21.15.64:0/4087211248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe74102ef0 0x7efe7419bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:09.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.771+0000 7efe73fff640 1 -- 172.21.15.64:0/4087211248 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe7419c410 con 0x7efe74100e20 2024-09-12T03:28:09.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.771+0000 7efe737fe640 1 --2- 172.21.15.64:0/4087211248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe74102570 0x7efe741979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:28:09.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.772+0000 7efe78bca640 1 --2- 172.21.15.64:0/4087211248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe74102ef0 0x7efe7419bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:28:09.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.772+0000 7efe73fff640 1 --2- 172.21.15.64:0/4087211248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe74100e20 0x7efe74197460 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efe6400b4a0 tx=0x7efe6400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:09.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.772+0000 7efe717fa640 1 -- 172.21.15.64:0/4087211248 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe64014070 con 0x7efe74100e20 2024-09-12T03:28:09.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.772+0000 7efe717fa640 1 -- 172.21.15.64:0/4087211248 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe64010ac0 con 0x7efe74100e20 2024-09-12T03:28:09.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.772+0000 7efe717fa640 1 -- 172.21.15.64:0/4087211248 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe64017c30 con 0x7efe74100e20 2024-09-12T03:28:09.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.772+0000 7efe7a62d640 1 -- 172.21.15.64:0/4087211248 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe7419c700 con 0x7efe74100e20 2024-09-12T03:28:09.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.772+0000 7efe7a62d640 1 -- 172.21.15.64:0/4087211248 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe74069f70 con 0x7efe74100e20 2024-09-12T03:28:09.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.773+0000 7efe7a62d640 1 -- 172.21.15.64:0/4087211248 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efe7406a580 con 0x7efe74100e20 2024-09-12T03:28:09.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.774+0000 7efe717fa640 1 -- 172.21.15.64:0/4087211248 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7efe64017070 con 0x7efe74100e20 2024-09-12T03:28:09.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.775+0000 7efe717fa640 1 --2- 172.21.15.64:0/4087211248 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7efe5c0a2c10 0x7efe5c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:09.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.775+0000 7efe737fe640 1 --2- 172.21.15.64:0/4087211248 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7efe5c0a2c10 0x7efe5c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:09.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.775+0000 7efe717fa640 1 -- 172.21.15.64:0/4087211248 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7efe640cecc0 con 0x7efe74100e20 2024-09-12T03:28:09.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.775+0000 7efe717fa640 1 --2- 172.21.15.64:0/4087211248 >> [v2:172.21.15.115:6808/446665227,v1:172.21.15.115:6809/446665227] conn(0x7efe5c0adde0 0x7efe5c0b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:09.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.775+0000 7efe717fa640 1 -- 172.21.15.64:0/4087211248 --> [v2:172.21.15.115:6808/446665227,v1:172.21.15.115:6809/446665227] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efe6400ee50 con 0x7efe5c0adde0 2024-09-12T03:28:09.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.775+0000 7efe717fa640 1 -- 172.21.15.64:0/4087211248 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7efe640cf110 con 0x7efe74100e20 2024-09-12T03:28:09.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.775+0000 7efe737fe640 1 --2- 172.21.15.64:0/4087211248 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7efe5c0a2c10 0x7efe5c0a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efe68006fd0 tx=0x7efe68008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:09.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.775+0000 7efe78bca640 1 --2- 172.21.15.64:0/4087211248 >> [v2:172.21.15.115:6808/446665227,v1:172.21.15.115:6809/446665227] conn(0x7efe5c0adde0 0x7efe5c0b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:09.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.776+0000 7efe78bca640 1 --2- 172.21.15.64:0/4087211248 >> [v2:172.21.15.115:6808/446665227,v1:172.21.15.115:6809/446665227] conn(0x7efe5c0adde0 0x7efe5c0b0280 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:09.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.777+0000 7efe717fa640 1 -- 172.21.15.64:0/4087211248 <== osd.4 v2:172.21.15.115:6808/446665227 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efe6400ee50 con 0x7efe5c0adde0 2024-09-12T03:28:09.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.790+0000 7efe7a62d640 1 -- 172.21.15.64:0/4087211248 --> [v2:172.21.15.115:6808/446665227,v1:172.21.15.115:6809/446665227] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efe7406aa00 con 0x7efe5c0adde0 2024-09-12T03:28:09.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.791+0000 7efe717fa640 1 -- 172.21.15.64:0/4087211248 <== osd.4 v2:172.21.15.115:6808/446665227 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7efe7406aa00 con 0x7efe5c0adde0 2024-09-12T03:28:09.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.791+0000 7efe7a62d640 1 -- 172.21.15.64:0/4087211248 >> [v2:172.21.15.115:6808/446665227,v1:172.21.15.115:6809/446665227] conn(0x7efe5c0adde0 msgr2=0x7efe5c0b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:09.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.792+0000 7efe7a62d640 1 --2- 172.21.15.64:0/4087211248 >> [v2:172.21.15.115:6808/446665227,v1:172.21.15.115:6809/446665227] conn(0x7efe5c0adde0 0x7efe5c0b0280 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:09.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.792+0000 7efe7a62d640 1 -- 172.21.15.64:0/4087211248 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7efe5c0a2c10 msgr2=0x7efe5c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:09.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.792+0000 7efe7a62d640 1 --2- 172.21.15.64:0/4087211248 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7efe5c0a2c10 0x7efe5c0a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efe68006fd0 tx=0x7efe68008040 comp rx=0 tx=0).stop 2024-09-12T03:28:09.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.792+0000 7efe7a62d640 1 -- 172.21.15.64:0/4087211248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe74100e20 msgr2=0x7efe74197460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:09.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.792+0000 7efe7a62d640 1 --2- 172.21.15.64:0/4087211248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe74100e20 0x7efe74197460 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efe6400b4a0 tx=0x7efe6400b960 comp rx=0 tx=0).stop 2024-09-12T03:28:09.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.792+0000 7efe78bca640 1 -- 172.21.15.64:0/4087211248 reap_dead start 2024-09-12T03:28:09.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.792+0000 7efe7a62d640 1 -- 172.21.15.64:0/4087211248 shutdown_connections 2024-09-12T03:28:09.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.792+0000 7efe7a62d640 1 -- 172.21.15.64:0/4087211248 >> 172.21.15.64:0/4087211248 conn(0x7efe740fc7d0 msgr2=0x7efe740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:09.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.792+0000 7efe7a62d640 1 -- 172.21.15.64:0/4087211248 shutdown_connections 2024-09-12T03:28:09.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:09.792+0000 7efe7a62d640 1 -- 172.21.15.64:0/4087211248 wait complete. 2024-09-12T03:28:10.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.002+0000 7f04c2770640 1 -- 172.21.15.64:0/1335173889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04bc102ef0 msgr2=0x7f04bc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:10.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.002+0000 7f04c2770640 1 --2- 172.21.15.64:0/1335173889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04bc102ef0 0x7f04bc10da40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f04b4009ca0 tx=0x7f04b402a950 comp rx=0 tx=0).stop 2024-09-12T03:28:10.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.002+0000 7f04c2770640 1 -- 172.21.15.64:0/1335173889 shutdown_connections 2024-09-12T03:28:10.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.002+0000 7f04c2770640 1 --2- 172.21.15.64:0/1335173889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04bc102ef0 0x7f04bc10da40 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:10.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.002+0000 7f04c2770640 1 --2- 172.21.15.64:0/1335173889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04bc102570 0x7f04bc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:10.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.002+0000 7f04c2770640 1 --2- 172.21.15.64:0/1335173889 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04bc100e20 0x7f04bc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:10.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.002+0000 7f04c2770640 1 -- 172.21.15.64:0/1335173889 >> 172.21.15.64:0/1335173889 conn(0x7f04bc0fc7d0 msgr2=0x7f04bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:10.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.002+0000 7f04c2770640 1 -- 172.21.15.64:0/1335173889 shutdown_connections 2024-09-12T03:28:10.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.002+0000 7f04c2770640 1 -- 172.21.15.64:0/1335173889 wait complete. 2024-09-12T03:28:10.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.003+0000 7f04c2770640 1 Processor -- start 2024-09-12T03:28:10.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.003+0000 7f04c2770640 1 -- start start 2024-09-12T03:28:10.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.003+0000 7f04c2770640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04bc100e20 0x7f04bc198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:10.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.003+0000 7f04c2770640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04bc102570 0x7f04bc1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:10.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.003+0000 7f04c2770640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04bc102ef0 0x7f04bc19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:10.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.003+0000 7f04c2770640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc110e80 con 0x7f04bc100e20 2024-09-12T03:28:10.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.003+0000 7f04c2770640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc110d00 con 0x7f04bc102570 2024-09-12T03:28:10.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.003+0000 7f04c2770640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc111000 con 0x7f04bc102ef0 2024-09-12T03:28:10.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.003+0000 7f04bbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04bc100e20 0x7f04bc198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:10.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.003+0000 7f04bbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04bc100e20 0x7f04bc198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41914/0 (socket says 172.21.15.64:41914) 2024-09-12T03:28:10.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.003+0000 7f04bbfff640 1 -- 172.21.15.64:0/633514723 learned_addr learned my addr 172.21.15.64:0/633514723 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:10.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.004+0000 7f04c0d0d640 1 --2- 172.21.15.64:0/633514723 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04bc102ef0 0x7f04bc19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:10.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.004+0000 7f04bbfff640 1 -- 172.21.15.64:0/633514723 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04bc102ef0 msgr2=0x7f04bc19cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:10.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.004+0000 7f04bbfff640 1 --2- 172.21.15.64:0/633514723 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04bc102ef0 0x7f04bc19cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:10.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.004+0000 7f04bbfff640 1 -- 172.21.15.64:0/633514723 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04bc102570 msgr2=0x7f04bc1989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:10.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.004+0000 7f04bbfff640 1 --2- 172.21.15.64:0/633514723 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04bc102570 0x7f04bc1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:10.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.004+0000 7f04bbfff640 1 -- 172.21.15.64:0/633514723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04bc19d430 con 0x7f04bc100e20 2024-09-12T03:28:10.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.004+0000 7f04c0d0d640 1 --2- 172.21.15.64:0/633514723 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04bc102ef0 0x7f04bc19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:28:10.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.004+0000 7f04bbfff640 1 --2- 172.21.15.64:0/633514723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04bc100e20 0x7f04bc198480 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f04ac00d950 tx=0x7f04ac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:10.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.004+0000 7f04b9ffb640 1 -- 172.21.15.64:0/633514723 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac017070 con 0x7f04bc100e20 2024-09-12T03:28:10.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.004+0000 7f04c2770640 1 -- 172.21.15.64:0/633514723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04bc19d720 con 0x7f04bc100e20 2024-09-12T03:28:10.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.004+0000 7f04b9ffb640 1 -- 172.21.15.64:0/633514723 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04ac009c30 con 0x7f04bc100e20 2024-09-12T03:28:10.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.004+0000 7f04b9ffb640 1 -- 172.21.15.64:0/633514723 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac00f620 con 0x7f04bc100e20 2024-09-12T03:28:10.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.004+0000 7f04c2770640 1 -- 172.21.15.64:0/633514723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04bc1a5000 con 0x7f04bc100e20 2024-09-12T03:28:10.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.005+0000 7f04c2770640 1 -- 172.21.15.64:0/633514723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f04bc062940 con 0x7f04bc100e20 2024-09-12T03:28:10.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.006+0000 7f04b9ffb640 1 -- 172.21.15.64:0/633514723 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f04ac0109a0 con 0x7f04bc100e20 2024-09-12T03:28:10.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.007+0000 7f04b9ffb640 1 --2- 172.21.15.64:0/633514723 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f04a80a2c10 0x7f04a80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:10.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.007+0000 7f04b3fff640 1 --2- 172.21.15.64:0/633514723 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f04a80a2c10 0x7f04a80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:10.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.007+0000 7f04b9ffb640 1 -- 172.21.15.64:0/633514723 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f04ac0cf5b0 con 0x7f04bc100e20 2024-09-12T03:28:10.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.007+0000 7f04b9ffb640 1 --2- 172.21.15.64:0/633514723 >> [v2:172.21.15.115:6824/2977887627,v1:172.21.15.115:6825/2977887627] conn(0x7f04a80adde0 0x7f04a80b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:10.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.007+0000 7f04b9ffb640 1 -- 172.21.15.64:0/633514723 --> [v2:172.21.15.115:6824/2977887627,v1:172.21.15.115:6825/2977887627] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f04a80b0950 con 0x7f04a80adde0 2024-09-12T03:28:10.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.007+0000 7f04b9ffb640 1 -- 172.21.15.64:0/633514723 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f04ac0cfa00 con 0x7f04bc100e20 2024-09-12T03:28:10.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.008+0000 7f04b3fff640 1 --2- 172.21.15.64:0/633514723 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f04a80a2c10 0x7f04a80a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f04a4005d80 tx=0x7f04a4005c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:10.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.008+0000 7f04c0d0d640 1 --2- 172.21.15.64:0/633514723 >> [v2:172.21.15.115:6824/2977887627,v1:172.21.15.115:6825/2977887627] conn(0x7f04a80adde0 0x7f04a80b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:10.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.008+0000 7f04c0d0d640 1 --2- 172.21.15.64:0/633514723 >> [v2:172.21.15.115:6824/2977887627,v1:172.21.15.115:6825/2977887627] conn(0x7f04a80adde0 0x7f04a80b0280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:10.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.009+0000 7f04b9ffb640 1 -- 172.21.15.64:0/633514723 <== osd.10 v2:172.21.15.115:6824/2977887627 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f04a80b0950 con 0x7f04a80adde0 2024-09-12T03:28:10.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.023+0000 7f04c2770640 1 -- 172.21.15.64:0/633514723 --> [v2:172.21.15.115:6824/2977887627,v1:172.21.15.115:6825/2977887627] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f04bc062d40 con 0x7f04a80adde0 2024-09-12T03:28:10.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.023+0000 7f04b9ffb640 1 -- 172.21.15.64:0/633514723 <== osd.10 v2:172.21.15.115:6824/2977887627 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f04bc062d40 con 0x7f04a80adde0 2024-09-12T03:28:10.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.024+0000 7f04c2770640 1 -- 172.21.15.64:0/633514723 >> [v2:172.21.15.115:6824/2977887627,v1:172.21.15.115:6825/2977887627] conn(0x7f04a80adde0 msgr2=0x7f04a80b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:10.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.024+0000 7f04c2770640 1 --2- 172.21.15.64:0/633514723 >> [v2:172.21.15.115:6824/2977887627,v1:172.21.15.115:6825/2977887627] conn(0x7f04a80adde0 0x7f04a80b0280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:10.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.024+0000 7f04c2770640 1 -- 172.21.15.64:0/633514723 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f04a80a2c10 msgr2=0x7f04a80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:10.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.024+0000 7f04c2770640 1 --2- 172.21.15.64:0/633514723 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f04a80a2c10 0x7f04a80a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f04a4005d80 tx=0x7f04a4005c90 comp rx=0 tx=0).stop 2024-09-12T03:28:10.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.024+0000 7f04c2770640 1 -- 172.21.15.64:0/633514723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04bc100e20 msgr2=0x7f04bc198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:10.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.024+0000 7f04c2770640 1 --2- 172.21.15.64:0/633514723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04bc100e20 0x7f04bc198480 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f04ac00d950 tx=0x7f04ac00de10 comp rx=0 tx=0).stop 2024-09-12T03:28:10.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.024+0000 7f04c0d0d640 1 -- 172.21.15.64:0/633514723 reap_dead start 2024-09-12T03:28:10.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.024+0000 7f04c2770640 1 -- 172.21.15.64:0/633514723 shutdown_connections 2024-09-12T03:28:10.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.024+0000 7f04c2770640 1 -- 172.21.15.64:0/633514723 >> 172.21.15.64:0/633514723 conn(0x7f04bc0fc7d0 msgr2=0x7f04bc06a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:10.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.024+0000 7f04c2770640 1 -- 172.21.15.64:0/633514723 shutdown_connections 2024-09-12T03:28:10.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:10.024+0000 7f04c2770640 1 -- 172.21.15.64:0/633514723 wait complete. 2024-09-12T03:28:10.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:09 smithi115 ceph-mon[34801]: osdmap e90: 12 total, 12 up, 12 in 2024-09-12T03:28:10.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:09 smithi195 ceph-mon[34786]: osdmap e90: 12 total, 12 up, 12 in 2024-09-12T03:28:10.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:09 smithi064 ceph-mon[30952]: osdmap e90: 12 total, 12 up, 12 in 2024-09-12T03:28:10.936 INFO:teuthology.orchestra.run.smithi064.stdout:103079215170 2024-09-12T03:28:10.936 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-12T03:28:11.133 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:10 smithi064 ceph-mon[30952]: pgmap v321: 129 pgs: 1 peering, 128 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:28:11.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:10 smithi115 ceph-mon[34801]: pgmap v321: 129 pgs: 1 peering, 128 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:28:11.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:10 smithi195 ceph-mon[34786]: pgmap v321: 129 pgs: 1 peering, 128 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:28:11.676 INFO:teuthology.orchestra.run.smithi064.stdout:34359738450 2024-09-12T03:28:11.677 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-12T03:28:12.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.052+0000 7f2951340640 1 -- 172.21.15.64:0/362173327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f294c106a80 msgr2=0x7f294c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:12.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.052+0000 7f2951340640 1 --2- 172.21.15.64:0/362173327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f294c106a80 0x7f294c106e60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f293c009930 tx=0x7f293c02a420 comp rx=0 tx=0).stop 2024-09-12T03:28:12.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.052+0000 7f2951340640 1 -- 172.21.15.64:0/362173327 shutdown_connections 2024-09-12T03:28:12.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.052+0000 7f2951340640 1 --2- 172.21.15.64:0/362173327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f294c101420 0x7f294c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.052+0000 7f2951340640 1 --2- 172.21.15.64:0/362173327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f294c100a80 0x7f294c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.052+0000 7f2951340640 1 --2- 172.21.15.64:0/362173327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f294c106a80 0x7f294c106e60 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.052+0000 7f2951340640 1 -- 172.21.15.64:0/362173327 >> 172.21.15.64:0/362173327 conn(0x7f294c0fc7a0 msgr2=0x7f294c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:12.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.052+0000 7f2951340640 1 -- 172.21.15.64:0/362173327 shutdown_connections 2024-09-12T03:28:12.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.052+0000 7f2951340640 1 -- 172.21.15.64:0/362173327 wait complete. 2024-09-12T03:28:12.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.053+0000 7f2951340640 1 Processor -- start 2024-09-12T03:28:12.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.053+0000 7f2951340640 1 -- start start 2024-09-12T03:28:12.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.053+0000 7f2951340640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f294c100a80 0x7f294c198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:12.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.053+0000 7f2951340640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f294c101420 0x7f294c198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:12.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.053+0000 7f2951340640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f294c106a80 0x7f294c19cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:12.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.053+0000 7f2951340640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f294c110cf0 con 0x7f294c106a80 2024-09-12T03:28:12.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.053+0000 7f2951340640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f294c110b70 con 0x7f294c100a80 2024-09-12T03:28:12.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.053+0000 7f2951340640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f294c110e70 con 0x7f294c101420 2024-09-12T03:28:12.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.053+0000 7f294b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f294c106a80 0x7f294c19cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:12.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.053+0000 7f294b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f294c106a80 0x7f294c19cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41922/0 (socket says 172.21.15.64:41922) 2024-09-12T03:28:12.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.053+0000 7f294b7fe640 1 -- 172.21.15.64:0/3613520936 learned_addr learned my addr 172.21.15.64:0/3613520936 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:12.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.053+0000 7f294affd640 1 --2- 172.21.15.64:0/3613520936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f294c100a80 0x7f294c198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:12.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.053+0000 7f2943fff640 1 --2- 172.21.15.64:0/3613520936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f294c101420 0x7f294c198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:12.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.053+0000 7f294b7fe640 1 -- 172.21.15.64:0/3613520936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f294c101420 msgr2=0x7f294c198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:12.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.053+0000 7f294b7fe640 1 --2- 172.21.15.64:0/3613520936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f294c101420 0x7f294c198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.053+0000 7f294b7fe640 1 -- 172.21.15.64:0/3613520936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f294c100a80 msgr2=0x7f294c198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:12.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.053+0000 7f294b7fe640 1 --2- 172.21.15.64:0/3613520936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f294c100a80 0x7f294c198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.054+0000 7f294b7fe640 1 -- 172.21.15.64:0/3613520936 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f294c19d2c0 con 0x7f294c106a80 2024-09-12T03:28:12.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.054+0000 7f2943fff640 1 --2- 172.21.15.64:0/3613520936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f294c101420 0x7f294c198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:28:12.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.054+0000 7f294b7fe640 1 --2- 172.21.15.64:0/3613520936 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f294c106a80 0x7f294c19cbe0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f294400eaa0 tx=0x7f294400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:12.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.054+0000 7f2948ff9640 1 -- 172.21.15.64:0/3613520936 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294400cdb0 con 0x7f294c106a80 2024-09-12T03:28:12.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.054+0000 7f2948ff9640 1 -- 172.21.15.64:0/3613520936 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2944005020 con 0x7f294c106a80 2024-09-12T03:28:12.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.054+0000 7f2951340640 1 -- 172.21.15.64:0/3613520936 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f294c19d5b0 con 0x7f294c106a80 2024-09-12T03:28:12.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.054+0000 7f2948ff9640 1 -- 172.21.15.64:0/3613520936 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29440116d0 con 0x7f294c106a80 2024-09-12T03:28:12.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.054+0000 7f2951340640 1 -- 172.21.15.64:0/3613520936 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f294c1a4e90 con 0x7f294c106a80 2024-09-12T03:28:12.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.055+0000 7f2951340640 1 -- 172.21.15.64:0/3613520936 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f294c062940 con 0x7f294c106a80 2024-09-12T03:28:12.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.056+0000 7f2948ff9640 1 -- 172.21.15.64:0/3613520936 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f29440051d0 con 0x7f294c106a80 2024-09-12T03:28:12.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.057+0000 7f2948ff9640 1 --2- 172.21.15.64:0/3613520936 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f29380a2c90 0x7f29380a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:12.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.057+0000 7f294affd640 1 --2- 172.21.15.64:0/3613520936 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f29380a2c90 0x7f29380a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:12.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.057+0000 7f2948ff9640 1 -- 172.21.15.64:0/3613520936 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f29440cfa00 con 0x7f294c106a80 2024-09-12T03:28:12.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.057+0000 7f2948ff9640 1 --2- 172.21.15.64:0/3613520936 >> [v2:172.21.15.64:6826/561728112,v1:172.21.15.64:6827/561728112] conn(0x7f29380ade60 0x7f29380b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:12.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.057+0000 7f2948ff9640 1 -- 172.21.15.64:0/3613520936 --> [v2:172.21.15.64:6826/561728112,v1:172.21.15.64:6827/561728112] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2944014ec0 con 0x7f29380ade60 2024-09-12T03:28:12.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.057+0000 7f2948ff9640 1 -- 172.21.15.64:0/3613520936 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f29440cfe50 con 0x7f294c106a80 2024-09-12T03:28:12.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.057+0000 7f2943fff640 1 --2- 172.21.15.64:0/3613520936 >> [v2:172.21.15.64:6826/561728112,v1:172.21.15.64:6827/561728112] conn(0x7f29380ade60 0x7f29380b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:12.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.058+0000 7f294affd640 1 --2- 172.21.15.64:0/3613520936 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f29380a2c90 0x7f29380a5150 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f293c009900 tx=0x7f293c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:12.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.058+0000 7f2943fff640 1 --2- 172.21.15.64:0/3613520936 >> [v2:172.21.15.64:6826/561728112,v1:172.21.15.64:6827/561728112] conn(0x7f29380ade60 0x7f29380b0300 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:12.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.058+0000 7f2948ff9640 1 -- 172.21.15.64:0/3613520936 <== osd.9 v2:172.21.15.64:6826/561728112 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2944014ec0 con 0x7f29380ade60 2024-09-12T03:28:12.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.072+0000 7f2951340640 1 -- 172.21.15.64:0/3613520936 --> [v2:172.21.15.64:6826/561728112,v1:172.21.15.64:6827/561728112] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f294c062dc0 con 0x7f29380ade60 2024-09-12T03:28:12.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.072+0000 7f2948ff9640 1 -- 172.21.15.64:0/3613520936 <== osd.9 v2:172.21.15.64:6826/561728112 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f294c062dc0 con 0x7f29380ade60 2024-09-12T03:28:12.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.073+0000 7f2951340640 1 -- 172.21.15.64:0/3613520936 >> [v2:172.21.15.64:6826/561728112,v1:172.21.15.64:6827/561728112] conn(0x7f29380ade60 msgr2=0x7f29380b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:12.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.073+0000 7f2951340640 1 --2- 172.21.15.64:0/3613520936 >> [v2:172.21.15.64:6826/561728112,v1:172.21.15.64:6827/561728112] conn(0x7f29380ade60 0x7f29380b0300 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.073+0000 7f2951340640 1 -- 172.21.15.64:0/3613520936 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f29380a2c90 msgr2=0x7f29380a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:12.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.073+0000 7f2951340640 1 --2- 172.21.15.64:0/3613520936 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f29380a2c90 0x7f29380a5150 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f293c009900 tx=0x7f293c000bc0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.073+0000 7f2951340640 1 -- 172.21.15.64:0/3613520936 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f294c106a80 msgr2=0x7f294c19cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:12.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.073+0000 7f2951340640 1 --2- 172.21.15.64:0/3613520936 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f294c106a80 0x7f294c19cbe0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f294400eaa0 tx=0x7f294400ef60 comp rx=0 tx=0).stop 2024-09-12T03:28:12.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.073+0000 7f294b7fe640 1 -- 172.21.15.64:0/3613520936 reap_dead start 2024-09-12T03:28:12.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.073+0000 7f2951340640 1 -- 172.21.15.64:0/3613520936 shutdown_connections 2024-09-12T03:28:12.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.073+0000 7f2951340640 1 -- 172.21.15.64:0/3613520936 >> 172.21.15.64:0/3613520936 conn(0x7f294c0fc7a0 msgr2=0x7f294c06aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:12.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.073+0000 7f2951340640 1 -- 172.21.15.64:0/3613520936 shutdown_connections 2024-09-12T03:28:12.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.073+0000 7f2951340640 1 -- 172.21.15.64:0/3613520936 wait complete. 2024-09-12T03:28:12.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.488+0000 7fcc3c81a640 1 -- 172.21.15.64:0/456708311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc38102d40 msgr2=0x7fcc38103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:12.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.488+0000 7fcc3c81a640 1 --2- 172.21.15.64:0/456708311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc38102d40 0x7fcc38103120 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcc20009900 tx=0x7fcc2002a480 comp rx=0 tx=0).stop 2024-09-12T03:28:12.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.489+0000 7fcc3c81a640 1 -- 172.21.15.64:0/456708311 shutdown_connections 2024-09-12T03:28:12.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.489+0000 7fcc3c81a640 1 --2- 172.21.15.64:0/456708311 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc3810c170 0x7fcc3810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.489+0000 7fcc3c81a640 1 --2- 172.21.15.64:0/456708311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc38103660 0x7fcc3810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.489+0000 7fcc3c81a640 1 --2- 172.21.15.64:0/456708311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc38102d40 0x7fcc38103120 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.489+0000 7fcc3c81a640 1 -- 172.21.15.64:0/456708311 >> 172.21.15.64:0/456708311 conn(0x7fcc380fc7a0 msgr2=0x7fcc380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:12.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.489+0000 7fcc3c81a640 1 -- 172.21.15.64:0/456708311 shutdown_connections 2024-09-12T03:28:12.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.489+0000 7fcc3c81a640 1 -- 172.21.15.64:0/456708311 wait complete. 2024-09-12T03:28:12.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.489+0000 7fcc3c81a640 1 Processor -- start 2024-09-12T03:28:12.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.489+0000 7fcc3c81a640 1 -- start start 2024-09-12T03:28:12.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.490+0000 7fcc3c81a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc38102d40 0x7fcc381009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:12.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.490+0000 7fcc3c81a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc38103660 0x7fcc38100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:12.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.490+0000 7fcc3c81a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc3810c170 0x7fcc38101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:12.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.490+0000 7fcc3c81a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcc38111e20 con 0x7fcc38103660 2024-09-12T03:28:12.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.490+0000 7fcc3c81a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcc38111ca0 con 0x7fcc3810c170 2024-09-12T03:28:12.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.490+0000 7fcc3c81a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcc38111fa0 con 0x7fcc38102d40 2024-09-12T03:28:12.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.490+0000 7fcc35d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc38103660 0x7fcc38100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:12.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.490+0000 7fcc35d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc38103660 0x7fcc38100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41938/0 (socket says 172.21.15.64:41938) 2024-09-12T03:28:12.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.490+0000 7fcc35d9b640 1 -- 172.21.15.64:0/1940959990 learned_addr learned my addr 172.21.15.64:0/1940959990 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:12.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.490+0000 7fcc3659c640 1 --2- 172.21.15.64:0/1940959990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc38102d40 0x7fcc381009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:12.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.490+0000 7fcc36d9d640 1 --2- 172.21.15.64:0/1940959990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc3810c170 0x7fcc38101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:12.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.490+0000 7fcc35d9b640 1 -- 172.21.15.64:0/1940959990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc38102d40 msgr2=0x7fcc381009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:12.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.490+0000 7fcc35d9b640 1 --2- 172.21.15.64:0/1940959990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc38102d40 0x7fcc381009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.490+0000 7fcc35d9b640 1 -- 172.21.15.64:0/1940959990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc3810c170 msgr2=0x7fcc38101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:12.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.490+0000 7fcc35d9b640 1 --2- 172.21.15.64:0/1940959990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc3810c170 0x7fcc38101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.490+0000 7fcc35d9b640 1 -- 172.21.15.64:0/1940959990 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc381a6830 con 0x7fcc38103660 2024-09-12T03:28:12.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.490+0000 7fcc35d9b640 1 --2- 172.21.15.64:0/1940959990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc38103660 0x7fcc38100ee0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcc2800e9f0 tx=0x7fcc2800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:12.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.490+0000 7fcc2f7fe640 1 -- 172.21.15.64:0/1940959990 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2800ccc0 con 0x7fcc38103660 2024-09-12T03:28:12.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.490+0000 7fcc3c81a640 1 -- 172.21.15.64:0/1940959990 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc381a6b20 con 0x7fcc38103660 2024-09-12T03:28:12.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.490+0000 7fcc2f7fe640 1 -- 172.21.15.64:0/1940959990 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc280108e0 con 0x7fcc38103660 2024-09-12T03:28:12.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.491+0000 7fcc3c81a640 1 -- 172.21.15.64:0/1940959990 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc381a7060 con 0x7fcc38103660 2024-09-12T03:28:12.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.491+0000 7fcc2f7fe640 1 -- 172.21.15.64:0/1940959990 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc28018760 con 0x7fcc38103660 2024-09-12T03:28:12.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.491+0000 7fcc3c81a640 1 -- 172.21.15.64:0/1940959990 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcc38062900 con 0x7fcc38103660 2024-09-12T03:28:12.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.493+0000 7fcc2f7fe640 1 -- 172.21.15.64:0/1940959990 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fcc28010430 con 0x7fcc38103660 2024-09-12T03:28:12.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.494+0000 7fcc2f7fe640 1 --2- 172.21.15.64:0/1940959990 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fcc240a2c10 0x7fcc240a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:12.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.494+0000 7fcc3659c640 1 --2- 172.21.15.64:0/1940959990 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fcc240a2c10 0x7fcc240a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:12.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.494+0000 7fcc2f7fe640 1 -- 172.21.15.64:0/1940959990 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fcc280cfdd0 con 0x7fcc38103660 2024-09-12T03:28:12.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.494+0000 7fcc2f7fe640 1 --2- 172.21.15.64:0/1940959990 >> [v2:172.21.15.115:6800/3123728622,v1:172.21.15.115:6801/3123728622] conn(0x7fcc240adde0 0x7fcc240b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:12.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.494+0000 7fcc2f7fe640 1 -- 172.21.15.64:0/1940959990 --> [v2:172.21.15.115:6800/3123728622,v1:172.21.15.115:6801/3123728622] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcc240b0950 con 0x7fcc240adde0 2024-09-12T03:28:12.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.494+0000 7fcc2f7fe640 1 -- 172.21.15.64:0/1940959990 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fcc280d0220 con 0x7fcc38103660 2024-09-12T03:28:12.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.494+0000 7fcc3659c640 1 --2- 172.21.15.64:0/1940959990 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fcc240a2c10 0x7fcc240a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcc20031440 tx=0x7fcc200095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:12.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.494+0000 7fcc36d9d640 1 --2- 172.21.15.64:0/1940959990 >> [v2:172.21.15.115:6800/3123728622,v1:172.21.15.115:6801/3123728622] conn(0x7fcc240adde0 0x7fcc240b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:12.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.495+0000 7fcc36d9d640 1 --2- 172.21.15.64:0/1940959990 >> [v2:172.21.15.115:6800/3123728622,v1:172.21.15.115:6801/3123728622] conn(0x7fcc240adde0 0x7fcc240b0280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:12.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.496+0000 7fcc2f7fe640 1 -- 172.21.15.64:0/1940959990 <== osd.1 v2:172.21.15.115:6800/3123728622 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcc240b0950 con 0x7fcc240adde0 2024-09-12T03:28:12.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.509+0000 7fcc3c81a640 1 -- 172.21.15.64:0/1940959990 --> [v2:172.21.15.115:6800/3123728622,v1:172.21.15.115:6801/3123728622] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcc38062d00 con 0x7fcc240adde0 2024-09-12T03:28:12.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.509+0000 7fcc2f7fe640 1 -- 172.21.15.64:0/1940959990 <== osd.1 v2:172.21.15.115:6800/3123728622 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fcc38062d00 con 0x7fcc240adde0 2024-09-12T03:28:12.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.510+0000 7fcc3c81a640 1 -- 172.21.15.64:0/1940959990 >> [v2:172.21.15.115:6800/3123728622,v1:172.21.15.115:6801/3123728622] conn(0x7fcc240adde0 msgr2=0x7fcc240b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:12.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.510+0000 7fcc3c81a640 1 --2- 172.21.15.64:0/1940959990 >> [v2:172.21.15.115:6800/3123728622,v1:172.21.15.115:6801/3123728622] conn(0x7fcc240adde0 0x7fcc240b0280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.510+0000 7fcc3c81a640 1 -- 172.21.15.64:0/1940959990 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fcc240a2c10 msgr2=0x7fcc240a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:12.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.510+0000 7fcc3c81a640 1 --2- 172.21.15.64:0/1940959990 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fcc240a2c10 0x7fcc240a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcc20031440 tx=0x7fcc200095f0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.510+0000 7fcc3c81a640 1 -- 172.21.15.64:0/1940959990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc38103660 msgr2=0x7fcc38100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:12.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.510+0000 7fcc3c81a640 1 --2- 172.21.15.64:0/1940959990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc38103660 0x7fcc38100ee0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcc2800e9f0 tx=0x7fcc2800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.510+0000 7fcc36d9d640 1 -- 172.21.15.64:0/1940959990 reap_dead start 2024-09-12T03:28:12.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.510+0000 7fcc3c81a640 1 -- 172.21.15.64:0/1940959990 shutdown_connections 2024-09-12T03:28:12.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.510+0000 7fcc3c81a640 1 -- 172.21.15.64:0/1940959990 >> 172.21.15.64:0/1940959990 conn(0x7fcc380fc7a0 msgr2=0x7fcc380fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:12.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.510+0000 7fcc3c81a640 1 -- 172.21.15.64:0/1940959990 shutdown_connections 2024-09-12T03:28:12.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:12.510+0000 7fcc3c81a640 1 -- 172.21.15.64:0/1940959990 wait complete. 2024-09-12T03:28:13.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:12 smithi115 ceph-mon[34801]: pgmap v322: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-09-12T03:28:13.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:12 smithi195 ceph-mon[34786]: pgmap v322: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-09-12T03:28:13.221 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:12 smithi064 ceph-mon[30952]: pgmap v322: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-09-12T03:28:13.255 INFO:teuthology.orchestra.run.smithi064.stdout:137438953534 2024-09-12T03:28:13.255 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-12T03:28:13.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.439+0000 7f67890fb640 1 -- 172.21.15.64:0/354676936 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6784106f10 msgr2=0x7f678410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:13.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.439+0000 7f67890fb640 1 --2- 172.21.15.64:0/354676936 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6784106f10 0x7f678410ea90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f677c009ca0 tx=0x7f677c02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:28:13.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.440+0000 7f67890fb640 1 -- 172.21.15.64:0/354676936 shutdown_connections 2024-09-12T03:28:13.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.440+0000 7f67890fb640 1 --2- 172.21.15.64:0/354676936 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6784106f10 0x7f678410ea90 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:13.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.440+0000 7f67890fb640 1 --2- 172.21.15.64:0/354676936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6784102790 0x7f67841067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:13.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.440+0000 7f67890fb640 1 --2- 172.21.15.64:0/354676936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6784101de0 0x7f67841021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:13.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.440+0000 7f67890fb640 1 -- 172.21.15.64:0/354676936 >> 172.21.15.64:0/354676936 conn(0x7f67840fd8c0 msgr2=0x7f67840ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:13.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.440+0000 7f67890fb640 1 -- 172.21.15.64:0/354676936 shutdown_connections 2024-09-12T03:28:13.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.440+0000 7f67890fb640 1 -- 172.21.15.64:0/354676936 wait complete. 2024-09-12T03:28:13.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.441+0000 7f67890fb640 1 Processor -- start 2024-09-12T03:28:13.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.441+0000 7f67890fb640 1 -- start start 2024-09-12T03:28:13.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.442+0000 7f67890fb640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6784101de0 0x7f6784198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:13.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.442+0000 7f67890fb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6784102790 0x7f6784198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:13.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.442+0000 7f67890fb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6784106f10 0x7f678419cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:13.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.442+0000 7f67890fb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6784111e20 con 0x7f6784102790 2024-09-12T03:28:13.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.442+0000 7f67890fb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6784111ca0 con 0x7f6784106f10 2024-09-12T03:28:13.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.442+0000 7f67890fb640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6784111fa0 con 0x7f6784101de0 2024-09-12T03:28:13.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.442+0000 7f678259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6784102790 0x7f6784198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:13.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.442+0000 7f678259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6784102790 0x7f6784198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41958/0 (socket says 172.21.15.64:41958) 2024-09-12T03:28:13.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.442+0000 7f678259c640 1 -- 172.21.15.64:0/1505341543 learned_addr learned my addr 172.21.15.64:0/1505341543 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:13.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.442+0000 7f678359e640 1 --2- 172.21.15.64:0/1505341543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6784106f10 0x7f678419cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:13.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.442+0000 7f6782d9d640 1 --2- 172.21.15.64:0/1505341543 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6784101de0 0x7f6784198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:13.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.442+0000 7f678259c640 1 -- 172.21.15.64:0/1505341543 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6784101de0 msgr2=0x7f6784198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:13.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.442+0000 7f678259c640 1 --2- 172.21.15.64:0/1505341543 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6784101de0 0x7f6784198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:13.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.442+0000 7f678259c640 1 -- 172.21.15.64:0/1505341543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6784106f10 msgr2=0x7f678419cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:13.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.442+0000 7f678259c640 1 --2- 172.21.15.64:0/1505341543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6784106f10 0x7f678419cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:13.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.442+0000 7f678259c640 1 -- 172.21.15.64:0/1505341543 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f678419d2d0 con 0x7f6784102790 2024-09-12T03:28:13.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.442+0000 7f6782d9d640 1 --2- 172.21.15.64:0/1505341543 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6784101de0 0x7f6784198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:28:13.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.442+0000 7f678259c640 1 --2- 172.21.15.64:0/1505341543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6784102790 0x7f6784198860 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f677800e9f0 tx=0x7f677800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:13.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.443+0000 7f676bfff640 1 -- 172.21.15.64:0/1505341543 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677800cd80 con 0x7f6784102790 2024-09-12T03:28:13.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.443+0000 7f67890fb640 1 -- 172.21.15.64:0/1505341543 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f678419d5c0 con 0x7f6784102790 2024-09-12T03:28:13.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.443+0000 7f676bfff640 1 -- 172.21.15.64:0/1505341543 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67780108e0 con 0x7f6784102790 2024-09-12T03:28:13.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.443+0000 7f676bfff640 1 -- 172.21.15.64:0/1505341543 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6778018920 con 0x7f6784102790 2024-09-12T03:28:13.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.443+0000 7f67890fb640 1 -- 172.21.15.64:0/1505341543 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6784074420 con 0x7f6784102790 2024-09-12T03:28:13.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.444+0000 7f67890fb640 1 -- 172.21.15.64:0/1505341543 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6784062980 con 0x7f6784102790 2024-09-12T03:28:13.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.446+0000 7f676bfff640 1 -- 172.21.15.64:0/1505341543 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6778010a90 con 0x7f6784102790 2024-09-12T03:28:13.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.446+0000 7f676bfff640 1 --2- 172.21.15.64:0/1505341543 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f676c0a7020 0x7f676c0a94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:13.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.446+0000 7f6782d9d640 1 --2- 172.21.15.64:0/1505341543 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f676c0a7020 0x7f676c0a94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:13.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.446+0000 7f676bfff640 1 -- 172.21.15.64:0/1505341543 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f67780d0050 con 0x7f6784102790 2024-09-12T03:28:13.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.446+0000 7f676bfff640 1 --2- 172.21.15.64:0/1505341543 >> [v2:172.21.15.115:6816/3075734174,v1:172.21.15.115:6817/3075734174] conn(0x7f676c0b21f0 0x7f676c0b4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:13.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.446+0000 7f676bfff640 1 -- 172.21.15.64:0/1505341543 --> [v2:172.21.15.115:6816/3075734174,v1:172.21.15.115:6817/3075734174] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f676c0b4d60 con 0x7f676c0b21f0 2024-09-12T03:28:13.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.446+0000 7f676bfff640 1 -- 172.21.15.64:0/1505341543 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f67780d04a0 con 0x7f6784102790 2024-09-12T03:28:13.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.447+0000 7f6782d9d640 1 --2- 172.21.15.64:0/1505341543 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f676c0a7020 0x7f676c0a94e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f67740058c0 tx=0x7f6774005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:13.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.447+0000 7f678359e640 1 --2- 172.21.15.64:0/1505341543 >> [v2:172.21.15.115:6816/3075734174,v1:172.21.15.115:6817/3075734174] conn(0x7f676c0b21f0 0x7f676c0b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:13.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.447+0000 7f678359e640 1 --2- 172.21.15.64:0/1505341543 >> [v2:172.21.15.115:6816/3075734174,v1:172.21.15.115:6817/3075734174] conn(0x7f676c0b21f0 0x7f676c0b4690 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:13.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.448+0000 7f676bfff640 1 -- 172.21.15.64:0/1505341543 <== osd.7 v2:172.21.15.115:6816/3075734174 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f676c0b4d60 con 0x7f676c0b21f0 2024-09-12T03:28:13.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.470+0000 7f67890fb640 1 -- 172.21.15.64:0/1505341543 --> [v2:172.21.15.115:6816/3075734174,v1:172.21.15.115:6817/3075734174] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6784062d80 con 0x7f676c0b21f0 2024-09-12T03:28:13.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.471+0000 7f676bfff640 1 -- 172.21.15.64:0/1505341543 <== osd.7 v2:172.21.15.115:6816/3075734174 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6784062d80 con 0x7f676c0b21f0 2024-09-12T03:28:13.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.471+0000 7f67890fb640 1 -- 172.21.15.64:0/1505341543 >> [v2:172.21.15.115:6816/3075734174,v1:172.21.15.115:6817/3075734174] conn(0x7f676c0b21f0 msgr2=0x7f676c0b4690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:13.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.471+0000 7f67890fb640 1 --2- 172.21.15.64:0/1505341543 >> [v2:172.21.15.115:6816/3075734174,v1:172.21.15.115:6817/3075734174] conn(0x7f676c0b21f0 0x7f676c0b4690 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:13.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.471+0000 7f67890fb640 1 -- 172.21.15.64:0/1505341543 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f676c0a7020 msgr2=0x7f676c0a94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:13.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.471+0000 7f67890fb640 1 --2- 172.21.15.64:0/1505341543 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f676c0a7020 0x7f676c0a94e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f67740058c0 tx=0x7f6774005850 comp rx=0 tx=0).stop 2024-09-12T03:28:13.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.471+0000 7f67890fb640 1 -- 172.21.15.64:0/1505341543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6784102790 msgr2=0x7f6784198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:13.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.471+0000 7f67890fb640 1 --2- 172.21.15.64:0/1505341543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6784102790 0x7f6784198860 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f677800e9f0 tx=0x7f677800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:28:13.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.471+0000 7f678359e640 1 -- 172.21.15.64:0/1505341543 reap_dead start 2024-09-12T03:28:13.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.471+0000 7f67890fb640 1 -- 172.21.15.64:0/1505341543 shutdown_connections 2024-09-12T03:28:13.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.471+0000 7f67890fb640 1 -- 172.21.15.64:0/1505341543 >> 172.21.15.64:0/1505341543 conn(0x7f67840fd8c0 msgr2=0x7f67840fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:13.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.472+0000 7f67890fb640 1 -- 172.21.15.64:0/1505341543 shutdown_connections 2024-09-12T03:28:13.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:13.472+0000 7f67890fb640 1 -- 172.21.15.64:0/1505341543 wait complete. 2024-09-12T03:28:13.915 INFO:teuthology.orchestra.run.smithi064.stdout:270582939686 2024-09-12T03:28:13.915 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-12T03:28:14.589 INFO:teuthology.orchestra.run.smithi064.stdout:292057776160 2024-09-12T03:28:14.589 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-12T03:28:15.097 INFO:teuthology.orchestra.run.smithi064.stdout:55834574926 2024-09-12T03:28:15.098 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-12T03:28:15.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:14 smithi064 ceph-mon[30952]: pgmap v323: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:28:15.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:14 smithi115 ceph-mon[34801]: pgmap v323: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:28:15.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:14 smithi195 ceph-mon[34786]: pgmap v323: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:28:15.549 INFO:teuthology.orchestra.run.smithi064.stdout:206158430256 2024-09-12T03:28:15.549 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-12T03:28:15.576 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:16.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.121+0000 7f893759d640 1 -- 172.21.15.64:0/1447856896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8930100e20 msgr2=0x7f8930101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:16.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.121+0000 7f893759d640 1 --2- 172.21.15.64:0/1447856896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8930100e20 0x7f8930101200 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f892c009900 tx=0x7f892c02a480 comp rx=0 tx=0).stop 2024-09-12T03:28:16.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.121+0000 7f893759d640 1 -- 172.21.15.64:0/1447856896 shutdown_connections 2024-09-12T03:28:16.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.121+0000 7f893759d640 1 --2- 172.21.15.64:0/1447856896 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8930102ef0 0x7f893010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:16.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.121+0000 7f893759d640 1 --2- 172.21.15.64:0/1447856896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8930102570 0x7f89301029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:16.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.121+0000 7f893759d640 1 --2- 172.21.15.64:0/1447856896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8930100e20 0x7f8930101200 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:16.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.121+0000 7f893759d640 1 -- 172.21.15.64:0/1447856896 >> 172.21.15.64:0/1447856896 conn(0x7f89300fc7d0 msgr2=0x7f89300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:16.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.121+0000 7f893759d640 1 -- 172.21.15.64:0/1447856896 shutdown_connections 2024-09-12T03:28:16.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.121+0000 7f893759d640 1 -- 172.21.15.64:0/1447856896 wait complete. 2024-09-12T03:28:16.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.122+0000 7f893759d640 1 Processor -- start 2024-09-12T03:28:16.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.122+0000 7f893759d640 1 -- start start 2024-09-12T03:28:16.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.122+0000 7f893759d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8930100e20 0x7f89301983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:16.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.122+0000 7f893759d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8930102570 0x7f8930198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:16.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.122+0000 7f893759d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8930102ef0 0x7f893019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:16.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.122+0000 7f893759d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8930110e50 con 0x7f8930102ef0 2024-09-12T03:28:16.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.122+0000 7f893759d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8930110cd0 con 0x7f8930102570 2024-09-12T03:28:16.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.122+0000 7f893759d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8930110fd0 con 0x7f8930100e20 2024-09-12T03:28:16.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.123+0000 7f8935b3a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8930102ef0 0x7f893019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:16.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.123+0000 7f8935b3a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8930102ef0 0x7f893019cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41984/0 (socket says 172.21.15.64:41984) 2024-09-12T03:28:16.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.123+0000 7f8935b3a640 1 -- 172.21.15.64:0/1800294597 learned_addr learned my addr 172.21.15.64:0/1800294597 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:16.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.123+0000 7f8935339640 1 --2- 172.21.15.64:0/1800294597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8930100e20 0x7f89301983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:16.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.123+0000 7f8934b38640 1 --2- 172.21.15.64:0/1800294597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8930102570 0x7f8930198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:16.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.123+0000 7f8935b3a640 1 -- 172.21.15.64:0/1800294597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8930100e20 msgr2=0x7f89301983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:16.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.123+0000 7f8935b3a640 1 --2- 172.21.15.64:0/1800294597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8930100e20 0x7f89301983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:16.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.123+0000 7f8935b3a640 1 -- 172.21.15.64:0/1800294597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8930102570 msgr2=0x7f8930198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:16.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.123+0000 7f8935b3a640 1 --2- 172.21.15.64:0/1800294597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8930102570 0x7f8930198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:16.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.123+0000 7f8935b3a640 1 -- 172.21.15.64:0/1800294597 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f893019d380 con 0x7f8930102ef0 2024-09-12T03:28:16.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.123+0000 7f8935b3a640 1 --2- 172.21.15.64:0/1800294597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8930102ef0 0x7f893019cca0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f892800b970 tx=0x7f892800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:16.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.123+0000 7f89267fc640 1 -- 172.21.15.64:0/1800294597 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8928019070 con 0x7f8930102ef0 2024-09-12T03:28:16.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.123+0000 7f893759d640 1 -- 172.21.15.64:0/1800294597 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f893019d670 con 0x7f8930102ef0 2024-09-12T03:28:16.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.123+0000 7f89267fc640 1 -- 172.21.15.64:0/1800294597 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8928014440 con 0x7f8930102ef0 2024-09-12T03:28:16.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.123+0000 7f89267fc640 1 -- 172.21.15.64:0/1800294597 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89280075d0 con 0x7f8930102ef0 2024-09-12T03:28:16.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.123+0000 7f893759d640 1 -- 172.21.15.64:0/1800294597 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89301a4f50 con 0x7f8930102ef0 2024-09-12T03:28:16.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.124+0000 7f893759d640 1 -- 172.21.15.64:0/1800294597 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8930101200 con 0x7f8930102ef0 2024-09-12T03:28:16.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.126+0000 7f89267fc640 1 -- 172.21.15.64:0/1800294597 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f892801c070 con 0x7f8930102ef0 2024-09-12T03:28:16.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.126+0000 7f89267fc640 1 --2- 172.21.15.64:0/1800294597 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f891c0a2c10 0x7f891c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:16.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.126+0000 7f8935339640 1 --2- 172.21.15.64:0/1800294597 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f891c0a2c10 0x7f891c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:16.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.126+0000 7f89267fc640 1 -- 172.21.15.64:0/1800294597 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f89280ceaa0 con 0x7f8930102ef0 2024-09-12T03:28:16.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.127+0000 7f8935339640 1 --2- 172.21.15.64:0/1800294597 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f891c0a2c10 0x7f891c0a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f892c005ab0 tx=0x7f892c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:16.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.130+0000 7f89267fc640 1 -- 172.21.15.64:0/1800294597 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f892809b650 con 0x7f8930102ef0 2024-09-12T03:28:16.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.309+0000 7f893759d640 1 -- 172.21.15.64:0/1800294597 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f893010a0a0 con 0x7f8930102ef0 2024-09-12T03:28:16.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.310+0000 7f89267fc640 1 -- 172.21.15.64:0/1800294597 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f892809af50 con 0x7f8930102ef0 2024-09-12T03:28:16.311 INFO:teuthology.orchestra.run.smithi064.stdout:236223201323 2024-09-12T03:28:16.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.312+0000 7f893759d640 1 -- 172.21.15.64:0/1800294597 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f891c0a2c10 msgr2=0x7f891c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:16.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.312+0000 7f893759d640 1 --2- 172.21.15.64:0/1800294597 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f891c0a2c10 0x7f891c0a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f892c005ab0 tx=0x7f892c005a20 comp rx=0 tx=0).stop 2024-09-12T03:28:16.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.312+0000 7f893759d640 1 -- 172.21.15.64:0/1800294597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8930102ef0 msgr2=0x7f893019cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:16.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.312+0000 7f893759d640 1 --2- 172.21.15.64:0/1800294597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8930102ef0 0x7f893019cca0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f892800b970 tx=0x7f892800be30 comp rx=0 tx=0).stop 2024-09-12T03:28:16.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.312+0000 7f893759d640 1 -- 172.21.15.64:0/1800294597 shutdown_connections 2024-09-12T03:28:16.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.312+0000 7f893759d640 1 --2- 172.21.15.64:0/1800294597 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f891c0a2c10 0x7f891c0a50d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:16.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.312+0000 7f893759d640 1 --2- 172.21.15.64:0/1800294597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8930102ef0 0x7f893019cca0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:16.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.312+0000 7f893759d640 1 --2- 172.21.15.64:0/1800294597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8930102570 0x7f8930198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:16.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.312+0000 7f893759d640 1 --2- 172.21.15.64:0/1800294597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8930100e20 0x7f89301983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:16.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.312+0000 7f893759d640 1 -- 172.21.15.64:0/1800294597 >> 172.21.15.64:0/1800294597 conn(0x7f89300fc7d0 msgr2=0x7f893006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:16.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.312+0000 7f893759d640 1 -- 172.21.15.64:0/1800294597 shutdown_connections 2024-09-12T03:28:16.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:16.312+0000 7f893759d640 1 -- 172.21.15.64:0/1800294597 wait complete. 2024-09-12T03:28:16.582 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:16.584 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:16.585 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:16.586 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:16.589 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:16.590 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:16.590 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:16.596 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:17.006 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201321 got 236223201323 for osd.8 2024-09-12T03:28:17.006 DEBUG:teuthology.parallel:result is None 2024-09-12T03:28:17.025 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:17.025 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:17.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:16 smithi115 ceph-mon[34801]: pgmap v324: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:28:17.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:16 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/1800294597' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T03:28:17.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:16 smithi064 ceph-mon[30952]: pgmap v324: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:28:17.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:16 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/1800294597' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T03:28:17.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:16 smithi195 ceph-mon[34786]: pgmap v324: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:28:17.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:16 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/1800294597' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T03:28:17.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:17.954+0000 7f501dc68640 1 -- 172.21.15.64:0/2998711567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5018106f30 msgr2=0x7f501810eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:17.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:17.954+0000 7f501dc68640 1 --2- 172.21.15.64:0/2998711567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5018106f30 0x7f501810eab0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5010009ca0 tx=0x7f501002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:28:17.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:17.954+0000 7f501dc68640 1 -- 172.21.15.64:0/2998711567 shutdown_connections 2024-09-12T03:28:17.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:17.954+0000 7f501dc68640 1 --2- 172.21.15.64:0/2998711567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5018106f30 0x7f501810eab0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:17.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:17.954+0000 7f501dc68640 1 --2- 172.21.15.64:0/2998711567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5018100cc0 0x7f5018106800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:17.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:17.954+0000 7f501dc68640 1 --2- 172.21.15.64:0/2998711567 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5018100310 0x7f50181006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:17.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:17.954+0000 7f501dc68640 1 -- 172.21.15.64:0/2998711567 >> 172.21.15.64:0/2998711567 conn(0x7f5018077210 msgr2=0x7f50180fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:17.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:17.954+0000 7f501dc68640 1 -- 172.21.15.64:0/2998711567 shutdown_connections 2024-09-12T03:28:17.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:17.954+0000 7f501dc68640 1 -- 172.21.15.64:0/2998711567 wait complete. 2024-09-12T03:28:17.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:17.954+0000 7f501dc68640 1 Processor -- start 2024-09-12T03:28:17.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:17.954+0000 7f501dc68640 1 -- start start 2024-09-12T03:28:17.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:17.955+0000 7f501dc68640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5018100310 0x7f5018101290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:17.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:17.955+0000 7f501dc68640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5018100cc0 0x7f50181017d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:17.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:17.955+0000 7f501dc68640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5018106f30 0x7f5018101d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:17.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:17.955+0000 7f501dc68640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5018111e40 con 0x7f5018100cc0 2024-09-12T03:28:17.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:17.955+0000 7f501dc68640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5018111cc0 con 0x7f5018100310 2024-09-12T03:28:17.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:17.955+0000 7f501dc68640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5018111fc0 con 0x7f5018106f30 2024-09-12T03:28:17.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:17.955+0000 7f5016ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5018100cc0 0x7f50181017d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:17.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:17.955+0000 7f5016ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5018100cc0 0x7f50181017d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40832/0 (socket says 172.21.15.64:40832) 2024-09-12T03:28:17.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:17.955+0000 7f5016ffd640 1 -- 172.21.15.64:0/3095274334 learned_addr learned my addr 172.21.15.64:0/3095274334 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:18.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:17.955+0000 7f5017fff640 1 --2- 172.21.15.64:0/3095274334 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5018106f30 0x7f5018101d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:18.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:17.955+0000 7f50177fe640 1 --2- 172.21.15.64:0/3095274334 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5018100310 0x7f5018101290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:18.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:17.955+0000 7f5016ffd640 1 -- 172.21.15.64:0/3095274334 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5018106f30 msgr2=0x7f5018101d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:18.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:17.955+0000 7f5016ffd640 1 --2- 172.21.15.64:0/3095274334 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5018106f30 0x7f5018101d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:18.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:17.955+0000 7f5016ffd640 1 -- 172.21.15.64:0/3095274334 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5018100310 msgr2=0x7f5018101290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:18.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:17.955+0000 7f5016ffd640 1 --2- 172.21.15.64:0/3095274334 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5018100310 0x7f5018101290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:18.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:17.955+0000 7f5016ffd640 1 -- 172.21.15.64:0/3095274334 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50181a6880 con 0x7f5018100cc0 2024-09-12T03:28:18.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:17.955+0000 7f50177fe640 1 --2- 172.21.15.64:0/3095274334 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5018100310 0x7f5018101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:28:18.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:17.955+0000 7f5016ffd640 1 --2- 172.21.15.64:0/3095274334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5018100cc0 0x7f50181017d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f500000d770 tx=0x7f500000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:18.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:17.956+0000 7f5014ff9640 1 -- 172.21.15.64:0/3095274334 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5000017070 con 0x7f5018100cc0 2024-09-12T03:28:18.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:17.956+0000 7f501dc68640 1 -- 172.21.15.64:0/3095274334 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50181a6b70 con 0x7f5018100cc0 2024-09-12T03:28:18.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:17.956+0000 7f5014ff9640 1 -- 172.21.15.64:0/3095274334 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50000054d0 con 0x7f5018100cc0 2024-09-12T03:28:18.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:17.956+0000 7f5014ff9640 1 -- 172.21.15.64:0/3095274334 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5000004cf0 con 0x7f5018100cc0 2024-09-12T03:28:18.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:17.956+0000 7f501dc68640 1 -- 172.21.15.64:0/3095274334 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50181a7060 con 0x7f5018100cc0 2024-09-12T03:28:18.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:17.957+0000 7f501dc68640 1 -- 172.21.15.64:0/3095274334 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50181006f0 con 0x7f5018100cc0 2024-09-12T03:28:18.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:17.958+0000 7f5014ff9640 1 -- 172.21.15.64:0/3095274334 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5000004e90 con 0x7f5018100cc0 2024-09-12T03:28:18.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:17.959+0000 7f5014ff9640 1 --2- 172.21.15.64:0/3095274334 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f50040a2c10 0x7f50040a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:18.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:17.959+0000 7f50177fe640 1 --2- 172.21.15.64:0/3095274334 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f50040a2c10 0x7f50040a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:18.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:17.959+0000 7f5014ff9640 1 -- 172.21.15.64:0/3095274334 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f50000cef00 con 0x7f5018100cc0 2024-09-12T03:28:18.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:17.959+0000 7f50177fe640 1 --2- 172.21.15.64:0/3095274334 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f50040a2c10 0x7f50040a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5008005fd0 tx=0x7f500800b5f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:18.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:17.963+0000 7f5014ff9640 1 -- 172.21.15.64:0/3095274334 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f500009b9a0 con 0x7f5018100cc0 2024-09-12T03:28:18.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.143+0000 7f501dc68640 1 -- 172.21.15.64:0/3095274334 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f501810a140 con 0x7f5018100cc0 2024-09-12T03:28:18.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.144+0000 7f5014ff9640 1 -- 172.21.15.64:0/3095274334 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f500009b2a0 con 0x7f5018100cc0 2024-09-12T03:28:18.145 INFO:teuthology.orchestra.run.smithi064.stdout:77309411403 2024-09-12T03:28:18.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.146+0000 7f501dc68640 1 -- 172.21.15.64:0/3095274334 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f50040a2c10 msgr2=0x7f50040a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:18.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.146+0000 7f501dc68640 1 --2- 172.21.15.64:0/3095274334 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f50040a2c10 0x7f50040a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5008005fd0 tx=0x7f500800b5f0 comp rx=0 tx=0).stop 2024-09-12T03:28:18.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.146+0000 7f501dc68640 1 -- 172.21.15.64:0/3095274334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5018100cc0 msgr2=0x7f50181017d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:18.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.146+0000 7f501dc68640 1 --2- 172.21.15.64:0/3095274334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5018100cc0 0x7f50181017d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f500000d770 tx=0x7f500000dc30 comp rx=0 tx=0).stop 2024-09-12T03:28:18.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.146+0000 7f501dc68640 1 -- 172.21.15.64:0/3095274334 shutdown_connections 2024-09-12T03:28:18.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.146+0000 7f501dc68640 1 --2- 172.21.15.64:0/3095274334 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f50040a2c10 0x7f50040a50d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:18.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.146+0000 7f501dc68640 1 --2- 172.21.15.64:0/3095274334 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5018106f30 0x7f5018101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:18.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.146+0000 7f501dc68640 1 --2- 172.21.15.64:0/3095274334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5018100cc0 0x7f50181017d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:18.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.146+0000 7f501dc68640 1 --2- 172.21.15.64:0/3095274334 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5018100310 0x7f5018101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:18.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.146+0000 7f501dc68640 1 -- 172.21.15.64:0/3095274334 >> 172.21.15.64:0/3095274334 conn(0x7f5018077210 msgr2=0x7f5018105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:18.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.146+0000 7f501dc68640 1 -- 172.21.15.64:0/3095274334 shutdown_connections 2024-09-12T03:28:18.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.146+0000 7f501dc68640 1 -- 172.21.15.64:0/3095274334 wait complete. 2024-09-12T03:28:19.016 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:18 smithi064 ceph-mon[30952]: pgmap v325: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-09-12T03:28:19.016 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:18 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/3095274334' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T03:28:19.016 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:18 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/3997400187' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T03:28:19.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:18 smithi115 ceph-mon[34801]: pgmap v325: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-09-12T03:28:19.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:18 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/3095274334' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T03:28:19.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:18 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/3997400187' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T03:28:19.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:18 smithi195 ceph-mon[34786]: pgmap v325: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-09-12T03:28:19.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:18 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/3095274334' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T03:28:19.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:18 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/3997400187' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T03:28:19.499 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411400 got 77309411403 for osd.2 2024-09-12T03:28:19.499 DEBUG:teuthology.parallel:result is None 2024-09-12T03:28:19.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.416+0000 7f1fda4c4640 1 -- 172.21.15.64:0/3225761435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fd4100e20 msgr2=0x7f1fd4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:19.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.416+0000 7f1fda4c4640 1 --2- 172.21.15.64:0/3225761435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fd4100e20 0x7f1fd4101200 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009900 tx=0x7f1fc402a480 comp rx=0 tx=0).stop 2024-09-12T03:28:19.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.416+0000 7f1fda4c4640 1 -- 172.21.15.64:0/3225761435 shutdown_connections 2024-09-12T03:28:19.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.416+0000 7f1fda4c4640 1 --2- 172.21.15.64:0/3225761435 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fd4102ef0 0x7f1fd410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:19.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.416+0000 7f1fda4c4640 1 --2- 172.21.15.64:0/3225761435 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fd4102570 0x7f1fd41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:19.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.416+0000 7f1fda4c4640 1 --2- 172.21.15.64:0/3225761435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fd4100e20 0x7f1fd4101200 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:19.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.416+0000 7f1fda4c4640 1 -- 172.21.15.64:0/3225761435 >> 172.21.15.64:0/3225761435 conn(0x7f1fd40fc7d0 msgr2=0x7f1fd40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:19.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.416+0000 7f1fda4c4640 1 -- 172.21.15.64:0/3225761435 shutdown_connections 2024-09-12T03:28:19.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.416+0000 7f1fda4c4640 1 -- 172.21.15.64:0/3225761435 wait complete. 2024-09-12T03:28:19.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.417+0000 7f1fda4c4640 1 Processor -- start 2024-09-12T03:28:19.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.417+0000 7f1fda4c4640 1 -- start start 2024-09-12T03:28:19.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.417+0000 7f1fda4c4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fd4100e20 0x7f1fd4198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:19.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.417+0000 7f1fda4c4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fd4102570 0x7f1fd41988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:19.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.417+0000 7f1fda4c4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fd4102ef0 0x7f1fd419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:19.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.417+0000 7f1fda4c4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd4110e30 con 0x7f1fd4102ef0 2024-09-12T03:28:19.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.417+0000 7f1fda4c4640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd4110cb0 con 0x7f1fd4100e20 2024-09-12T03:28:19.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.417+0000 7f1fda4c4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd4110fb0 con 0x7f1fd4102570 2024-09-12T03:28:19.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.417+0000 7f1fd8a61640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fd4102ef0 0x7f1fd419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:19.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.417+0000 7f1fd8a61640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fd4102ef0 0x7f1fd419cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40850/0 (socket says 172.21.15.64:40850) 2024-09-12T03:28:19.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.417+0000 7f1fd8a61640 1 -- 172.21.15.64:0/3997400187 learned_addr learned my addr 172.21.15.64:0/3997400187 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:19.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.417+0000 7f1fd3fff640 1 --2- 172.21.15.64:0/3997400187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fd4100e20 0x7f1fd4198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:19.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.417+0000 7f1fcbfff640 1 --2- 172.21.15.64:0/3997400187 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fd4102570 0x7f1fd41988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:19.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.417+0000 7f1fd8a61640 1 -- 172.21.15.64:0/3997400187 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fd4102570 msgr2=0x7f1fd41988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:19.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.417+0000 7f1fd8a61640 1 --2- 172.21.15.64:0/3997400187 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fd4102570 0x7f1fd41988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:19.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.417+0000 7f1fd8a61640 1 -- 172.21.15.64:0/3997400187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fd4100e20 msgr2=0x7f1fd4198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:19.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.417+0000 7f1fd8a61640 1 --2- 172.21.15.64:0/3997400187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fd4100e20 0x7f1fd4198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:19.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.417+0000 7f1fd8a61640 1 -- 172.21.15.64:0/3997400187 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fd419d330 con 0x7f1fd4102ef0 2024-09-12T03:28:19.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.418+0000 7f1fcbfff640 1 --2- 172.21.15.64:0/3997400187 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fd4102570 0x7f1fd41988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:28:19.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.418+0000 7f1fd8a61640 1 --2- 172.21.15.64:0/3997400187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fd4102ef0 0x7f1fd419cc50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc00eaa0 tx=0x7f1fcc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:19.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.418+0000 7f1fd1ffb640 1 -- 172.21.15.64:0/3997400187 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc00cdb0 con 0x7f1fd4102ef0 2024-09-12T03:28:19.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.418+0000 7f1fd1ffb640 1 -- 172.21.15.64:0/3997400187 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fcc005020 con 0x7f1fd4102ef0 2024-09-12T03:28:19.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.418+0000 7f1fda4c4640 1 -- 172.21.15.64:0/3997400187 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fd419d620 con 0x7f1fd4102ef0 2024-09-12T03:28:19.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.418+0000 7f1fd1ffb640 1 -- 172.21.15.64:0/3997400187 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc0116d0 con 0x7f1fd4102ef0 2024-09-12T03:28:19.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.418+0000 7f1fda4c4640 1 -- 172.21.15.64:0/3997400187 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fd41a4f00 con 0x7f1fd4102ef0 2024-09-12T03:28:19.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.419+0000 7f1fda4c4640 1 -- 172.21.15.64:0/3997400187 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fd4101200 con 0x7f1fd4102ef0 2024-09-12T03:28:19.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.420+0000 7f1fd1ffb640 1 -- 172.21.15.64:0/3997400187 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1fcc0051d0 con 0x7f1fd4102ef0 2024-09-12T03:28:19.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.421+0000 7f1fd1ffb640 1 --2- 172.21.15.64:0/3997400187 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f1fc00a2c90 0x7f1fc00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:19.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.421+0000 7f1fd3fff640 1 --2- 172.21.15.64:0/3997400187 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f1fc00a2c90 0x7f1fc00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:19.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.421+0000 7f1fd1ffb640 1 -- 172.21.15.64:0/3997400187 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f1fcc0cf6d0 con 0x7f1fd4102ef0 2024-09-12T03:28:19.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.421+0000 7f1fd3fff640 1 --2- 172.21.15.64:0/3997400187 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f1fc00a2c90 0x7f1fc00a5150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009230 tx=0x7f1fc40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:19.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.425+0000 7f1fd1ffb640 1 -- 172.21.15.64:0/3997400187 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1fcc016030 con 0x7f1fd4102ef0 2024-09-12T03:28:19.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.602+0000 7f1fda4c4640 1 -- 172.21.15.64:0/3997400187 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f1fd41a5130 con 0x7f1fd4102ef0 2024-09-12T03:28:19.971 INFO:teuthology.orchestra.run.smithi064.stdout:55834574927 2024-09-12T03:28:19.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.602+0000 7f1fd1ffb640 1 -- 172.21.15.64:0/3997400187 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1fcc09c170 con 0x7f1fd4102ef0 2024-09-12T03:28:19.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.604+0000 7f1fda4c4640 1 -- 172.21.15.64:0/3997400187 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f1fc00a2c90 msgr2=0x7f1fc00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:19.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.605+0000 7f1fda4c4640 1 --2- 172.21.15.64:0/3997400187 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f1fc00a2c90 0x7f1fc00a5150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009230 tx=0x7f1fc40058d0 comp rx=0 tx=0).stop 2024-09-12T03:28:19.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.605+0000 7f1fda4c4640 1 -- 172.21.15.64:0/3997400187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fd4102ef0 msgr2=0x7f1fd419cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:19.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.605+0000 7f1fda4c4640 1 --2- 172.21.15.64:0/3997400187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fd4102ef0 0x7f1fd419cc50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc00eaa0 tx=0x7f1fcc00ef60 comp rx=0 tx=0).stop 2024-09-12T03:28:19.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.605+0000 7f1fda4c4640 1 -- 172.21.15.64:0/3997400187 shutdown_connections 2024-09-12T03:28:19.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.605+0000 7f1fda4c4640 1 --2- 172.21.15.64:0/3997400187 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f1fc00a2c90 0x7f1fc00a5150 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:19.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.605+0000 7f1fda4c4640 1 --2- 172.21.15.64:0/3997400187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fd4102ef0 0x7f1fd419cc50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:19.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.605+0000 7f1fda4c4640 1 --2- 172.21.15.64:0/3997400187 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fd4102570 0x7f1fd41988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:19.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.605+0000 7f1fda4c4640 1 --2- 172.21.15.64:0/3997400187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fd4100e20 0x7f1fd4198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:19.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.605+0000 7f1fda4c4640 1 -- 172.21.15.64:0/3997400187 >> 172.21.15.64:0/3997400187 conn(0x7f1fd40fc7d0 msgr2=0x7f1fd406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:19.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.605+0000 7f1fda4c4640 1 -- 172.21.15.64:0/3997400187 shutdown_connections 2024-09-12T03:28:19.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:18.605+0000 7f1fda4c4640 1 -- 172.21.15.64:0/3997400187 wait complete. 2024-09-12T03:28:20.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.518+0000 7f8627ed3640 1 -- 172.21.15.64:0/2425373630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8620101bc0 msgr2=0x7f862010d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:20.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.518+0000 7f8627ed3640 1 --2- 172.21.15.64:0/2425373630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8620101bc0 0x7f862010d910 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8618009ca0 tx=0x7f861802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.518+0000 7f8627ed3640 1 -- 172.21.15.64:0/2425373630 shutdown_connections 2024-09-12T03:28:20.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.518+0000 7f8627ed3640 1 --2- 172.21.15.64:0/2425373630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8620101bc0 0x7f862010d910 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.518+0000 7f8627ed3640 1 --2- 172.21.15.64:0/2425373630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8620101240 0x7f8620101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.518+0000 7f8627ed3640 1 --2- 172.21.15.64:0/2425373630 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8620105a90 0x7f8620105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.518+0000 7f8627ed3640 1 -- 172.21.15.64:0/2425373630 >> 172.21.15.64:0/2425373630 conn(0x7f8620077210 msgr2=0x7f86200fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:20.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.518+0000 7f8627ed3640 1 -- 172.21.15.64:0/2425373630 shutdown_connections 2024-09-12T03:28:20.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.518+0000 7f8627ed3640 1 -- 172.21.15.64:0/2425373630 wait complete. 2024-09-12T03:28:20.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.519+0000 7f8627ed3640 1 Processor -- start 2024-09-12T03:28:20.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.519+0000 7f8627ed3640 1 -- start start 2024-09-12T03:28:20.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.519+0000 7f8627ed3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8620101240 0x7f8620199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:20.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.520+0000 7f8627ed3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8620101bc0 0x7f8620199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:20.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.520+0000 7f8627ed3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8620105a90 0x7f86201a4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:20.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.520+0000 7f8627ed3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8620110e20 con 0x7f8620105a90 2024-09-12T03:28:20.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.520+0000 7f8627ed3640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8620110ca0 con 0x7f8620101bc0 2024-09-12T03:28:20.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.520+0000 7f8627ed3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8620110fa0 con 0x7f8620101240 2024-09-12T03:28:20.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.520+0000 7f8626470640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8620105a90 0x7f86201a4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:20.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.520+0000 7f8626470640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8620105a90 0x7f86201a4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40880/0 (socket says 172.21.15.64:40880) 2024-09-12T03:28:20.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.520+0000 7f8626470640 1 -- 172.21.15.64:0/1204765805 learned_addr learned my addr 172.21.15.64:0/1204765805 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:20.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.520+0000 7f8625c6f640 1 --2- 172.21.15.64:0/1204765805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8620101240 0x7f8620199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:20.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.520+0000 7f862546e640 1 --2- 172.21.15.64:0/1204765805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8620101bc0 0x7f8620199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:20.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.520+0000 7f8626470640 1 -- 172.21.15.64:0/1204765805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8620101240 msgr2=0x7f8620199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:20.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.520+0000 7f8626470640 1 --2- 172.21.15.64:0/1204765805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8620101240 0x7f8620199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.520+0000 7f8626470640 1 -- 172.21.15.64:0/1204765805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8620101bc0 msgr2=0x7f8620199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:20.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.520+0000 7f8626470640 1 --2- 172.21.15.64:0/1204765805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8620101bc0 0x7f8620199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.520+0000 7f8626470640 1 -- 172.21.15.64:0/1204765805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86201a47d0 con 0x7f8620105a90 2024-09-12T03:28:20.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.520+0000 7f862546e640 1 --2- 172.21.15.64:0/1204765805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8620101bc0 0x7f8620199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:28:20.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.520+0000 7f8626470640 1 --2- 172.21.15.64:0/1204765805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8620105a90 0x7f86201a4290 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8618009dc0 tx=0x7f8618032fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:20.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.520+0000 7f8612ffd640 1 -- 172.21.15.64:0/1204765805 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8618003cf0 con 0x7f8620105a90 2024-09-12T03:28:20.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.520+0000 7f8627ed3640 1 -- 172.21.15.64:0/1204765805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86201a49a0 con 0x7f8620105a90 2024-09-12T03:28:20.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.520+0000 7f8612ffd640 1 -- 172.21.15.64:0/1204765805 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86180044f0 con 0x7f8620105a90 2024-09-12T03:28:20.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.520+0000 7f8612ffd640 1 -- 172.21.15.64:0/1204765805 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8618030bf0 con 0x7f8620105a90 2024-09-12T03:28:20.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.521+0000 7f8627ed3640 1 -- 172.21.15.64:0/1204765805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86201a4e50 con 0x7f8620105a90 2024-09-12T03:28:20.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.522+0000 7f8627ed3640 1 -- 172.21.15.64:0/1204765805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8620101680 con 0x7f8620105a90 2024-09-12T03:28:20.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.523+0000 7f8612ffd640 1 -- 172.21.15.64:0/1204765805 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8618004040 con 0x7f8620105a90 2024-09-12T03:28:20.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.523+0000 7f8612ffd640 1 --2- 172.21.15.64:0/1204765805 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f860c0a2c10 0x7f860c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:20.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.524+0000 7f8625c6f640 1 --2- 172.21.15.64:0/1204765805 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f860c0a2c10 0x7f860c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:20.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.524+0000 7f8612ffd640 1 -- 172.21.15.64:0/1204765805 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f86180f0c00 con 0x7f8620105a90 2024-09-12T03:28:20.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.524+0000 7f8625c6f640 1 --2- 172.21.15.64:0/1204765805 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f860c0a2c10 0x7f860c0a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8614005fd0 tx=0x7f861400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:20.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.527+0000 7f8612ffd640 1 -- 172.21.15.64:0/1204765805 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f86180bd7b0 con 0x7f8620105a90 2024-09-12T03:28:20.546 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:20.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.708+0000 7f8627ed3640 1 -- 172.21.15.64:0/1204765805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f8620108fb0 con 0x7f8620105a90 2024-09-12T03:28:20.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.709+0000 7f8612ffd640 1 -- 172.21.15.64:0/1204765805 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f86180bd0b0 con 0x7f8620105a90 2024-09-12T03:28:20.710 INFO:teuthology.orchestra.run.smithi064.stdout:292057776162 2024-09-12T03:28:20.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.711+0000 7f8627ed3640 1 -- 172.21.15.64:0/1204765805 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f860c0a2c10 msgr2=0x7f860c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:20.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.711+0000 7f8627ed3640 1 --2- 172.21.15.64:0/1204765805 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f860c0a2c10 0x7f860c0a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8614005fd0 tx=0x7f861400b390 comp rx=0 tx=0).stop 2024-09-12T03:28:20.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.711+0000 7f8627ed3640 1 -- 172.21.15.64:0/1204765805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8620105a90 msgr2=0x7f86201a4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:20.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.711+0000 7f8627ed3640 1 --2- 172.21.15.64:0/1204765805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8620105a90 0x7f86201a4290 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8618009dc0 tx=0x7f8618032fe0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.711+0000 7f8627ed3640 1 -- 172.21.15.64:0/1204765805 shutdown_connections 2024-09-12T03:28:20.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.711+0000 7f8627ed3640 1 --2- 172.21.15.64:0/1204765805 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f860c0a2c10 0x7f860c0a50d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.711+0000 7f8627ed3640 1 --2- 172.21.15.64:0/1204765805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8620105a90 0x7f86201a4290 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.711+0000 7f8627ed3640 1 --2- 172.21.15.64:0/1204765805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8620101bc0 0x7f8620199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.711+0000 7f8627ed3640 1 --2- 172.21.15.64:0/1204765805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8620101240 0x7f8620199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.711+0000 7f8627ed3640 1 -- 172.21.15.64:0/1204765805 >> 172.21.15.64:0/1204765805 conn(0x7f8620077210 msgr2=0x7f86201041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:20.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.711+0000 7f8627ed3640 1 -- 172.21.15.64:0/1204765805 shutdown_connections 2024-09-12T03:28:20.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.711+0000 7f8627ed3640 1 -- 172.21.15.64:0/1204765805 wait complete. 2024-09-12T03:28:20.987 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:20 smithi064 ceph-mon[30952]: pgmap v326: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-12T03:28:20.987 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:20 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/81993587' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T03:28:20.987 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:20 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/1204765805' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T03:28:21.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:20 smithi115 ceph-mon[34801]: pgmap v326: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-12T03:28:21.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:20 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/81993587' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T03:28:21.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:20 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/1204765805' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T03:28:21.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:20 smithi195 ceph-mon[34786]: pgmap v326: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-12T03:28:21.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:20 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/81993587' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T03:28:21.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:20 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/1204765805' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T03:28:21.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.209+0000 7f469371b640 1 -- 172.21.15.64:0/2769777605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f468c100a80 msgr2=0x7f468c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:21.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.209+0000 7f469371b640 1 --2- 172.21.15.64:0/2769777605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f468c100a80 0x7f468c100ee0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4680009930 tx=0x7f468002a420 comp rx=0 tx=0).stop 2024-09-12T03:28:21.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.209+0000 7f469371b640 1 -- 172.21.15.64:0/2769777605 shutdown_connections 2024-09-12T03:28:21.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.209+0000 7f469371b640 1 --2- 172.21.15.64:0/2769777605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f468c101420 0x7f468c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.209+0000 7f469371b640 1 --2- 172.21.15.64:0/2769777605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f468c100a80 0x7f468c100ee0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.209+0000 7f469371b640 1 --2- 172.21.15.64:0/2769777605 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f468c106a80 0x7f468c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.209+0000 7f469371b640 1 -- 172.21.15.64:0/2769777605 >> 172.21.15.64:0/2769777605 conn(0x7f468c0fc7a0 msgr2=0x7f468c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:21.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.210+0000 7f469371b640 1 -- 172.21.15.64:0/2769777605 shutdown_connections 2024-09-12T03:28:21.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.210+0000 7f469371b640 1 -- 172.21.15.64:0/2769777605 wait complete. 2024-09-12T03:28:21.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.210+0000 7f469371b640 1 Processor -- start 2024-09-12T03:28:21.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.210+0000 7f469371b640 1 -- start start 2024-09-12T03:28:21.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.210+0000 7f469371b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f468c100a80 0x7f468c198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:21.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.210+0000 7f469371b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f468c101420 0x7f468c1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:21.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.210+0000 7f469371b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f468c106a80 0x7f468c19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:21.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.210+0000 7f469371b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f468c110d30 con 0x7f468c100a80 2024-09-12T03:28:21.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.210+0000 7f469371b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f468c110bb0 con 0x7f468c106a80 2024-09-12T03:28:21.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.210+0000 7f469371b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f468c110eb0 con 0x7f468c101420 2024-09-12T03:28:21.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.211+0000 7f46914b7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f468c100a80 0x7f468c198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:21.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.211+0000 7f46914b7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f468c100a80 0x7f468c198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40942/0 (socket says 172.21.15.64:40942) 2024-09-12T03:28:21.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.211+0000 7f46914b7640 1 -- 172.21.15.64:0/789837001 learned_addr learned my addr 172.21.15.64:0/789837001 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:21.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.211+0000 7f4690cb6640 1 --2- 172.21.15.64:0/789837001 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f468c101420 0x7f468c1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:21.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.211+0000 7f4691cb8640 1 --2- 172.21.15.64:0/789837001 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f468c106a80 0x7f468c19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:21.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.211+0000 7f46914b7640 1 -- 172.21.15.64:0/789837001 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f468c101420 msgr2=0x7f468c1988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:21.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.211+0000 7f46914b7640 1 --2- 172.21.15.64:0/789837001 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f468c101420 0x7f468c1988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.211+0000 7f46914b7640 1 -- 172.21.15.64:0/789837001 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f468c106a80 msgr2=0x7f468c19cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:21.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.211+0000 7f46914b7640 1 --2- 172.21.15.64:0/789837001 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f468c106a80 0x7f468c19cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.211+0000 7f46914b7640 1 -- 172.21.15.64:0/789837001 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f468c19d320 con 0x7f468c100a80 2024-09-12T03:28:21.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.211+0000 7f46914b7640 1 --2- 172.21.15.64:0/789837001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f468c100a80 0x7f468c198370 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f468800c890 tx=0x7f468800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:21.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.211+0000 7f467e7fc640 1 -- 172.21.15.64:0/789837001 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4688004220 con 0x7f468c100a80 2024-09-12T03:28:21.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.211+0000 7f467e7fc640 1 -- 172.21.15.64:0/789837001 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46880043c0 con 0x7f468c100a80 2024-09-12T03:28:21.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.211+0000 7f469371b640 1 -- 172.21.15.64:0/789837001 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f468c19d610 con 0x7f468c100a80 2024-09-12T03:28:21.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.211+0000 7f467e7fc640 1 -- 172.21.15.64:0/789837001 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4688004a00 con 0x7f468c100a80 2024-09-12T03:28:21.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.211+0000 7f469371b640 1 -- 172.21.15.64:0/789837001 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f468c1a4ef0 con 0x7f468c100a80 2024-09-12T03:28:21.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.212+0000 7f469371b640 1 -- 172.21.15.64:0/789837001 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f468c100ee0 con 0x7f468c100a80 2024-09-12T03:28:21.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.213+0000 7f467e7fc640 1 -- 172.21.15.64:0/789837001 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4688004ba0 con 0x7f468c100a80 2024-09-12T03:28:21.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.214+0000 7f467e7fc640 1 --2- 172.21.15.64:0/789837001 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f46780a2c10 0x7f46780a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:21.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.214+0000 7f4690cb6640 1 --2- 172.21.15.64:0/789837001 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f46780a2c10 0x7f46780a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:21.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.214+0000 7f467e7fc640 1 -- 172.21.15.64:0/789837001 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f46880cf110 con 0x7f468c100a80 2024-09-12T03:28:21.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.215+0000 7f4690cb6640 1 --2- 172.21.15.64:0/789837001 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f46780a2c10 0x7f46780a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4680009900 tx=0x7f4680000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:21.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.218+0000 7f467e7fc640 1 -- 172.21.15.64:0/789837001 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4688014030 con 0x7f468c100a80 2024-09-12T03:28:21.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.398+0000 7f469371b640 1 -- 172.21.15.64:0/789837001 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f468c109a70 con 0x7f468c100a80 2024-09-12T03:28:21.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.398+0000 7f467e7fc640 1 -- 172.21.15.64:0/789837001 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f468809b560 con 0x7f468c100a80 2024-09-12T03:28:21.399 INFO:teuthology.orchestra.run.smithi064.stdout:326417514526 2024-09-12T03:28:21.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.401+0000 7f469371b640 1 -- 172.21.15.64:0/789837001 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f46780a2c10 msgr2=0x7f46780a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:21.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.401+0000 7f469371b640 1 --2- 172.21.15.64:0/789837001 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f46780a2c10 0x7f46780a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4680009900 tx=0x7f4680000bc0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.401+0000 7f469371b640 1 -- 172.21.15.64:0/789837001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f468c100a80 msgr2=0x7f468c198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:21.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.401+0000 7f469371b640 1 --2- 172.21.15.64:0/789837001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f468c100a80 0x7f468c198370 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f468800c890 tx=0x7f468800cd50 comp rx=0 tx=0).stop 2024-09-12T03:28:21.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.401+0000 7f469371b640 1 -- 172.21.15.64:0/789837001 shutdown_connections 2024-09-12T03:28:21.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.401+0000 7f469371b640 1 --2- 172.21.15.64:0/789837001 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f46780a2c10 0x7f46780a50d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.401+0000 7f469371b640 1 --2- 172.21.15.64:0/789837001 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f468c106a80 0x7f468c19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.401+0000 7f469371b640 1 --2- 172.21.15.64:0/789837001 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f468c101420 0x7f468c1988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.401+0000 7f469371b640 1 --2- 172.21.15.64:0/789837001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f468c100a80 0x7f468c198370 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.401+0000 7f469371b640 1 -- 172.21.15.64:0/789837001 >> 172.21.15.64:0/789837001 conn(0x7f468c0fc7a0 msgr2=0x7f468c06ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:21.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.401+0000 7f469371b640 1 -- 172.21.15.64:0/789837001 shutdown_connections 2024-09-12T03:28:21.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.401+0000 7f469371b640 1 -- 172.21.15.64:0/789837001 wait complete. 2024-09-12T03:28:22.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:21 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/1029773887' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T03:28:22.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:21 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/1935597628' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T03:28:22.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:21 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/789837001' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T03:28:22.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:21 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/1029773887' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T03:28:22.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:21 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/1935597628' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T03:28:22.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:21 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/789837001' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T03:28:22.247 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:21 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/1029773887' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T03:28:22.247 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:21 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/1935597628' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T03:28:22.247 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:21 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/789837001' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T03:28:22.278 INFO:tasks.cephadm.ceph_manager.ceph:need seq 326417514523 got 326417514526 for osd.11 2024-09-12T03:28:22.278 DEBUG:teuthology.parallel:result is None 2024-09-12T03:28:22.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.457+0000 7fb022eaf640 1 -- 172.21.15.64:0/1170642122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb01c101de0 msgr2=0x7fb01c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:22.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.457+0000 7fb022eaf640 1 --2- 172.21.15.64:0/1170642122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb01c101de0 0x7fb01c1021c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb00c009900 tx=0x7fb00c02a410 comp rx=0 tx=0).stop 2024-09-12T03:28:22.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.457+0000 7fb022eaf640 1 -- 172.21.15.64:0/1170642122 shutdown_connections 2024-09-12T03:28:22.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.457+0000 7fb022eaf640 1 --2- 172.21.15.64:0/1170642122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb01c106f10 0x7fb01c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.457+0000 7fb022eaf640 1 --2- 172.21.15.64:0/1170642122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb01c102790 0x7fb01c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.457+0000 7fb022eaf640 1 --2- 172.21.15.64:0/1170642122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb01c101de0 0x7fb01c1021c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.457+0000 7fb022eaf640 1 -- 172.21.15.64:0/1170642122 >> 172.21.15.64:0/1170642122 conn(0x7fb01c0fd8c0 msgr2=0x7fb01c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:22.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.457+0000 7fb022eaf640 1 -- 172.21.15.64:0/1170642122 shutdown_connections 2024-09-12T03:28:22.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.457+0000 7fb022eaf640 1 -- 172.21.15.64:0/1170642122 wait complete. 2024-09-12T03:28:22.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.457+0000 7fb022eaf640 1 Processor -- start 2024-09-12T03:28:22.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.457+0000 7fb022eaf640 1 -- start start 2024-09-12T03:28:22.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.458+0000 7fb022eaf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb01c101de0 0x7fb01c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:22.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.458+0000 7fb022eaf640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb01c102790 0x7fb01c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:22.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.458+0000 7fb022eaf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb01c106f10 0x7fb01c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:22.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.458+0000 7fb022eaf640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c111e70 con 0x7fb01c106f10 2024-09-12T03:28:22.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.458+0000 7fb022eaf640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c111cf0 con 0x7fb01c101de0 2024-09-12T03:28:22.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.458+0000 7fb022eaf640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c111ff0 con 0x7fb01c102790 2024-09-12T03:28:22.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.458+0000 7fb02144c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb01c106f10 0x7fb01c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:22.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.458+0000 7fb02144c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb01c106f10 0x7fb01c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40954/0 (socket says 172.21.15.64:40954) 2024-09-12T03:28:22.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.458+0000 7fb02144c640 1 -- 172.21.15.64:0/4275340989 learned_addr learned my addr 172.21.15.64:0/4275340989 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:22.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.458+0000 7fb01bfff640 1 --2- 172.21.15.64:0/4275340989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb01c102790 0x7fb01c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:22.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.458+0000 7fb020c4b640 1 --2- 172.21.15.64:0/4275340989 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb01c101de0 0x7fb01c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:22.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.458+0000 7fb02144c640 1 -- 172.21.15.64:0/4275340989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb01c102790 msgr2=0x7fb01c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:22.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.458+0000 7fb02144c640 1 --2- 172.21.15.64:0/4275340989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb01c102790 0x7fb01c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.458+0000 7fb02144c640 1 -- 172.21.15.64:0/4275340989 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb01c101de0 msgr2=0x7fb01c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:22.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.458+0000 7fb02144c640 1 --2- 172.21.15.64:0/4275340989 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb01c101de0 0x7fb01c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.458+0000 7fb02144c640 1 -- 172.21.15.64:0/4275340989 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb01c1a6810 con 0x7fb01c106f10 2024-09-12T03:28:22.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.458+0000 7fb02144c640 1 --2- 172.21.15.64:0/4275340989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb01c106f10 0x7fb01c074f70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb01400cce0 tx=0x7fb01400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:22.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.458+0000 7fb019ffb640 1 -- 172.21.15.64:0/4275340989 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01400f450 con 0x7fb01c106f10 2024-09-12T03:28:22.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.459+0000 7fb022eaf640 1 -- 172.21.15.64:0/4275340989 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb01c1a6b00 con 0x7fb01c106f10 2024-09-12T03:28:22.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.459+0000 7fb019ffb640 1 -- 172.21.15.64:0/4275340989 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb01400faa0 con 0x7fb01c106f10 2024-09-12T03:28:22.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.459+0000 7fb019ffb640 1 -- 172.21.15.64:0/4275340989 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb014018b70 con 0x7fb01c106f10 2024-09-12T03:28:22.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.459+0000 7fb022eaf640 1 -- 172.21.15.64:0/4275340989 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb01c1a6ff0 con 0x7fb01c106f10 2024-09-12T03:28:22.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.460+0000 7fb022eaf640 1 -- 172.21.15.64:0/4275340989 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb01c1021c0 con 0x7fb01c106f10 2024-09-12T03:28:22.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.461+0000 7fb019ffb640 1 -- 172.21.15.64:0/4275340989 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb014018d10 con 0x7fb01c106f10 2024-09-12T03:28:22.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.462+0000 7fb019ffb640 1 --2- 172.21.15.64:0/4275340989 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fb0040a2c10 0x7fb0040a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:22.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.462+0000 7fb020c4b640 1 --2- 172.21.15.64:0/4275340989 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fb0040a2c10 0x7fb0040a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:22.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.462+0000 7fb019ffb640 1 -- 172.21.15.64:0/4275340989 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fb0140cf850 con 0x7fb01c106f10 2024-09-12T03:28:22.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.462+0000 7fb020c4b640 1 --2- 172.21.15.64:0/4275340989 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fb0040a2c10 0x7fb0040a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb00c005ab0 tx=0x7fb00c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:22.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.466+0000 7fb019ffb640 1 -- 172.21.15.64:0/4275340989 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb014014030 con 0x7fb01c106f10 2024-09-12T03:28:22.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.748+0000 7f8cbc95b640 1 -- 172.21.15.64:0/1631846992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cb8100a70 msgr2=0x7f8cb8100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:22.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.748+0000 7f8cbc95b640 1 --2- 172.21.15.64:0/1631846992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cb8100a70 0x7f8cb8100ed0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8cac009930 tx=0x7f8cac02a420 comp rx=0 tx=0).stop 2024-09-12T03:28:22.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.749+0000 7f8cbc95b640 1 -- 172.21.15.64:0/1631846992 shutdown_connections 2024-09-12T03:28:22.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.749+0000 7f8cbc95b640 1 --2- 172.21.15.64:0/1631846992 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8cb8101410 0x7f8cb810d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.749+0000 7f8cbc95b640 1 --2- 172.21.15.64:0/1631846992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cb8100a70 0x7f8cb8100ed0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.749+0000 7f8cbc95b640 1 --2- 172.21.15.64:0/1631846992 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cb8106a70 0x7f8cb8106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.749+0000 7f8cbc95b640 1 -- 172.21.15.64:0/1631846992 >> 172.21.15.64:0/1631846992 conn(0x7f8cb80fc790 msgr2=0x7f8cb80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:22.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.749+0000 7f8cbc95b640 1 -- 172.21.15.64:0/1631846992 shutdown_connections 2024-09-12T03:28:22.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.749+0000 7f8cbc95b640 1 -- 172.21.15.64:0/1631846992 wait complete. 2024-09-12T03:28:22.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.749+0000 7f8cbc95b640 1 Processor -- start 2024-09-12T03:28:22.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.749+0000 7f8cbc95b640 1 -- start start 2024-09-12T03:28:22.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.750+0000 7f8cbc95b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cb8100a70 0x7f8cb81991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:22.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.750+0000 7f8cbc95b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cb8101410 0x7f8cb8199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:22.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.750+0000 7f8cbc95b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8cb8106a70 0x7f8cb81a4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:22.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.750+0000 7f8cbc95b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb8110ce0 con 0x7f8cb8101410 2024-09-12T03:28:22.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.750+0000 7f8cbc95b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb8110b60 con 0x7f8cb8100a70 2024-09-12T03:28:22.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.750+0000 7f8cbc95b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb8110e60 con 0x7f8cb8106a70 2024-09-12T03:28:22.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.750+0000 7f8cb5d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cb8101410 0x7f8cb8199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:22.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.750+0000 7f8cb5d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cb8101410 0x7f8cb8199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40896/0 (socket says 172.21.15.64:40896) 2024-09-12T03:28:22.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.750+0000 7f8cb5d9b640 1 -- 172.21.15.64:0/1029773887 learned_addr learned my addr 172.21.15.64:0/1029773887 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:22.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.750+0000 7f8cb6d9d640 1 --2- 172.21.15.64:0/1029773887 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8cb8106a70 0x7f8cb81a4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:22.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.750+0000 7f8cb659c640 1 --2- 172.21.15.64:0/1029773887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cb8100a70 0x7f8cb81991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:22.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.750+0000 7f8cb5d9b640 1 -- 172.21.15.64:0/1029773887 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8cb8106a70 msgr2=0x7f8cb81a4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:22.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.750+0000 7f8cb5d9b640 1 --2- 172.21.15.64:0/1029773887 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8cb8106a70 0x7f8cb81a4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.750+0000 7f8cb5d9b640 1 -- 172.21.15.64:0/1029773887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cb8100a70 msgr2=0x7f8cb81991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:22.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.750+0000 7f8cb5d9b640 1 --2- 172.21.15.64:0/1029773887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cb8100a70 0x7f8cb81991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.750+0000 7f8cb5d9b640 1 -- 172.21.15.64:0/1029773887 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cb81a4790 con 0x7f8cb8101410 2024-09-12T03:28:22.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.750+0000 7f8cb5d9b640 1 --2- 172.21.15.64:0/1029773887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cb8101410 0x7f8cb8199710 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8cac004770 tx=0x7f8cac0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:22.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.750+0000 7f8ca77fe640 1 -- 172.21.15.64:0/1029773887 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cac03f070 con 0x7f8cb8101410 2024-09-12T03:28:22.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.750+0000 7f8ca77fe640 1 -- 172.21.15.64:0/1029773887 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cac03b460 con 0x7f8cb8101410 2024-09-12T03:28:22.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.750+0000 7f8cbc95b640 1 -- 172.21.15.64:0/1029773887 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cb81a4960 con 0x7f8cb8101410 2024-09-12T03:28:22.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.750+0000 7f8ca77fe640 1 -- 172.21.15.64:0/1029773887 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cac035070 con 0x7f8cb8101410 2024-09-12T03:28:22.624 INFO:teuthology.orchestra.run.smithi064.stdout:103079215173 2024-09-12T03:28:22.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.751+0000 7f8cbc95b640 1 -- 172.21.15.64:0/1029773887 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cb81a4e10 con 0x7f8cb8101410 2024-09-12T03:28:22.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.751+0000 7f8cbc95b640 1 -- 172.21.15.64:0/1029773887 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cb8100ed0 con 0x7f8cb8101410 2024-09-12T03:28:22.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.753+0000 7f8ca77fe640 1 -- 172.21.15.64:0/1029773887 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8cac03b610 con 0x7f8cb8101410 2024-09-12T03:28:22.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.753+0000 7f8ca77fe640 1 --2- 172.21.15.64:0/1029773887 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f8ca00a2c90 0x7f8ca00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:22.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.754+0000 7f8cb659c640 1 --2- 172.21.15.64:0/1029773887 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f8ca00a2c90 0x7f8ca00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:22.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.754+0000 7f8ca77fe640 1 -- 172.21.15.64:0/1029773887 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f8cac0efc00 con 0x7f8cb8101410 2024-09-12T03:28:22.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.754+0000 7f8cb659c640 1 --2- 172.21.15.64:0/1029773887 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f8ca00a2c90 0x7f8ca00a5150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8005e00 tx=0x7f8ca8005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:22.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.757+0000 7f8ca77fe640 1 -- 172.21.15.64:0/1029773887 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8cac0bc6a0 con 0x7f8cb8101410 2024-09-12T03:28:22.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.941+0000 7f8cbc95b640 1 -- 172.21.15.64:0/1029773887 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f8cb8109030 con 0x7f8cb8101410 2024-09-12T03:28:22.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.942+0000 7f8ca77fe640 1 -- 172.21.15.64:0/1029773887 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8cac0bbfa0 con 0x7f8cb8101410 2024-09-12T03:28:22.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.944+0000 7f8cbc95b640 1 -- 172.21.15.64:0/1029773887 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f8ca00a2c90 msgr2=0x7f8ca00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:22.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.944+0000 7f8cbc95b640 1 --2- 172.21.15.64:0/1029773887 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f8ca00a2c90 0x7f8ca00a5150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8005e00 tx=0x7f8ca8005c50 comp rx=0 tx=0).stop 2024-09-12T03:28:22.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.944+0000 7f8cbc95b640 1 -- 172.21.15.64:0/1029773887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cb8101410 msgr2=0x7f8cb8199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:22.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.944+0000 7f8cbc95b640 1 --2- 172.21.15.64:0/1029773887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cb8101410 0x7f8cb8199710 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8cac004770 tx=0x7f8cac0042f0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.944+0000 7f8cbc95b640 1 -- 172.21.15.64:0/1029773887 shutdown_connections 2024-09-12T03:28:22.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.944+0000 7f8cbc95b640 1 --2- 172.21.15.64:0/1029773887 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f8ca00a2c90 0x7f8ca00a5150 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.944+0000 7f8cbc95b640 1 --2- 172.21.15.64:0/1029773887 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8cb8106a70 0x7f8cb81a4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.944+0000 7f8cbc95b640 1 --2- 172.21.15.64:0/1029773887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cb8101410 0x7f8cb8199710 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.944+0000 7f8cbc95b640 1 --2- 172.21.15.64:0/1029773887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cb8100a70 0x7f8cb81991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.944+0000 7f8cbc95b640 1 -- 172.21.15.64:0/1029773887 >> 172.21.15.64:0/1029773887 conn(0x7f8cb80fc790 msgr2=0x7f8cb80fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:22.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.944+0000 7f8cbc95b640 1 -- 172.21.15.64:0/1029773887 shutdown_connections 2024-09-12T03:28:22.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.944+0000 7f8cbc95b640 1 -- 172.21.15.64:0/1029773887 wait complete. 2024-09-12T03:28:22.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.650+0000 7fb022eaf640 1 -- 172.21.15.64:0/4275340989 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fb01c10a0f0 con 0x7fb01c106f10 2024-09-12T03:28:22.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.651+0000 7fb019ffb640 1 -- 172.21.15.64:0/4275340989 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb01409c2f0 con 0x7fb01c106f10 2024-09-12T03:28:22.652 INFO:teuthology.orchestra.run.smithi064.stdout:184683593783 2024-09-12T03:28:22.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.653+0000 7fb022eaf640 1 -- 172.21.15.64:0/4275340989 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fb0040a2c10 msgr2=0x7fb0040a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:22.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.653+0000 7fb022eaf640 1 --2- 172.21.15.64:0/4275340989 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fb0040a2c10 0x7fb0040a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb00c005ab0 tx=0x7fb00c005a20 comp rx=0 tx=0).stop 2024-09-12T03:28:22.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.653+0000 7fb022eaf640 1 -- 172.21.15.64:0/4275340989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb01c106f10 msgr2=0x7fb01c074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:22.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.653+0000 7fb022eaf640 1 --2- 172.21.15.64:0/4275340989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb01c106f10 0x7fb01c074f70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb01400cce0 tx=0x7fb01400a620 comp rx=0 tx=0).stop 2024-09-12T03:28:22.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.653+0000 7fb022eaf640 1 -- 172.21.15.64:0/4275340989 shutdown_connections 2024-09-12T03:28:22.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.653+0000 7fb022eaf640 1 --2- 172.21.15.64:0/4275340989 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fb0040a2c10 0x7fb0040a50d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.653+0000 7fb022eaf640 1 --2- 172.21.15.64:0/4275340989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb01c106f10 0x7fb01c074f70 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.653+0000 7fb022eaf640 1 --2- 172.21.15.64:0/4275340989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb01c102790 0x7fb01c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.653+0000 7fb022eaf640 1 --2- 172.21.15.64:0/4275340989 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb01c101de0 0x7fb01c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.653+0000 7fb022eaf640 1 -- 172.21.15.64:0/4275340989 >> 172.21.15.64:0/4275340989 conn(0x7fb01c0fd8c0 msgr2=0x7fb01c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:22.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.653+0000 7fb022eaf640 1 -- 172.21.15.64:0/4275340989 shutdown_connections 2024-09-12T03:28:22.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:22.653+0000 7fb022eaf640 1 -- 172.21.15.64:0/4275340989 wait complete. 2024-09-12T03:28:23.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:22 smithi115 ceph-mon[34801]: pgmap v327: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-12T03:28:23.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:22 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/4275340989' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T03:28:23.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:22 smithi195 ceph-mon[34786]: pgmap v327: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-12T03:28:23.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:22 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/4275340989' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T03:28:23.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:22 smithi064 ceph-mon[30952]: pgmap v327: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-12T03:28:23.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:22 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/4275340989' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T03:28:23.330 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215170 got 103079215173 for osd.3 2024-09-12T03:28:23.331 DEBUG:teuthology.parallel:result is None 2024-09-12T03:28:23.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.864+0000 7f5a4c9ec640 1 -- 172.21.15.64:0/35550179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a481082a0 msgr2=0x7f5a4810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:23.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.864+0000 7f5a4c9ec640 1 --2- 172.21.15.64:0/35550179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a481082a0 0x7f5a4810ec40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5a40009cd0 tx=0x7f5a4002a8e0 comp rx=0 tx=0).stop 2024-09-12T03:28:23.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.864+0000 7f5a4c9ec640 1 -- 172.21.15.64:0/35550179 shutdown_connections 2024-09-12T03:28:23.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.864+0000 7f5a4c9ec640 1 --2- 172.21.15.64:0/35550179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a481082a0 0x7f5a4810ec40 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:23.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.864+0000 7f5a4c9ec640 1 --2- 172.21.15.64:0/35550179 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a481036e0 0x7f5a48107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:23.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.864+0000 7f5a4c9ec640 1 --2- 172.21.15.64:0/35550179 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a48102d30 0x7f5a48103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:23.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.864+0000 7f5a4c9ec640 1 -- 172.21.15.64:0/35550179 >> 172.21.15.64:0/35550179 conn(0x7f5a480fc7d0 msgr2=0x7f5a480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:23.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.865+0000 7f5a4c9ec640 1 -- 172.21.15.64:0/35550179 shutdown_connections 2024-09-12T03:28:23.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.865+0000 7f5a4c9ec640 1 -- 172.21.15.64:0/35550179 wait complete. 2024-09-12T03:28:23.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.865+0000 7f5a4c9ec640 1 Processor -- start 2024-09-12T03:28:23.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.866+0000 7f5a4c9ec640 1 -- start start 2024-09-12T03:28:23.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.866+0000 7f5a4c9ec640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a48102d30 0x7f5a48198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:23.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.866+0000 7f5a4c9ec640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a481036e0 0x7f5a481989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:23.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.866+0000 7f5a4c9ec640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a481082a0 0x7f5a4819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:23.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.866+0000 7f5a4c9ec640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5a48111f50 con 0x7f5a48102d30 2024-09-12T03:28:23.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.866+0000 7f5a4c9ec640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5a48111dd0 con 0x7f5a481036e0 2024-09-12T03:28:23.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.866+0000 7f5a4c9ec640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5a481120d0 con 0x7f5a481082a0 2024-09-12T03:28:23.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.866+0000 7f5a4659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a48102d30 0x7f5a48198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:23.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.866+0000 7f5a4659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a48102d30 0x7f5a48198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40922/0 (socket says 172.21.15.64:40922) 2024-09-12T03:28:23.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.866+0000 7f5a4659c640 1 -- 172.21.15.64:0/1935597628 learned_addr learned my addr 172.21.15.64:0/1935597628 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:23.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.866+0000 7f5a3dd9b640 1 --2- 172.21.15.64:0/1935597628 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a481036e0 0x7f5a481989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:23.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.866+0000 7f5a46d9d640 1 --2- 172.21.15.64:0/1935597628 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a481082a0 0x7f5a4819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:23.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.867+0000 7f5a4659c640 1 -- 172.21.15.64:0/1935597628 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a481082a0 msgr2=0x7f5a4819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:23.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.867+0000 7f5a4659c640 1 --2- 172.21.15.64:0/1935597628 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a481082a0 0x7f5a4819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:23.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.867+0000 7f5a4659c640 1 -- 172.21.15.64:0/1935597628 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a481036e0 msgr2=0x7f5a481989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:23.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.867+0000 7f5a4659c640 1 --2- 172.21.15.64:0/1935597628 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a481036e0 0x7f5a481989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:23.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.867+0000 7f5a4659c640 1 -- 172.21.15.64:0/1935597628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a4819d410 con 0x7f5a48102d30 2024-09-12T03:28:23.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.867+0000 7f5a3dd9b640 1 --2- 172.21.15.64:0/1935597628 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a481036e0 0x7f5a481989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:23.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.867+0000 7f5a4659c640 1 --2- 172.21.15.64:0/1935597628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a48102d30 0x7f5a48198460 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800e9f0 tx=0x7f5a3800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:23.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.870+0000 7f5a3ffff640 1 -- 172.21.15.64:0/1935597628 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3800cd80 con 0x7f5a48102d30 2024-09-12T03:28:23.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.870+0000 7f5a3ffff640 1 -- 172.21.15.64:0/1935597628 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a380108e0 con 0x7f5a48102d30 2024-09-12T03:28:23.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.870+0000 7f5a3ffff640 1 -- 172.21.15.64:0/1935597628 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a38018920 con 0x7f5a48102d30 2024-09-12T03:28:23.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.871+0000 7f5a4c9ec640 1 -- 172.21.15.64:0/1935597628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a4819d700 con 0x7f5a48102d30 2024-09-12T03:28:23.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.871+0000 7f5a4c9ec640 1 -- 172.21.15.64:0/1935597628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a481a4fe0 con 0x7f5a48102d30 2024-09-12T03:28:23.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.872+0000 7f5a4c9ec640 1 -- 172.21.15.64:0/1935597628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a48103110 con 0x7f5a48102d30 2024-09-12T03:28:23.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.873+0000 7f5a3ffff640 1 -- 172.21.15.64:0/1935597628 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5a38010a90 con 0x7f5a48102d30 2024-09-12T03:28:23.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.874+0000 7f5a3ffff640 1 --2- 172.21.15.64:0/1935597628 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f5a340a2c90 0x7f5a340a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:23.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.874+0000 7f5a3dd9b640 1 --2- 172.21.15.64:0/1935597628 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f5a340a2c90 0x7f5a340a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:23.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.874+0000 7f5a3ffff640 1 -- 172.21.15.64:0/1935597628 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f5a380cf810 con 0x7f5a48102d30 2024-09-12T03:28:23.425 INFO:teuthology.orchestra.run.smithi064.stdout:154618822715 2024-09-12T03:28:23.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.875+0000 7f5a3dd9b640 1 --2- 172.21.15.64:0/1935597628 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f5a340a2c90 0x7f5a340a5150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5a480fee80 tx=0x7f5a3000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:23.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.878+0000 7f5a3ffff640 1 -- 172.21.15.64:0/1935597628 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5a3809bc00 con 0x7f5a48102d30 2024-09-12T03:28:23.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.063+0000 7f5a4c9ec640 1 -- 172.21.15.64:0/1935597628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f5a481997c0 con 0x7f5a48102d30 2024-09-12T03:28:23.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.063+0000 7f5a3ffff640 1 -- 172.21.15.64:0/1935597628 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5a3809ff00 con 0x7f5a48102d30 2024-09-12T03:28:23.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.065+0000 7f5a4c9ec640 1 -- 172.21.15.64:0/1935597628 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f5a340a2c90 msgr2=0x7f5a340a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:23.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.065+0000 7f5a4c9ec640 1 --2- 172.21.15.64:0/1935597628 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f5a340a2c90 0x7f5a340a5150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5a480fee80 tx=0x7f5a3000b390 comp rx=0 tx=0).stop 2024-09-12T03:28:23.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.065+0000 7f5a4c9ec640 1 -- 172.21.15.64:0/1935597628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a48102d30 msgr2=0x7f5a48198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:23.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.065+0000 7f5a4c9ec640 1 --2- 172.21.15.64:0/1935597628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a48102d30 0x7f5a48198460 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800e9f0 tx=0x7f5a3800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:28:23.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.065+0000 7f5a4c9ec640 1 -- 172.21.15.64:0/1935597628 shutdown_connections 2024-09-12T03:28:23.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.065+0000 7f5a4c9ec640 1 --2- 172.21.15.64:0/1935597628 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f5a340a2c90 0x7f5a340a5150 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:23.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.065+0000 7f5a4c9ec640 1 --2- 172.21.15.64:0/1935597628 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a481082a0 0x7f5a4819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:23.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.065+0000 7f5a4c9ec640 1 --2- 172.21.15.64:0/1935597628 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a481036e0 0x7f5a481989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:23.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.065+0000 7f5a4c9ec640 1 --2- 172.21.15.64:0/1935597628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a48102d30 0x7f5a48198460 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:23.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.065+0000 7f5a4c9ec640 1 -- 172.21.15.64:0/1935597628 >> 172.21.15.64:0/1935597628 conn(0x7f5a480fc7d0 msgr2=0x7f5a48105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:23.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.065+0000 7f5a4c9ec640 1 -- 172.21.15.64:0/1935597628 shutdown_connections 2024-09-12T03:28:23.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:21.065+0000 7f5a4c9ec640 1 -- 172.21.15.64:0/1935597628 wait complete. 2024-09-12T03:28:23.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.614+0000 7f15f1048640 1 -- 172.21.15.64:0/670746193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15ec106a80 msgr2=0x7f15ec106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:23.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.614+0000 7f15f1048640 1 --2- 172.21.15.64:0/670746193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15ec106a80 0x7f15ec106e60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f15e4009cd0 tx=0x7f15e402a8e0 comp rx=0 tx=0).stop 2024-09-12T03:28:23.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.614+0000 7f15f1048640 1 -- 172.21.15.64:0/670746193 shutdown_connections 2024-09-12T03:28:23.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.614+0000 7f15f1048640 1 --2- 172.21.15.64:0/670746193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15ec101420 0x7f15ec10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:23.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.614+0000 7f15f1048640 1 --2- 172.21.15.64:0/670746193 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15ec100a80 0x7f15ec100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:23.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.614+0000 7f15f1048640 1 --2- 172.21.15.64:0/670746193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15ec106a80 0x7f15ec106e60 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:23.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.614+0000 7f15f1048640 1 -- 172.21.15.64:0/670746193 >> 172.21.15.64:0/670746193 conn(0x7f15ec0fc7a0 msgr2=0x7f15ec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:23.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.614+0000 7f15f1048640 1 -- 172.21.15.64:0/670746193 shutdown_connections 2024-09-12T03:28:23.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.614+0000 7f15f1048640 1 -- 172.21.15.64:0/670746193 wait complete. 2024-09-12T03:28:23.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.615+0000 7f15f1048640 1 Processor -- start 2024-09-12T03:28:23.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.615+0000 7f15f1048640 1 -- start start 2024-09-12T03:28:23.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.615+0000 7f15f1048640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15ec100a80 0x7f15ec1a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:23.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.615+0000 7f15f1048640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15ec101420 0x7f15ec1a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:23.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.615+0000 7f15f1048640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15ec106a80 0x7f15ec19ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:23.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.615+0000 7f15f1048640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f15ec110d50 con 0x7f15ec101420 2024-09-12T03:28:23.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.615+0000 7f15f1048640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f15ec110bd0 con 0x7f15ec100a80 2024-09-12T03:28:23.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.615+0000 7f15f1048640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f15ec110ed0 con 0x7f15ec106a80 2024-09-12T03:28:23.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.615+0000 7f15ea59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15ec101420 0x7f15ec1a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:23.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.615+0000 7f15ea59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15ec101420 0x7f15ec1a4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40980/0 (socket says 172.21.15.64:40980) 2024-09-12T03:28:23.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.615+0000 7f15ea59c640 1 -- 172.21.15.64:0/3113824997 learned_addr learned my addr 172.21.15.64:0/3113824997 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:23.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.615+0000 7f15ead9d640 1 --2- 172.21.15.64:0/3113824997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15ec100a80 0x7f15ec1a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:23.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.615+0000 7f15eb59e640 1 --2- 172.21.15.64:0/3113824997 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15ec106a80 0x7f15ec19ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:23.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.616+0000 7f15ea59c640 1 -- 172.21.15.64:0/3113824997 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15ec106a80 msgr2=0x7f15ec19ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:23.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.616+0000 7f15ea59c640 1 --2- 172.21.15.64:0/3113824997 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15ec106a80 0x7f15ec19ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:23.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.616+0000 7f15ea59c640 1 -- 172.21.15.64:0/3113824997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15ec100a80 msgr2=0x7f15ec1a4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:23.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.616+0000 7f15ea59c640 1 --2- 172.21.15.64:0/3113824997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15ec100a80 0x7f15ec1a4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:23.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.616+0000 7f15ea59c640 1 -- 172.21.15.64:0/3113824997 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15ec19f1d0 con 0x7f15ec101420 2024-09-12T03:28:23.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.616+0000 7f15eb59e640 1 --2- 172.21.15.64:0/3113824997 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15ec106a80 0x7f15ec19ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:28:23.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.616+0000 7f15ea59c640 1 --2- 172.21.15.64:0/3113824997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15ec101420 0x7f15ec1a4ba0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f15d400c890 tx=0x7f15d400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:23.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.616+0000 7f15e3fff640 1 -- 172.21.15.64:0/3113824997 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d4004220 con 0x7f15ec101420 2024-09-12T03:28:23.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.616+0000 7f15f1048640 1 -- 172.21.15.64:0/3113824997 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15ec19f4c0 con 0x7f15ec101420 2024-09-12T03:28:23.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.616+0000 7f15e3fff640 1 -- 172.21.15.64:0/3113824997 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15d40043c0 con 0x7f15ec101420 2024-09-12T03:28:23.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.616+0000 7f15e3fff640 1 -- 172.21.15.64:0/3113824997 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d4004a00 con 0x7f15ec101420 2024-09-12T03:28:23.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.616+0000 7f15f1048640 1 -- 172.21.15.64:0/3113824997 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15ec1ab4c0 con 0x7f15ec101420 2024-09-12T03:28:23.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.617+0000 7f15f1048640 1 -- 172.21.15.64:0/3113824997 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15ec1021c0 con 0x7f15ec101420 2024-09-12T03:28:23.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.618+0000 7f15e3fff640 1 -- 172.21.15.64:0/3113824997 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f15d4004ba0 con 0x7f15ec101420 2024-09-12T03:28:23.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.619+0000 7f15e3fff640 1 --2- 172.21.15.64:0/3113824997 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f15d80a2c10 0x7f15d80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:23.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.619+0000 7f15ead9d640 1 --2- 172.21.15.64:0/3113824997 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f15d80a2c10 0x7f15d80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:23.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.619+0000 7f15e3fff640 1 -- 172.21.15.64:0/3113824997 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f15d40cf1f0 con 0x7f15ec101420 2024-09-12T03:28:23.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.620+0000 7f15ead9d640 1 --2- 172.21.15.64:0/3113824997 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f15d80a2c10 0x7f15d80a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f15e4006010 tx=0x7f15e4005f70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:23.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.623+0000 7f15e3fff640 1 -- 172.21.15.64:0/3113824997 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f15d4014030 con 0x7f15ec101420 2024-09-12T03:28:23.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.801+0000 7f15f1048640 1 -- 172.21.15.64:0/3113824997 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f15ec1ab750 con 0x7f15ec101420 2024-09-12T03:28:23.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.802+0000 7f15e3fff640 1 -- 172.21.15.64:0/3113824997 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f15d409bc90 con 0x7f15ec101420 2024-09-12T03:28:23.803 INFO:teuthology.orchestra.run.smithi064.stdout:137438953536 2024-09-12T03:28:23.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.803+0000 7f15f1048640 1 -- 172.21.15.64:0/3113824997 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f15d80a2c10 msgr2=0x7f15d80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:23.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.803+0000 7f15f1048640 1 --2- 172.21.15.64:0/3113824997 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f15d80a2c10 0x7f15d80a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f15e4006010 tx=0x7f15e4005f70 comp rx=0 tx=0).stop 2024-09-12T03:28:23.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.803+0000 7f15f1048640 1 -- 172.21.15.64:0/3113824997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15ec101420 msgr2=0x7f15ec1a4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:23.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.803+0000 7f15f1048640 1 --2- 172.21.15.64:0/3113824997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15ec101420 0x7f15ec1a4ba0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f15d400c890 tx=0x7f15d400cd50 comp rx=0 tx=0).stop 2024-09-12T03:28:23.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.804+0000 7f15f1048640 1 -- 172.21.15.64:0/3113824997 shutdown_connections 2024-09-12T03:28:23.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.804+0000 7f15f1048640 1 --2- 172.21.15.64:0/3113824997 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f15d80a2c10 0x7f15d80a50d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:23.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.804+0000 7f15f1048640 1 --2- 172.21.15.64:0/3113824997 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15ec106a80 0x7f15ec19ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:23.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.804+0000 7f15f1048640 1 --2- 172.21.15.64:0/3113824997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15ec101420 0x7f15ec1a4ba0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:23.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.804+0000 7f15f1048640 1 --2- 172.21.15.64:0/3113824997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15ec100a80 0x7f15ec1a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:23.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.804+0000 7f15f1048640 1 -- 172.21.15.64:0/3113824997 >> 172.21.15.64:0/3113824997 conn(0x7f15ec0fc7a0 msgr2=0x7f15ec06a950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:23.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.804+0000 7f15f1048640 1 -- 172.21.15.64:0/3113824997 shutdown_connections 2024-09-12T03:28:23.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:23.804+0000 7f15f1048640 1 -- 172.21.15.64:0/3113824997 wait complete. 2024-09-12T03:28:24.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:23 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/3113824997' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T03:28:24.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:23 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/3113824997' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T03:28:24.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:23 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/3113824997' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T03:28:24.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:19.948+0000 7f0cf2806640 1 -- 172.21.15.64:0/3177085898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cec100e20 msgr2=0x7f0cec101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:24.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:19.948+0000 7f0cf2806640 1 --2- 172.21.15.64:0/3177085898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cec100e20 0x7f0cec101200 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc009900 tx=0x7f0cdc02a480 comp rx=0 tx=0).stop 2024-09-12T03:28:24.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:19.949+0000 7f0cf2806640 1 -- 172.21.15.64:0/3177085898 shutdown_connections 2024-09-12T03:28:24.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:19.949+0000 7f0cf2806640 1 --2- 172.21.15.64:0/3177085898 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0cec102ef0 0x7f0cec10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:24.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:19.949+0000 7f0cf2806640 1 --2- 172.21.15.64:0/3177085898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0cec102570 0x7f0cec1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:24.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:19.949+0000 7f0cf2806640 1 --2- 172.21.15.64:0/3177085898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cec100e20 0x7f0cec101200 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:24.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:19.949+0000 7f0cf2806640 1 -- 172.21.15.64:0/3177085898 >> 172.21.15.64:0/3177085898 conn(0x7f0cec0fc7d0 msgr2=0x7f0cec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:24.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:19.949+0000 7f0cf2806640 1 -- 172.21.15.64:0/3177085898 shutdown_connections 2024-09-12T03:28:24.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:19.949+0000 7f0cf2806640 1 -- 172.21.15.64:0/3177085898 wait complete. 2024-09-12T03:28:24.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:19.949+0000 7f0cf2806640 1 Processor -- start 2024-09-12T03:28:24.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:19.949+0000 7f0cf2806640 1 -- start start 2024-09-12T03:28:24.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:19.950+0000 7f0cf2806640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cec100e20 0x7f0cec198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:24.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:19.950+0000 7f0cf2806640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0cec102570 0x7f0cec1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:24.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:19.950+0000 7f0cf2806640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0cec102ef0 0x7f0cec19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:24.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:19.950+0000 7f0cf2806640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0cec111f70 con 0x7f0cec100e20 2024-09-12T03:28:24.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:19.950+0000 7f0cf2806640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0cec111df0 con 0x7f0cec102570 2024-09-12T03:28:24.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:19.950+0000 7f0cf2806640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0cec1120f0 con 0x7f0cec102ef0 2024-09-12T03:28:24.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:19.950+0000 7f0cebfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cec100e20 0x7f0cec198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:24.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:19.950+0000 7f0cebfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cec100e20 0x7f0cec198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40864/0 (socket says 172.21.15.64:40864) 2024-09-12T03:28:24.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:19.950+0000 7f0cebfff640 1 -- 172.21.15.64:0/81993587 learned_addr learned my addr 172.21.15.64:0/81993587 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:24.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:19.950+0000 7f0ceb7fe640 1 --2- 172.21.15.64:0/81993587 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0cec102570 0x7f0cec1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:24.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:19.950+0000 7f0cf0da3640 1 --2- 172.21.15.64:0/81993587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0cec102ef0 0x7f0cec19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:24.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:19.950+0000 7f0cebfff640 1 -- 172.21.15.64:0/81993587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0cec102ef0 msgr2=0x7f0cec19cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:24.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:19.950+0000 7f0cebfff640 1 --2- 172.21.15.64:0/81993587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0cec102ef0 0x7f0cec19cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:24.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:19.950+0000 7f0cebfff640 1 -- 172.21.15.64:0/81993587 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0cec102570 msgr2=0x7f0cec1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:24.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:19.950+0000 7f0cebfff640 1 --2- 172.21.15.64:0/81993587 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0cec102570 0x7f0cec1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:24.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:19.950+0000 7f0cebfff640 1 -- 172.21.15.64:0/81993587 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cec19d420 con 0x7f0cec100e20 2024-09-12T03:28:24.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:19.950+0000 7f0cebfff640 1 --2- 172.21.15.64:0/81993587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cec100e20 0x7f0cec198470 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc006010 tx=0x7f0cdc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:24.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:19.950+0000 7f0ce97fa640 1 -- 172.21.15.64:0/81993587 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc03f030 con 0x7f0cec100e20 2024-09-12T03:28:24.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:19.950+0000 7f0ce97fa640 1 -- 172.21.15.64:0/81993587 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0cdc003cb0 con 0x7f0cec100e20 2024-09-12T03:28:24.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:19.950+0000 7f0cf2806640 1 -- 172.21.15.64:0/81993587 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cec19d6b0 con 0x7f0cec100e20 2024-09-12T03:28:24.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:19.951+0000 7f0ce97fa640 1 -- 172.21.15.64:0/81993587 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc035070 con 0x7f0cec100e20 2024-09-12T03:28:24.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:19.951+0000 7f0cf2806640 1 -- 172.21.15.64:0/81993587 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cec1a4ff0 con 0x7f0cec100e20 2024-09-12T03:28:24.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:19.952+0000 7f0cf2806640 1 -- 172.21.15.64:0/81993587 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cec1021b0 con 0x7f0cec100e20 2024-09-12T03:28:24.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:19.953+0000 7f0ce97fa640 1 -- 172.21.15.64:0/81993587 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0cdc042030 con 0x7f0cec100e20 2024-09-12T03:28:24.505 INFO:teuthology.orchestra.run.smithi064.stdout:34359738453 2024-09-12T03:28:24.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:19.954+0000 7f0ce97fa640 1 --2- 172.21.15.64:0/81993587 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f0cd40a2c10 0x7f0cd40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:24.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:19.954+0000 7f0ceb7fe640 1 --2- 172.21.15.64:0/81993587 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f0cd40a2c10 0x7f0cd40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:24.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:19.954+0000 7f0ce97fa640 1 -- 172.21.15.64:0/81993587 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f0cdc0ef9f0 con 0x7f0cec100e20 2024-09-12T03:28:24.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:19.954+0000 7f0ceb7fe640 1 --2- 172.21.15.64:0/81993587 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f0cd40a2c10 0x7f0cd40a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0cec1036c0 tx=0x7f0ce000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:24.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:19.958+0000 7f0ce97fa640 1 -- 172.21.15.64:0/81993587 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0cdc0bbe40 con 0x7f0cec100e20 2024-09-12T03:28:24.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.136+0000 7f0cf2806640 1 -- 172.21.15.64:0/81993587 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f0cec109090 con 0x7f0cec100e20 2024-09-12T03:28:24.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.137+0000 7f0ce97fa640 1 -- 172.21.15.64:0/81993587 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0cdc0c0140 con 0x7f0cec100e20 2024-09-12T03:28:24.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.139+0000 7f0cf2806640 1 -- 172.21.15.64:0/81993587 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f0cd40a2c10 msgr2=0x7f0cd40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:24.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.139+0000 7f0cf2806640 1 --2- 172.21.15.64:0/81993587 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f0cd40a2c10 0x7f0cd40a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0cec1036c0 tx=0x7f0ce000b390 comp rx=0 tx=0).stop 2024-09-12T03:28:24.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.139+0000 7f0cf2806640 1 -- 172.21.15.64:0/81993587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cec100e20 msgr2=0x7f0cec198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:24.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.139+0000 7f0cf2806640 1 --2- 172.21.15.64:0/81993587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cec100e20 0x7f0cec198470 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc006010 tx=0x7f0cdc0042f0 comp rx=0 tx=0).stop 2024-09-12T03:28:24.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.139+0000 7f0cf2806640 1 -- 172.21.15.64:0/81993587 shutdown_connections 2024-09-12T03:28:24.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.139+0000 7f0cf2806640 1 --2- 172.21.15.64:0/81993587 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f0cd40a2c10 0x7f0cd40a50d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:24.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.139+0000 7f0cf2806640 1 --2- 172.21.15.64:0/81993587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0cec102ef0 0x7f0cec19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:24.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.139+0000 7f0cf2806640 1 --2- 172.21.15.64:0/81993587 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0cec102570 0x7f0cec1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:24.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.139+0000 7f0cf2806640 1 --2- 172.21.15.64:0/81993587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cec100e20 0x7f0cec198470 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:24.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.139+0000 7f0cf2806640 1 -- 172.21.15.64:0/81993587 >> 172.21.15.64:0/81993587 conn(0x7f0cec0fc7d0 msgr2=0x7f0cec0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:24.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.139+0000 7f0cf2806640 1 -- 172.21.15.64:0/81993587 shutdown_connections 2024-09-12T03:28:24.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:20.139+0000 7f0cf2806640 1 -- 172.21.15.64:0/81993587 wait complete. 2024-09-12T03:28:25.109 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:24 smithi064 ceph-mon[30952]: pgmap v328: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-12T03:28:25.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:24 smithi115 ceph-mon[34801]: pgmap v328: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-12T03:28:25.143 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738450 got 34359738453 for osd.0 2024-09-12T03:28:25.144 DEBUG:teuthology.parallel:result is None 2024-09-12T03:28:25.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:24 smithi195 ceph-mon[34786]: pgmap v328: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-12T03:28:25.445 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776160 got 292057776162 for osd.10 2024-09-12T03:28:25.445 DEBUG:teuthology.parallel:result is None 2024-09-12T03:28:25.918 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953534 got 137438953536 for osd.4 2024-09-12T03:28:25.918 DEBUG:teuthology.parallel:result is None 2024-09-12T03:28:26.875 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822712 got 154618822715 for osd.5 2024-09-12T03:28:26.875 DEBUG:teuthology.parallel:result is None 2024-09-12T03:28:27.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:26 smithi115 ceph-mon[34801]: pgmap v329: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-09-12T03:28:27.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:26 smithi195 ceph-mon[34786]: pgmap v329: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-09-12T03:28:27.252 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:26 smithi064 ceph-mon[30952]: pgmap v329: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-09-12T03:28:27.370 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574926 got 55834574927 for osd.1 2024-09-12T03:28:27.370 DEBUG:teuthology.parallel:result is None 2024-09-12T03:28:27.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.453+0000 7fa7ab53b640 1 -- 172.21.15.64:0/2508426977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7a4102d40 msgr2=0x7fa7a4103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:27.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.453+0000 7fa7ab53b640 1 --2- 172.21.15.64:0/2508426977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7a4102d40 0x7fa7a4103120 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0009900 tx=0x7fa7a002a480 comp rx=0 tx=0).stop 2024-09-12T03:28:27.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.453+0000 7fa7ab53b640 1 -- 172.21.15.64:0/2508426977 shutdown_connections 2024-09-12T03:28:27.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.453+0000 7fa7ab53b640 1 --2- 172.21.15.64:0/2508426977 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7a410c170 0x7fa7a410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:27.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.453+0000 7fa7ab53b640 1 --2- 172.21.15.64:0/2508426977 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7a4103660 0x7fa7a410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:27.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.453+0000 7fa7ab53b640 1 --2- 172.21.15.64:0/2508426977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7a4102d40 0x7fa7a4103120 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:27.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.453+0000 7fa7ab53b640 1 -- 172.21.15.64:0/2508426977 >> 172.21.15.64:0/2508426977 conn(0x7fa7a40fc7a0 msgr2=0x7fa7a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:27.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.453+0000 7fa7ab53b640 1 -- 172.21.15.64:0/2508426977 shutdown_connections 2024-09-12T03:28:27.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.453+0000 7fa7ab53b640 1 -- 172.21.15.64:0/2508426977 wait complete. 2024-09-12T03:28:27.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.453+0000 7fa7ab53b640 1 Processor -- start 2024-09-12T03:28:27.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.454+0000 7fa7ab53b640 1 -- start start 2024-09-12T03:28:27.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.454+0000 7fa7ab53b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7a4102d40 0x7fa7a4198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:27.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.454+0000 7fa7ab53b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7a4103660 0x7fa7a41988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:27.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.454+0000 7fa7ab53b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7a410c170 0x7fa7a419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:27.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.454+0000 7fa7ab53b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a4111e20 con 0x7fa7a410c170 2024-09-12T03:28:27.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.454+0000 7fa7ab53b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a4111ca0 con 0x7fa7a4102d40 2024-09-12T03:28:27.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.454+0000 7fa7ab53b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a4111fa0 con 0x7fa7a4103660 2024-09-12T03:28:27.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.454+0000 7fa7a9ad8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7a410c170 0x7fa7a419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:27.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.454+0000 7fa7a9ad8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7a410c170 0x7fa7a419cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41000/0 (socket says 172.21.15.64:41000) 2024-09-12T03:28:27.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.454+0000 7fa7a9ad8640 1 -- 172.21.15.64:0/3441868651 learned_addr learned my addr 172.21.15.64:0/3441868651 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:27.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.454+0000 7fa7a92d7640 1 --2- 172.21.15.64:0/3441868651 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7a4102d40 0x7fa7a4198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:27.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.454+0000 7fa7a8ad6640 1 --2- 172.21.15.64:0/3441868651 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7a4103660 0x7fa7a41988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:27.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.454+0000 7fa7a9ad8640 1 -- 172.21.15.64:0/3441868651 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7a4103660 msgr2=0x7fa7a41988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:27.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.454+0000 7fa7a9ad8640 1 --2- 172.21.15.64:0/3441868651 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7a4103660 0x7fa7a41988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:27.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.454+0000 7fa7a9ad8640 1 -- 172.21.15.64:0/3441868651 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7a4102d40 msgr2=0x7fa7a4198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:27.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.454+0000 7fa7a9ad8640 1 --2- 172.21.15.64:0/3441868651 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7a4102d40 0x7fa7a4198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:27.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.454+0000 7fa7a9ad8640 1 -- 172.21.15.64:0/3441868651 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7a419d320 con 0x7fa7a410c170 2024-09-12T03:28:27.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.455+0000 7fa7a9ad8640 1 --2- 172.21.15.64:0/3441868651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7a410c170 0x7fa7a419cc40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa79c00eaa0 tx=0x7fa79c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:27.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.455+0000 7fa79a7fc640 1 -- 172.21.15.64:0/3441868651 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79c00cdb0 con 0x7fa7a410c170 2024-09-12T03:28:27.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.455+0000 7fa79a7fc640 1 -- 172.21.15.64:0/3441868651 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa79c005020 con 0x7fa7a410c170 2024-09-12T03:28:27.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.455+0000 7fa7ab53b640 1 -- 172.21.15.64:0/3441868651 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7a419d610 con 0x7fa7a410c170 2024-09-12T03:28:27.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.455+0000 7fa79a7fc640 1 -- 172.21.15.64:0/3441868651 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79c0116d0 con 0x7fa7a410c170 2024-09-12T03:28:27.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.455+0000 7fa7ab53b640 1 -- 172.21.15.64:0/3441868651 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7a41a4ef0 con 0x7fa7a410c170 2024-09-12T03:28:27.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.456+0000 7fa7ab53b640 1 -- 172.21.15.64:0/3441868651 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7a4103120 con 0x7fa7a410c170 2024-09-12T03:28:27.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.457+0000 7fa79a7fc640 1 -- 172.21.15.64:0/3441868651 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa79c0051d0 con 0x7fa7a410c170 2024-09-12T03:28:27.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.458+0000 7fa79a7fc640 1 --2- 172.21.15.64:0/3441868651 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fa7900a2c10 0x7fa7900a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:27.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.458+0000 7fa7a92d7640 1 --2- 172.21.15.64:0/3441868651 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fa7900a2c10 0x7fa7900a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:27.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.458+0000 7fa79a7fc640 1 -- 172.21.15.64:0/3441868651 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fa79c0cf5e0 con 0x7fa7a410c170 2024-09-12T03:28:27.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.458+0000 7fa7a92d7640 1 --2- 172.21.15.64:0/3441868651 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fa7900a2c10 0x7fa7900a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0031440 tx=0x7fa7a0005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:27.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.462+0000 7fa79a7fc640 1 -- 172.21.15.64:0/3441868651 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa79c016030 con 0x7fa7a410c170 2024-09-12T03:28:27.514 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593780 got 184683593783 for osd.6 2024-09-12T03:28:27.514 DEBUG:teuthology.parallel:result is None 2024-09-12T03:28:27.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.645+0000 7fa7ab53b640 1 -- 172.21.15.64:0/3441868651 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fa7a4106c80 con 0x7fa7a410c170 2024-09-12T03:28:27.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.646+0000 7fa79a7fc640 1 -- 172.21.15.64:0/3441868651 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa79c09ba30 con 0x7fa7a410c170 2024-09-12T03:28:27.647 INFO:teuthology.orchestra.run.smithi064.stdout:270582939689 2024-09-12T03:28:27.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.648+0000 7fa7ab53b640 1 -- 172.21.15.64:0/3441868651 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fa7900a2c10 msgr2=0x7fa7900a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:27.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.648+0000 7fa7ab53b640 1 --2- 172.21.15.64:0/3441868651 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fa7900a2c10 0x7fa7900a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0031440 tx=0x7fa7a0005b50 comp rx=0 tx=0).stop 2024-09-12T03:28:27.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.648+0000 7fa7ab53b640 1 -- 172.21.15.64:0/3441868651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7a410c170 msgr2=0x7fa7a419cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:27.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.648+0000 7fa7ab53b640 1 --2- 172.21.15.64:0/3441868651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7a410c170 0x7fa7a419cc40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa79c00eaa0 tx=0x7fa79c00ef60 comp rx=0 tx=0).stop 2024-09-12T03:28:27.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.648+0000 7fa7ab53b640 1 -- 172.21.15.64:0/3441868651 shutdown_connections 2024-09-12T03:28:27.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.648+0000 7fa7ab53b640 1 --2- 172.21.15.64:0/3441868651 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fa7900a2c10 0x7fa7900a50d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:27.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.648+0000 7fa7ab53b640 1 --2- 172.21.15.64:0/3441868651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7a410c170 0x7fa7a419cc40 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:27.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.648+0000 7fa7ab53b640 1 --2- 172.21.15.64:0/3441868651 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7a4103660 0x7fa7a41988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:27.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.648+0000 7fa7ab53b640 1 --2- 172.21.15.64:0/3441868651 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7a4102d40 0x7fa7a4198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:27.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.648+0000 7fa7ab53b640 1 -- 172.21.15.64:0/3441868651 >> 172.21.15.64:0/3441868651 conn(0x7fa7a40fc7a0 msgr2=0x7fa7a40fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:27.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.648+0000 7fa7ab53b640 1 -- 172.21.15.64:0/3441868651 shutdown_connections 2024-09-12T03:28:27.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:27.648+0000 7fa7ab53b640 1 -- 172.21.15.64:0/3441868651 wait complete. 2024-09-12T03:28:28.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:27 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/3441868651' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T03:28:28.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:27 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/3441868651' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T03:28:28.271 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:27 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/3441868651' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T03:28:28.348 INFO:tasks.cephadm.ceph_manager.ceph:need seq 270582939686 got 270582939689 for osd.9 2024-09-12T03:28:28.349 DEBUG:teuthology.parallel:result is None 2024-09-12T03:28:28.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.606+0000 7f2da7a20640 1 -- 172.21.15.64:0/964872647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2da0102de0 msgr2=0x7f2da01031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:28.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.606+0000 7f2da7a20640 1 --2- 172.21.15.64:0/964872647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2da0102de0 0x7f2da01031c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c009900 tx=0x7f2d9c02a480 comp rx=0 tx=0).stop 2024-09-12T03:28:28.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.606+0000 7f2da7a20640 1 -- 172.21.15.64:0/964872647 shutdown_connections 2024-09-12T03:28:28.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.606+0000 7f2da7a20640 1 --2- 172.21.15.64:0/964872647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2da010c180 0x7f2da010e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:28.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.606+0000 7f2da7a20640 1 --2- 172.21.15.64:0/964872647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2da0103700 0x7f2da010bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:28.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.606+0000 7f2da7a20640 1 --2- 172.21.15.64:0/964872647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2da0102de0 0x7f2da01031c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:28.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.606+0000 7f2da7a20640 1 -- 172.21.15.64:0/964872647 >> 172.21.15.64:0/964872647 conn(0x7f2da00fc780 msgr2=0x7f2da00feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:28.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.606+0000 7f2da7a20640 1 -- 172.21.15.64:0/964872647 shutdown_connections 2024-09-12T03:28:28.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.606+0000 7f2da7a20640 1 -- 172.21.15.64:0/964872647 wait complete. 2024-09-12T03:28:28.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.607+0000 7f2da7a20640 1 Processor -- start 2024-09-12T03:28:28.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.607+0000 7f2da7a20640 1 -- start start 2024-09-12T03:28:28.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.607+0000 7f2da7a20640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2da0102de0 0x7f2da019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:28.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.607+0000 7f2da7a20640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2da0103700 0x7f2da019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:28.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.607+0000 7f2da7a20640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2da010c180 0x7f2da019ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:28.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.607+0000 7f2da7a20640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2da0111e50 con 0x7f2da0102de0 2024-09-12T03:28:28.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.607+0000 7f2da7a20640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2da0111cd0 con 0x7f2da0103700 2024-09-12T03:28:28.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.607+0000 7f2da7a20640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2da0111fd0 con 0x7f2da010c180 2024-09-12T03:28:28.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.607+0000 7f2da57bc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2da0102de0 0x7f2da019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:28.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.607+0000 7f2da57bc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2da0102de0 0x7f2da019a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39834/0 (socket says 172.21.15.64:39834) 2024-09-12T03:28:28.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.607+0000 7f2da57bc640 1 -- 172.21.15.64:0/3726174154 learned_addr learned my addr 172.21.15.64:0/3726174154 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:28.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.608+0000 7f2da5fbd640 1 --2- 172.21.15.64:0/3726174154 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2da010c180 0x7f2da019ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:28.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.608+0000 7f2da4fbb640 1 --2- 172.21.15.64:0/3726174154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2da0103700 0x7f2da019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:28.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.608+0000 7f2da57bc640 1 -- 172.21.15.64:0/3726174154 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2da010c180 msgr2=0x7f2da019ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:28.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.608+0000 7f2da57bc640 1 --2- 172.21.15.64:0/3726174154 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2da010c180 0x7f2da019ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:28.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.608+0000 7f2da57bc640 1 -- 172.21.15.64:0/3726174154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2da0103700 msgr2=0x7f2da019aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:28.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.608+0000 7f2da57bc640 1 --2- 172.21.15.64:0/3726174154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2da0103700 0x7f2da019aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:28.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.608+0000 7f2da57bc640 1 -- 172.21.15.64:0/3726174154 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da019f500 con 0x7f2da0102de0 2024-09-12T03:28:28.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.608+0000 7f2da4fbb640 1 --2- 172.21.15.64:0/3726174154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2da0103700 0x7f2da019aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:28:28.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.608+0000 7f2da57bc640 1 --2- 172.21.15.64:0/3726174154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2da0102de0 0x7f2da019a550 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c005ab0 tx=0x7f2d9c003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:28.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.608+0000 7f2d967fc640 1 -- 172.21.15.64:0/3726174154 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c03f070 con 0x7f2da0102de0 2024-09-12T03:28:28.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.608+0000 7f2da7a20640 1 -- 172.21.15.64:0/3726174154 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2da019f790 con 0x7f2da0102de0 2024-09-12T03:28:28.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.608+0000 7f2d967fc640 1 -- 172.21.15.64:0/3726174154 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d9c031d60 con 0x7f2da0102de0 2024-09-12T03:28:28.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.608+0000 7f2d967fc640 1 -- 172.21.15.64:0/3726174154 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c035030 con 0x7f2da0102de0 2024-09-12T03:28:28.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.608+0000 7f2da7a20640 1 -- 172.21.15.64:0/3726174154 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2da01a70d0 con 0x7f2da0102de0 2024-09-12T03:28:28.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.610+0000 7f2da7a20640 1 -- 172.21.15.64:0/3726174154 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2da01031c0 con 0x7f2da0102de0 2024-09-12T03:28:28.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.611+0000 7f2d967fc640 1 -- 172.21.15.64:0/3726174154 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2d9c042070 con 0x7f2da0102de0 2024-09-12T03:28:28.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.611+0000 7f2d967fc640 1 --2- 172.21.15.64:0/3726174154 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f2d8c0a2c10 0x7f2d8c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:28.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.612+0000 7f2da4fbb640 1 --2- 172.21.15.64:0/3726174154 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f2d8c0a2c10 0x7f2d8c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:28.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.612+0000 7f2d967fc640 1 -- 172.21.15.64:0/3726174154 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f2d9c0efe30 con 0x7f2da0102de0 2024-09-12T03:28:28.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.612+0000 7f2da4fbb640 1 --2- 172.21.15.64:0/3726174154 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f2d8c0a2c10 0x7f2d8c0a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2d900058c0 tx=0x7f2d90005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:28.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.616+0000 7f2d967fc640 1 -- 172.21.15.64:0/3726174154 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2d9c0bc8d0 con 0x7f2da0102de0 2024-09-12T03:28:28.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.794+0000 7f2da7a20640 1 -- 172.21.15.64:0/3726174154 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f2da01a72a0 con 0x7f2da0102de0 2024-09-12T03:28:28.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.794+0000 7f2d967fc640 1 -- 172.21.15.64:0/3726174154 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2d9c0bc1d0 con 0x7f2da0102de0 2024-09-12T03:28:28.795 INFO:teuthology.orchestra.run.smithi064.stdout:206158430259 2024-09-12T03:28:28.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.796+0000 7f2da7a20640 1 -- 172.21.15.64:0/3726174154 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f2d8c0a2c10 msgr2=0x7f2d8c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:28.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.796+0000 7f2da7a20640 1 --2- 172.21.15.64:0/3726174154 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f2d8c0a2c10 0x7f2d8c0a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2d900058c0 tx=0x7f2d90005850 comp rx=0 tx=0).stop 2024-09-12T03:28:28.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.796+0000 7f2da7a20640 1 -- 172.21.15.64:0/3726174154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2da0102de0 msgr2=0x7f2da019a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:28.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.796+0000 7f2da7a20640 1 --2- 172.21.15.64:0/3726174154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2da0102de0 0x7f2da019a550 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c005ab0 tx=0x7f2d9c003cd0 comp rx=0 tx=0).stop 2024-09-12T03:28:28.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.796+0000 7f2da7a20640 1 -- 172.21.15.64:0/3726174154 shutdown_connections 2024-09-12T03:28:28.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.796+0000 7f2da7a20640 1 --2- 172.21.15.64:0/3726174154 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f2d8c0a2c10 0x7f2d8c0a50d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:28.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.796+0000 7f2da7a20640 1 --2- 172.21.15.64:0/3726174154 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2da010c180 0x7f2da019ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:28.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.796+0000 7f2da7a20640 1 --2- 172.21.15.64:0/3726174154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2da0103700 0x7f2da019aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:28.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.796+0000 7f2da7a20640 1 --2- 172.21.15.64:0/3726174154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2da0102de0 0x7f2da019a550 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:28.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.796+0000 7f2da7a20640 1 -- 172.21.15.64:0/3726174154 >> 172.21.15.64:0/3726174154 conn(0x7f2da00fc780 msgr2=0x7f2da01009d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:28.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.796+0000 7f2da7a20640 1 -- 172.21.15.64:0/3726174154 shutdown_connections 2024-09-12T03:28:28.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:28.796+0000 7f2da7a20640 1 -- 172.21.15.64:0/3726174154 wait complete. 2024-09-12T03:28:29.173 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:28 smithi064 ceph-mon[30952]: pgmap v330: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-12T03:28:29.174 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:28 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/3726174154' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T03:28:29.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:28 smithi195 ceph-mon[34786]: pgmap v330: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-12T03:28:29.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:28 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/3726174154' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T03:28:29.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:28 smithi115 ceph-mon[34801]: pgmap v330: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-12T03:28:29.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:28 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/3726174154' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T03:28:29.521 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430256 got 206158430259 for osd.7 2024-09-12T03:28:29.522 DEBUG:teuthology.parallel:result is None 2024-09-12T03:28:29.522 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-12T03:28:29.522 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T03:28:29.784 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:30.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.276+0000 7f20e0839640 1 -- 172.21.15.64:0/1469016172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20dc106a80 msgr2=0x7f20dc106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:30.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.276+0000 7f20e0839640 1 --2- 172.21.15.64:0/1469016172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20dc106a80 0x7f20dc106e60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f20cc009930 tx=0x7f20cc02a420 comp rx=0 tx=0).stop 2024-09-12T03:28:30.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.276+0000 7f20e0839640 1 -- 172.21.15.64:0/1469016172 shutdown_connections 2024-09-12T03:28:30.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.276+0000 7f20e0839640 1 --2- 172.21.15.64:0/1469016172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20dc101420 0x7f20dc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:30.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.276+0000 7f20e0839640 1 --2- 172.21.15.64:0/1469016172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20dc100a80 0x7f20dc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:30.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.276+0000 7f20e0839640 1 --2- 172.21.15.64:0/1469016172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20dc106a80 0x7f20dc106e60 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:30.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.276+0000 7f20e0839640 1 -- 172.21.15.64:0/1469016172 >> 172.21.15.64:0/1469016172 conn(0x7f20dc0fc7a0 msgr2=0x7f20dc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:30.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.276+0000 7f20e0839640 1 -- 172.21.15.64:0/1469016172 shutdown_connections 2024-09-12T03:28:30.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.276+0000 7f20e0839640 1 -- 172.21.15.64:0/1469016172 wait complete. 2024-09-12T03:28:30.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.277+0000 7f20e0839640 1 Processor -- start 2024-09-12T03:28:30.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.277+0000 7f20e0839640 1 -- start start 2024-09-12T03:28:30.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.277+0000 7f20e0839640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20dc100a80 0x7f20dc19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:30.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.277+0000 7f20e0839640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20dc101420 0x7f20dc19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:30.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.277+0000 7f20e0839640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20dc106a80 0x7f20dc19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:30.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.277+0000 7f20e0839640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc110d50 con 0x7f20dc101420 2024-09-12T03:28:30.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.277+0000 7f20e0839640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc110bd0 con 0x7f20dc106a80 2024-09-12T03:28:30.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.277+0000 7f20e0839640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc110ed0 con 0x7f20dc100a80 2024-09-12T03:28:30.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.277+0000 7f20d9d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20dc101420 0x7f20dc19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:30.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.277+0000 7f20d9d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20dc101420 0x7f20dc19ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39864/0 (socket says 172.21.15.64:39864) 2024-09-12T03:28:30.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.277+0000 7f20d9d9b640 1 -- 172.21.15.64:0/2953433927 learned_addr learned my addr 172.21.15.64:0/2953433927 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:30.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.278+0000 7f20dad9d640 1 --2- 172.21.15.64:0/2953433927 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20dc106a80 0x7f20dc19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:30.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.278+0000 7f20da59c640 1 --2- 172.21.15.64:0/2953433927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20dc100a80 0x7f20dc19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:30.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.278+0000 7f20d9d9b640 1 -- 172.21.15.64:0/2953433927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20dc100a80 msgr2=0x7f20dc19a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:30.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.278+0000 7f20d9d9b640 1 --2- 172.21.15.64:0/2953433927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20dc100a80 0x7f20dc19a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:30.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.278+0000 7f20d9d9b640 1 -- 172.21.15.64:0/2953433927 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20dc106a80 msgr2=0x7f20dc19eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:30.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.278+0000 7f20d9d9b640 1 --2- 172.21.15.64:0/2953433927 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20dc106a80 0x7f20dc19eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:30.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.278+0000 7f20d9d9b640 1 -- 172.21.15.64:0/2953433927 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20dc19f590 con 0x7f20dc101420 2024-09-12T03:28:30.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.278+0000 7f20dad9d640 1 --2- 172.21.15.64:0/2953433927 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20dc106a80 0x7f20dc19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:28:30.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.278+0000 7f20d9d9b640 1 --2- 172.21.15.64:0/2953433927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20dc101420 0x7f20dc19ab20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f20d000d950 tx=0x7f20d000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:30.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.278+0000 7f20cb7fe640 1 -- 172.21.15.64:0/2953433927 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d0017070 con 0x7f20dc101420 2024-09-12T03:28:30.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.278+0000 7f20cb7fe640 1 -- 172.21.15.64:0/2953433927 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20d0009c30 con 0x7f20dc101420 2024-09-12T03:28:30.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.278+0000 7f20e0839640 1 -- 172.21.15.64:0/2953433927 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20dc19f880 con 0x7f20dc101420 2024-09-12T03:28:30.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.278+0000 7f20cb7fe640 1 -- 172.21.15.64:0/2953433927 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d000f620 con 0x7f20dc101420 2024-09-12T03:28:30.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.278+0000 7f20e0839640 1 -- 172.21.15.64:0/2953433927 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20dc1a7160 con 0x7f20dc101420 2024-09-12T03:28:30.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.279+0000 7f20e0839640 1 -- 172.21.15.64:0/2953433927 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20dc100ee0 con 0x7f20dc101420 2024-09-12T03:28:30.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.280+0000 7f20cb7fe640 1 -- 172.21.15.64:0/2953433927 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f20d00109a0 con 0x7f20dc101420 2024-09-12T03:28:30.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.281+0000 7f20cb7fe640 1 --2- 172.21.15.64:0/2953433927 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f20c40a2c10 0x7f20c40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:30.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.281+0000 7f20da59c640 1 --2- 172.21.15.64:0/2953433927 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f20c40a2c10 0x7f20c40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:30.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.281+0000 7f20cb7fe640 1 -- 172.21.15.64:0/2953433927 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f20d00cf220 con 0x7f20dc101420 2024-09-12T03:28:30.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.281+0000 7f20da59c640 1 --2- 172.21.15.64:0/2953433927 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f20c40a2c10 0x7f20c40a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f20cc000c00 tx=0x7f20cc032c80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:30.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.285+0000 7f20cb7fe640 1 -- 172.21.15.64:0/2953433927 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f20d0009de0 con 0x7f20dc101420 2024-09-12T03:28:30.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.458+0000 7f20e0839640 1 -- 172.21.15.64:0/2953433927 --> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f20dc06af20 con 0x7f20c40a2c10 2024-09-12T03:28:30.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.467+0000 7f20cb7fe640 1 -- 172.21.15.64:0/2953433927 <== mgr.14150 v2:172.21.15.64:6800/759931848 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340596 (secure 0 0 0) 0x7f20dc06af20 con 0x7f20c40a2c10 2024-09-12T03:28:30.472 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:28:30.474 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-09-12T03:28:30.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.470+0000 7f20e0839640 1 -- 172.21.15.64:0/2953433927 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f20c40a2c10 msgr2=0x7f20c40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:30.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.470+0000 7f20e0839640 1 --2- 172.21.15.64:0/2953433927 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f20c40a2c10 0x7f20c40a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f20cc000c00 tx=0x7f20cc032c80 comp rx=0 tx=0).stop 2024-09-12T03:28:30.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.470+0000 7f20e0839640 1 -- 172.21.15.64:0/2953433927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20dc101420 msgr2=0x7f20dc19ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:30.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.470+0000 7f20e0839640 1 --2- 172.21.15.64:0/2953433927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20dc101420 0x7f20dc19ab20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f20d000d950 tx=0x7f20d000de10 comp rx=0 tx=0).stop 2024-09-12T03:28:30.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.470+0000 7f20e0839640 1 -- 172.21.15.64:0/2953433927 shutdown_connections 2024-09-12T03:28:30.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.470+0000 7f20e0839640 1 --2- 172.21.15.64:0/2953433927 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f20c40a2c10 0x7f20c40a50d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:30.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.470+0000 7f20e0839640 1 --2- 172.21.15.64:0/2953433927 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20dc106a80 0x7f20dc19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:30.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.470+0000 7f20e0839640 1 --2- 172.21.15.64:0/2953433927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20dc101420 0x7f20dc19ab20 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:30.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.470+0000 7f20e0839640 1 --2- 172.21.15.64:0/2953433927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20dc100a80 0x7f20dc19a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:30.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.470+0000 7f20e0839640 1 -- 172.21.15.64:0/2953433927 >> 172.21.15.64:0/2953433927 conn(0x7f20dc0fc7a0 msgr2=0x7f20dc06a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:30.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.470+0000 7f20e0839640 1 -- 172.21.15.64:0/2953433927 shutdown_connections 2024-09-12T03:28:30.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:30.471+0000 7f20e0839640 1 -- 172.21.15.64:0/2953433927 wait complete. 2024-09-12T03:28:31.125 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:30 smithi064 ceph-mon[30952]: pgmap v331: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-12T03:28:31.157 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-09-12T03:28:29.515452+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656681,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":475,"num_read_kb":359,"num_write":356,"num_write_kb":3057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":285,"ondisk_log_size":285,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365580,"kb_used_data":12104,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576244,"statfs":{"total":1151940427776,"available":1151566073856,"internally_reserved":0,"allocated":12394496,"data_stored":8046017,"data_compressed":390576,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":33,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004174"},"pg_stats":[{"pgid":"2.1f","version":"88'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.844104+0000","last_change":"2024-09-12T03:26:56.956415+0000","last_active":"2024-09-12T03:28:08.844104+0000","last_peered":"2024-09-12T03:28:08.844104+0000","last_clean":"2024-09-12T03:28:08.844104+0000","last_became_active":"2024-09-12T03:26:56.956089+0000","last_became_peered":"2024-09-12T03:26:56.956089+0000","last_unstale":"2024-09-12T03:28:08.844104+0000","last_undegraded":"2024-09-12T03:28:08.844104+0000","last_fullsized":"2024-09-12T03:28:08.844104+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:32:22.089322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.928111+0000","last_change":"2024-09-12T03:26:59.024102+0000","last_active":"2024-09-12T03:28:08.928111+0000","last_peered":"2024-09-12T03:28:08.928111+0000","last_clean":"2024-09-12T03:28:08.928111+0000","last_became_active":"2024-09-12T03:26:59.023795+0000","last_became_peered":"2024-09-12T03:26:59.023795+0000","last_unstale":"2024-09-12T03:28:08.928111+0000","last_undegraded":"2024-09-12T03:28:08.928111+0000","last_fullsized":"2024-09-12T03:28:08.928111+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:25:55.392622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.842900+0000","last_change":"2024-09-12T03:27:06.052565+0000","last_active":"2024-09-12T03:28:08.842900+0000","last_peered":"2024-09-12T03:28:08.842900+0000","last_clean":"2024-09-12T03:28:08.842900+0000","last_became_active":"2024-09-12T03:27:06.052209+0000","last_became_peered":"2024-09-12T03:27:06.052209+0000","last_unstale":"2024-09-12T03:28:08.842900+0000","last_undegraded":"2024-09-12T03:28:08.842900+0000","last_fullsized":"2024-09-12T03:28:08.842900+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:24:28.153740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.928248+0000","last_change":"2024-09-12T03:26:57.018418+0000","last_active":"2024-09-12T03:28:08.928248+0000","last_peered":"2024-09-12T03:28:08.928248+0000","last_clean":"2024-09-12T03:28:08.928248+0000","last_became_active":"2024-09-12T03:26:57.018081+0000","last_became_peered":"2024-09-12T03:26:57.018081+0000","last_unstale":"2024-09-12T03:28:08.928248+0000","last_undegraded":"2024-09-12T03:28:08.928248+0000","last_fullsized":"2024-09-12T03:28:08.928248+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:19.954332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.846845+0000","last_change":"2024-09-12T03:26:59.019632+0000","last_active":"2024-09-12T03:28:08.846845+0000","last_peered":"2024-09-12T03:28:08.846845+0000","last_clean":"2024-09-12T03:28:08.846845+0000","last_became_active":"2024-09-12T03:26:59.016847+0000","last_became_peered":"2024-09-12T03:26:59.016847+0000","last_unstale":"2024-09-12T03:28:08.846845+0000","last_undegraded":"2024-09-12T03:28:08.846845+0000","last_fullsized":"2024-09-12T03:28:08.846845+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:32:28.261954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.845288+0000","last_change":"2024-09-12T03:27:06.045628+0000","last_active":"2024-09-12T03:28:08.845288+0000","last_peered":"2024-09-12T03:28:08.845288+0000","last_clean":"2024-09-12T03:28:08.845288+0000","last_became_active":"2024-09-12T03:27:06.045141+0000","last_became_peered":"2024-09-12T03:27:06.045141+0000","last_unstale":"2024-09-12T03:28:08.845288+0000","last_undegraded":"2024-09-12T03:28:08.845288+0000","last_fullsized":"2024-09-12T03:28:08.845288+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:10:26.154446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.844027+0000","last_change":"2024-09-12T03:26:56.958615+0000","last_active":"2024-09-12T03:28:08.844027+0000","last_peered":"2024-09-12T03:28:08.844027+0000","last_clean":"2024-09-12T03:28:08.844027+0000","last_became_active":"2024-09-12T03:26:56.958295+0000","last_became_peered":"2024-09-12T03:26:56.958295+0000","last_unstale":"2024-09-12T03:28:08.844027+0000","last_undegraded":"2024-09-12T03:28:08.844027+0000","last_fullsized":"2024-09-12T03:28:08.844027+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:13:09.031983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.843042+0000","last_change":"2024-09-12T03:26:59.015376+0000","last_active":"2024-09-12T03:28:08.843042+0000","last_peered":"2024-09-12T03:28:08.843042+0000","last_clean":"2024-09-12T03:28:08.843042+0000","last_became_active":"2024-09-12T03:26:59.015055+0000","last_became_peered":"2024-09-12T03:26:59.015055+0000","last_unstale":"2024-09-12T03:28:08.843042+0000","last_undegraded":"2024-09-12T03:28:08.843042+0000","last_fullsized":"2024-09-12T03:28:08.843042+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:44:38.077558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.846460+0000","last_change":"2024-09-12T03:27:06.042785+0000","last_active":"2024-09-12T03:28:08.846460+0000","last_peered":"2024-09-12T03:28:08.846460+0000","last_clean":"2024-09-12T03:28:08.846460+0000","last_became_active":"2024-09-12T03:27:06.042379+0000","last_became_peered":"2024-09-12T03:27:06.042379+0000","last_unstale":"2024-09-12T03:28:08.846460+0000","last_undegraded":"2024-09-12T03:28:08.846460+0000","last_fullsized":"2024-09-12T03:28:08.846460+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:08:51.683470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.844454+0000","last_change":"2024-09-12T03:26:56.955846+0000","last_active":"2024-09-12T03:28:08.844454+0000","last_peered":"2024-09-12T03:28:08.844454+0000","last_clean":"2024-09-12T03:28:08.844454+0000","last_became_active":"2024-09-12T03:26:56.955476+0000","last_became_peered":"2024-09-12T03:26:56.955476+0000","last_unstale":"2024-09-12T03:28:08.844454+0000","last_undegraded":"2024-09-12T03:28:08.844454+0000","last_fullsized":"2024-09-12T03:28:08.844454+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:10:46.811039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.841131+0000","last_change":"2024-09-12T03:26:59.021498+0000","last_active":"2024-09-12T03:28:08.841131+0000","last_peered":"2024-09-12T03:28:08.841131+0000","last_clean":"2024-09-12T03:28:08.841131+0000","last_became_active":"2024-09-12T03:26:59.021196+0000","last_became_peered":"2024-09-12T03:26:59.021196+0000","last_unstale":"2024-09-12T03:28:08.841131+0000","last_undegraded":"2024-09-12T03:28:08.841131+0000","last_fullsized":"2024-09-12T03:28:08.841131+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:13:17.537648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.845349+0000","last_change":"2024-09-12T03:27:06.040597+0000","last_active":"2024-09-12T03:28:08.845349+0000","last_peered":"2024-09-12T03:28:08.845349+0000","last_clean":"2024-09-12T03:28:08.845349+0000","last_became_active":"2024-09-12T03:27:06.040149+0000","last_became_peered":"2024-09-12T03:27:06.040149+0000","last_unstale":"2024-09-12T03:28:08.845349+0000","last_undegraded":"2024-09-12T03:28:08.845349+0000","last_fullsized":"2024-09-12T03:28:08.845349+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:27.192367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.843662+0000","last_change":"2024-09-12T03:26:59.020542+0000","last_active":"2024-09-12T03:28:08.843662+0000","last_peered":"2024-09-12T03:28:08.843662+0000","last_clean":"2024-09-12T03:28:08.843662+0000","last_became_active":"2024-09-12T03:26:59.020189+0000","last_became_peered":"2024-09-12T03:26:59.020189+0000","last_unstale":"2024-09-12T03:28:08.843662+0000","last_undegraded":"2024-09-12T03:28:08.843662+0000","last_fullsized":"2024-09-12T03:28:08.843662+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:06:12.761331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.842698+0000","last_change":"2024-09-12T03:26:56.958320+0000","last_active":"2024-09-12T03:28:08.842698+0000","last_peered":"2024-09-12T03:28:08.842698+0000","last_clean":"2024-09-12T03:28:08.842698+0000","last_became_active":"2024-09-12T03:26:56.956965+0000","last_became_peered":"2024-09-12T03:26:56.956965+0000","last_unstale":"2024-09-12T03:28:08.842698+0000","last_undegraded":"2024-09-12T03:28:08.842698+0000","last_fullsized":"2024-09-12T03:28:08.842698+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:46.685994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.845734+0000","last_change":"2024-09-12T03:27:06.189973+0000","last_active":"2024-09-12T03:28:08.845734+0000","last_peered":"2024-09-12T03:28:08.845734+0000","last_clean":"2024-09-12T03:28:08.845734+0000","last_became_active":"2024-09-12T03:27:06.189454+0000","last_became_peered":"2024-09-12T03:27:06.189454+0000","last_unstale":"2024-09-12T03:28:08.845734+0000","last_undegraded":"2024-09-12T03:28:08.845734+0000","last_fullsized":"2024-09-12T03:28:08.845734+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:25.130566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.927982+0000","last_change":"2024-09-12T03:26:59.014586+0000","last_active":"2024-09-12T03:28:08.927982+0000","last_peered":"2024-09-12T03:28:08.927982+0000","last_clean":"2024-09-12T03:28:08.927982+0000","last_became_active":"2024-09-12T03:26:59.013764+0000","last_became_peered":"2024-09-12T03:26:59.013764+0000","last_unstale":"2024-09-12T03:28:08.927982+0000","last_undegraded":"2024-09-12T03:28:08.927982+0000","last_fullsized":"2024-09-12T03:28:08.927982+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:55:34.136770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.842601+0000","last_change":"2024-09-12T03:26:56.960415+0000","last_active":"2024-09-12T03:28:08.842601+0000","last_peered":"2024-09-12T03:28:08.842601+0000","last_clean":"2024-09-12T03:28:08.842601+0000","last_became_active":"2024-09-12T03:26:56.960043+0000","last_became_peered":"2024-09-12T03:26:56.960043+0000","last_unstale":"2024-09-12T03:28:08.842601+0000","last_undegraded":"2024-09-12T03:28:08.842601+0000","last_fullsized":"2024-09-12T03:28:08.842601+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:02:12.209929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.845523+0000","last_change":"2024-09-12T03:27:06.047002+0000","last_active":"2024-09-12T03:28:08.845523+0000","last_peered":"2024-09-12T03:28:08.845523+0000","last_clean":"2024-09-12T03:28:08.845523+0000","last_became_active":"2024-09-12T03:27:06.046520+0000","last_became_peered":"2024-09-12T03:27:06.046520+0000","last_unstale":"2024-09-12T03:28:08.845523+0000","last_undegraded":"2024-09-12T03:28:08.845523+0000","last_fullsized":"2024-09-12T03:28:08.845523+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:30.332831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.928022+0000","last_change":"2024-09-12T03:26:59.016811+0000","last_active":"2024-09-12T03:28:08.928022+0000","last_peered":"2024-09-12T03:28:08.928022+0000","last_clean":"2024-09-12T03:28:08.928022+0000","last_became_active":"2024-09-12T03:26:59.014667+0000","last_became_peered":"2024-09-12T03:26:59.014667+0000","last_unstale":"2024-09-12T03:28:08.928022+0000","last_undegraded":"2024-09-12T03:28:08.928022+0000","last_fullsized":"2024-09-12T03:28:08.928022+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:28:51.128895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.927764+0000","last_change":"2024-09-12T03:26:56.956177+0000","last_active":"2024-09-12T03:28:08.927764+0000","last_peered":"2024-09-12T03:28:08.927764+0000","last_clean":"2024-09-12T03:28:08.927764+0000","last_became_active":"2024-09-12T03:26:56.955730+0000","last_became_peered":"2024-09-12T03:26:56.955730+0000","last_unstale":"2024-09-12T03:28:08.927764+0000","last_undegraded":"2024-09-12T03:28:08.927764+0000","last_fullsized":"2024-09-12T03:28:08.927764+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:05:18.890917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.843898+0000","last_change":"2024-09-12T03:27:06.050894+0000","last_active":"2024-09-12T03:28:08.843898+0000","last_peered":"2024-09-12T03:28:08.843898+0000","last_clean":"2024-09-12T03:28:08.843898+0000","last_became_active":"2024-09-12T03:27:06.050440+0000","last_became_peered":"2024-09-12T03:27:06.050440+0000","last_unstale":"2024-09-12T03:28:08.843898+0000","last_undegraded":"2024-09-12T03:28:08.843898+0000","last_fullsized":"2024-09-12T03:28:08.843898+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:31:25.559665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.842439+0000","last_change":"2024-09-12T03:26:59.018167+0000","last_active":"2024-09-12T03:28:08.842439+0000","last_peered":"2024-09-12T03:28:08.842439+0000","last_clean":"2024-09-12T03:28:08.842439+0000","last_became_active":"2024-09-12T03:26:59.017718+0000","last_became_peered":"2024-09-12T03:26:59.017718+0000","last_unstale":"2024-09-12T03:28:08.842439+0000","last_undegraded":"2024-09-12T03:28:08.842439+0000","last_fullsized":"2024-09-12T03:28:08.842439+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:36:12.742570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.842821+0000","last_change":"2024-09-12T03:26:57.018829+0000","last_active":"2024-09-12T03:28:08.842821+0000","last_peered":"2024-09-12T03:28:08.842821+0000","last_clean":"2024-09-12T03:28:08.842821+0000","last_became_active":"2024-09-12T03:26:57.018467+0000","last_became_peered":"2024-09-12T03:26:57.018467+0000","last_unstale":"2024-09-12T03:28:08.842821+0000","last_undegraded":"2024-09-12T03:28:08.842821+0000","last_fullsized":"2024-09-12T03:28:08.842821+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:33:37.318703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.845899+0000","last_change":"2024-09-12T03:27:06.190560+0000","last_active":"2024-09-12T03:28:08.845899+0000","last_peered":"2024-09-12T03:28:08.845899+0000","last_clean":"2024-09-12T03:28:08.845899+0000","last_became_active":"2024-09-12T03:27:06.190008+0000","last_became_peered":"2024-09-12T03:27:06.190008+0000","last_unstale":"2024-09-12T03:28:08.845899+0000","last_undegraded":"2024-09-12T03:28:08.845899+0000","last_fullsized":"2024-09-12T03:28:08.845899+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:14.423372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.846689+0000","last_change":"2024-09-12T03:26:59.022149+0000","last_active":"2024-09-12T03:28:08.846689+0000","last_peered":"2024-09-12T03:28:08.846689+0000","last_clean":"2024-09-12T03:28:08.846689+0000","last_became_active":"2024-09-12T03:26:59.021830+0000","last_became_peered":"2024-09-12T03:26:59.021830+0000","last_unstale":"2024-09-12T03:28:08.846689+0000","last_undegraded":"2024-09-12T03:28:08.846689+0000","last_fullsized":"2024-09-12T03:28:08.846689+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:07.096908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.846205+0000","last_change":"2024-09-12T03:26:57.018335+0000","last_active":"2024-09-12T03:28:08.846205+0000","last_peered":"2024-09-12T03:28:08.846205+0000","last_clean":"2024-09-12T03:28:08.846205+0000","last_became_active":"2024-09-12T03:26:57.017993+0000","last_became_peered":"2024-09-12T03:26:57.017993+0000","last_unstale":"2024-09-12T03:28:08.846205+0000","last_undegraded":"2024-09-12T03:28:08.846205+0000","last_fullsized":"2024-09-12T03:28:08.846205+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:27:52.828836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.842912+0000","last_change":"2024-09-12T03:27:06.051146+0000","last_active":"2024-09-12T03:28:08.842912+0000","last_peered":"2024-09-12T03:28:08.842912+0000","last_clean":"2024-09-12T03:28:08.842912+0000","last_became_active":"2024-09-12T03:27:06.050599+0000","last_became_peered":"2024-09-12T03:27:06.050599+0000","last_unstale":"2024-09-12T03:28:08.842912+0000","last_undegraded":"2024-09-12T03:28:08.842912+0000","last_fullsized":"2024-09-12T03:28:08.842912+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:17:57.543994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.928221+0000","last_change":"2024-09-12T03:26:59.024832+0000","last_active":"2024-09-12T03:28:08.928221+0000","last_peered":"2024-09-12T03:28:08.928221+0000","last_clean":"2024-09-12T03:28:08.928221+0000","last_became_active":"2024-09-12T03:26:59.024473+0000","last_became_peered":"2024-09-12T03:26:59.024473+0000","last_unstale":"2024-09-12T03:28:08.928221+0000","last_undegraded":"2024-09-12T03:28:08.928221+0000","last_fullsized":"2024-09-12T03:28:08.928221+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:05:54.568496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'14","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.844093+0000","last_change":"2024-09-12T03:26:56.955213+0000","last_active":"2024-09-12T03:28:08.844093+0000","last_peered":"2024-09-12T03:28:08.844093+0000","last_clean":"2024-09-12T03:28:08.844093+0000","last_became_active":"2024-09-12T03:26:56.954746+0000","last_became_peered":"2024-09-12T03:26:56.954746+0000","last_unstale":"2024-09-12T03:28:08.844093+0000","last_undegraded":"2024-09-12T03:28:08.844093+0000","last_fullsized":"2024-09-12T03:28:08.844093+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:03.303931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.927417+0000","last_change":"2024-09-12T03:27:06.190398+0000","last_active":"2024-09-12T03:28:08.927417+0000","last_peered":"2024-09-12T03:28:08.927417+0000","last_clean":"2024-09-12T03:28:08.927417+0000","last_became_active":"2024-09-12T03:27:06.189861+0000","last_became_peered":"2024-09-12T03:27:06.189861+0000","last_unstale":"2024-09-12T03:28:08.927417+0000","last_undegraded":"2024-09-12T03:28:08.927417+0000","last_fullsized":"2024-09-12T03:28:08.927417+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:36:57.366931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.841792+0000","last_change":"2024-09-12T03:26:59.012702+0000","last_active":"2024-09-12T03:28:08.841792+0000","last_peered":"2024-09-12T03:28:08.841792+0000","last_clean":"2024-09-12T03:28:08.841792+0000","last_became_active":"2024-09-12T03:26:59.012421+0000","last_became_peered":"2024-09-12T03:26:59.012421+0000","last_unstale":"2024-09-12T03:28:08.841792+0000","last_undegraded":"2024-09-12T03:28:08.841792+0000","last_fullsized":"2024-09-12T03:28:08.841792+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:21:23.866131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.844078+0000","last_change":"2024-09-12T03:26:57.019744+0000","last_active":"2024-09-12T03:28:08.844078+0000","last_peered":"2024-09-12T03:28:08.844078+0000","last_clean":"2024-09-12T03:28:08.844078+0000","last_became_active":"2024-09-12T03:26:57.019397+0000","last_became_peered":"2024-09-12T03:26:57.019397+0000","last_unstale":"2024-09-12T03:28:08.844078+0000","last_undegraded":"2024-09-12T03:28:08.844078+0000","last_fullsized":"2024-09-12T03:28:08.844078+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:44.891829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.927763+0000","last_change":"2024-09-12T03:27:06.048452+0000","last_active":"2024-09-12T03:28:08.927763+0000","last_peered":"2024-09-12T03:28:08.927763+0000","last_clean":"2024-09-12T03:28:08.927763+0000","last_became_active":"2024-09-12T03:27:06.048011+0000","last_became_peered":"2024-09-12T03:27:06.048011+0000","last_unstale":"2024-09-12T03:28:08.927763+0000","last_undegraded":"2024-09-12T03:28:08.927763+0000","last_fullsized":"2024-09-12T03:28:08.927763+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:20:38.360428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.974833+0000","last_change":"2024-09-12T03:26:59.017212+0000","last_active":"2024-09-12T03:28:08.974833+0000","last_peered":"2024-09-12T03:28:08.974833+0000","last_clean":"2024-09-12T03:28:08.974833+0000","last_became_active":"2024-09-12T03:26:59.016605+0000","last_became_peered":"2024-09-12T03:26:59.016605+0000","last_unstale":"2024-09-12T03:28:08.974833+0000","last_undegraded":"2024-09-12T03:28:08.974833+0000","last_fullsized":"2024-09-12T03:28:08.974833+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:34.639464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.844194+0000","last_change":"2024-09-12T03:26:57.020236+0000","last_active":"2024-09-12T03:28:08.844194+0000","last_peered":"2024-09-12T03:28:08.844194+0000","last_clean":"2024-09-12T03:28:08.844194+0000","last_became_active":"2024-09-12T03:26:57.019899+0000","last_became_peered":"2024-09-12T03:26:57.019899+0000","last_unstale":"2024-09-12T03:28:08.844194+0000","last_undegraded":"2024-09-12T03:28:08.844194+0000","last_fullsized":"2024-09-12T03:28:08.844194+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:03:16.184419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.975992+0000","last_change":"2024-09-12T03:28:08.975992+0000","last_active":"2024-09-12T03:28:08.975992+0000","last_peered":"2024-09-12T03:28:08.975992+0000","last_clean":"2024-09-12T03:28:08.975992+0000","last_became_active":"2024-09-12T03:28:08.975516+0000","last_became_peered":"2024-09-12T03:28:08.975516+0000","last_unstale":"2024-09-12T03:28:08.975992+0000","last_undegraded":"2024-09-12T03:28:08.975992+0000","last_fullsized":"2024-09-12T03:28:08.975992+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:37:23.151370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.974751+0000","last_change":"2024-09-12T03:26:59.014332+0000","last_active":"2024-09-12T03:28:08.974751+0000","last_peered":"2024-09-12T03:28:08.974751+0000","last_clean":"2024-09-12T03:28:08.974751+0000","last_became_active":"2024-09-12T03:26:59.014019+0000","last_became_peered":"2024-09-12T03:26:59.014019+0000","last_unstale":"2024-09-12T03:28:08.974751+0000","last_undegraded":"2024-09-12T03:28:08.974751+0000","last_fullsized":"2024-09-12T03:28:08.974751+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:03:18.020391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.843887+0000","last_change":"2024-09-12T03:26:56.956591+0000","last_active":"2024-09-12T03:28:08.843887+0000","last_peered":"2024-09-12T03:28:08.843887+0000","last_clean":"2024-09-12T03:28:08.843887+0000","last_became_active":"2024-09-12T03:26:56.956028+0000","last_became_peered":"2024-09-12T03:26:56.956028+0000","last_unstale":"2024-09-12T03:28:08.843887+0000","last_undegraded":"2024-09-12T03:28:08.843887+0000","last_fullsized":"2024-09-12T03:28:08.843887+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:55:20.500536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.841580+0000","last_change":"2024-09-12T03:27:06.052630+0000","last_active":"2024-09-12T03:28:08.841580+0000","last_peered":"2024-09-12T03:28:08.841580+0000","last_clean":"2024-09-12T03:28:08.841580+0000","last_became_active":"2024-09-12T03:27:06.052240+0000","last_became_peered":"2024-09-12T03:27:06.052240+0000","last_unstale":"2024-09-12T03:28:08.841580+0000","last_undegraded":"2024-09-12T03:28:08.841580+0000","last_fullsized":"2024-09-12T03:28:08.841580+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:26:49.356330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.840281+0000","last_change":"2024-09-12T03:26:59.022054+0000","last_active":"2024-09-12T03:28:08.840281+0000","last_peered":"2024-09-12T03:28:08.840281+0000","last_clean":"2024-09-12T03:28:08.840281+0000","last_became_active":"2024-09-12T03:26:59.021673+0000","last_became_peered":"2024-09-12T03:26:59.021673+0000","last_unstale":"2024-09-12T03:28:08.840281+0000","last_undegraded":"2024-09-12T03:28:08.840281+0000","last_fullsized":"2024-09-12T03:28:08.840281+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:07:07.581766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":91,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:26.216914+0000","last_change":"2024-09-12T03:26:57.018607+0000","last_active":"2024-09-12T03:28:26.216914+0000","last_peered":"2024-09-12T03:28:26.216914+0000","last_clean":"2024-09-12T03:28:26.216914+0000","last_became_active":"2024-09-12T03:26:57.018293+0000","last_became_peered":"2024-09-12T03:26:57.018293+0000","last_unstale":"2024-09-12T03:28:26.216914+0000","last_undegraded":"2024-09-12T03:28:26.216914+0000","last_fullsized":"2024-09-12T03:28:26.216914+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:23:57.245446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.841617+0000","last_change":"2024-09-12T03:27:06.190452+0000","last_active":"2024-09-12T03:28:08.841617+0000","last_peered":"2024-09-12T03:28:08.841617+0000","last_clean":"2024-09-12T03:28:08.841617+0000","last_became_active":"2024-09-12T03:27:06.189940+0000","last_became_peered":"2024-09-12T03:27:06.189940+0000","last_unstale":"2024-09-12T03:28:08.841617+0000","last_undegraded":"2024-09-12T03:28:08.841617+0000","last_fullsized":"2024-09-12T03:28:08.841617+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:25:23.114124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.841121+0000","last_change":"2024-09-12T03:26:59.019294+0000","last_active":"2024-09-12T03:28:08.841121+0000","last_peered":"2024-09-12T03:28:08.841121+0000","last_clean":"2024-09-12T03:28:08.841121+0000","last_became_active":"2024-09-12T03:26:59.018976+0000","last_became_peered":"2024-09-12T03:26:59.018976+0000","last_unstale":"2024-09-12T03:28:08.841121+0000","last_undegraded":"2024-09-12T03:28:08.841121+0000","last_fullsized":"2024-09-12T03:28:08.841121+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:06:38.571171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.842508+0000","last_change":"2024-09-12T03:26:56.960077+0000","last_active":"2024-09-12T03:28:08.842508+0000","last_peered":"2024-09-12T03:28:08.842508+0000","last_clean":"2024-09-12T03:28:08.842508+0000","last_became_active":"2024-09-12T03:26:56.958892+0000","last_became_peered":"2024-09-12T03:26:56.958892+0000","last_unstale":"2024-09-12T03:28:08.842508+0000","last_undegraded":"2024-09-12T03:28:08.842508+0000","last_fullsized":"2024-09-12T03:28:08.842508+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:27:04.779026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.975849+0000","last_change":"2024-09-12T03:28:08.975198+0000","last_active":"2024-09-12T03:28:08.975849+0000","last_peered":"2024-09-12T03:28:08.975849+0000","last_clean":"2024-09-12T03:28:08.975849+0000","last_became_active":"2024-09-12T03:28:08.974727+0000","last_became_peered":"2024-09-12T03:28:08.974727+0000","last_unstale":"2024-09-12T03:28:08.975849+0000","last_undegraded":"2024-09-12T03:28:08.975849+0000","last_fullsized":"2024-09-12T03:28:08.975849+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:04:45.121889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.927570+0000","last_change":"2024-09-12T03:26:59.024060+0000","last_active":"2024-09-12T03:28:08.927570+0000","last_peered":"2024-09-12T03:28:08.927570+0000","last_clean":"2024-09-12T03:28:08.927570+0000","last_became_active":"2024-09-12T03:26:59.023726+0000","last_became_peered":"2024-09-12T03:26:59.023726+0000","last_unstale":"2024-09-12T03:28:08.927570+0000","last_undegraded":"2024-09-12T03:28:08.927570+0000","last_fullsized":"2024-09-12T03:28:08.927570+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:49:44.984188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.846866+0000","last_change":"2024-09-12T03:26:56.957847+0000","last_active":"2024-09-12T03:28:08.846866+0000","last_peered":"2024-09-12T03:28:08.846866+0000","last_clean":"2024-09-12T03:28:08.846866+0000","last_became_active":"2024-09-12T03:26:56.957491+0000","last_became_peered":"2024-09-12T03:26:56.957491+0000","last_unstale":"2024-09-12T03:28:08.846866+0000","last_undegraded":"2024-09-12T03:28:08.846866+0000","last_fullsized":"2024-09-12T03:28:08.846866+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:06:23.028881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":362,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.842558+0000","last_change":"2024-09-12T03:25:45.211570+0000","last_active":"2024-09-12T03:28:08.842558+0000","last_peered":"2024-09-12T03:28:08.842558+0000","last_clean":"2024-09-12T03:28:08.842558+0000","last_became_active":"2024-09-12T03:25:45.211052+0000","last_became_peered":"2024-09-12T03:25:45.211052+0000","last_unstale":"2024-09-12T03:28:08.842558+0000","last_undegraded":"2024-09-12T03:28:08.842558+0000","last_fullsized":"2024-09-12T03:28:08.842558+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:21.915950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:21.915950+0000","last_clean_scrub_stamp":"2024-09-12T03:22:21.915950+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:03:06.560617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.843924+0000","last_change":"2024-09-12T03:27:06.046319+0000","last_active":"2024-09-12T03:28:08.843924+0000","last_peered":"2024-09-12T03:28:08.843924+0000","last_clean":"2024-09-12T03:28:08.843924+0000","last_became_active":"2024-09-12T03:27:06.045864+0000","last_became_peered":"2024-09-12T03:27:06.045864+0000","last_unstale":"2024-09-12T03:28:08.843924+0000","last_undegraded":"2024-09-12T03:28:08.843924+0000","last_fullsized":"2024-09-12T03:28:08.843924+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:45:25.224775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.844185+0000","last_change":"2024-09-12T03:26:59.019197+0000","last_active":"2024-09-12T03:28:08.844185+0000","last_peered":"2024-09-12T03:28:08.844185+0000","last_clean":"2024-09-12T03:28:08.844185+0000","last_became_active":"2024-09-12T03:26:59.018868+0000","last_became_peered":"2024-09-12T03:26:59.018868+0000","last_unstale":"2024-09-12T03:28:08.844185+0000","last_undegraded":"2024-09-12T03:28:08.844185+0000","last_fullsized":"2024-09-12T03:28:08.844185+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:50:51.707556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.843494+0000","last_change":"2024-09-12T03:26:57.019888+0000","last_active":"2024-09-12T03:28:08.843494+0000","last_peered":"2024-09-12T03:28:08.843494+0000","last_clean":"2024-09-12T03:28:08.843494+0000","last_became_active":"2024-09-12T03:26:57.019378+0000","last_became_peered":"2024-09-12T03:26:57.019378+0000","last_unstale":"2024-09-12T03:28:08.843494+0000","last_undegraded":"2024-09-12T03:28:08.843494+0000","last_fullsized":"2024-09-12T03:28:08.843494+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:44:02.298726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.842731+0000","last_change":"2024-09-12T03:27:06.043780+0000","last_active":"2024-09-12T03:28:08.842731+0000","last_peered":"2024-09-12T03:28:08.842731+0000","last_clean":"2024-09-12T03:28:08.842731+0000","last_became_active":"2024-09-12T03:27:06.042770+0000","last_became_peered":"2024-09-12T03:27:06.042770+0000","last_unstale":"2024-09-12T03:28:08.842731+0000","last_undegraded":"2024-09-12T03:28:08.842731+0000","last_fullsized":"2024-09-12T03:28:08.842731+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:32:40.801908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.843955+0000","last_change":"2024-09-12T03:26:59.022996+0000","last_active":"2024-09-12T03:28:08.843955+0000","last_peered":"2024-09-12T03:28:08.843955+0000","last_clean":"2024-09-12T03:28:08.843955+0000","last_became_active":"2024-09-12T03:26:59.022675+0000","last_became_peered":"2024-09-12T03:26:59.022675+0000","last_unstale":"2024-09-12T03:28:08.843955+0000","last_undegraded":"2024-09-12T03:28:08.843955+0000","last_fullsized":"2024-09-12T03:28:08.843955+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:09.674971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.841256+0000","last_change":"2024-09-12T03:26:56.957101+0000","last_active":"2024-09-12T03:28:08.841256+0000","last_peered":"2024-09-12T03:28:08.841256+0000","last_clean":"2024-09-12T03:28:08.841256+0000","last_became_active":"2024-09-12T03:26:56.956694+0000","last_became_peered":"2024-09-12T03:26:56.956694+0000","last_unstale":"2024-09-12T03:28:08.841256+0000","last_undegraded":"2024-09-12T03:28:08.841256+0000","last_fullsized":"2024-09-12T03:28:08.841256+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:29.793229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.846038+0000","last_change":"2024-09-12T03:27:06.049382+0000","last_active":"2024-09-12T03:28:08.846038+0000","last_peered":"2024-09-12T03:28:08.846038+0000","last_clean":"2024-09-12T03:28:08.846038+0000","last_became_active":"2024-09-12T03:27:06.048961+0000","last_became_peered":"2024-09-12T03:27:06.048961+0000","last_unstale":"2024-09-12T03:28:08.846038+0000","last_undegraded":"2024-09-12T03:28:08.846038+0000","last_fullsized":"2024-09-12T03:28:08.846038+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:32.400328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.844226+0000","last_change":"2024-09-12T03:26:59.020146+0000","last_active":"2024-09-12T03:28:08.844226+0000","last_peered":"2024-09-12T03:28:08.844226+0000","last_clean":"2024-09-12T03:28:08.844226+0000","last_became_active":"2024-09-12T03:26:59.019854+0000","last_became_peered":"2024-09-12T03:26:59.019854+0000","last_unstale":"2024-09-12T03:28:08.844226+0000","last_undegraded":"2024-09-12T03:28:08.844226+0000","last_fullsized":"2024-09-12T03:28:08.844226+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:38:47.476070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.841950+0000","last_change":"2024-09-12T03:26:56.956237+0000","last_active":"2024-09-12T03:28:08.841950+0000","last_peered":"2024-09-12T03:28:08.841950+0000","last_clean":"2024-09-12T03:28:08.841950+0000","last_became_active":"2024-09-12T03:26:56.955941+0000","last_became_peered":"2024-09-12T03:26:56.955941+0000","last_unstale":"2024-09-12T03:28:08.841950+0000","last_undegraded":"2024-09-12T03:28:08.841950+0000","last_fullsized":"2024-09-12T03:28:08.841950+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:08:53.422429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.840107+0000","last_change":"2024-09-12T03:27:06.050916+0000","last_active":"2024-09-12T03:28:08.840107+0000","last_peered":"2024-09-12T03:28:08.840107+0000","last_clean":"2024-09-12T03:28:08.840107+0000","last_became_active":"2024-09-12T03:27:06.050494+0000","last_became_peered":"2024-09-12T03:27:06.050494+0000","last_unstale":"2024-09-12T03:28:08.840107+0000","last_undegraded":"2024-09-12T03:28:08.840107+0000","last_fullsized":"2024-09-12T03:28:08.840107+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:58:40.830354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.928008+0000","last_change":"2024-09-12T03:26:59.025310+0000","last_active":"2024-09-12T03:28:08.928008+0000","last_peered":"2024-09-12T03:28:08.928008+0000","last_clean":"2024-09-12T03:28:08.928008+0000","last_became_active":"2024-09-12T03:26:59.024917+0000","last_became_peered":"2024-09-12T03:26:59.024917+0000","last_unstale":"2024-09-12T03:28:08.928008+0000","last_undegraded":"2024-09-12T03:28:08.928008+0000","last_fullsized":"2024-09-12T03:28:08.928008+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:14:19.786496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.843881+0000","last_change":"2024-09-12T03:26:56.959434+0000","last_active":"2024-09-12T03:28:08.843881+0000","last_peered":"2024-09-12T03:28:08.843881+0000","last_clean":"2024-09-12T03:28:08.843881+0000","last_became_active":"2024-09-12T03:26:56.959095+0000","last_became_peered":"2024-09-12T03:26:56.959095+0000","last_unstale":"2024-09-12T03:28:08.843881+0000","last_undegraded":"2024-09-12T03:28:08.843881+0000","last_fullsized":"2024-09-12T03:28:08.843881+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:43:39.291022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.840153+0000","last_change":"2024-09-12T03:27:06.054028+0000","last_active":"2024-09-12T03:28:08.840153+0000","last_peered":"2024-09-12T03:28:08.840153+0000","last_clean":"2024-09-12T03:28:08.840153+0000","last_became_active":"2024-09-12T03:27:06.053573+0000","last_became_peered":"2024-09-12T03:27:06.053573+0000","last_unstale":"2024-09-12T03:28:08.840153+0000","last_undegraded":"2024-09-12T03:28:08.840153+0000","last_fullsized":"2024-09-12T03:28:08.840153+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:05.977501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.840384+0000","last_change":"2024-09-12T03:26:59.022978+0000","last_active":"2024-09-12T03:28:08.840384+0000","last_peered":"2024-09-12T03:28:08.840384+0000","last_clean":"2024-09-12T03:28:08.840384+0000","last_became_active":"2024-09-12T03:26:59.022555+0000","last_became_peered":"2024-09-12T03:26:59.022555+0000","last_unstale":"2024-09-12T03:28:08.840384+0000","last_undegraded":"2024-09-12T03:28:08.840384+0000","last_fullsized":"2024-09-12T03:28:08.840384+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:51.903788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.842846+0000","last_change":"2024-09-12T03:26:57.018735+0000","last_active":"2024-09-12T03:28:08.842846+0000","last_peered":"2024-09-12T03:28:08.842846+0000","last_clean":"2024-09-12T03:28:08.842846+0000","last_became_active":"2024-09-12T03:26:57.018367+0000","last_became_peered":"2024-09-12T03:26:57.018367+0000","last_unstale":"2024-09-12T03:28:08.842846+0000","last_undegraded":"2024-09-12T03:28:08.842846+0000","last_fullsized":"2024-09-12T03:28:08.842846+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:25.420324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.841856+0000","last_change":"2024-09-12T03:27:06.046626+0000","last_active":"2024-09-12T03:28:08.841856+0000","last_peered":"2024-09-12T03:28:08.841856+0000","last_clean":"2024-09-12T03:28:08.841856+0000","last_became_active":"2024-09-12T03:27:06.046234+0000","last_became_peered":"2024-09-12T03:27:06.046234+0000","last_unstale":"2024-09-12T03:28:08.841856+0000","last_undegraded":"2024-09-12T03:28:08.841856+0000","last_fullsized":"2024-09-12T03:28:08.841856+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:05:57.353206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.842361+0000","last_change":"2024-09-12T03:26:59.023541+0000","last_active":"2024-09-12T03:28:08.842361+0000","last_peered":"2024-09-12T03:28:08.842361+0000","last_clean":"2024-09-12T03:28:08.842361+0000","last_became_active":"2024-09-12T03:26:59.023229+0000","last_became_peered":"2024-09-12T03:26:59.023229+0000","last_unstale":"2024-09-12T03:28:08.842361+0000","last_undegraded":"2024-09-12T03:28:08.842361+0000","last_fullsized":"2024-09-12T03:28:08.842361+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:27:58.514616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'4","reported_seq":94,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:26.217681+0000","last_change":"2024-09-12T03:26:56.962703+0000","last_active":"2024-09-12T03:28:26.217681+0000","last_peered":"2024-09-12T03:28:26.217681+0000","last_clean":"2024-09-12T03:28:26.217681+0000","last_became_active":"2024-09-12T03:26:56.962280+0000","last_became_peered":"2024-09-12T03:26:56.962280+0000","last_unstale":"2024-09-12T03:28:26.217681+0000","last_undegraded":"2024-09-12T03:28:26.217681+0000","last_fullsized":"2024-09-12T03:28:26.217681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:01.045200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.843959+0000","last_change":"2024-09-12T03:27:06.054791+0000","last_active":"2024-09-12T03:28:08.843959+0000","last_peered":"2024-09-12T03:28:08.843959+0000","last_clean":"2024-09-12T03:28:08.843959+0000","last_became_active":"2024-09-12T03:27:06.054311+0000","last_became_peered":"2024-09-12T03:27:06.054311+0000","last_unstale":"2024-09-12T03:28:08.843959+0000","last_undegraded":"2024-09-12T03:28:08.843959+0000","last_fullsized":"2024-09-12T03:28:08.843959+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:21:59.115200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.842867+0000","last_change":"2024-09-12T03:26:59.023971+0000","last_active":"2024-09-12T03:28:08.842867+0000","last_peered":"2024-09-12T03:28:08.842867+0000","last_clean":"2024-09-12T03:28:08.842867+0000","last_became_active":"2024-09-12T03:26:59.023653+0000","last_became_peered":"2024-09-12T03:26:59.023653+0000","last_unstale":"2024-09-12T03:28:08.842867+0000","last_undegraded":"2024-09-12T03:28:08.842867+0000","last_fullsized":"2024-09-12T03:28:08.842867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:10:08.571177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.840423+0000","last_change":"2024-09-12T03:26:56.962459+0000","last_active":"2024-09-12T03:28:08.840423+0000","last_peered":"2024-09-12T03:28:08.840423+0000","last_clean":"2024-09-12T03:28:08.840423+0000","last_became_active":"2024-09-12T03:26:56.962039+0000","last_became_peered":"2024-09-12T03:26:56.962039+0000","last_unstale":"2024-09-12T03:28:08.840423+0000","last_undegraded":"2024-09-12T03:28:08.840423+0000","last_fullsized":"2024-09-12T03:28:08.840423+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:08:44.214788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.927632+0000","last_change":"2024-09-12T03:27:06.049058+0000","last_active":"2024-09-12T03:28:08.927632+0000","last_peered":"2024-09-12T03:28:08.927632+0000","last_clean":"2024-09-12T03:28:08.927632+0000","last_became_active":"2024-09-12T03:27:06.048606+0000","last_became_peered":"2024-09-12T03:27:06.048606+0000","last_unstale":"2024-09-12T03:28:08.927632+0000","last_undegraded":"2024-09-12T03:28:08.927632+0000","last_fullsized":"2024-09-12T03:28:08.927632+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:23:14.571184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.841292+0000","last_change":"2024-09-12T03:26:59.017832+0000","last_active":"2024-09-12T03:28:08.841292+0000","last_peered":"2024-09-12T03:28:08.841292+0000","last_clean":"2024-09-12T03:28:08.841292+0000","last_became_active":"2024-09-12T03:26:59.017497+0000","last_became_peered":"2024-09-12T03:26:59.017497+0000","last_unstale":"2024-09-12T03:28:08.841292+0000","last_undegraded":"2024-09-12T03:28:08.841292+0000","last_fullsized":"2024-09-12T03:28:08.841292+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:49.753556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.842849+0000","last_change":"2024-09-12T03:26:56.957877+0000","last_active":"2024-09-12T03:28:08.842849+0000","last_peered":"2024-09-12T03:28:08.842849+0000","last_clean":"2024-09-12T03:28:08.842849+0000","last_became_active":"2024-09-12T03:26:56.957502+0000","last_became_peered":"2024-09-12T03:26:56.957502+0000","last_unstale":"2024-09-12T03:28:08.842849+0000","last_undegraded":"2024-09-12T03:28:08.842849+0000","last_fullsized":"2024-09-12T03:28:08.842849+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:35.851957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.840767+0000","last_change":"2024-09-12T03:27:06.047070+0000","last_active":"2024-09-12T03:28:08.840767+0000","last_peered":"2024-09-12T03:28:08.840767+0000","last_clean":"2024-09-12T03:28:08.840767+0000","last_became_active":"2024-09-12T03:27:06.046612+0000","last_became_peered":"2024-09-12T03:27:06.046612+0000","last_unstale":"2024-09-12T03:28:08.840767+0000","last_undegraded":"2024-09-12T03:28:08.840767+0000","last_fullsized":"2024-09-12T03:28:08.840767+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:36:53.960456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.843815+0000","last_change":"2024-09-12T03:26:59.019856+0000","last_active":"2024-09-12T03:28:08.843815+0000","last_peered":"2024-09-12T03:28:08.843815+0000","last_clean":"2024-09-12T03:28:08.843815+0000","last_became_active":"2024-09-12T03:26:59.019498+0000","last_became_peered":"2024-09-12T03:26:59.019498+0000","last_unstale":"2024-09-12T03:28:08.843815+0000","last_undegraded":"2024-09-12T03:28:08.843815+0000","last_fullsized":"2024-09-12T03:28:08.843815+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:02:18.000549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'33","reported_seq":178,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:25.216237+0000","last_change":"2024-09-12T03:26:56.958919+0000","last_active":"2024-09-12T03:28:25.216237+0000","last_peered":"2024-09-12T03:28:25.216237+0000","last_clean":"2024-09-12T03:28:25.216237+0000","last_became_active":"2024-09-12T03:26:56.958455+0000","last_became_peered":"2024-09-12T03:26:56.958455+0000","last_unstale":"2024-09-12T03:28:25.216237+0000","last_undegraded":"2024-09-12T03:28:25.216237+0000","last_fullsized":"2024-09-12T03:28:25.216237+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:05:19.318922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62076,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":111,"num_read_kb":117,"num_write":34,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.843172+0000","last_change":"2024-09-12T03:27:06.047094+0000","last_active":"2024-09-12T03:28:08.843172+0000","last_peered":"2024-09-12T03:28:08.843172+0000","last_clean":"2024-09-12T03:28:08.843172+0000","last_became_active":"2024-09-12T03:27:06.046702+0000","last_became_peered":"2024-09-12T03:27:06.046702+0000","last_unstale":"2024-09-12T03:28:08.843172+0000","last_undegraded":"2024-09-12T03:28:08.843172+0000","last_fullsized":"2024-09-12T03:28:08.843172+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:46:57.557056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.843360+0000","last_change":"2024-09-12T03:26:59.018905+0000","last_active":"2024-09-12T03:28:08.843360+0000","last_peered":"2024-09-12T03:28:08.843360+0000","last_clean":"2024-09-12T03:28:08.843360+0000","last_became_active":"2024-09-12T03:26:59.018492+0000","last_became_peered":"2024-09-12T03:26:59.018492+0000","last_unstale":"2024-09-12T03:28:08.843360+0000","last_undegraded":"2024-09-12T03:28:08.843360+0000","last_fullsized":"2024-09-12T03:28:08.843360+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:11.953999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.841210+0000","last_change":"2024-09-12T03:26:57.018607+0000","last_active":"2024-09-12T03:28:08.841210+0000","last_peered":"2024-09-12T03:28:08.841210+0000","last_clean":"2024-09-12T03:28:08.841210+0000","last_became_active":"2024-09-12T03:26:57.018255+0000","last_became_peered":"2024-09-12T03:26:57.018255+0000","last_unstale":"2024-09-12T03:28:08.841210+0000","last_undegraded":"2024-09-12T03:28:08.841210+0000","last_fullsized":"2024-09-12T03:28:08.841210+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:39.029308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.975859+0000","last_change":"2024-09-12T03:28:08.975859+0000","last_active":"2024-09-12T03:28:08.975859+0000","last_peered":"2024-09-12T03:28:08.975859+0000","last_clean":"2024-09-12T03:28:08.975859+0000","last_became_active":"2024-09-12T03:28:08.975408+0000","last_became_peered":"2024-09-12T03:28:08.975408+0000","last_unstale":"2024-09-12T03:28:08.975859+0000","last_undegraded":"2024-09-12T03:28:08.975859+0000","last_fullsized":"2024-09-12T03:28:08.975859+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:13:01.181897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.846908+0000","last_change":"2024-09-12T03:26:59.025406+0000","last_active":"2024-09-12T03:28:08.846908+0000","last_peered":"2024-09-12T03:28:08.846908+0000","last_clean":"2024-09-12T03:28:08.846908+0000","last_became_active":"2024-09-12T03:26:59.025091+0000","last_became_peered":"2024-09-12T03:26:59.025091+0000","last_unstale":"2024-09-12T03:28:08.846908+0000","last_undegraded":"2024-09-12T03:28:08.846908+0000","last_fullsized":"2024-09-12T03:28:08.846908+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:39:08.297785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.844127+0000","last_change":"2024-09-12T03:26:56.961325+0000","last_active":"2024-09-12T03:28:08.844127+0000","last_peered":"2024-09-12T03:28:08.844127+0000","last_clean":"2024-09-12T03:28:08.844127+0000","last_became_active":"2024-09-12T03:26:56.960806+0000","last_became_peered":"2024-09-12T03:26:56.960806+0000","last_unstale":"2024-09-12T03:28:08.844127+0000","last_undegraded":"2024-09-12T03:28:08.844127+0000","last_fullsized":"2024-09-12T03:28:08.844127+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:44:11.209394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.843094+0000","last_change":"2024-09-12T03:27:06.055331+0000","last_active":"2024-09-12T03:28:08.843094+0000","last_peered":"2024-09-12T03:28:08.843094+0000","last_clean":"2024-09-12T03:28:08.843094+0000","last_became_active":"2024-09-12T03:27:06.054227+0000","last_became_peered":"2024-09-12T03:27:06.054227+0000","last_unstale":"2024-09-12T03:28:08.843094+0000","last_undegraded":"2024-09-12T03:28:08.843094+0000","last_fullsized":"2024-09-12T03:28:08.843094+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:35:10.446231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.843574+0000","last_change":"2024-09-12T03:26:59.012095+0000","last_active":"2024-09-12T03:28:08.843574+0000","last_peered":"2024-09-12T03:28:08.843574+0000","last_clean":"2024-09-12T03:28:08.843574+0000","last_became_active":"2024-09-12T03:26:59.011592+0000","last_became_peered":"2024-09-12T03:26:59.011592+0000","last_unstale":"2024-09-12T03:28:08.843574+0000","last_undegraded":"2024-09-12T03:28:08.843574+0000","last_fullsized":"2024-09-12T03:28:08.843574+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:49:33.026996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":88,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:24.214814+0000","last_change":"2024-09-12T03:26:56.959353+0000","last_active":"2024-09-12T03:28:24.214814+0000","last_peered":"2024-09-12T03:28:24.214814+0000","last_clean":"2024-09-12T03:28:24.214814+0000","last_became_active":"2024-09-12T03:26:56.958767+0000","last_became_peered":"2024-09-12T03:26:56.958767+0000","last_unstale":"2024-09-12T03:28:24.214814+0000","last_undegraded":"2024-09-12T03:28:24.214814+0000","last_fullsized":"2024-09-12T03:28:24.214814+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:37:51.518907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":51,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.927484+0000","last_change":"2024-09-12T03:27:06.043745+0000","last_active":"2024-09-12T03:28:08.927484+0000","last_peered":"2024-09-12T03:28:08.927484+0000","last_clean":"2024-09-12T03:28:08.927484+0000","last_became_active":"2024-09-12T03:27:06.042855+0000","last_became_peered":"2024-09-12T03:27:06.042855+0000","last_unstale":"2024-09-12T03:28:08.927484+0000","last_undegraded":"2024-09-12T03:28:08.927484+0000","last_fullsized":"2024-09-12T03:28:08.927484+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:14:34.939593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.846475+0000","last_change":"2024-09-12T03:27:06.052561+0000","last_active":"2024-09-12T03:28:08.846475+0000","last_peered":"2024-09-12T03:28:08.846475+0000","last_clean":"2024-09-12T03:28:08.846475+0000","last_became_active":"2024-09-12T03:27:06.052192+0000","last_became_peered":"2024-09-12T03:27:06.052192+0000","last_unstale":"2024-09-12T03:28:08.846475+0000","last_undegraded":"2024-09-12T03:28:08.846475+0000","last_fullsized":"2024-09-12T03:28:08.846475+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:31:31.800188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.842792+0000","last_change":"2024-09-12T03:26:59.012762+0000","last_active":"2024-09-12T03:28:08.842792+0000","last_peered":"2024-09-12T03:28:08.842792+0000","last_clean":"2024-09-12T03:28:08.842792+0000","last_became_active":"2024-09-12T03:26:59.012400+0000","last_became_peered":"2024-09-12T03:26:59.012400+0000","last_unstale":"2024-09-12T03:28:08.842792+0000","last_undegraded":"2024-09-12T03:28:08.842792+0000","last_fullsized":"2024-09-12T03:28:08.842792+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:35:13.573284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.840534+0000","last_change":"2024-09-12T03:26:56.962454+0000","last_active":"2024-09-12T03:28:08.840534+0000","last_peered":"2024-09-12T03:28:08.840534+0000","last_clean":"2024-09-12T03:28:08.840534+0000","last_became_active":"2024-09-12T03:26:56.962026+0000","last_became_peered":"2024-09-12T03:26:56.962026+0000","last_unstale":"2024-09-12T03:28:08.840534+0000","last_undegraded":"2024-09-12T03:28:08.840534+0000","last_fullsized":"2024-09-12T03:28:08.840534+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:00:23.501942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.928066+0000","last_change":"2024-09-12T03:27:06.053946+0000","last_active":"2024-09-12T03:28:08.928066+0000","last_peered":"2024-09-12T03:28:08.928066+0000","last_clean":"2024-09-12T03:28:08.928066+0000","last_became_active":"2024-09-12T03:27:06.053494+0000","last_became_peered":"2024-09-12T03:27:06.053494+0000","last_unstale":"2024-09-12T03:28:08.928066+0000","last_undegraded":"2024-09-12T03:28:08.928066+0000","last_fullsized":"2024-09-12T03:28:08.928066+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:13:11.537918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.929840+0000","last_change":"2024-09-12T03:28:08.929156+0000","last_active":"2024-09-12T03:28:08.929840+0000","last_peered":"2024-09-12T03:28:08.929840+0000","last_clean":"2024-09-12T03:28:08.929840+0000","last_became_active":"2024-09-12T03:28:08.928761+0000","last_became_peered":"2024-09-12T03:28:08.928761+0000","last_unstale":"2024-09-12T03:28:08.929840+0000","last_undegraded":"2024-09-12T03:28:08.929840+0000","last_fullsized":"2024-09-12T03:28:08.929840+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:20:00.775981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.841127+0000","last_change":"2024-09-12T03:26:56.958944+0000","last_active":"2024-09-12T03:28:08.841127+0000","last_peered":"2024-09-12T03:28:08.841127+0000","last_clean":"2024-09-12T03:28:08.841127+0000","last_became_active":"2024-09-12T03:26:56.958511+0000","last_became_peered":"2024-09-12T03:26:56.958511+0000","last_unstale":"2024-09-12T03:28:08.841127+0000","last_undegraded":"2024-09-12T03:28:08.841127+0000","last_fullsized":"2024-09-12T03:28:08.841127+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:52:37.871482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.973581+0000","last_change":"2024-09-12T03:26:56.961442+0000","last_active":"2024-09-12T03:28:08.973581+0000","last_peered":"2024-09-12T03:28:08.973581+0000","last_clean":"2024-09-12T03:28:08.973581+0000","last_became_active":"2024-09-12T03:26:56.961078+0000","last_became_peered":"2024-09-12T03:26:56.961078+0000","last_unstale":"2024-09-12T03:28:08.973581+0000","last_undegraded":"2024-09-12T03:28:08.973581+0000","last_fullsized":"2024-09-12T03:28:08.973581+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:17.185990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.840543+0000","last_change":"2024-09-12T03:26:59.022873+0000","last_active":"2024-09-12T03:28:08.840543+0000","last_peered":"2024-09-12T03:28:08.840543+0000","last_clean":"2024-09-12T03:28:08.840543+0000","last_became_active":"2024-09-12T03:26:59.022391+0000","last_became_peered":"2024-09-12T03:26:59.022391+0000","last_unstale":"2024-09-12T03:28:08.840543+0000","last_undegraded":"2024-09-12T03:28:08.840543+0000","last_fullsized":"2024-09-12T03:28:08.840543+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:05:08.138480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.847395+0000","last_change":"2024-09-12T03:28:08.847395+0000","last_active":"2024-09-12T03:28:08.847395+0000","last_peered":"2024-09-12T03:28:08.847395+0000","last_clean":"2024-09-12T03:28:08.847395+0000","last_became_active":"2024-09-12T03:28:08.846580+0000","last_became_peered":"2024-09-12T03:28:08.846580+0000","last_unstale":"2024-09-12T03:28:08.847395+0000","last_undegraded":"2024-09-12T03:28:08.847395+0000","last_fullsized":"2024-09-12T03:28:08.847395+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:21:55.047085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":93,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:25.217007+0000","last_change":"2024-09-12T03:26:56.962474+0000","last_active":"2024-09-12T03:28:25.217007+0000","last_peered":"2024-09-12T03:28:25.217007+0000","last_clean":"2024-09-12T03:28:25.217007+0000","last_became_active":"2024-09-12T03:26:56.962105+0000","last_became_peered":"2024-09-12T03:26:56.962105+0000","last_unstale":"2024-09-12T03:28:25.217007+0000","last_undegraded":"2024-09-12T03:28:25.217007+0000","last_fullsized":"2024-09-12T03:28:25.217007+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:40:41.905790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.844069+0000","last_change":"2024-09-12T03:26:59.011561+0000","last_active":"2024-09-12T03:28:08.844069+0000","last_peered":"2024-09-12T03:28:08.844069+0000","last_clean":"2024-09-12T03:28:08.844069+0000","last_became_active":"2024-09-12T03:26:59.011006+0000","last_became_peered":"2024-09-12T03:26:59.011006+0000","last_unstale":"2024-09-12T03:28:08.844069+0000","last_undegraded":"2024-09-12T03:28:08.844069+0000","last_fullsized":"2024-09-12T03:28:08.844069+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:09:20.221637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.843376+0000","last_change":"2024-09-12T03:27:06.050816+0000","last_active":"2024-09-12T03:28:08.843376+0000","last_peered":"2024-09-12T03:28:08.843376+0000","last_clean":"2024-09-12T03:28:08.843376+0000","last_became_active":"2024-09-12T03:27:06.050403+0000","last_became_peered":"2024-09-12T03:27:06.050403+0000","last_unstale":"2024-09-12T03:28:08.843376+0000","last_undegraded":"2024-09-12T03:28:08.843376+0000","last_fullsized":"2024-09-12T03:28:08.843376+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:02:24.285363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.927854+0000","last_change":"2024-09-12T03:26:59.013900+0000","last_active":"2024-09-12T03:28:08.927854+0000","last_peered":"2024-09-12T03:28:08.927854+0000","last_clean":"2024-09-12T03:28:08.927854+0000","last_became_active":"2024-09-12T03:26:59.013308+0000","last_became_peered":"2024-09-12T03:26:59.013308+0000","last_unstale":"2024-09-12T03:28:08.927854+0000","last_undegraded":"2024-09-12T03:28:08.927854+0000","last_fullsized":"2024-09-12T03:28:08.927854+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:50.943991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.847105+0000","last_change":"2024-09-12T03:28:08.847105+0000","last_active":"2024-09-12T03:28:08.847105+0000","last_peered":"2024-09-12T03:28:08.847105+0000","last_clean":"2024-09-12T03:28:08.847105+0000","last_became_active":"2024-09-12T03:28:08.846639+0000","last_became_peered":"2024-09-12T03:28:08.846639+0000","last_unstale":"2024-09-12T03:28:08.847105+0000","last_undegraded":"2024-09-12T03:28:08.847105+0000","last_fullsized":"2024-09-12T03:28:08.847105+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:45.898536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.842508+0000","last_change":"2024-09-12T03:26:59.017779+0000","last_active":"2024-09-12T03:28:08.842508+0000","last_peered":"2024-09-12T03:28:08.842508+0000","last_clean":"2024-09-12T03:28:08.842508+0000","last_became_active":"2024-09-12T03:26:59.017438+0000","last_became_peered":"2024-09-12T03:26:59.017438+0000","last_unstale":"2024-09-12T03:28:08.842508+0000","last_undegraded":"2024-09-12T03:28:08.842508+0000","last_fullsized":"2024-09-12T03:28:08.842508+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:29.224776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.846213+0000","last_change":"2024-09-12T03:26:59.017542+0000","last_active":"2024-09-12T03:28:08.846213+0000","last_peered":"2024-09-12T03:28:08.846213+0000","last_clean":"2024-09-12T03:28:08.846213+0000","last_became_active":"2024-09-12T03:26:59.017253+0000","last_became_peered":"2024-09-12T03:26:59.017253+0000","last_unstale":"2024-09-12T03:28:08.846213+0000","last_undegraded":"2024-09-12T03:28:08.846213+0000","last_fullsized":"2024-09-12T03:28:08.846213+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:06:51.389627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.841742+0000","last_change":"2024-09-12T03:26:59.013574+0000","last_active":"2024-09-12T03:28:08.841742+0000","last_peered":"2024-09-12T03:28:08.841742+0000","last_clean":"2024-09-12T03:28:08.841742+0000","last_became_active":"2024-09-12T03:26:59.013253+0000","last_became_peered":"2024-09-12T03:26:59.013253+0000","last_unstale":"2024-09-12T03:28:08.841742+0000","last_undegraded":"2024-09-12T03:28:08.841742+0000","last_fullsized":"2024-09-12T03:28:08.841742+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:37.610909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.927943+0000","last_change":"2024-09-12T03:26:59.012430+0000","last_active":"2024-09-12T03:28:08.927943+0000","last_peered":"2024-09-12T03:28:08.927943+0000","last_clean":"2024-09-12T03:28:08.927943+0000","last_became_active":"2024-09-12T03:26:59.011672+0000","last_became_peered":"2024-09-12T03:26:59.011672+0000","last_unstale":"2024-09-12T03:28:08.927943+0000","last_undegraded":"2024-09-12T03:28:08.927943+0000","last_fullsized":"2024-09-12T03:28:08.927943+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:50:44.888834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.843735+0000","last_change":"2024-09-12T03:26:59.015704+0000","last_active":"2024-09-12T03:28:08.843735+0000","last_peered":"2024-09-12T03:28:08.843735+0000","last_clean":"2024-09-12T03:28:08.843735+0000","last_became_active":"2024-09-12T03:26:59.015389+0000","last_became_peered":"2024-09-12T03:26:59.015389+0000","last_unstale":"2024-09-12T03:28:08.843735+0000","last_undegraded":"2024-09-12T03:28:08.843735+0000","last_fullsized":"2024-09-12T03:28:08.843735+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:19:52.917902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.928189+0000","last_change":"2024-09-12T03:26:59.026325+0000","last_active":"2024-09-12T03:28:08.928189+0000","last_peered":"2024-09-12T03:28:08.928189+0000","last_clean":"2024-09-12T03:28:08.928189+0000","last_became_active":"2024-09-12T03:26:59.026014+0000","last_became_peered":"2024-09-12T03:26:59.026014+0000","last_unstale":"2024-09-12T03:28:08.928189+0000","last_undegraded":"2024-09-12T03:28:08.928189+0000","last_fullsized":"2024-09-12T03:28:08.928189+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:32:50.778286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.929290+0000","last_change":"2024-09-12T03:28:08.928903+0000","last_active":"2024-09-12T03:28:08.929290+0000","last_peered":"2024-09-12T03:28:08.929290+0000","last_clean":"2024-09-12T03:28:08.929290+0000","last_became_active":"2024-09-12T03:28:08.928547+0000","last_became_peered":"2024-09-12T03:28:08.928547+0000","last_unstale":"2024-09-12T03:28:08.929290+0000","last_undegraded":"2024-09-12T03:28:08.929290+0000","last_fullsized":"2024-09-12T03:28:08.929290+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:54:47.701399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.927591+0000","last_change":"2024-09-12T03:26:59.023143+0000","last_active":"2024-09-12T03:28:08.927591+0000","last_peered":"2024-09-12T03:28:08.927591+0000","last_clean":"2024-09-12T03:28:08.927591+0000","last_became_active":"2024-09-12T03:26:59.022282+0000","last_became_peered":"2024-09-12T03:26:59.022282+0000","last_unstale":"2024-09-12T03:28:08.927591+0000","last_undegraded":"2024-09-12T03:28:08.927591+0000","last_fullsized":"2024-09-12T03:28:08.927591+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:59.864192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.843099+0000","last_change":"2024-09-12T03:26:59.025505+0000","last_active":"2024-09-12T03:28:08.843099+0000","last_peered":"2024-09-12T03:28:08.843099+0000","last_clean":"2024-09-12T03:28:08.843099+0000","last_became_active":"2024-09-12T03:26:59.025081+0000","last_became_peered":"2024-09-12T03:26:59.025081+0000","last_unstale":"2024-09-12T03:28:08.843099+0000","last_undegraded":"2024-09-12T03:28:08.843099+0000","last_fullsized":"2024-09-12T03:28:08.843099+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:42:18.012217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.841848+0000","last_change":"2024-09-12T03:26:59.011315+0000","last_active":"2024-09-12T03:28:08.841848+0000","last_peered":"2024-09-12T03:28:08.841848+0000","last_clean":"2024-09-12T03:28:08.841848+0000","last_became_active":"2024-09-12T03:26:59.011028+0000","last_became_peered":"2024-09-12T03:26:59.011028+0000","last_unstale":"2024-09-12T03:28:08.841848+0000","last_undegraded":"2024-09-12T03:28:08.841848+0000","last_fullsized":"2024-09-12T03:28:08.841848+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:18:10.541578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.927652+0000","last_change":"2024-09-12T03:26:59.024518+0000","last_active":"2024-09-12T03:28:08.927652+0000","last_peered":"2024-09-12T03:28:08.927652+0000","last_clean":"2024-09-12T03:28:08.927652+0000","last_became_active":"2024-09-12T03:26:59.024197+0000","last_became_peered":"2024-09-12T03:26:59.024197+0000","last_unstale":"2024-09-12T03:28:08.927652+0000","last_undegraded":"2024-09-12T03:28:08.927652+0000","last_fullsized":"2024-09-12T03:28:08.927652+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:52.113096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.843728+0000","last_change":"2024-09-12T03:26:59.018707+0000","last_active":"2024-09-12T03:28:08.843728+0000","last_peered":"2024-09-12T03:28:08.843728+0000","last_clean":"2024-09-12T03:28:08.843728+0000","last_became_active":"2024-09-12T03:26:59.017997+0000","last_became_peered":"2024-09-12T03:26:59.017997+0000","last_unstale":"2024-09-12T03:28:08.843728+0000","last_undegraded":"2024-09-12T03:28:08.843728+0000","last_fullsized":"2024-09-12T03:28:08.843728+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:37:58.896056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.843975+0000","last_change":"2024-09-12T03:26:59.013523+0000","last_active":"2024-09-12T03:28:08.843975+0000","last_peered":"2024-09-12T03:28:08.843975+0000","last_clean":"2024-09-12T03:28:08.843975+0000","last_became_active":"2024-09-12T03:26:59.013210+0000","last_became_peered":"2024-09-12T03:26:59.013210+0000","last_unstale":"2024-09-12T03:28:08.843975+0000","last_undegraded":"2024-09-12T03:28:08.843975+0000","last_fullsized":"2024-09-12T03:28:08.843975+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:03:42.390871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.928332+0000","last_change":"2024-09-12T03:26:59.016719+0000","last_active":"2024-09-12T03:28:08.928332+0000","last_peered":"2024-09-12T03:28:08.928332+0000","last_clean":"2024-09-12T03:28:08.928332+0000","last_became_active":"2024-09-12T03:26:59.015245+0000","last_became_peered":"2024-09-12T03:26:59.015245+0000","last_unstale":"2024-09-12T03:28:08.928332+0000","last_undegraded":"2024-09-12T03:28:08.928332+0000","last_fullsized":"2024-09-12T03:28:08.928332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:12.163847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.842697+0000","last_change":"2024-09-12T03:26:59.024317+0000","last_active":"2024-09-12T03:28:08.842697+0000","last_peered":"2024-09-12T03:28:08.842697+0000","last_clean":"2024-09-12T03:28:08.842697+0000","last_became_active":"2024-09-12T03:26:59.023998+0000","last_became_peered":"2024-09-12T03:26:59.023998+0000","last_unstale":"2024-09-12T03:28:08.842697+0000","last_undegraded":"2024-09-12T03:28:08.842697+0000","last_fullsized":"2024-09-12T03:28:08.842697+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:29:11.713342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.848298+0000","last_change":"2024-09-12T03:28:08.848298+0000","last_active":"2024-09-12T03:28:08.848298+0000","last_peered":"2024-09-12T03:28:08.848298+0000","last_clean":"2024-09-12T03:28:08.848298+0000","last_became_active":"2024-09-12T03:28:08.847989+0000","last_became_peered":"2024-09-12T03:28:08.847989+0000","last_unstale":"2024-09-12T03:28:08.848298+0000","last_undegraded":"2024-09-12T03:28:08.848298+0000","last_fullsized":"2024-09-12T03:28:08.848298+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:02:21.380751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.845774+0000","last_change":"2024-09-12T03:26:59.011340+0000","last_active":"2024-09-12T03:28:08.845774+0000","last_peered":"2024-09-12T03:28:08.845774+0000","last_clean":"2024-09-12T03:28:08.845774+0000","last_became_active":"2024-09-12T03:26:59.011042+0000","last_became_peered":"2024-09-12T03:26:59.011042+0000","last_unstale":"2024-09-12T03:28:08.845774+0000","last_undegraded":"2024-09-12T03:28:08.845774+0000","last_fullsized":"2024-09-12T03:28:08.845774+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:49.568438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.841822+0000","last_change":"2024-09-12T03:26:59.016449+0000","last_active":"2024-09-12T03:28:08.841822+0000","last_peered":"2024-09-12T03:28:08.841822+0000","last_clean":"2024-09-12T03:28:08.841822+0000","last_became_active":"2024-09-12T03:26:59.016087+0000","last_became_peered":"2024-09-12T03:26:59.016087+0000","last_unstale":"2024-09-12T03:28:08.841822+0000","last_undegraded":"2024-09-12T03:28:08.841822+0000","last_fullsized":"2024-09-12T03:28:08.841822+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:13:40.683471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.975871+0000","last_change":"2024-09-12T03:28:08.975304+0000","last_active":"2024-09-12T03:28:08.975871+0000","last_peered":"2024-09-12T03:28:08.975871+0000","last_clean":"2024-09-12T03:28:08.975871+0000","last_became_active":"2024-09-12T03:28:08.974962+0000","last_became_peered":"2024-09-12T03:28:08.974962+0000","last_unstale":"2024-09-12T03:28:08.975871+0000","last_undegraded":"2024-09-12T03:28:08.975871+0000","last_fullsized":"2024-09-12T03:28:08.975871+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:14.369832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.844084+0000","last_change":"2024-09-12T03:26:59.021434+0000","last_active":"2024-09-12T03:28:08.844084+0000","last_peered":"2024-09-12T03:28:08.844084+0000","last_clean":"2024-09-12T03:28:08.844084+0000","last_became_active":"2024-09-12T03:26:59.021071+0000","last_became_peered":"2024-09-12T03:26:59.021071+0000","last_unstale":"2024-09-12T03:28:08.844084+0000","last_undegraded":"2024-09-12T03:28:08.844084+0000","last_fullsized":"2024-09-12T03:28:08.844084+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:14:46.774992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.975119+0000","last_change":"2024-09-12T03:26:59.019310+0000","last_active":"2024-09-12T03:28:08.975119+0000","last_peered":"2024-09-12T03:28:08.975119+0000","last_clean":"2024-09-12T03:28:08.975119+0000","last_became_active":"2024-09-12T03:26:59.018967+0000","last_became_peered":"2024-09-12T03:26:59.018967+0000","last_unstale":"2024-09-12T03:28:08.975119+0000","last_undegraded":"2024-09-12T03:28:08.975119+0000","last_fullsized":"2024-09-12T03:28:08.975119+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:37:10.830500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.843822+0000","last_change":"2024-09-12T03:26:59.022088+0000","last_active":"2024-09-12T03:28:08.843822+0000","last_peered":"2024-09-12T03:28:08.843822+0000","last_clean":"2024-09-12T03:28:08.843822+0000","last_became_active":"2024-09-12T03:26:59.021655+0000","last_became_peered":"2024-09-12T03:26:59.021655+0000","last_unstale":"2024-09-12T03:28:08.843822+0000","last_undegraded":"2024-09-12T03:28:08.843822+0000","last_fullsized":"2024-09-12T03:28:08.843822+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:39.490597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.927621+0000","last_change":"2024-09-12T03:26:59.024898+0000","last_active":"2024-09-12T03:28:08.927621+0000","last_peered":"2024-09-12T03:28:08.927621+0000","last_clean":"2024-09-12T03:28:08.927621+0000","last_became_active":"2024-09-12T03:26:59.024511+0000","last_became_peered":"2024-09-12T03:26:59.024511+0000","last_unstale":"2024-09-12T03:28:08.927621+0000","last_undegraded":"2024-09-12T03:28:08.927621+0000","last_fullsized":"2024-09-12T03:28:08.927621+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:24.191365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.846391+0000","last_change":"2024-09-12T03:28:08.846391+0000","last_active":"2024-09-12T03:28:08.846391+0000","last_peered":"2024-09-12T03:28:08.846391+0000","last_clean":"2024-09-12T03:28:08.846391+0000","last_became_active":"2024-09-12T03:28:08.845889+0000","last_became_peered":"2024-09-12T03:28:08.845889+0000","last_unstale":"2024-09-12T03:28:08.846391+0000","last_undegraded":"2024-09-12T03:28:08.846391+0000","last_fullsized":"2024-09-12T03:28:08.846391+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:00.687744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.974749+0000","last_change":"2024-09-12T03:26:59.014536+0000","last_active":"2024-09-12T03:28:08.974749+0000","last_peered":"2024-09-12T03:28:08.974749+0000","last_clean":"2024-09-12T03:28:08.974749+0000","last_became_active":"2024-09-12T03:26:59.013910+0000","last_became_peered":"2024-09-12T03:26:59.013910+0000","last_unstale":"2024-09-12T03:28:08.974749+0000","last_undegraded":"2024-09-12T03:28:08.974749+0000","last_fullsized":"2024-09-12T03:28:08.974749+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:27.519367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.842587+0000","last_change":"2024-09-12T03:26:59.016656+0000","last_active":"2024-09-12T03:28:08.842587+0000","last_peered":"2024-09-12T03:28:08.842587+0000","last_clean":"2024-09-12T03:28:08.842587+0000","last_became_active":"2024-09-12T03:26:59.016217+0000","last_became_peered":"2024-09-12T03:26:59.016217+0000","last_unstale":"2024-09-12T03:28:08.842587+0000","last_undegraded":"2024-09-12T03:28:08.842587+0000","last_fullsized":"2024-09-12T03:28:08.842587+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:07:42.718394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.845739+0000","last_change":"2024-09-12T03:26:59.015940+0000","last_active":"2024-09-12T03:28:08.845739+0000","last_peered":"2024-09-12T03:28:08.845739+0000","last_clean":"2024-09-12T03:28:08.845739+0000","last_became_active":"2024-09-12T03:26:59.015578+0000","last_became_peered":"2024-09-12T03:26:59.015578+0000","last_unstale":"2024-09-12T03:28:08.845739+0000","last_undegraded":"2024-09-12T03:28:08.845739+0000","last_fullsized":"2024-09-12T03:28:08.845739+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:07:46.559935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.846252+0000","last_change":"2024-09-12T03:26:59.016283+0000","last_active":"2024-09-12T03:28:08.846252+0000","last_peered":"2024-09-12T03:28:08.846252+0000","last_clean":"2024-09-12T03:28:08.846252+0000","last_became_active":"2024-09-12T03:26:59.015109+0000","last_became_peered":"2024-09-12T03:26:59.015109+0000","last_unstale":"2024-09-12T03:28:08.846252+0000","last_undegraded":"2024-09-12T03:28:08.846252+0000","last_fullsized":"2024-09-12T03:28:08.846252+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:02:44.493393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.846573+0000","last_change":"2024-09-12T03:26:59.020161+0000","last_active":"2024-09-12T03:28:08.846573+0000","last_peered":"2024-09-12T03:28:08.846573+0000","last_clean":"2024-09-12T03:28:08.846573+0000","last_became_active":"2024-09-12T03:26:59.019796+0000","last_became_peered":"2024-09-12T03:26:59.019796+0000","last_unstale":"2024-09-12T03:28:08.846573+0000","last_undegraded":"2024-09-12T03:28:08.846573+0000","last_fullsized":"2024-09-12T03:28:08.846573+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:04:02.817005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.928268+0000","last_change":"2024-09-12T03:26:59.015763+0000","last_active":"2024-09-12T03:28:08.928268+0000","last_peered":"2024-09-12T03:28:08.928268+0000","last_clean":"2024-09-12T03:28:08.928268+0000","last_became_active":"2024-09-12T03:26:59.014979+0000","last_became_peered":"2024-09-12T03:26:59.014979+0000","last_unstale":"2024-09-12T03:28:08.928268+0000","last_undegraded":"2024-09-12T03:28:08.928268+0000","last_fullsized":"2024-09-12T03:28:08.928268+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:49:26.959019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66079,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":339,"num_read_kb":240,"num_write":173,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":198237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":247982,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514528,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":991232,"data_stored":508337,"data_compressed":23516,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776164,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32296,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712856,"statfs":{"total":95995035648,"available":95961964544,"internally_reserved":0,"allocated":1134592,"data_stored":1158074,"data_compressed":59644,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939690,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1011712,"data_stored":512669,"data_compressed":23516,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201326,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1007616,"data_stored":508493,"data_compressed":23516,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430259,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1056768,"data_stored":1104404,"data_compressed":59644,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593785,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":987136,"data_stored":503749,"data_compressed":23516,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822717,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1064960,"data_stored":1099599,"data_compressed":59644,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953538,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":999424,"data_stored":508792,"data_compressed":23516,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215175,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":999424,"data_stored":503943,"data_compressed":23516,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411405,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1077248,"data_stored":567085,"data_compressed":23516,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1052672,"data_stored":565735,"data_compressed":23516,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738455,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1011712,"data_stored":505137,"data_compressed":23516,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34490,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34490,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34490,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36128,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36128,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36128,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36128,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:28:31.164 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T03:28:31.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:30 smithi195 ceph-mon[34786]: pgmap v331: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-12T03:28:31.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:30 smithi115 ceph-mon[34801]: pgmap v331: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-12T03:28:31.423 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:31.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:31.895+0000 7fc6b1795640 1 -- 172.21.15.64:0/1042584490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6ac102d30 msgr2=0x7fc6ac103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:31.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:31.895+0000 7fc6b1795640 1 --2- 172.21.15.64:0/1042584490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6ac102d30 0x7fc6ac103110 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4009cd0 tx=0x7fc6a402a8e0 comp rx=0 tx=0).stop 2024-09-12T03:28:31.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:31.895+0000 7fc6b1795640 1 -- 172.21.15.64:0/1042584490 shutdown_connections 2024-09-12T03:28:31.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:31.895+0000 7fc6b1795640 1 --2- 172.21.15.64:0/1042584490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6ac1082a0 0x7fc6ac10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:31.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:31.895+0000 7fc6b1795640 1 --2- 172.21.15.64:0/1042584490 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6ac1036e0 0x7fc6ac107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:31.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:31.895+0000 7fc6b1795640 1 --2- 172.21.15.64:0/1042584490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6ac102d30 0x7fc6ac103110 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:31.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:31.895+0000 7fc6b1795640 1 -- 172.21.15.64:0/1042584490 >> 172.21.15.64:0/1042584490 conn(0x7fc6ac0fc7d0 msgr2=0x7fc6ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:31.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:31.895+0000 7fc6b1795640 1 -- 172.21.15.64:0/1042584490 shutdown_connections 2024-09-12T03:28:31.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:31.895+0000 7fc6b1795640 1 -- 172.21.15.64:0/1042584490 wait complete. 2024-09-12T03:28:31.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:31.895+0000 7fc6b1795640 1 Processor -- start 2024-09-12T03:28:31.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:31.896+0000 7fc6b1795640 1 -- start start 2024-09-12T03:28:31.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:31.896+0000 7fc6b1795640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6ac102d30 0x7fc6ac198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:31.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:31.896+0000 7fc6b1795640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6ac1036e0 0x7fc6ac198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:31.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:31.896+0000 7fc6b1795640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6ac1082a0 0x7fc6ac19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:31.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:31.896+0000 7fc6b1795640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ac111f50 con 0x7fc6ac1082a0 2024-09-12T03:28:31.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:31.896+0000 7fc6b1795640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ac111dd0 con 0x7fc6ac1036e0 2024-09-12T03:28:31.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:31.896+0000 7fc6b1795640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ac1120d0 con 0x7fc6ac102d30 2024-09-12T03:28:31.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:31.896+0000 7fc6ab7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6ac1082a0 0x7fc6ac19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:31.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:31.896+0000 7fc6ab7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6ac1082a0 0x7fc6ac19cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39886/0 (socket says 172.21.15.64:39886) 2024-09-12T03:28:31.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:31.896+0000 7fc6ab7fe640 1 -- 172.21.15.64:0/4087777844 learned_addr learned my addr 172.21.15.64:0/4087777844 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:31.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:31.896+0000 7fc6aa7fc640 1 --2- 172.21.15.64:0/4087777844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6ac1036e0 0x7fc6ac198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:31.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:31.896+0000 7fc6aaffd640 1 --2- 172.21.15.64:0/4087777844 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6ac102d30 0x7fc6ac198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:31.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:31.896+0000 7fc6ab7fe640 1 -- 172.21.15.64:0/4087777844 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6ac102d30 msgr2=0x7fc6ac198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:31.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:31.896+0000 7fc6ab7fe640 1 --2- 172.21.15.64:0/4087777844 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6ac102d30 0x7fc6ac198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:31.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:31.896+0000 7fc6ab7fe640 1 -- 172.21.15.64:0/4087777844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6ac1036e0 msgr2=0x7fc6ac198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:31.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:31.896+0000 7fc6ab7fe640 1 --2- 172.21.15.64:0/4087777844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6ac1036e0 0x7fc6ac198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:31.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:31.896+0000 7fc6ab7fe640 1 -- 172.21.15.64:0/4087777844 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6ac19d4c0 con 0x7fc6ac1082a0 2024-09-12T03:28:31.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:31.896+0000 7fc6aa7fc640 1 --2- 172.21.15.64:0/4087777844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6ac1036e0 0x7fc6ac198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:31.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:31.896+0000 7fc6ab7fe640 1 --2- 172.21.15.64:0/4087777844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6ac1082a0 0x7fc6ac19cde0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc69c00e970 tx=0x7fc69c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:31.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:31.898+0000 7fc6a3fff640 1 -- 172.21.15.64:0/4087777844 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69c00ccc0 con 0x7fc6ac1082a0 2024-09-12T03:28:31.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:31.898+0000 7fc6a3fff640 1 -- 172.21.15.64:0/4087777844 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc69c00ce60 con 0x7fc6ac1082a0 2024-09-12T03:28:31.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:31.898+0000 7fc6b1795640 1 -- 172.21.15.64:0/4087777844 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6ac19d7b0 con 0x7fc6ac1082a0 2024-09-12T03:28:31.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:31.898+0000 7fc6a3fff640 1 -- 172.21.15.64:0/4087777844 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69c010430 con 0x7fc6ac1082a0 2024-09-12T03:28:31.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:31.898+0000 7fc6b1795640 1 -- 172.21.15.64:0/4087777844 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6ac1a5090 con 0x7fc6ac1082a0 2024-09-12T03:28:31.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:31.900+0000 7fc6b1795640 1 -- 172.21.15.64:0/4087777844 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6ac06a850 con 0x7fc6ac1082a0 2024-09-12T03:28:31.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:31.901+0000 7fc6a3fff640 1 -- 172.21.15.64:0/4087777844 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc69c004ff0 con 0x7fc6ac1082a0 2024-09-12T03:28:31.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:31.902+0000 7fc6a3fff640 1 --2- 172.21.15.64:0/4087777844 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fc6980a2c10 0x7fc6980a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:31.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:31.902+0000 7fc6aaffd640 1 --2- 172.21.15.64:0/4087777844 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fc6980a2c10 0x7fc6980a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:31.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:31.902+0000 7fc6a3fff640 1 -- 172.21.15.64:0/4087777844 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fc69c0cf8d0 con 0x7fc6ac1082a0 2024-09-12T03:28:31.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:31.903+0000 7fc6aaffd640 1 --2- 172.21.15.64:0/4087777844 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fc6980a2c10 0x7fc6980a50d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4006010 tx=0x7fc6a4031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:31.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:31.908+0000 7fc6a3fff640 1 -- 172.21.15.64:0/4087777844 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc69c09c370 con 0x7fc6ac1082a0 2024-09-12T03:28:32.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:32.085+0000 7fc6b1795640 1 -- 172.21.15.64:0/4087777844 --> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc6ac10a320 con 0x7fc6980a2c10 2024-09-12T03:28:32.085 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:31 smithi064 ceph-mon[30952]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:28:32.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:32.093+0000 7fc6a3fff640 1 -- 172.21.15.64:0/4087777844 <== mgr.14150 v2:172.21.15.64:6800/759931848 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340596 (secure 0 0 0) 0x7fc6a4036040 con 0x7fc6980a2c10 2024-09-12T03:28:32.097 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:28:32.098 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-09-12T03:28:32.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:32.097+0000 7fc6b1795640 1 -- 172.21.15.64:0/4087777844 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fc6980a2c10 msgr2=0x7fc6980a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:32.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:32.097+0000 7fc6b1795640 1 --2- 172.21.15.64:0/4087777844 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fc6980a2c10 0x7fc6980a50d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4006010 tx=0x7fc6a4031510 comp rx=0 tx=0).stop 2024-09-12T03:28:32.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:32.097+0000 7fc6b1795640 1 -- 172.21.15.64:0/4087777844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6ac1082a0 msgr2=0x7fc6ac19cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:32.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:32.097+0000 7fc6b1795640 1 --2- 172.21.15.64:0/4087777844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6ac1082a0 0x7fc6ac19cde0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc69c00e970 tx=0x7fc69c00ed70 comp rx=0 tx=0).stop 2024-09-12T03:28:32.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:32.097+0000 7fc6b1795640 1 -- 172.21.15.64:0/4087777844 shutdown_connections 2024-09-12T03:28:32.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:32.097+0000 7fc6b1795640 1 --2- 172.21.15.64:0/4087777844 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fc6980a2c10 0x7fc6980a50d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:32.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:32.097+0000 7fc6b1795640 1 --2- 172.21.15.64:0/4087777844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6ac1082a0 0x7fc6ac19cde0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:32.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:32.097+0000 7fc6b1795640 1 --2- 172.21.15.64:0/4087777844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6ac1036e0 0x7fc6ac198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:32.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:32.097+0000 7fc6b1795640 1 --2- 172.21.15.64:0/4087777844 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6ac102d30 0x7fc6ac198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:32.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:32.097+0000 7fc6b1795640 1 -- 172.21.15.64:0/4087777844 >> 172.21.15.64:0/4087777844 conn(0x7fc6ac0fc7d0 msgr2=0x7fc6ac105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:32.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:32.097+0000 7fc6b1795640 1 -- 172.21.15.64:0/4087777844 shutdown_connections 2024-09-12T03:28:32.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:32.097+0000 7fc6b1795640 1 -- 172.21.15.64:0/4087777844 wait complete. 2024-09-12T03:28:32.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:31 smithi195 ceph-mon[34786]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:28:32.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:31 smithi115 ceph-mon[34801]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:28:32.789 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-09-12T03:28:31.515934+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656681,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":495,"num_read_kb":369,"num_write":356,"num_write_kb":3057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":285,"ondisk_log_size":285,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365580,"kb_used_data":12104,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576244,"statfs":{"total":1151940427776,"available":1151566073856,"internally_reserved":0,"allocated":12394496,"data_stored":8046017,"data_compressed":390576,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":77,"num_read_kb":38,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003790"},"pg_stats":[{"pgid":"2.1f","version":"88'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.844104+0000","last_change":"2024-09-12T03:26:56.956415+0000","last_active":"2024-09-12T03:28:08.844104+0000","last_peered":"2024-09-12T03:28:08.844104+0000","last_clean":"2024-09-12T03:28:08.844104+0000","last_became_active":"2024-09-12T03:26:56.956089+0000","last_became_peered":"2024-09-12T03:26:56.956089+0000","last_unstale":"2024-09-12T03:28:08.844104+0000","last_undegraded":"2024-09-12T03:28:08.844104+0000","last_fullsized":"2024-09-12T03:28:08.844104+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:32:22.089322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.928111+0000","last_change":"2024-09-12T03:26:59.024102+0000","last_active":"2024-09-12T03:28:08.928111+0000","last_peered":"2024-09-12T03:28:08.928111+0000","last_clean":"2024-09-12T03:28:08.928111+0000","last_became_active":"2024-09-12T03:26:59.023795+0000","last_became_peered":"2024-09-12T03:26:59.023795+0000","last_unstale":"2024-09-12T03:28:08.928111+0000","last_undegraded":"2024-09-12T03:28:08.928111+0000","last_fullsized":"2024-09-12T03:28:08.928111+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:25:55.392622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.842900+0000","last_change":"2024-09-12T03:27:06.052565+0000","last_active":"2024-09-12T03:28:08.842900+0000","last_peered":"2024-09-12T03:28:08.842900+0000","last_clean":"2024-09-12T03:28:08.842900+0000","last_became_active":"2024-09-12T03:27:06.052209+0000","last_became_peered":"2024-09-12T03:27:06.052209+0000","last_unstale":"2024-09-12T03:28:08.842900+0000","last_undegraded":"2024-09-12T03:28:08.842900+0000","last_fullsized":"2024-09-12T03:28:08.842900+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:24:28.153740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.928248+0000","last_change":"2024-09-12T03:26:57.018418+0000","last_active":"2024-09-12T03:28:08.928248+0000","last_peered":"2024-09-12T03:28:08.928248+0000","last_clean":"2024-09-12T03:28:08.928248+0000","last_became_active":"2024-09-12T03:26:57.018081+0000","last_became_peered":"2024-09-12T03:26:57.018081+0000","last_unstale":"2024-09-12T03:28:08.928248+0000","last_undegraded":"2024-09-12T03:28:08.928248+0000","last_fullsized":"2024-09-12T03:28:08.928248+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:19.954332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.846845+0000","last_change":"2024-09-12T03:26:59.019632+0000","last_active":"2024-09-12T03:28:08.846845+0000","last_peered":"2024-09-12T03:28:08.846845+0000","last_clean":"2024-09-12T03:28:08.846845+0000","last_became_active":"2024-09-12T03:26:59.016847+0000","last_became_peered":"2024-09-12T03:26:59.016847+0000","last_unstale":"2024-09-12T03:28:08.846845+0000","last_undegraded":"2024-09-12T03:28:08.846845+0000","last_fullsized":"2024-09-12T03:28:08.846845+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:32:28.261954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.845288+0000","last_change":"2024-09-12T03:27:06.045628+0000","last_active":"2024-09-12T03:28:08.845288+0000","last_peered":"2024-09-12T03:28:08.845288+0000","last_clean":"2024-09-12T03:28:08.845288+0000","last_became_active":"2024-09-12T03:27:06.045141+0000","last_became_peered":"2024-09-12T03:27:06.045141+0000","last_unstale":"2024-09-12T03:28:08.845288+0000","last_undegraded":"2024-09-12T03:28:08.845288+0000","last_fullsized":"2024-09-12T03:28:08.845288+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:10:26.154446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.844027+0000","last_change":"2024-09-12T03:26:56.958615+0000","last_active":"2024-09-12T03:28:08.844027+0000","last_peered":"2024-09-12T03:28:08.844027+0000","last_clean":"2024-09-12T03:28:08.844027+0000","last_became_active":"2024-09-12T03:26:56.958295+0000","last_became_peered":"2024-09-12T03:26:56.958295+0000","last_unstale":"2024-09-12T03:28:08.844027+0000","last_undegraded":"2024-09-12T03:28:08.844027+0000","last_fullsized":"2024-09-12T03:28:08.844027+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:13:09.031983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.843042+0000","last_change":"2024-09-12T03:26:59.015376+0000","last_active":"2024-09-12T03:28:08.843042+0000","last_peered":"2024-09-12T03:28:08.843042+0000","last_clean":"2024-09-12T03:28:08.843042+0000","last_became_active":"2024-09-12T03:26:59.015055+0000","last_became_peered":"2024-09-12T03:26:59.015055+0000","last_unstale":"2024-09-12T03:28:08.843042+0000","last_undegraded":"2024-09-12T03:28:08.843042+0000","last_fullsized":"2024-09-12T03:28:08.843042+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:44:38.077558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.846460+0000","last_change":"2024-09-12T03:27:06.042785+0000","last_active":"2024-09-12T03:28:08.846460+0000","last_peered":"2024-09-12T03:28:08.846460+0000","last_clean":"2024-09-12T03:28:08.846460+0000","last_became_active":"2024-09-12T03:27:06.042379+0000","last_became_peered":"2024-09-12T03:27:06.042379+0000","last_unstale":"2024-09-12T03:28:08.846460+0000","last_undegraded":"2024-09-12T03:28:08.846460+0000","last_fullsized":"2024-09-12T03:28:08.846460+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:08:51.683470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.844454+0000","last_change":"2024-09-12T03:26:56.955846+0000","last_active":"2024-09-12T03:28:08.844454+0000","last_peered":"2024-09-12T03:28:08.844454+0000","last_clean":"2024-09-12T03:28:08.844454+0000","last_became_active":"2024-09-12T03:26:56.955476+0000","last_became_peered":"2024-09-12T03:26:56.955476+0000","last_unstale":"2024-09-12T03:28:08.844454+0000","last_undegraded":"2024-09-12T03:28:08.844454+0000","last_fullsized":"2024-09-12T03:28:08.844454+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:10:46.811039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.841131+0000","last_change":"2024-09-12T03:26:59.021498+0000","last_active":"2024-09-12T03:28:08.841131+0000","last_peered":"2024-09-12T03:28:08.841131+0000","last_clean":"2024-09-12T03:28:08.841131+0000","last_became_active":"2024-09-12T03:26:59.021196+0000","last_became_peered":"2024-09-12T03:26:59.021196+0000","last_unstale":"2024-09-12T03:28:08.841131+0000","last_undegraded":"2024-09-12T03:28:08.841131+0000","last_fullsized":"2024-09-12T03:28:08.841131+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:13:17.537648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.845349+0000","last_change":"2024-09-12T03:27:06.040597+0000","last_active":"2024-09-12T03:28:08.845349+0000","last_peered":"2024-09-12T03:28:08.845349+0000","last_clean":"2024-09-12T03:28:08.845349+0000","last_became_active":"2024-09-12T03:27:06.040149+0000","last_became_peered":"2024-09-12T03:27:06.040149+0000","last_unstale":"2024-09-12T03:28:08.845349+0000","last_undegraded":"2024-09-12T03:28:08.845349+0000","last_fullsized":"2024-09-12T03:28:08.845349+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:27.192367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.843662+0000","last_change":"2024-09-12T03:26:59.020542+0000","last_active":"2024-09-12T03:28:08.843662+0000","last_peered":"2024-09-12T03:28:08.843662+0000","last_clean":"2024-09-12T03:28:08.843662+0000","last_became_active":"2024-09-12T03:26:59.020189+0000","last_became_peered":"2024-09-12T03:26:59.020189+0000","last_unstale":"2024-09-12T03:28:08.843662+0000","last_undegraded":"2024-09-12T03:28:08.843662+0000","last_fullsized":"2024-09-12T03:28:08.843662+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:06:12.761331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.842698+0000","last_change":"2024-09-12T03:26:56.958320+0000","last_active":"2024-09-12T03:28:08.842698+0000","last_peered":"2024-09-12T03:28:08.842698+0000","last_clean":"2024-09-12T03:28:08.842698+0000","last_became_active":"2024-09-12T03:26:56.956965+0000","last_became_peered":"2024-09-12T03:26:56.956965+0000","last_unstale":"2024-09-12T03:28:08.842698+0000","last_undegraded":"2024-09-12T03:28:08.842698+0000","last_fullsized":"2024-09-12T03:28:08.842698+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:46.685994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.845734+0000","last_change":"2024-09-12T03:27:06.189973+0000","last_active":"2024-09-12T03:28:08.845734+0000","last_peered":"2024-09-12T03:28:08.845734+0000","last_clean":"2024-09-12T03:28:08.845734+0000","last_became_active":"2024-09-12T03:27:06.189454+0000","last_became_peered":"2024-09-12T03:27:06.189454+0000","last_unstale":"2024-09-12T03:28:08.845734+0000","last_undegraded":"2024-09-12T03:28:08.845734+0000","last_fullsized":"2024-09-12T03:28:08.845734+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:25.130566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.927982+0000","last_change":"2024-09-12T03:26:59.014586+0000","last_active":"2024-09-12T03:28:08.927982+0000","last_peered":"2024-09-12T03:28:08.927982+0000","last_clean":"2024-09-12T03:28:08.927982+0000","last_became_active":"2024-09-12T03:26:59.013764+0000","last_became_peered":"2024-09-12T03:26:59.013764+0000","last_unstale":"2024-09-12T03:28:08.927982+0000","last_undegraded":"2024-09-12T03:28:08.927982+0000","last_fullsized":"2024-09-12T03:28:08.927982+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:55:34.136770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.842601+0000","last_change":"2024-09-12T03:26:56.960415+0000","last_active":"2024-09-12T03:28:08.842601+0000","last_peered":"2024-09-12T03:28:08.842601+0000","last_clean":"2024-09-12T03:28:08.842601+0000","last_became_active":"2024-09-12T03:26:56.960043+0000","last_became_peered":"2024-09-12T03:26:56.960043+0000","last_unstale":"2024-09-12T03:28:08.842601+0000","last_undegraded":"2024-09-12T03:28:08.842601+0000","last_fullsized":"2024-09-12T03:28:08.842601+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:02:12.209929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.845523+0000","last_change":"2024-09-12T03:27:06.047002+0000","last_active":"2024-09-12T03:28:08.845523+0000","last_peered":"2024-09-12T03:28:08.845523+0000","last_clean":"2024-09-12T03:28:08.845523+0000","last_became_active":"2024-09-12T03:27:06.046520+0000","last_became_peered":"2024-09-12T03:27:06.046520+0000","last_unstale":"2024-09-12T03:28:08.845523+0000","last_undegraded":"2024-09-12T03:28:08.845523+0000","last_fullsized":"2024-09-12T03:28:08.845523+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:30.332831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.928022+0000","last_change":"2024-09-12T03:26:59.016811+0000","last_active":"2024-09-12T03:28:08.928022+0000","last_peered":"2024-09-12T03:28:08.928022+0000","last_clean":"2024-09-12T03:28:08.928022+0000","last_became_active":"2024-09-12T03:26:59.014667+0000","last_became_peered":"2024-09-12T03:26:59.014667+0000","last_unstale":"2024-09-12T03:28:08.928022+0000","last_undegraded":"2024-09-12T03:28:08.928022+0000","last_fullsized":"2024-09-12T03:28:08.928022+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:28:51.128895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.927764+0000","last_change":"2024-09-12T03:26:56.956177+0000","last_active":"2024-09-12T03:28:08.927764+0000","last_peered":"2024-09-12T03:28:08.927764+0000","last_clean":"2024-09-12T03:28:08.927764+0000","last_became_active":"2024-09-12T03:26:56.955730+0000","last_became_peered":"2024-09-12T03:26:56.955730+0000","last_unstale":"2024-09-12T03:28:08.927764+0000","last_undegraded":"2024-09-12T03:28:08.927764+0000","last_fullsized":"2024-09-12T03:28:08.927764+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:05:18.890917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.843898+0000","last_change":"2024-09-12T03:27:06.050894+0000","last_active":"2024-09-12T03:28:08.843898+0000","last_peered":"2024-09-12T03:28:08.843898+0000","last_clean":"2024-09-12T03:28:08.843898+0000","last_became_active":"2024-09-12T03:27:06.050440+0000","last_became_peered":"2024-09-12T03:27:06.050440+0000","last_unstale":"2024-09-12T03:28:08.843898+0000","last_undegraded":"2024-09-12T03:28:08.843898+0000","last_fullsized":"2024-09-12T03:28:08.843898+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:31:25.559665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.842439+0000","last_change":"2024-09-12T03:26:59.018167+0000","last_active":"2024-09-12T03:28:08.842439+0000","last_peered":"2024-09-12T03:28:08.842439+0000","last_clean":"2024-09-12T03:28:08.842439+0000","last_became_active":"2024-09-12T03:26:59.017718+0000","last_became_peered":"2024-09-12T03:26:59.017718+0000","last_unstale":"2024-09-12T03:28:08.842439+0000","last_undegraded":"2024-09-12T03:28:08.842439+0000","last_fullsized":"2024-09-12T03:28:08.842439+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:36:12.742570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.842821+0000","last_change":"2024-09-12T03:26:57.018829+0000","last_active":"2024-09-12T03:28:08.842821+0000","last_peered":"2024-09-12T03:28:08.842821+0000","last_clean":"2024-09-12T03:28:08.842821+0000","last_became_active":"2024-09-12T03:26:57.018467+0000","last_became_peered":"2024-09-12T03:26:57.018467+0000","last_unstale":"2024-09-12T03:28:08.842821+0000","last_undegraded":"2024-09-12T03:28:08.842821+0000","last_fullsized":"2024-09-12T03:28:08.842821+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:33:37.318703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.845899+0000","last_change":"2024-09-12T03:27:06.190560+0000","last_active":"2024-09-12T03:28:08.845899+0000","last_peered":"2024-09-12T03:28:08.845899+0000","last_clean":"2024-09-12T03:28:08.845899+0000","last_became_active":"2024-09-12T03:27:06.190008+0000","last_became_peered":"2024-09-12T03:27:06.190008+0000","last_unstale":"2024-09-12T03:28:08.845899+0000","last_undegraded":"2024-09-12T03:28:08.845899+0000","last_fullsized":"2024-09-12T03:28:08.845899+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:14.423372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.846689+0000","last_change":"2024-09-12T03:26:59.022149+0000","last_active":"2024-09-12T03:28:08.846689+0000","last_peered":"2024-09-12T03:28:08.846689+0000","last_clean":"2024-09-12T03:28:08.846689+0000","last_became_active":"2024-09-12T03:26:59.021830+0000","last_became_peered":"2024-09-12T03:26:59.021830+0000","last_unstale":"2024-09-12T03:28:08.846689+0000","last_undegraded":"2024-09-12T03:28:08.846689+0000","last_fullsized":"2024-09-12T03:28:08.846689+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:07.096908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.846205+0000","last_change":"2024-09-12T03:26:57.018335+0000","last_active":"2024-09-12T03:28:08.846205+0000","last_peered":"2024-09-12T03:28:08.846205+0000","last_clean":"2024-09-12T03:28:08.846205+0000","last_became_active":"2024-09-12T03:26:57.017993+0000","last_became_peered":"2024-09-12T03:26:57.017993+0000","last_unstale":"2024-09-12T03:28:08.846205+0000","last_undegraded":"2024-09-12T03:28:08.846205+0000","last_fullsized":"2024-09-12T03:28:08.846205+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:27:52.828836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.842912+0000","last_change":"2024-09-12T03:27:06.051146+0000","last_active":"2024-09-12T03:28:08.842912+0000","last_peered":"2024-09-12T03:28:08.842912+0000","last_clean":"2024-09-12T03:28:08.842912+0000","last_became_active":"2024-09-12T03:27:06.050599+0000","last_became_peered":"2024-09-12T03:27:06.050599+0000","last_unstale":"2024-09-12T03:28:08.842912+0000","last_undegraded":"2024-09-12T03:28:08.842912+0000","last_fullsized":"2024-09-12T03:28:08.842912+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:17:57.543994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.928221+0000","last_change":"2024-09-12T03:26:59.024832+0000","last_active":"2024-09-12T03:28:08.928221+0000","last_peered":"2024-09-12T03:28:08.928221+0000","last_clean":"2024-09-12T03:28:08.928221+0000","last_became_active":"2024-09-12T03:26:59.024473+0000","last_became_peered":"2024-09-12T03:26:59.024473+0000","last_unstale":"2024-09-12T03:28:08.928221+0000","last_undegraded":"2024-09-12T03:28:08.928221+0000","last_fullsized":"2024-09-12T03:28:08.928221+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:05:54.568496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'14","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.844093+0000","last_change":"2024-09-12T03:26:56.955213+0000","last_active":"2024-09-12T03:28:08.844093+0000","last_peered":"2024-09-12T03:28:08.844093+0000","last_clean":"2024-09-12T03:28:08.844093+0000","last_became_active":"2024-09-12T03:26:56.954746+0000","last_became_peered":"2024-09-12T03:26:56.954746+0000","last_unstale":"2024-09-12T03:28:08.844093+0000","last_undegraded":"2024-09-12T03:28:08.844093+0000","last_fullsized":"2024-09-12T03:28:08.844093+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:03.303931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.927417+0000","last_change":"2024-09-12T03:27:06.190398+0000","last_active":"2024-09-12T03:28:08.927417+0000","last_peered":"2024-09-12T03:28:08.927417+0000","last_clean":"2024-09-12T03:28:08.927417+0000","last_became_active":"2024-09-12T03:27:06.189861+0000","last_became_peered":"2024-09-12T03:27:06.189861+0000","last_unstale":"2024-09-12T03:28:08.927417+0000","last_undegraded":"2024-09-12T03:28:08.927417+0000","last_fullsized":"2024-09-12T03:28:08.927417+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:36:57.366931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.841792+0000","last_change":"2024-09-12T03:26:59.012702+0000","last_active":"2024-09-12T03:28:08.841792+0000","last_peered":"2024-09-12T03:28:08.841792+0000","last_clean":"2024-09-12T03:28:08.841792+0000","last_became_active":"2024-09-12T03:26:59.012421+0000","last_became_peered":"2024-09-12T03:26:59.012421+0000","last_unstale":"2024-09-12T03:28:08.841792+0000","last_undegraded":"2024-09-12T03:28:08.841792+0000","last_fullsized":"2024-09-12T03:28:08.841792+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:21:23.866131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.844078+0000","last_change":"2024-09-12T03:26:57.019744+0000","last_active":"2024-09-12T03:28:08.844078+0000","last_peered":"2024-09-12T03:28:08.844078+0000","last_clean":"2024-09-12T03:28:08.844078+0000","last_became_active":"2024-09-12T03:26:57.019397+0000","last_became_peered":"2024-09-12T03:26:57.019397+0000","last_unstale":"2024-09-12T03:28:08.844078+0000","last_undegraded":"2024-09-12T03:28:08.844078+0000","last_fullsized":"2024-09-12T03:28:08.844078+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:44.891829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.927763+0000","last_change":"2024-09-12T03:27:06.048452+0000","last_active":"2024-09-12T03:28:08.927763+0000","last_peered":"2024-09-12T03:28:08.927763+0000","last_clean":"2024-09-12T03:28:08.927763+0000","last_became_active":"2024-09-12T03:27:06.048011+0000","last_became_peered":"2024-09-12T03:27:06.048011+0000","last_unstale":"2024-09-12T03:28:08.927763+0000","last_undegraded":"2024-09-12T03:28:08.927763+0000","last_fullsized":"2024-09-12T03:28:08.927763+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:20:38.360428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.974833+0000","last_change":"2024-09-12T03:26:59.017212+0000","last_active":"2024-09-12T03:28:08.974833+0000","last_peered":"2024-09-12T03:28:08.974833+0000","last_clean":"2024-09-12T03:28:08.974833+0000","last_became_active":"2024-09-12T03:26:59.016605+0000","last_became_peered":"2024-09-12T03:26:59.016605+0000","last_unstale":"2024-09-12T03:28:08.974833+0000","last_undegraded":"2024-09-12T03:28:08.974833+0000","last_fullsized":"2024-09-12T03:28:08.974833+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:34.639464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.844194+0000","last_change":"2024-09-12T03:26:57.020236+0000","last_active":"2024-09-12T03:28:08.844194+0000","last_peered":"2024-09-12T03:28:08.844194+0000","last_clean":"2024-09-12T03:28:08.844194+0000","last_became_active":"2024-09-12T03:26:57.019899+0000","last_became_peered":"2024-09-12T03:26:57.019899+0000","last_unstale":"2024-09-12T03:28:08.844194+0000","last_undegraded":"2024-09-12T03:28:08.844194+0000","last_fullsized":"2024-09-12T03:28:08.844194+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:03:16.184419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.975992+0000","last_change":"2024-09-12T03:28:08.975992+0000","last_active":"2024-09-12T03:28:08.975992+0000","last_peered":"2024-09-12T03:28:08.975992+0000","last_clean":"2024-09-12T03:28:08.975992+0000","last_became_active":"2024-09-12T03:28:08.975516+0000","last_became_peered":"2024-09-12T03:28:08.975516+0000","last_unstale":"2024-09-12T03:28:08.975992+0000","last_undegraded":"2024-09-12T03:28:08.975992+0000","last_fullsized":"2024-09-12T03:28:08.975992+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:37:23.151370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.974751+0000","last_change":"2024-09-12T03:26:59.014332+0000","last_active":"2024-09-12T03:28:08.974751+0000","last_peered":"2024-09-12T03:28:08.974751+0000","last_clean":"2024-09-12T03:28:08.974751+0000","last_became_active":"2024-09-12T03:26:59.014019+0000","last_became_peered":"2024-09-12T03:26:59.014019+0000","last_unstale":"2024-09-12T03:28:08.974751+0000","last_undegraded":"2024-09-12T03:28:08.974751+0000","last_fullsized":"2024-09-12T03:28:08.974751+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:03:18.020391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.843887+0000","last_change":"2024-09-12T03:26:56.956591+0000","last_active":"2024-09-12T03:28:08.843887+0000","last_peered":"2024-09-12T03:28:08.843887+0000","last_clean":"2024-09-12T03:28:08.843887+0000","last_became_active":"2024-09-12T03:26:56.956028+0000","last_became_peered":"2024-09-12T03:26:56.956028+0000","last_unstale":"2024-09-12T03:28:08.843887+0000","last_undegraded":"2024-09-12T03:28:08.843887+0000","last_fullsized":"2024-09-12T03:28:08.843887+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:55:20.500536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.841580+0000","last_change":"2024-09-12T03:27:06.052630+0000","last_active":"2024-09-12T03:28:08.841580+0000","last_peered":"2024-09-12T03:28:08.841580+0000","last_clean":"2024-09-12T03:28:08.841580+0000","last_became_active":"2024-09-12T03:27:06.052240+0000","last_became_peered":"2024-09-12T03:27:06.052240+0000","last_unstale":"2024-09-12T03:28:08.841580+0000","last_undegraded":"2024-09-12T03:28:08.841580+0000","last_fullsized":"2024-09-12T03:28:08.841580+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:26:49.356330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.840281+0000","last_change":"2024-09-12T03:26:59.022054+0000","last_active":"2024-09-12T03:28:08.840281+0000","last_peered":"2024-09-12T03:28:08.840281+0000","last_clean":"2024-09-12T03:28:08.840281+0000","last_became_active":"2024-09-12T03:26:59.021673+0000","last_became_peered":"2024-09-12T03:26:59.021673+0000","last_unstale":"2024-09-12T03:28:08.840281+0000","last_undegraded":"2024-09-12T03:28:08.840281+0000","last_fullsized":"2024-09-12T03:28:08.840281+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:07:07.581766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":91,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:26.216914+0000","last_change":"2024-09-12T03:26:57.018607+0000","last_active":"2024-09-12T03:28:26.216914+0000","last_peered":"2024-09-12T03:28:26.216914+0000","last_clean":"2024-09-12T03:28:26.216914+0000","last_became_active":"2024-09-12T03:26:57.018293+0000","last_became_peered":"2024-09-12T03:26:57.018293+0000","last_unstale":"2024-09-12T03:28:26.216914+0000","last_undegraded":"2024-09-12T03:28:26.216914+0000","last_fullsized":"2024-09-12T03:28:26.216914+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:23:57.245446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.841617+0000","last_change":"2024-09-12T03:27:06.190452+0000","last_active":"2024-09-12T03:28:08.841617+0000","last_peered":"2024-09-12T03:28:08.841617+0000","last_clean":"2024-09-12T03:28:08.841617+0000","last_became_active":"2024-09-12T03:27:06.189940+0000","last_became_peered":"2024-09-12T03:27:06.189940+0000","last_unstale":"2024-09-12T03:28:08.841617+0000","last_undegraded":"2024-09-12T03:28:08.841617+0000","last_fullsized":"2024-09-12T03:28:08.841617+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:25:23.114124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.841121+0000","last_change":"2024-09-12T03:26:59.019294+0000","last_active":"2024-09-12T03:28:08.841121+0000","last_peered":"2024-09-12T03:28:08.841121+0000","last_clean":"2024-09-12T03:28:08.841121+0000","last_became_active":"2024-09-12T03:26:59.018976+0000","last_became_peered":"2024-09-12T03:26:59.018976+0000","last_unstale":"2024-09-12T03:28:08.841121+0000","last_undegraded":"2024-09-12T03:28:08.841121+0000","last_fullsized":"2024-09-12T03:28:08.841121+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:06:38.571171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.842508+0000","last_change":"2024-09-12T03:26:56.960077+0000","last_active":"2024-09-12T03:28:08.842508+0000","last_peered":"2024-09-12T03:28:08.842508+0000","last_clean":"2024-09-12T03:28:08.842508+0000","last_became_active":"2024-09-12T03:26:56.958892+0000","last_became_peered":"2024-09-12T03:26:56.958892+0000","last_unstale":"2024-09-12T03:28:08.842508+0000","last_undegraded":"2024-09-12T03:28:08.842508+0000","last_fullsized":"2024-09-12T03:28:08.842508+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:27:04.779026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.975849+0000","last_change":"2024-09-12T03:28:08.975198+0000","last_active":"2024-09-12T03:28:08.975849+0000","last_peered":"2024-09-12T03:28:08.975849+0000","last_clean":"2024-09-12T03:28:08.975849+0000","last_became_active":"2024-09-12T03:28:08.974727+0000","last_became_peered":"2024-09-12T03:28:08.974727+0000","last_unstale":"2024-09-12T03:28:08.975849+0000","last_undegraded":"2024-09-12T03:28:08.975849+0000","last_fullsized":"2024-09-12T03:28:08.975849+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:04:45.121889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.927570+0000","last_change":"2024-09-12T03:26:59.024060+0000","last_active":"2024-09-12T03:28:08.927570+0000","last_peered":"2024-09-12T03:28:08.927570+0000","last_clean":"2024-09-12T03:28:08.927570+0000","last_became_active":"2024-09-12T03:26:59.023726+0000","last_became_peered":"2024-09-12T03:26:59.023726+0000","last_unstale":"2024-09-12T03:28:08.927570+0000","last_undegraded":"2024-09-12T03:28:08.927570+0000","last_fullsized":"2024-09-12T03:28:08.927570+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:49:44.984188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.846866+0000","last_change":"2024-09-12T03:26:56.957847+0000","last_active":"2024-09-12T03:28:08.846866+0000","last_peered":"2024-09-12T03:28:08.846866+0000","last_clean":"2024-09-12T03:28:08.846866+0000","last_became_active":"2024-09-12T03:26:56.957491+0000","last_became_peered":"2024-09-12T03:26:56.957491+0000","last_unstale":"2024-09-12T03:28:08.846866+0000","last_undegraded":"2024-09-12T03:28:08.846866+0000","last_fullsized":"2024-09-12T03:28:08.846866+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:06:23.028881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":363,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.842614+0000","last_change":"2024-09-12T03:25:45.211570+0000","last_active":"2024-09-12T03:28:08.842614+0000","last_peered":"2024-09-12T03:28:08.842614+0000","last_clean":"2024-09-12T03:28:08.842614+0000","last_became_active":"2024-09-12T03:25:45.211052+0000","last_became_peered":"2024-09-12T03:25:45.211052+0000","last_unstale":"2024-09-12T03:28:08.842614+0000","last_undegraded":"2024-09-12T03:28:08.842614+0000","last_fullsized":"2024-09-12T03:28:08.842614+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:21.915950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:21.915950+0000","last_clean_scrub_stamp":"2024-09-12T03:22:21.915950+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:03:06.560617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.843924+0000","last_change":"2024-09-12T03:27:06.046319+0000","last_active":"2024-09-12T03:28:08.843924+0000","last_peered":"2024-09-12T03:28:08.843924+0000","last_clean":"2024-09-12T03:28:08.843924+0000","last_became_active":"2024-09-12T03:27:06.045864+0000","last_became_peered":"2024-09-12T03:27:06.045864+0000","last_unstale":"2024-09-12T03:28:08.843924+0000","last_undegraded":"2024-09-12T03:28:08.843924+0000","last_fullsized":"2024-09-12T03:28:08.843924+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:45:25.224775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.844185+0000","last_change":"2024-09-12T03:26:59.019197+0000","last_active":"2024-09-12T03:28:08.844185+0000","last_peered":"2024-09-12T03:28:08.844185+0000","last_clean":"2024-09-12T03:28:08.844185+0000","last_became_active":"2024-09-12T03:26:59.018868+0000","last_became_peered":"2024-09-12T03:26:59.018868+0000","last_unstale":"2024-09-12T03:28:08.844185+0000","last_undegraded":"2024-09-12T03:28:08.844185+0000","last_fullsized":"2024-09-12T03:28:08.844185+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:50:51.707556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.843494+0000","last_change":"2024-09-12T03:26:57.019888+0000","last_active":"2024-09-12T03:28:08.843494+0000","last_peered":"2024-09-12T03:28:08.843494+0000","last_clean":"2024-09-12T03:28:08.843494+0000","last_became_active":"2024-09-12T03:26:57.019378+0000","last_became_peered":"2024-09-12T03:26:57.019378+0000","last_unstale":"2024-09-12T03:28:08.843494+0000","last_undegraded":"2024-09-12T03:28:08.843494+0000","last_fullsized":"2024-09-12T03:28:08.843494+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:44:02.298726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.842731+0000","last_change":"2024-09-12T03:27:06.043780+0000","last_active":"2024-09-12T03:28:08.842731+0000","last_peered":"2024-09-12T03:28:08.842731+0000","last_clean":"2024-09-12T03:28:08.842731+0000","last_became_active":"2024-09-12T03:27:06.042770+0000","last_became_peered":"2024-09-12T03:27:06.042770+0000","last_unstale":"2024-09-12T03:28:08.842731+0000","last_undegraded":"2024-09-12T03:28:08.842731+0000","last_fullsized":"2024-09-12T03:28:08.842731+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:32:40.801908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.843955+0000","last_change":"2024-09-12T03:26:59.022996+0000","last_active":"2024-09-12T03:28:08.843955+0000","last_peered":"2024-09-12T03:28:08.843955+0000","last_clean":"2024-09-12T03:28:08.843955+0000","last_became_active":"2024-09-12T03:26:59.022675+0000","last_became_peered":"2024-09-12T03:26:59.022675+0000","last_unstale":"2024-09-12T03:28:08.843955+0000","last_undegraded":"2024-09-12T03:28:08.843955+0000","last_fullsized":"2024-09-12T03:28:08.843955+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:09.674971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.841256+0000","last_change":"2024-09-12T03:26:56.957101+0000","last_active":"2024-09-12T03:28:08.841256+0000","last_peered":"2024-09-12T03:28:08.841256+0000","last_clean":"2024-09-12T03:28:08.841256+0000","last_became_active":"2024-09-12T03:26:56.956694+0000","last_became_peered":"2024-09-12T03:26:56.956694+0000","last_unstale":"2024-09-12T03:28:08.841256+0000","last_undegraded":"2024-09-12T03:28:08.841256+0000","last_fullsized":"2024-09-12T03:28:08.841256+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:29.793229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.846038+0000","last_change":"2024-09-12T03:27:06.049382+0000","last_active":"2024-09-12T03:28:08.846038+0000","last_peered":"2024-09-12T03:28:08.846038+0000","last_clean":"2024-09-12T03:28:08.846038+0000","last_became_active":"2024-09-12T03:27:06.048961+0000","last_became_peered":"2024-09-12T03:27:06.048961+0000","last_unstale":"2024-09-12T03:28:08.846038+0000","last_undegraded":"2024-09-12T03:28:08.846038+0000","last_fullsized":"2024-09-12T03:28:08.846038+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:32.400328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.844226+0000","last_change":"2024-09-12T03:26:59.020146+0000","last_active":"2024-09-12T03:28:08.844226+0000","last_peered":"2024-09-12T03:28:08.844226+0000","last_clean":"2024-09-12T03:28:08.844226+0000","last_became_active":"2024-09-12T03:26:59.019854+0000","last_became_peered":"2024-09-12T03:26:59.019854+0000","last_unstale":"2024-09-12T03:28:08.844226+0000","last_undegraded":"2024-09-12T03:28:08.844226+0000","last_fullsized":"2024-09-12T03:28:08.844226+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:38:47.476070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.841950+0000","last_change":"2024-09-12T03:26:56.956237+0000","last_active":"2024-09-12T03:28:08.841950+0000","last_peered":"2024-09-12T03:28:08.841950+0000","last_clean":"2024-09-12T03:28:08.841950+0000","last_became_active":"2024-09-12T03:26:56.955941+0000","last_became_peered":"2024-09-12T03:26:56.955941+0000","last_unstale":"2024-09-12T03:28:08.841950+0000","last_undegraded":"2024-09-12T03:28:08.841950+0000","last_fullsized":"2024-09-12T03:28:08.841950+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:08:53.422429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.840107+0000","last_change":"2024-09-12T03:27:06.050916+0000","last_active":"2024-09-12T03:28:08.840107+0000","last_peered":"2024-09-12T03:28:08.840107+0000","last_clean":"2024-09-12T03:28:08.840107+0000","last_became_active":"2024-09-12T03:27:06.050494+0000","last_became_peered":"2024-09-12T03:27:06.050494+0000","last_unstale":"2024-09-12T03:28:08.840107+0000","last_undegraded":"2024-09-12T03:28:08.840107+0000","last_fullsized":"2024-09-12T03:28:08.840107+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:58:40.830354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.928008+0000","last_change":"2024-09-12T03:26:59.025310+0000","last_active":"2024-09-12T03:28:08.928008+0000","last_peered":"2024-09-12T03:28:08.928008+0000","last_clean":"2024-09-12T03:28:08.928008+0000","last_became_active":"2024-09-12T03:26:59.024917+0000","last_became_peered":"2024-09-12T03:26:59.024917+0000","last_unstale":"2024-09-12T03:28:08.928008+0000","last_undegraded":"2024-09-12T03:28:08.928008+0000","last_fullsized":"2024-09-12T03:28:08.928008+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:14:19.786496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.843881+0000","last_change":"2024-09-12T03:26:56.959434+0000","last_active":"2024-09-12T03:28:08.843881+0000","last_peered":"2024-09-12T03:28:08.843881+0000","last_clean":"2024-09-12T03:28:08.843881+0000","last_became_active":"2024-09-12T03:26:56.959095+0000","last_became_peered":"2024-09-12T03:26:56.959095+0000","last_unstale":"2024-09-12T03:28:08.843881+0000","last_undegraded":"2024-09-12T03:28:08.843881+0000","last_fullsized":"2024-09-12T03:28:08.843881+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:43:39.291022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.840153+0000","last_change":"2024-09-12T03:27:06.054028+0000","last_active":"2024-09-12T03:28:08.840153+0000","last_peered":"2024-09-12T03:28:08.840153+0000","last_clean":"2024-09-12T03:28:08.840153+0000","last_became_active":"2024-09-12T03:27:06.053573+0000","last_became_peered":"2024-09-12T03:27:06.053573+0000","last_unstale":"2024-09-12T03:28:08.840153+0000","last_undegraded":"2024-09-12T03:28:08.840153+0000","last_fullsized":"2024-09-12T03:28:08.840153+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:05.977501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.840384+0000","last_change":"2024-09-12T03:26:59.022978+0000","last_active":"2024-09-12T03:28:08.840384+0000","last_peered":"2024-09-12T03:28:08.840384+0000","last_clean":"2024-09-12T03:28:08.840384+0000","last_became_active":"2024-09-12T03:26:59.022555+0000","last_became_peered":"2024-09-12T03:26:59.022555+0000","last_unstale":"2024-09-12T03:28:08.840384+0000","last_undegraded":"2024-09-12T03:28:08.840384+0000","last_fullsized":"2024-09-12T03:28:08.840384+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:51.903788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.842846+0000","last_change":"2024-09-12T03:26:57.018735+0000","last_active":"2024-09-12T03:28:08.842846+0000","last_peered":"2024-09-12T03:28:08.842846+0000","last_clean":"2024-09-12T03:28:08.842846+0000","last_became_active":"2024-09-12T03:26:57.018367+0000","last_became_peered":"2024-09-12T03:26:57.018367+0000","last_unstale":"2024-09-12T03:28:08.842846+0000","last_undegraded":"2024-09-12T03:28:08.842846+0000","last_fullsized":"2024-09-12T03:28:08.842846+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:25.420324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.841856+0000","last_change":"2024-09-12T03:27:06.046626+0000","last_active":"2024-09-12T03:28:08.841856+0000","last_peered":"2024-09-12T03:28:08.841856+0000","last_clean":"2024-09-12T03:28:08.841856+0000","last_became_active":"2024-09-12T03:27:06.046234+0000","last_became_peered":"2024-09-12T03:27:06.046234+0000","last_unstale":"2024-09-12T03:28:08.841856+0000","last_undegraded":"2024-09-12T03:28:08.841856+0000","last_fullsized":"2024-09-12T03:28:08.841856+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:05:57.353206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.842361+0000","last_change":"2024-09-12T03:26:59.023541+0000","last_active":"2024-09-12T03:28:08.842361+0000","last_peered":"2024-09-12T03:28:08.842361+0000","last_clean":"2024-09-12T03:28:08.842361+0000","last_became_active":"2024-09-12T03:26:59.023229+0000","last_became_peered":"2024-09-12T03:26:59.023229+0000","last_unstale":"2024-09-12T03:28:08.842361+0000","last_undegraded":"2024-09-12T03:28:08.842361+0000","last_fullsized":"2024-09-12T03:28:08.842361+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:27:58.514616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'4","reported_seq":94,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:26.217681+0000","last_change":"2024-09-12T03:26:56.962703+0000","last_active":"2024-09-12T03:28:26.217681+0000","last_peered":"2024-09-12T03:28:26.217681+0000","last_clean":"2024-09-12T03:28:26.217681+0000","last_became_active":"2024-09-12T03:26:56.962280+0000","last_became_peered":"2024-09-12T03:26:56.962280+0000","last_unstale":"2024-09-12T03:28:26.217681+0000","last_undegraded":"2024-09-12T03:28:26.217681+0000","last_fullsized":"2024-09-12T03:28:26.217681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:01.045200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.843959+0000","last_change":"2024-09-12T03:27:06.054791+0000","last_active":"2024-09-12T03:28:08.843959+0000","last_peered":"2024-09-12T03:28:08.843959+0000","last_clean":"2024-09-12T03:28:08.843959+0000","last_became_active":"2024-09-12T03:27:06.054311+0000","last_became_peered":"2024-09-12T03:27:06.054311+0000","last_unstale":"2024-09-12T03:28:08.843959+0000","last_undegraded":"2024-09-12T03:28:08.843959+0000","last_fullsized":"2024-09-12T03:28:08.843959+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:21:59.115200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.842867+0000","last_change":"2024-09-12T03:26:59.023971+0000","last_active":"2024-09-12T03:28:08.842867+0000","last_peered":"2024-09-12T03:28:08.842867+0000","last_clean":"2024-09-12T03:28:08.842867+0000","last_became_active":"2024-09-12T03:26:59.023653+0000","last_became_peered":"2024-09-12T03:26:59.023653+0000","last_unstale":"2024-09-12T03:28:08.842867+0000","last_undegraded":"2024-09-12T03:28:08.842867+0000","last_fullsized":"2024-09-12T03:28:08.842867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:10:08.571177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.840423+0000","last_change":"2024-09-12T03:26:56.962459+0000","last_active":"2024-09-12T03:28:08.840423+0000","last_peered":"2024-09-12T03:28:08.840423+0000","last_clean":"2024-09-12T03:28:08.840423+0000","last_became_active":"2024-09-12T03:26:56.962039+0000","last_became_peered":"2024-09-12T03:26:56.962039+0000","last_unstale":"2024-09-12T03:28:08.840423+0000","last_undegraded":"2024-09-12T03:28:08.840423+0000","last_fullsized":"2024-09-12T03:28:08.840423+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:08:44.214788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.927632+0000","last_change":"2024-09-12T03:27:06.049058+0000","last_active":"2024-09-12T03:28:08.927632+0000","last_peered":"2024-09-12T03:28:08.927632+0000","last_clean":"2024-09-12T03:28:08.927632+0000","last_became_active":"2024-09-12T03:27:06.048606+0000","last_became_peered":"2024-09-12T03:27:06.048606+0000","last_unstale":"2024-09-12T03:28:08.927632+0000","last_undegraded":"2024-09-12T03:28:08.927632+0000","last_fullsized":"2024-09-12T03:28:08.927632+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:23:14.571184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.841292+0000","last_change":"2024-09-12T03:26:59.017832+0000","last_active":"2024-09-12T03:28:08.841292+0000","last_peered":"2024-09-12T03:28:08.841292+0000","last_clean":"2024-09-12T03:28:08.841292+0000","last_became_active":"2024-09-12T03:26:59.017497+0000","last_became_peered":"2024-09-12T03:26:59.017497+0000","last_unstale":"2024-09-12T03:28:08.841292+0000","last_undegraded":"2024-09-12T03:28:08.841292+0000","last_fullsized":"2024-09-12T03:28:08.841292+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:49.753556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.842849+0000","last_change":"2024-09-12T03:26:56.957877+0000","last_active":"2024-09-12T03:28:08.842849+0000","last_peered":"2024-09-12T03:28:08.842849+0000","last_clean":"2024-09-12T03:28:08.842849+0000","last_became_active":"2024-09-12T03:26:56.957502+0000","last_became_peered":"2024-09-12T03:26:56.957502+0000","last_unstale":"2024-09-12T03:28:08.842849+0000","last_undegraded":"2024-09-12T03:28:08.842849+0000","last_fullsized":"2024-09-12T03:28:08.842849+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:35.851957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.840767+0000","last_change":"2024-09-12T03:27:06.047070+0000","last_active":"2024-09-12T03:28:08.840767+0000","last_peered":"2024-09-12T03:28:08.840767+0000","last_clean":"2024-09-12T03:28:08.840767+0000","last_became_active":"2024-09-12T03:27:06.046612+0000","last_became_peered":"2024-09-12T03:27:06.046612+0000","last_unstale":"2024-09-12T03:28:08.840767+0000","last_undegraded":"2024-09-12T03:28:08.840767+0000","last_fullsized":"2024-09-12T03:28:08.840767+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:36:53.960456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.843815+0000","last_change":"2024-09-12T03:26:59.019856+0000","last_active":"2024-09-12T03:28:08.843815+0000","last_peered":"2024-09-12T03:28:08.843815+0000","last_clean":"2024-09-12T03:28:08.843815+0000","last_became_active":"2024-09-12T03:26:59.019498+0000","last_became_peered":"2024-09-12T03:26:59.019498+0000","last_unstale":"2024-09-12T03:28:08.843815+0000","last_undegraded":"2024-09-12T03:28:08.843815+0000","last_fullsized":"2024-09-12T03:28:08.843815+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:02:18.000549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'33","reported_seq":188,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:30.223581+0000","last_change":"2024-09-12T03:26:56.958919+0000","last_active":"2024-09-12T03:28:30.223581+0000","last_peered":"2024-09-12T03:28:30.223581+0000","last_clean":"2024-09-12T03:28:30.223581+0000","last_became_active":"2024-09-12T03:26:56.958455+0000","last_became_peered":"2024-09-12T03:26:56.958455+0000","last_unstale":"2024-09-12T03:28:30.223581+0000","last_undegraded":"2024-09-12T03:28:30.223581+0000","last_fullsized":"2024-09-12T03:28:30.223581+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:05:19.318922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62076,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":121,"num_read_kb":122,"num_write":34,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.843172+0000","last_change":"2024-09-12T03:27:06.047094+0000","last_active":"2024-09-12T03:28:08.843172+0000","last_peered":"2024-09-12T03:28:08.843172+0000","last_clean":"2024-09-12T03:28:08.843172+0000","last_became_active":"2024-09-12T03:27:06.046702+0000","last_became_peered":"2024-09-12T03:27:06.046702+0000","last_unstale":"2024-09-12T03:28:08.843172+0000","last_undegraded":"2024-09-12T03:28:08.843172+0000","last_fullsized":"2024-09-12T03:28:08.843172+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:46:57.557056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.843360+0000","last_change":"2024-09-12T03:26:59.018905+0000","last_active":"2024-09-12T03:28:08.843360+0000","last_peered":"2024-09-12T03:28:08.843360+0000","last_clean":"2024-09-12T03:28:08.843360+0000","last_became_active":"2024-09-12T03:26:59.018492+0000","last_became_peered":"2024-09-12T03:26:59.018492+0000","last_unstale":"2024-09-12T03:28:08.843360+0000","last_undegraded":"2024-09-12T03:28:08.843360+0000","last_fullsized":"2024-09-12T03:28:08.843360+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:11.953999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.841210+0000","last_change":"2024-09-12T03:26:57.018607+0000","last_active":"2024-09-12T03:28:08.841210+0000","last_peered":"2024-09-12T03:28:08.841210+0000","last_clean":"2024-09-12T03:28:08.841210+0000","last_became_active":"2024-09-12T03:26:57.018255+0000","last_became_peered":"2024-09-12T03:26:57.018255+0000","last_unstale":"2024-09-12T03:28:08.841210+0000","last_undegraded":"2024-09-12T03:28:08.841210+0000","last_fullsized":"2024-09-12T03:28:08.841210+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:39.029308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.975859+0000","last_change":"2024-09-12T03:28:08.975859+0000","last_active":"2024-09-12T03:28:08.975859+0000","last_peered":"2024-09-12T03:28:08.975859+0000","last_clean":"2024-09-12T03:28:08.975859+0000","last_became_active":"2024-09-12T03:28:08.975408+0000","last_became_peered":"2024-09-12T03:28:08.975408+0000","last_unstale":"2024-09-12T03:28:08.975859+0000","last_undegraded":"2024-09-12T03:28:08.975859+0000","last_fullsized":"2024-09-12T03:28:08.975859+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:13:01.181897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.846908+0000","last_change":"2024-09-12T03:26:59.025406+0000","last_active":"2024-09-12T03:28:08.846908+0000","last_peered":"2024-09-12T03:28:08.846908+0000","last_clean":"2024-09-12T03:28:08.846908+0000","last_became_active":"2024-09-12T03:26:59.025091+0000","last_became_peered":"2024-09-12T03:26:59.025091+0000","last_unstale":"2024-09-12T03:28:08.846908+0000","last_undegraded":"2024-09-12T03:28:08.846908+0000","last_fullsized":"2024-09-12T03:28:08.846908+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:39:08.297785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.844127+0000","last_change":"2024-09-12T03:26:56.961325+0000","last_active":"2024-09-12T03:28:08.844127+0000","last_peered":"2024-09-12T03:28:08.844127+0000","last_clean":"2024-09-12T03:28:08.844127+0000","last_became_active":"2024-09-12T03:26:56.960806+0000","last_became_peered":"2024-09-12T03:26:56.960806+0000","last_unstale":"2024-09-12T03:28:08.844127+0000","last_undegraded":"2024-09-12T03:28:08.844127+0000","last_fullsized":"2024-09-12T03:28:08.844127+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:44:11.209394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.843094+0000","last_change":"2024-09-12T03:27:06.055331+0000","last_active":"2024-09-12T03:28:08.843094+0000","last_peered":"2024-09-12T03:28:08.843094+0000","last_clean":"2024-09-12T03:28:08.843094+0000","last_became_active":"2024-09-12T03:27:06.054227+0000","last_became_peered":"2024-09-12T03:27:06.054227+0000","last_unstale":"2024-09-12T03:28:08.843094+0000","last_undegraded":"2024-09-12T03:28:08.843094+0000","last_fullsized":"2024-09-12T03:28:08.843094+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:35:10.446231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.843574+0000","last_change":"2024-09-12T03:26:59.012095+0000","last_active":"2024-09-12T03:28:08.843574+0000","last_peered":"2024-09-12T03:28:08.843574+0000","last_clean":"2024-09-12T03:28:08.843574+0000","last_became_active":"2024-09-12T03:26:59.011592+0000","last_became_peered":"2024-09-12T03:26:59.011592+0000","last_unstale":"2024-09-12T03:28:08.843574+0000","last_undegraded":"2024-09-12T03:28:08.843574+0000","last_fullsized":"2024-09-12T03:28:08.843574+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:49:33.026996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":93,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:29.222014+0000","last_change":"2024-09-12T03:26:56.959353+0000","last_active":"2024-09-12T03:28:29.222014+0000","last_peered":"2024-09-12T03:28:29.222014+0000","last_clean":"2024-09-12T03:28:29.222014+0000","last_became_active":"2024-09-12T03:26:56.958767+0000","last_became_peered":"2024-09-12T03:26:56.958767+0000","last_unstale":"2024-09-12T03:28:29.222014+0000","last_undegraded":"2024-09-12T03:28:29.222014+0000","last_fullsized":"2024-09-12T03:28:29.222014+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:37:51.518907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.927484+0000","last_change":"2024-09-12T03:27:06.043745+0000","last_active":"2024-09-12T03:28:08.927484+0000","last_peered":"2024-09-12T03:28:08.927484+0000","last_clean":"2024-09-12T03:28:08.927484+0000","last_became_active":"2024-09-12T03:27:06.042855+0000","last_became_peered":"2024-09-12T03:27:06.042855+0000","last_unstale":"2024-09-12T03:28:08.927484+0000","last_undegraded":"2024-09-12T03:28:08.927484+0000","last_fullsized":"2024-09-12T03:28:08.927484+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:14:34.939593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.846475+0000","last_change":"2024-09-12T03:27:06.052561+0000","last_active":"2024-09-12T03:28:08.846475+0000","last_peered":"2024-09-12T03:28:08.846475+0000","last_clean":"2024-09-12T03:28:08.846475+0000","last_became_active":"2024-09-12T03:27:06.052192+0000","last_became_peered":"2024-09-12T03:27:06.052192+0000","last_unstale":"2024-09-12T03:28:08.846475+0000","last_undegraded":"2024-09-12T03:28:08.846475+0000","last_fullsized":"2024-09-12T03:28:08.846475+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:31:31.800188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.842792+0000","last_change":"2024-09-12T03:26:59.012762+0000","last_active":"2024-09-12T03:28:08.842792+0000","last_peered":"2024-09-12T03:28:08.842792+0000","last_clean":"2024-09-12T03:28:08.842792+0000","last_became_active":"2024-09-12T03:26:59.012400+0000","last_became_peered":"2024-09-12T03:26:59.012400+0000","last_unstale":"2024-09-12T03:28:08.842792+0000","last_undegraded":"2024-09-12T03:28:08.842792+0000","last_fullsized":"2024-09-12T03:28:08.842792+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:35:13.573284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.840534+0000","last_change":"2024-09-12T03:26:56.962454+0000","last_active":"2024-09-12T03:28:08.840534+0000","last_peered":"2024-09-12T03:28:08.840534+0000","last_clean":"2024-09-12T03:28:08.840534+0000","last_became_active":"2024-09-12T03:26:56.962026+0000","last_became_peered":"2024-09-12T03:26:56.962026+0000","last_unstale":"2024-09-12T03:28:08.840534+0000","last_undegraded":"2024-09-12T03:28:08.840534+0000","last_fullsized":"2024-09-12T03:28:08.840534+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:00:23.501942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.928066+0000","last_change":"2024-09-12T03:27:06.053946+0000","last_active":"2024-09-12T03:28:08.928066+0000","last_peered":"2024-09-12T03:28:08.928066+0000","last_clean":"2024-09-12T03:28:08.928066+0000","last_became_active":"2024-09-12T03:27:06.053494+0000","last_became_peered":"2024-09-12T03:27:06.053494+0000","last_unstale":"2024-09-12T03:28:08.928066+0000","last_undegraded":"2024-09-12T03:28:08.928066+0000","last_fullsized":"2024-09-12T03:28:08.928066+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:13:11.537918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.929840+0000","last_change":"2024-09-12T03:28:08.929156+0000","last_active":"2024-09-12T03:28:08.929840+0000","last_peered":"2024-09-12T03:28:08.929840+0000","last_clean":"2024-09-12T03:28:08.929840+0000","last_became_active":"2024-09-12T03:28:08.928761+0000","last_became_peered":"2024-09-12T03:28:08.928761+0000","last_unstale":"2024-09-12T03:28:08.929840+0000","last_undegraded":"2024-09-12T03:28:08.929840+0000","last_fullsized":"2024-09-12T03:28:08.929840+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:20:00.775981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.841127+0000","last_change":"2024-09-12T03:26:56.958944+0000","last_active":"2024-09-12T03:28:08.841127+0000","last_peered":"2024-09-12T03:28:08.841127+0000","last_clean":"2024-09-12T03:28:08.841127+0000","last_became_active":"2024-09-12T03:26:56.958511+0000","last_became_peered":"2024-09-12T03:26:56.958511+0000","last_unstale":"2024-09-12T03:28:08.841127+0000","last_undegraded":"2024-09-12T03:28:08.841127+0000","last_fullsized":"2024-09-12T03:28:08.841127+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:52:37.871482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.973581+0000","last_change":"2024-09-12T03:26:56.961442+0000","last_active":"2024-09-12T03:28:08.973581+0000","last_peered":"2024-09-12T03:28:08.973581+0000","last_clean":"2024-09-12T03:28:08.973581+0000","last_became_active":"2024-09-12T03:26:56.961078+0000","last_became_peered":"2024-09-12T03:26:56.961078+0000","last_unstale":"2024-09-12T03:28:08.973581+0000","last_undegraded":"2024-09-12T03:28:08.973581+0000","last_fullsized":"2024-09-12T03:28:08.973581+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:17.185990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.840543+0000","last_change":"2024-09-12T03:26:59.022873+0000","last_active":"2024-09-12T03:28:08.840543+0000","last_peered":"2024-09-12T03:28:08.840543+0000","last_clean":"2024-09-12T03:28:08.840543+0000","last_became_active":"2024-09-12T03:26:59.022391+0000","last_became_peered":"2024-09-12T03:26:59.022391+0000","last_unstale":"2024-09-12T03:28:08.840543+0000","last_undegraded":"2024-09-12T03:28:08.840543+0000","last_fullsized":"2024-09-12T03:28:08.840543+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:05:08.138480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.847395+0000","last_change":"2024-09-12T03:28:08.847395+0000","last_active":"2024-09-12T03:28:08.847395+0000","last_peered":"2024-09-12T03:28:08.847395+0000","last_clean":"2024-09-12T03:28:08.847395+0000","last_became_active":"2024-09-12T03:28:08.846580+0000","last_became_peered":"2024-09-12T03:28:08.846580+0000","last_unstale":"2024-09-12T03:28:08.847395+0000","last_undegraded":"2024-09-12T03:28:08.847395+0000","last_fullsized":"2024-09-12T03:28:08.847395+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:21:55.047085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":98,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:30.224311+0000","last_change":"2024-09-12T03:26:56.962474+0000","last_active":"2024-09-12T03:28:30.224311+0000","last_peered":"2024-09-12T03:28:30.224311+0000","last_clean":"2024-09-12T03:28:30.224311+0000","last_became_active":"2024-09-12T03:26:56.962105+0000","last_became_peered":"2024-09-12T03:26:56.962105+0000","last_unstale":"2024-09-12T03:28:30.224311+0000","last_undegraded":"2024-09-12T03:28:30.224311+0000","last_fullsized":"2024-09-12T03:28:30.224311+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:55.935260+0000","last_clean_scrub_stamp":"2024-09-12T03:26:55.935260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:40:41.905790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":59,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.844069+0000","last_change":"2024-09-12T03:26:59.011561+0000","last_active":"2024-09-12T03:28:08.844069+0000","last_peered":"2024-09-12T03:28:08.844069+0000","last_clean":"2024-09-12T03:28:08.844069+0000","last_became_active":"2024-09-12T03:26:59.011006+0000","last_became_peered":"2024-09-12T03:26:59.011006+0000","last_unstale":"2024-09-12T03:28:08.844069+0000","last_undegraded":"2024-09-12T03:28:08.844069+0000","last_fullsized":"2024-09-12T03:28:08.844069+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:09:20.221637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.843376+0000","last_change":"2024-09-12T03:27:06.050816+0000","last_active":"2024-09-12T03:28:08.843376+0000","last_peered":"2024-09-12T03:28:08.843376+0000","last_clean":"2024-09-12T03:28:08.843376+0000","last_became_active":"2024-09-12T03:27:06.050403+0000","last_became_peered":"2024-09-12T03:27:06.050403+0000","last_unstale":"2024-09-12T03:28:08.843376+0000","last_undegraded":"2024-09-12T03:28:08.843376+0000","last_fullsized":"2024-09-12T03:28:08.843376+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:27:05.020994+0000","last_clean_scrub_stamp":"2024-09-12T03:27:05.020994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:02:24.285363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.927854+0000","last_change":"2024-09-12T03:26:59.013900+0000","last_active":"2024-09-12T03:28:08.927854+0000","last_peered":"2024-09-12T03:28:08.927854+0000","last_clean":"2024-09-12T03:28:08.927854+0000","last_became_active":"2024-09-12T03:26:59.013308+0000","last_became_peered":"2024-09-12T03:26:59.013308+0000","last_unstale":"2024-09-12T03:28:08.927854+0000","last_undegraded":"2024-09-12T03:28:08.927854+0000","last_fullsized":"2024-09-12T03:28:08.927854+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:50.943991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.847105+0000","last_change":"2024-09-12T03:28:08.847105+0000","last_active":"2024-09-12T03:28:08.847105+0000","last_peered":"2024-09-12T03:28:08.847105+0000","last_clean":"2024-09-12T03:28:08.847105+0000","last_became_active":"2024-09-12T03:28:08.846639+0000","last_became_peered":"2024-09-12T03:28:08.846639+0000","last_unstale":"2024-09-12T03:28:08.847105+0000","last_undegraded":"2024-09-12T03:28:08.847105+0000","last_fullsized":"2024-09-12T03:28:08.847105+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:45.898536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.842508+0000","last_change":"2024-09-12T03:26:59.017779+0000","last_active":"2024-09-12T03:28:08.842508+0000","last_peered":"2024-09-12T03:28:08.842508+0000","last_clean":"2024-09-12T03:28:08.842508+0000","last_became_active":"2024-09-12T03:26:59.017438+0000","last_became_peered":"2024-09-12T03:26:59.017438+0000","last_unstale":"2024-09-12T03:28:08.842508+0000","last_undegraded":"2024-09-12T03:28:08.842508+0000","last_fullsized":"2024-09-12T03:28:08.842508+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:29.224776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.846213+0000","last_change":"2024-09-12T03:26:59.017542+0000","last_active":"2024-09-12T03:28:08.846213+0000","last_peered":"2024-09-12T03:28:08.846213+0000","last_clean":"2024-09-12T03:28:08.846213+0000","last_became_active":"2024-09-12T03:26:59.017253+0000","last_became_peered":"2024-09-12T03:26:59.017253+0000","last_unstale":"2024-09-12T03:28:08.846213+0000","last_undegraded":"2024-09-12T03:28:08.846213+0000","last_fullsized":"2024-09-12T03:28:08.846213+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:06:51.389627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.841742+0000","last_change":"2024-09-12T03:26:59.013574+0000","last_active":"2024-09-12T03:28:08.841742+0000","last_peered":"2024-09-12T03:28:08.841742+0000","last_clean":"2024-09-12T03:28:08.841742+0000","last_became_active":"2024-09-12T03:26:59.013253+0000","last_became_peered":"2024-09-12T03:26:59.013253+0000","last_unstale":"2024-09-12T03:28:08.841742+0000","last_undegraded":"2024-09-12T03:28:08.841742+0000","last_fullsized":"2024-09-12T03:28:08.841742+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:37.610909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.927943+0000","last_change":"2024-09-12T03:26:59.012430+0000","last_active":"2024-09-12T03:28:08.927943+0000","last_peered":"2024-09-12T03:28:08.927943+0000","last_clean":"2024-09-12T03:28:08.927943+0000","last_became_active":"2024-09-12T03:26:59.011672+0000","last_became_peered":"2024-09-12T03:26:59.011672+0000","last_unstale":"2024-09-12T03:28:08.927943+0000","last_undegraded":"2024-09-12T03:28:08.927943+0000","last_fullsized":"2024-09-12T03:28:08.927943+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:50:44.888834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.843735+0000","last_change":"2024-09-12T03:26:59.015704+0000","last_active":"2024-09-12T03:28:08.843735+0000","last_peered":"2024-09-12T03:28:08.843735+0000","last_clean":"2024-09-12T03:28:08.843735+0000","last_became_active":"2024-09-12T03:26:59.015389+0000","last_became_peered":"2024-09-12T03:26:59.015389+0000","last_unstale":"2024-09-12T03:28:08.843735+0000","last_undegraded":"2024-09-12T03:28:08.843735+0000","last_fullsized":"2024-09-12T03:28:08.843735+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:19:52.917902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.928189+0000","last_change":"2024-09-12T03:26:59.026325+0000","last_active":"2024-09-12T03:28:08.928189+0000","last_peered":"2024-09-12T03:28:08.928189+0000","last_clean":"2024-09-12T03:28:08.928189+0000","last_became_active":"2024-09-12T03:26:59.026014+0000","last_became_peered":"2024-09-12T03:26:59.026014+0000","last_unstale":"2024-09-12T03:28:08.928189+0000","last_undegraded":"2024-09-12T03:28:08.928189+0000","last_fullsized":"2024-09-12T03:28:08.928189+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:32:50.778286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.929290+0000","last_change":"2024-09-12T03:28:08.928903+0000","last_active":"2024-09-12T03:28:08.929290+0000","last_peered":"2024-09-12T03:28:08.929290+0000","last_clean":"2024-09-12T03:28:08.929290+0000","last_became_active":"2024-09-12T03:28:08.928547+0000","last_became_peered":"2024-09-12T03:28:08.928547+0000","last_unstale":"2024-09-12T03:28:08.929290+0000","last_undegraded":"2024-09-12T03:28:08.929290+0000","last_fullsized":"2024-09-12T03:28:08.929290+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:54:47.701399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.927591+0000","last_change":"2024-09-12T03:26:59.023143+0000","last_active":"2024-09-12T03:28:08.927591+0000","last_peered":"2024-09-12T03:28:08.927591+0000","last_clean":"2024-09-12T03:28:08.927591+0000","last_became_active":"2024-09-12T03:26:59.022282+0000","last_became_peered":"2024-09-12T03:26:59.022282+0000","last_unstale":"2024-09-12T03:28:08.927591+0000","last_undegraded":"2024-09-12T03:28:08.927591+0000","last_fullsized":"2024-09-12T03:28:08.927591+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:59.864192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.843099+0000","last_change":"2024-09-12T03:26:59.025505+0000","last_active":"2024-09-12T03:28:08.843099+0000","last_peered":"2024-09-12T03:28:08.843099+0000","last_clean":"2024-09-12T03:28:08.843099+0000","last_became_active":"2024-09-12T03:26:59.025081+0000","last_became_peered":"2024-09-12T03:26:59.025081+0000","last_unstale":"2024-09-12T03:28:08.843099+0000","last_undegraded":"2024-09-12T03:28:08.843099+0000","last_fullsized":"2024-09-12T03:28:08.843099+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:42:18.012217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.841848+0000","last_change":"2024-09-12T03:26:59.011315+0000","last_active":"2024-09-12T03:28:08.841848+0000","last_peered":"2024-09-12T03:28:08.841848+0000","last_clean":"2024-09-12T03:28:08.841848+0000","last_became_active":"2024-09-12T03:26:59.011028+0000","last_became_peered":"2024-09-12T03:26:59.011028+0000","last_unstale":"2024-09-12T03:28:08.841848+0000","last_undegraded":"2024-09-12T03:28:08.841848+0000","last_fullsized":"2024-09-12T03:28:08.841848+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:18:10.541578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.927652+0000","last_change":"2024-09-12T03:26:59.024518+0000","last_active":"2024-09-12T03:28:08.927652+0000","last_peered":"2024-09-12T03:28:08.927652+0000","last_clean":"2024-09-12T03:28:08.927652+0000","last_became_active":"2024-09-12T03:26:59.024197+0000","last_became_peered":"2024-09-12T03:26:59.024197+0000","last_unstale":"2024-09-12T03:28:08.927652+0000","last_undegraded":"2024-09-12T03:28:08.927652+0000","last_fullsized":"2024-09-12T03:28:08.927652+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:52.113096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.843728+0000","last_change":"2024-09-12T03:26:59.018707+0000","last_active":"2024-09-12T03:28:08.843728+0000","last_peered":"2024-09-12T03:28:08.843728+0000","last_clean":"2024-09-12T03:28:08.843728+0000","last_became_active":"2024-09-12T03:26:59.017997+0000","last_became_peered":"2024-09-12T03:26:59.017997+0000","last_unstale":"2024-09-12T03:28:08.843728+0000","last_undegraded":"2024-09-12T03:28:08.843728+0000","last_fullsized":"2024-09-12T03:28:08.843728+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:37:58.896056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.843975+0000","last_change":"2024-09-12T03:26:59.013523+0000","last_active":"2024-09-12T03:28:08.843975+0000","last_peered":"2024-09-12T03:28:08.843975+0000","last_clean":"2024-09-12T03:28:08.843975+0000","last_became_active":"2024-09-12T03:26:59.013210+0000","last_became_peered":"2024-09-12T03:26:59.013210+0000","last_unstale":"2024-09-12T03:28:08.843975+0000","last_undegraded":"2024-09-12T03:28:08.843975+0000","last_fullsized":"2024-09-12T03:28:08.843975+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:03:42.390871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.928332+0000","last_change":"2024-09-12T03:26:59.016719+0000","last_active":"2024-09-12T03:28:08.928332+0000","last_peered":"2024-09-12T03:28:08.928332+0000","last_clean":"2024-09-12T03:28:08.928332+0000","last_became_active":"2024-09-12T03:26:59.015245+0000","last_became_peered":"2024-09-12T03:26:59.015245+0000","last_unstale":"2024-09-12T03:28:08.928332+0000","last_undegraded":"2024-09-12T03:28:08.928332+0000","last_fullsized":"2024-09-12T03:28:08.928332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:12.163847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.842697+0000","last_change":"2024-09-12T03:26:59.024317+0000","last_active":"2024-09-12T03:28:08.842697+0000","last_peered":"2024-09-12T03:28:08.842697+0000","last_clean":"2024-09-12T03:28:08.842697+0000","last_became_active":"2024-09-12T03:26:59.023998+0000","last_became_peered":"2024-09-12T03:26:59.023998+0000","last_unstale":"2024-09-12T03:28:08.842697+0000","last_undegraded":"2024-09-12T03:28:08.842697+0000","last_fullsized":"2024-09-12T03:28:08.842697+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:29:11.713342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.848298+0000","last_change":"2024-09-12T03:28:08.848298+0000","last_active":"2024-09-12T03:28:08.848298+0000","last_peered":"2024-09-12T03:28:08.848298+0000","last_clean":"2024-09-12T03:28:08.848298+0000","last_became_active":"2024-09-12T03:28:08.847989+0000","last_became_peered":"2024-09-12T03:28:08.847989+0000","last_unstale":"2024-09-12T03:28:08.848298+0000","last_undegraded":"2024-09-12T03:28:08.848298+0000","last_fullsized":"2024-09-12T03:28:08.848298+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:02:21.380751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.845774+0000","last_change":"2024-09-12T03:26:59.011340+0000","last_active":"2024-09-12T03:28:08.845774+0000","last_peered":"2024-09-12T03:28:08.845774+0000","last_clean":"2024-09-12T03:28:08.845774+0000","last_became_active":"2024-09-12T03:26:59.011042+0000","last_became_peered":"2024-09-12T03:26:59.011042+0000","last_unstale":"2024-09-12T03:28:08.845774+0000","last_undegraded":"2024-09-12T03:28:08.845774+0000","last_fullsized":"2024-09-12T03:28:08.845774+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:49.568438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.841822+0000","last_change":"2024-09-12T03:26:59.016449+0000","last_active":"2024-09-12T03:28:08.841822+0000","last_peered":"2024-09-12T03:28:08.841822+0000","last_clean":"2024-09-12T03:28:08.841822+0000","last_became_active":"2024-09-12T03:26:59.016087+0000","last_became_peered":"2024-09-12T03:26:59.016087+0000","last_unstale":"2024-09-12T03:28:08.841822+0000","last_undegraded":"2024-09-12T03:28:08.841822+0000","last_fullsized":"2024-09-12T03:28:08.841822+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:13:40.683471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.975871+0000","last_change":"2024-09-12T03:28:08.975304+0000","last_active":"2024-09-12T03:28:08.975871+0000","last_peered":"2024-09-12T03:28:08.975871+0000","last_clean":"2024-09-12T03:28:08.975871+0000","last_became_active":"2024-09-12T03:28:08.974962+0000","last_became_peered":"2024-09-12T03:28:08.974962+0000","last_unstale":"2024-09-12T03:28:08.975871+0000","last_undegraded":"2024-09-12T03:28:08.975871+0000","last_fullsized":"2024-09-12T03:28:08.975871+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:14.369832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.844084+0000","last_change":"2024-09-12T03:26:59.021434+0000","last_active":"2024-09-12T03:28:08.844084+0000","last_peered":"2024-09-12T03:28:08.844084+0000","last_clean":"2024-09-12T03:28:08.844084+0000","last_became_active":"2024-09-12T03:26:59.021071+0000","last_became_peered":"2024-09-12T03:26:59.021071+0000","last_unstale":"2024-09-12T03:28:08.844084+0000","last_undegraded":"2024-09-12T03:28:08.844084+0000","last_fullsized":"2024-09-12T03:28:08.844084+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:14:46.774992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.975119+0000","last_change":"2024-09-12T03:26:59.019310+0000","last_active":"2024-09-12T03:28:08.975119+0000","last_peered":"2024-09-12T03:28:08.975119+0000","last_clean":"2024-09-12T03:28:08.975119+0000","last_became_active":"2024-09-12T03:26:59.018967+0000","last_became_peered":"2024-09-12T03:26:59.018967+0000","last_unstale":"2024-09-12T03:28:08.975119+0000","last_undegraded":"2024-09-12T03:28:08.975119+0000","last_fullsized":"2024-09-12T03:28:08.975119+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:37:10.830500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.843822+0000","last_change":"2024-09-12T03:26:59.022088+0000","last_active":"2024-09-12T03:28:08.843822+0000","last_peered":"2024-09-12T03:28:08.843822+0000","last_clean":"2024-09-12T03:28:08.843822+0000","last_became_active":"2024-09-12T03:26:59.021655+0000","last_became_peered":"2024-09-12T03:26:59.021655+0000","last_unstale":"2024-09-12T03:28:08.843822+0000","last_undegraded":"2024-09-12T03:28:08.843822+0000","last_fullsized":"2024-09-12T03:28:08.843822+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:39.490597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.927621+0000","last_change":"2024-09-12T03:26:59.024898+0000","last_active":"2024-09-12T03:28:08.927621+0000","last_peered":"2024-09-12T03:28:08.927621+0000","last_clean":"2024-09-12T03:28:08.927621+0000","last_became_active":"2024-09-12T03:26:59.024511+0000","last_became_peered":"2024-09-12T03:26:59.024511+0000","last_unstale":"2024-09-12T03:28:08.927621+0000","last_undegraded":"2024-09-12T03:28:08.927621+0000","last_fullsized":"2024-09-12T03:28:08.927621+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:09:24.191365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.846391+0000","last_change":"2024-09-12T03:28:08.846391+0000","last_active":"2024-09-12T03:28:08.846391+0000","last_peered":"2024-09-12T03:28:08.846391+0000","last_clean":"2024-09-12T03:28:08.846391+0000","last_became_active":"2024-09-12T03:28:08.845889+0000","last_became_peered":"2024-09-12T03:28:08.845889+0000","last_unstale":"2024-09-12T03:28:08.846391+0000","last_undegraded":"2024-09-12T03:28:08.846391+0000","last_fullsized":"2024-09-12T03:28:08.846391+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:00.687744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.974749+0000","last_change":"2024-09-12T03:26:59.014536+0000","last_active":"2024-09-12T03:28:08.974749+0000","last_peered":"2024-09-12T03:28:08.974749+0000","last_clean":"2024-09-12T03:28:08.974749+0000","last_became_active":"2024-09-12T03:26:59.013910+0000","last_became_peered":"2024-09-12T03:26:59.013910+0000","last_unstale":"2024-09-12T03:28:08.974749+0000","last_undegraded":"2024-09-12T03:28:08.974749+0000","last_fullsized":"2024-09-12T03:28:08.974749+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:27.519367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.842587+0000","last_change":"2024-09-12T03:26:59.016656+0000","last_active":"2024-09-12T03:28:08.842587+0000","last_peered":"2024-09-12T03:28:08.842587+0000","last_clean":"2024-09-12T03:28:08.842587+0000","last_became_active":"2024-09-12T03:26:59.016217+0000","last_became_peered":"2024-09-12T03:26:59.016217+0000","last_unstale":"2024-09-12T03:28:08.842587+0000","last_undegraded":"2024-09-12T03:28:08.842587+0000","last_fullsized":"2024-09-12T03:28:08.842587+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:07:42.718394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.845739+0000","last_change":"2024-09-12T03:26:59.015940+0000","last_active":"2024-09-12T03:28:08.845739+0000","last_peered":"2024-09-12T03:28:08.845739+0000","last_clean":"2024-09-12T03:28:08.845739+0000","last_became_active":"2024-09-12T03:26:59.015578+0000","last_became_peered":"2024-09-12T03:26:59.015578+0000","last_unstale":"2024-09-12T03:28:08.845739+0000","last_undegraded":"2024-09-12T03:28:08.845739+0000","last_fullsized":"2024-09-12T03:28:08.845739+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:07:46.559935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.846252+0000","last_change":"2024-09-12T03:26:59.016283+0000","last_active":"2024-09-12T03:28:08.846252+0000","last_peered":"2024-09-12T03:28:08.846252+0000","last_clean":"2024-09-12T03:28:08.846252+0000","last_became_active":"2024-09-12T03:26:59.015109+0000","last_became_peered":"2024-09-12T03:26:59.015109+0000","last_unstale":"2024-09-12T03:28:08.846252+0000","last_undegraded":"2024-09-12T03:28:08.846252+0000","last_fullsized":"2024-09-12T03:28:08.846252+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:02:44.493393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.846573+0000","last_change":"2024-09-12T03:26:59.020161+0000","last_active":"2024-09-12T03:28:08.846573+0000","last_peered":"2024-09-12T03:28:08.846573+0000","last_clean":"2024-09-12T03:28:08.846573+0000","last_became_active":"2024-09-12T03:26:59.019796+0000","last_became_peered":"2024-09-12T03:26:59.019796+0000","last_unstale":"2024-09-12T03:28:08.846573+0000","last_undegraded":"2024-09-12T03:28:08.846573+0000","last_fullsized":"2024-09-12T03:28:08.846573+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:04:02.817005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-12T03:28:08.928268+0000","last_change":"2024-09-12T03:26:59.015763+0000","last_active":"2024-09-12T03:28:08.928268+0000","last_peered":"2024-09-12T03:28:08.928268+0000","last_clean":"2024-09-12T03:28:08.928268+0000","last_became_active":"2024-09-12T03:26:59.014979+0000","last_became_peered":"2024-09-12T03:26:59.014979+0000","last_unstale":"2024-09-12T03:28:08.928268+0000","last_undegraded":"2024-09-12T03:28:08.928268+0000","last_fullsized":"2024-09-12T03:28:08.928268+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:26:57.984783+0000","last_clean_scrub_stamp":"2024-09-12T03:26:57.984783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:49:26.959019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66079,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":359,"num_read_kb":250,"num_write":173,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":198237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":247982,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514528,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":991232,"data_stored":508337,"data_compressed":23516,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776165,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32296,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712856,"statfs":{"total":95995035648,"available":95961964544,"internally_reserved":0,"allocated":1134592,"data_stored":1158074,"data_compressed":59644,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939690,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1011712,"data_stored":512669,"data_compressed":23516,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201327,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1007616,"data_stored":508493,"data_compressed":23516,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430260,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1056768,"data_stored":1104404,"data_compressed":59644,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593785,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":987136,"data_stored":503749,"data_compressed":23516,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822718,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1064960,"data_stored":1099599,"data_compressed":59644,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953538,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":999424,"data_stored":508792,"data_compressed":23516,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215176,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":999424,"data_stored":503943,"data_compressed":23516,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411406,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1077248,"data_stored":567085,"data_compressed":23516,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1052672,"data_stored":565735,"data_compressed":23516,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738455,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1011712,"data_stored":505137,"data_compressed":23516,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34490,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34490,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34490,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36128,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36128,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36128,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36128,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:28:32.795 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-12T03:28:32.795 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-12T03:28:32.795 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-12T03:28:32.795 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-12T03:28:33.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:32 smithi064 ceph-mon[30952]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-12T03:28:33.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:32 smithi064 ceph-mon[30952]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:28:33.056 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:33.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:32 smithi195 ceph-mon[34786]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-12T03:28:33.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:32 smithi195 ceph-mon[34786]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:28:33.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:32 smithi115 ceph-mon[34801]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-12T03:28:33.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:32 smithi115 ceph-mon[34801]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:28:33.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.536+0000 7faf360c7640 1 -- 172.21.15.64:0/1324420519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf3010c170 msgr2=0x7faf3010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:33.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.536+0000 7faf360c7640 1 --2- 172.21.15.64:0/1324420519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf3010c170 0x7faf3010e610 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7faf28009ca0 tx=0x7faf2802a950 comp rx=0 tx=0).stop 2024-09-12T03:28:33.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.536+0000 7faf360c7640 1 -- 172.21.15.64:0/1324420519 shutdown_connections 2024-09-12T03:28:33.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.536+0000 7faf360c7640 1 --2- 172.21.15.64:0/1324420519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf3010c170 0x7faf3010e610 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.536+0000 7faf360c7640 1 --2- 172.21.15.64:0/1324420519 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faf30103660 0x7faf3010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.536+0000 7faf360c7640 1 --2- 172.21.15.64:0/1324420519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf30102d40 0x7faf30103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.536+0000 7faf360c7640 1 -- 172.21.15.64:0/1324420519 >> 172.21.15.64:0/1324420519 conn(0x7faf300fc7a0 msgr2=0x7faf300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:33.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.536+0000 7faf360c7640 1 -- 172.21.15.64:0/1324420519 shutdown_connections 2024-09-12T03:28:33.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.536+0000 7faf360c7640 1 -- 172.21.15.64:0/1324420519 wait complete. 2024-09-12T03:28:33.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.537+0000 7faf360c7640 1 Processor -- start 2024-09-12T03:28:33.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.537+0000 7faf360c7640 1 -- start start 2024-09-12T03:28:33.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.537+0000 7faf360c7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf30102d40 0x7faf30198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:33.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.537+0000 7faf360c7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faf30103660 0x7faf30198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:33.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.537+0000 7faf360c7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf3010c170 0x7faf3019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:33.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.537+0000 7faf360c7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7faf30111e00 con 0x7faf3010c170 2024-09-12T03:28:33.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.537+0000 7faf360c7640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7faf30111c80 con 0x7faf30102d40 2024-09-12T03:28:33.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.537+0000 7faf360c7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7faf30111f80 con 0x7faf30103660 2024-09-12T03:28:33.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.537+0000 7faf2ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf3010c170 0x7faf3019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:33.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.537+0000 7faf2ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf3010c170 0x7faf3019cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39900/0 (socket says 172.21.15.64:39900) 2024-09-12T03:28:33.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.537+0000 7faf2ffff640 1 -- 172.21.15.64:0/2486440190 learned_addr learned my addr 172.21.15.64:0/2486440190 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:33.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.537+0000 7faf26ffd640 1 --2- 172.21.15.64:0/2486440190 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faf30103660 0x7faf30198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:33.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.537+0000 7faf2f7fe640 1 --2- 172.21.15.64:0/2486440190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf30102d40 0x7faf30198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:33.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.537+0000 7faf2ffff640 1 -- 172.21.15.64:0/2486440190 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faf30103660 msgr2=0x7faf30198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:33.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.537+0000 7faf2ffff640 1 --2- 172.21.15.64:0/2486440190 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faf30103660 0x7faf30198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.538+0000 7faf2ffff640 1 -- 172.21.15.64:0/2486440190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf30102d40 msgr2=0x7faf30198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:33.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.538+0000 7faf2ffff640 1 --2- 172.21.15.64:0/2486440190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf30102d40 0x7faf30198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.538+0000 7faf2ffff640 1 -- 172.21.15.64:0/2486440190 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf3019d2f0 con 0x7faf3010c170 2024-09-12T03:28:33.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.538+0000 7faf26ffd640 1 --2- 172.21.15.64:0/2486440190 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faf30103660 0x7faf30198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:28:33.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.538+0000 7faf2f7fe640 1 --2- 172.21.15.64:0/2486440190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf30102d40 0x7faf30198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:28:33.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.538+0000 7faf2ffff640 1 --2- 172.21.15.64:0/2486440190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf3010c170 0x7faf3019cc10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7faf2802ae70 tx=0x7faf2802f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:33.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.538+0000 7faf2d7fa640 1 -- 172.21.15.64:0/2486440190 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf28034040 con 0x7faf3010c170 2024-09-12T03:28:33.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.538+0000 7faf360c7640 1 -- 172.21.15.64:0/2486440190 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf3019d580 con 0x7faf3010c170 2024-09-12T03:28:33.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.538+0000 7faf2d7fa640 1 -- 172.21.15.64:0/2486440190 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf280308f0 con 0x7faf3010c170 2024-09-12T03:28:33.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.538+0000 7faf2d7fa640 1 -- 172.21.15.64:0/2486440190 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf280047f0 con 0x7faf3010c170 2024-09-12T03:28:33.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.538+0000 7faf360c7640 1 -- 172.21.15.64:0/2486440190 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf301a4ec0 con 0x7faf3010c170 2024-09-12T03:28:33.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.539+0000 7faf360c7640 1 -- 172.21.15.64:0/2486440190 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf30103120 con 0x7faf3010c170 2024-09-12T03:28:33.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.540+0000 7faf2d7fa640 1 -- 172.21.15.64:0/2486440190 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7faf28037020 con 0x7faf3010c170 2024-09-12T03:28:33.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.541+0000 7faf2d7fa640 1 --2- 172.21.15.64:0/2486440190 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7faf1c0a2c10 0x7faf1c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:33.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.541+0000 7faf2f7fe640 1 --2- 172.21.15.64:0/2486440190 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7faf1c0a2c10 0x7faf1c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:33.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.541+0000 7faf2d7fa640 1 -- 172.21.15.64:0/2486440190 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7faf280f0520 con 0x7faf3010c170 2024-09-12T03:28:33.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.542+0000 7faf2f7fe640 1 --2- 172.21.15.64:0/2486440190 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7faf1c0a2c10 0x7faf1c0a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7faf200058c0 tx=0x7faf20005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:33.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.545+0000 7faf2d7fa640 1 -- 172.21.15.64:0/2486440190 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faf280bd070 con 0x7faf3010c170 2024-09-12T03:28:33.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.758+0000 7faf360c7640 1 -- 172.21.15.64:0/2486440190 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7faf30107210 con 0x7faf3010c170 2024-09-12T03:28:33.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.758+0000 7faf2d7fa640 1 -- 172.21.15.64:0/2486440190 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7faf280bc970 con 0x7faf3010c170 2024-09-12T03:28:33.758 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:28:33.758 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-12T03:28:33.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.760+0000 7faf360c7640 1 -- 172.21.15.64:0/2486440190 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7faf1c0a2c10 msgr2=0x7faf1c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:33.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.760+0000 7faf360c7640 1 --2- 172.21.15.64:0/2486440190 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7faf1c0a2c10 0x7faf1c0a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7faf200058c0 tx=0x7faf20005d30 comp rx=0 tx=0).stop 2024-09-12T03:28:33.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.760+0000 7faf360c7640 1 -- 172.21.15.64:0/2486440190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf3010c170 msgr2=0x7faf3019cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:33.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.760+0000 7faf360c7640 1 --2- 172.21.15.64:0/2486440190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf3010c170 0x7faf3019cc10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7faf2802ae70 tx=0x7faf2802f5d0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.760+0000 7faf360c7640 1 -- 172.21.15.64:0/2486440190 shutdown_connections 2024-09-12T03:28:33.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.760+0000 7faf360c7640 1 --2- 172.21.15.64:0/2486440190 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7faf1c0a2c10 0x7faf1c0a50d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.760+0000 7faf360c7640 1 --2- 172.21.15.64:0/2486440190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf3010c170 0x7faf3019cc10 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.760+0000 7faf360c7640 1 --2- 172.21.15.64:0/2486440190 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faf30103660 0x7faf30198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.760+0000 7faf360c7640 1 --2- 172.21.15.64:0/2486440190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf30102d40 0x7faf30198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:33.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.760+0000 7faf360c7640 1 -- 172.21.15.64:0/2486440190 >> 172.21.15.64:0/2486440190 conn(0x7faf300fc7a0 msgr2=0x7faf300fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:33.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.760+0000 7faf360c7640 1 -- 172.21.15.64:0/2486440190 shutdown_connections 2024-09-12T03:28:33.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:33.760+0000 7faf360c7640 1 -- 172.21.15.64:0/2486440190 wait complete. 2024-09-12T03:28:34.021 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:33 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/2486440190' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T03:28:34.209 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:33 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/2486440190' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T03:28:34.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:33 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/2486440190' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T03:28:34.667 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-12T03:28:34.667 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:28:34.929 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:35.191 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:34 smithi064 ceph-mon[30952]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:28:35.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:34 smithi195 ceph-mon[34786]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:28:35.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:34 smithi115 ceph-mon[34801]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:28:35.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.440+0000 7f49badd3640 1 -- 172.21.15.64:0/539222097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49b4100e20 msgr2=0x7f49b4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:35.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.440+0000 7f49badd3640 1 --2- 172.21.15.64:0/539222097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49b4100e20 0x7f49b4101200 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f49a8009900 tx=0x7f49a802a480 comp rx=0 tx=0).stop 2024-09-12T03:28:35.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.440+0000 7f49badd3640 1 -- 172.21.15.64:0/539222097 shutdown_connections 2024-09-12T03:28:35.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.440+0000 7f49badd3640 1 --2- 172.21.15.64:0/539222097 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49b4102ef0 0x7f49b410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:35.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.440+0000 7f49badd3640 1 --2- 172.21.15.64:0/539222097 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49b4102570 0x7f49b41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:35.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.440+0000 7f49badd3640 1 --2- 172.21.15.64:0/539222097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49b4100e20 0x7f49b4101200 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:35.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.440+0000 7f49badd3640 1 -- 172.21.15.64:0/539222097 >> 172.21.15.64:0/539222097 conn(0x7f49b40fc7d0 msgr2=0x7f49b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:35.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.441+0000 7f49badd3640 1 -- 172.21.15.64:0/539222097 shutdown_connections 2024-09-12T03:28:35.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.441+0000 7f49badd3640 1 -- 172.21.15.64:0/539222097 wait complete. 2024-09-12T03:28:35.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.441+0000 7f49badd3640 1 Processor -- start 2024-09-12T03:28:35.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.441+0000 7f49badd3640 1 -- start start 2024-09-12T03:28:35.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.441+0000 7f49badd3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49b4100e20 0x7f49b419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:35.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.441+0000 7f49badd3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49b4102570 0x7f49b419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:35.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.441+0000 7f49badd3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49b4102ef0 0x7f49b419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:35.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.441+0000 7f49badd3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f49b4110e00 con 0x7f49b4100e20 2024-09-12T03:28:35.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.441+0000 7f49badd3640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f49b4110c80 con 0x7f49b4102ef0 2024-09-12T03:28:35.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.441+0000 7f49badd3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f49b4110f80 con 0x7f49b4102570 2024-09-12T03:28:35.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.442+0000 7f49b8b6f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49b4100e20 0x7f49b419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:35.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.442+0000 7f49b3fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49b4102570 0x7f49b419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:35.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.442+0000 7f49b8b6f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49b4100e20 0x7f49b419a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39918/0 (socket says 172.21.15.64:39918) 2024-09-12T03:28:35.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.442+0000 7f49b3fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49b4102570 0x7f49b419ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.64:40340/0 (socket says 172.21.15.64:40340) 2024-09-12T03:28:35.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.442+0000 7f49b8b6f640 1 -- 172.21.15.64:0/3398953672 learned_addr learned my addr 172.21.15.64:0/3398953672 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:35.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.442+0000 7f49b9370640 1 --2- 172.21.15.64:0/3398953672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49b4102ef0 0x7f49b419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:35.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.442+0000 7f49b3fff640 1 -- 172.21.15.64:0/3398953672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49b4102ef0 msgr2=0x7f49b419eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:35.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.442+0000 7f49b3fff640 1 --2- 172.21.15.64:0/3398953672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49b4102ef0 0x7f49b419eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:35.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.442+0000 7f49b3fff640 1 -- 172.21.15.64:0/3398953672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49b4100e20 msgr2=0x7f49b419a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:35.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.442+0000 7f49b3fff640 1 --2- 172.21.15.64:0/3398953672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49b4100e20 0x7f49b419a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:35.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.442+0000 7f49b3fff640 1 -- 172.21.15.64:0/3398953672 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b419f590 con 0x7f49b4102570 2024-09-12T03:28:35.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.442+0000 7f49b8b6f640 1 --2- 172.21.15.64:0/3398953672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49b4100e20 0x7f49b419a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:35.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.442+0000 7f49b3fff640 1 --2- 172.21.15.64:0/3398953672 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49b4102570 0x7f49b419ab20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f49a000d6d0 tx=0x7f49a000db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:35.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.443+0000 7f49b1ffb640 1 -- 172.21.15.64:0/3398953672 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a0017070 con 0x7f49b4102570 2024-09-12T03:28:35.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.443+0000 7f49b1ffb640 1 -- 172.21.15.64:0/3398953672 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49a0009820 con 0x7f49b4102570 2024-09-12T03:28:35.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.443+0000 7f49badd3640 1 -- 172.21.15.64:0/3398953672 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49b419f880 con 0x7f49b4102570 2024-09-12T03:28:35.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.443+0000 7f49b1ffb640 1 -- 172.21.15.64:0/3398953672 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a0004c40 con 0x7f49b4102570 2024-09-12T03:28:35.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.443+0000 7f49badd3640 1 -- 172.21.15.64:0/3398953672 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49b41a7160 con 0x7f49b4102570 2024-09-12T03:28:35.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.444+0000 7f49badd3640 1 -- 172.21.15.64:0/3398953672 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49b4101200 con 0x7f49b4102570 2024-09-12T03:28:35.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.446+0000 7f49b1ffb640 1 -- 172.21.15.64:0/3398953672 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f49a00099d0 con 0x7f49b4102570 2024-09-12T03:28:35.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.447+0000 7f49b1ffb640 1 --2- 172.21.15.64:0/3398953672 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f49a40a2c10 0x7f49a40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:35.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.447+0000 7f49b1ffb640 1 -- 172.21.15.64:0/3398953672 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f49a00cf100 con 0x7f49b4102570 2024-09-12T03:28:35.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.447+0000 7f49b8b6f640 1 --2- 172.21.15.64:0/3398953672 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f49a40a2c10 0x7f49a40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:35.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.447+0000 7f49b8b6f640 1 --2- 172.21.15.64:0/3398953672 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f49a40a2c10 0x7f49a40a50d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f49a80096c0 tx=0x7f49a8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:35.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.449+0000 7f49b1ffb640 1 -- 172.21.15.64:0/3398953672 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f49a009b550 con 0x7f49b4102570 2024-09-12T03:28:35.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.663+0000 7f49badd3640 1 -- 172.21.15.64:0/3398953672 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49b41a7330 con 0x7f49b4102570 2024-09-12T03:28:35.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.664+0000 7f49b1ffb640 1 -- 172.21.15.64:0/3398953672 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11921 (secure 0 0 0) 0x7f49a009f850 con 0x7f49b4102570 2024-09-12T03:28:35.664 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:28:35.664 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":24,"btime":"2024-09-12T03:27:37:139728+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2580078738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2580078738},{"type":"v1","addr":"172.21.15.64:6835","nonce":2580078738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2333415004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2333415004},{"type":"v1","addr":"172.21.15.64:6837","nonce":2333415004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/3030618395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":3030618395},{"type":"v1","addr":"172.21.15.64:6839","nonce":3030618395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3333057625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3333057625},{"type":"v1","addr":"172.21.15.64:6841","nonce":3333057625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2991933266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2991933266},{"type":"v1","addr":"172.21.15.195:6833","nonce":2991933266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1138960346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1138960346},{"type":"v1","addr":"172.21.15.115:6833","nonce":1138960346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:26:59.647657+0000","modified":"2024-09-12T03:27:36.865338+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24481,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/332737916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":332737916},{"type":"v1","addr":"172.21.15.115:6835","nonce":332737916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/2531045583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2531045583},{"type":"v1","addr":"172.21.15.195:6835","nonce":2531045583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/71647401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":71647401},{"type":"v1","addr":"172.21.15.195:6837","nonce":71647401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.115:6837/2915155705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2915155705},{"type":"v1","addr":"172.21.15.115:6837","nonce":2915155705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.195:6839/2574695134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":2574695134},{"type":"v1","addr":"172.21.15.195:6839","nonce":2574695134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6839/2238031796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2238031796},{"type":"v1","addr":"172.21.15.115:6839","nonce":2238031796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24470,24481,24500]},"id":1}]} 2024-09-12T03:28:35.664 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 24 2024-09-12T03:28:35.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.666+0000 7f49badd3640 1 -- 172.21.15.64:0/3398953672 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f49a40a2c10 msgr2=0x7f49a40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:35.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.666+0000 7f49badd3640 1 --2- 172.21.15.64:0/3398953672 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f49a40a2c10 0x7f49a40a50d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f49a80096c0 tx=0x7f49a8005a20 comp rx=0 tx=0).stop 2024-09-12T03:28:35.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.666+0000 7f49badd3640 1 -- 172.21.15.64:0/3398953672 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49b4102570 msgr2=0x7f49b419ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:35.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.666+0000 7f49badd3640 1 --2- 172.21.15.64:0/3398953672 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49b4102570 0x7f49b419ab20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f49a000d6d0 tx=0x7f49a000db90 comp rx=0 tx=0).stop 2024-09-12T03:28:35.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.666+0000 7f49badd3640 1 -- 172.21.15.64:0/3398953672 shutdown_connections 2024-09-12T03:28:35.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.666+0000 7f49badd3640 1 --2- 172.21.15.64:0/3398953672 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f49a40a2c10 0x7f49a40a50d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:35.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.666+0000 7f49badd3640 1 --2- 172.21.15.64:0/3398953672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49b4102ef0 0x7f49b419eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:35.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.666+0000 7f49badd3640 1 --2- 172.21.15.64:0/3398953672 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49b4102570 0x7f49b419ab20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:35.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.666+0000 7f49badd3640 1 --2- 172.21.15.64:0/3398953672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49b4100e20 0x7f49b419a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:35.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.666+0000 7f49badd3640 1 -- 172.21.15.64:0/3398953672 >> 172.21.15.64:0/3398953672 conn(0x7f49b40fc7d0 msgr2=0x7f49b406a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:35.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.666+0000 7f49badd3640 1 -- 172.21.15.64:0/3398953672 shutdown_connections 2024-09-12T03:28:35.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:35.666+0000 7f49badd3640 1 -- 172.21.15.64:0/3398953672 wait complete. 2024-09-12T03:28:36.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:35 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/3398953672' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:28:36.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:35 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/3398953672' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:28:36.348 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:28:36.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:35 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/3398953672' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:28:36.609 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:37.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:36 smithi064 ceph-mon[30952]: pgmap v334: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:28:37.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.120+0000 7ff145342640 1 -- 172.21.15.64:0/2768058800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff140101de0 msgr2=0x7ff1401021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:37.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.120+0000 7ff145342640 1 --2- 172.21.15.64:0/2768058800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff140101de0 0x7ff1401021c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff130009900 tx=0x7ff13002a410 comp rx=0 tx=0).stop 2024-09-12T03:28:37.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.120+0000 7ff145342640 1 -- 172.21.15.64:0/2768058800 shutdown_connections 2024-09-12T03:28:37.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.120+0000 7ff145342640 1 --2- 172.21.15.64:0/2768058800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff140106f10 0x7ff14010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:37.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.120+0000 7ff145342640 1 --2- 172.21.15.64:0/2768058800 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff140102790 0x7ff1401067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:37.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.120+0000 7ff145342640 1 --2- 172.21.15.64:0/2768058800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff140101de0 0x7ff1401021c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:37.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.120+0000 7ff145342640 1 -- 172.21.15.64:0/2768058800 >> 172.21.15.64:0/2768058800 conn(0x7ff1400fd8c0 msgr2=0x7ff1400ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:37.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.120+0000 7ff145342640 1 -- 172.21.15.64:0/2768058800 shutdown_connections 2024-09-12T03:28:37.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.120+0000 7ff145342640 1 -- 172.21.15.64:0/2768058800 wait complete. 2024-09-12T03:28:37.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.120+0000 7ff145342640 1 Processor -- start 2024-09-12T03:28:37.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.121+0000 7ff145342640 1 -- start start 2024-09-12T03:28:37.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.121+0000 7ff145342640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff140101de0 0x7ff140198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:37.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.121+0000 7ff145342640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff140102790 0x7ff1401988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:37.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.121+0000 7ff145342640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff140106f10 0x7ff14019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:37.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.121+0000 7ff145342640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff140111de0 con 0x7ff140101de0 2024-09-12T03:28:37.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.121+0000 7ff145342640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff140111c60 con 0x7ff140102790 2024-09-12T03:28:37.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.121+0000 7ff145342640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff140111f60 con 0x7ff140106f10 2024-09-12T03:28:37.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.121+0000 7ff13effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff140101de0 0x7ff140198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:37.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.121+0000 7ff13effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff140101de0 0x7ff140198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39928/0 (socket says 172.21.15.64:39928) 2024-09-12T03:28:37.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.121+0000 7ff13effd640 1 -- 172.21.15.64:0/1922352440 learned_addr learned my addr 172.21.15.64:0/1922352440 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:37.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.121+0000 7ff13f7fe640 1 --2- 172.21.15.64:0/1922352440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff140106f10 0x7ff14019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:37.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.121+0000 7ff13e7fc640 1 --2- 172.21.15.64:0/1922352440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff140102790 0x7ff1401988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:37.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.121+0000 7ff13effd640 1 -- 172.21.15.64:0/1922352440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff140106f10 msgr2=0x7ff14019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:37.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.121+0000 7ff13effd640 1 --2- 172.21.15.64:0/1922352440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff140106f10 0x7ff14019cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:37.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.121+0000 7ff13effd640 1 -- 172.21.15.64:0/1922352440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff140102790 msgr2=0x7ff1401988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:37.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.121+0000 7ff13effd640 1 --2- 172.21.15.64:0/1922352440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff140102790 0x7ff1401988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:37.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.121+0000 7ff13effd640 1 -- 172.21.15.64:0/1922352440 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff14019d330 con 0x7ff140101de0 2024-09-12T03:28:37.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.121+0000 7ff13e7fc640 1 --2- 172.21.15.64:0/1922352440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff140102790 0x7ff1401988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:28:37.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.121+0000 7ff13effd640 1 --2- 172.21.15.64:0/1922352440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff140101de0 0x7ff140198380 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff130005940 tx=0x7ff130004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:37.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.122+0000 7ff127fff640 1 -- 172.21.15.64:0/1922352440 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13003f030 con 0x7ff140101de0 2024-09-12T03:28:37.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.122+0000 7ff127fff640 1 -- 172.21.15.64:0/1922352440 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff130003aa0 con 0x7ff140101de0 2024-09-12T03:28:37.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.122+0000 7ff145342640 1 -- 172.21.15.64:0/1922352440 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff14019d5c0 con 0x7ff140101de0 2024-09-12T03:28:37.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.122+0000 7ff127fff640 1 -- 172.21.15.64:0/1922352440 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff130035070 con 0x7ff140101de0 2024-09-12T03:28:37.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.122+0000 7ff145342640 1 -- 172.21.15.64:0/1922352440 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1401a4f00 con 0x7ff140101de0 2024-09-12T03:28:37.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.123+0000 7ff145342640 1 -- 172.21.15.64:0/1922352440 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1401032c0 con 0x7ff140101de0 2024-09-12T03:28:37.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.124+0000 7ff127fff640 1 -- 172.21.15.64:0/1922352440 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff130042030 con 0x7ff140101de0 2024-09-12T03:28:37.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.124+0000 7ff127fff640 1 --2- 172.21.15.64:0/1922352440 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff1280a2c10 0x7ff1280a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:37.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.125+0000 7ff13e7fc640 1 --2- 172.21.15.64:0/1922352440 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff1280a2c10 0x7ff1280a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:37.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.125+0000 7ff127fff640 1 -- 172.21.15.64:0/1922352440 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff1300efd90 con 0x7ff140101de0 2024-09-12T03:28:37.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.125+0000 7ff13e7fc640 1 --2- 172.21.15.64:0/1922352440 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff1280a2c10 0x7ff1280a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff1400ffe30 tx=0x7ff134005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:37.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.129+0000 7ff127fff640 1 -- 172.21.15.64:0/1922352440 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff1300f3030 con 0x7ff140101de0 2024-09-12T03:28:37.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:36 smithi195 ceph-mon[34786]: pgmap v334: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:28:37.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.345+0000 7ff145342640 1 -- 172.21.15.64:0/1922352440 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff14010a140 con 0x7ff140101de0 2024-09-12T03:28:37.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.345+0000 7ff127fff640 1 -- 172.21.15.64:0/1922352440 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11921 (secure 0 0 0) 0x7ff14010a140 con 0x7ff140101de0 2024-09-12T03:28:37.345 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:28:37.346 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":24,"btime":"2024-09-12T03:27:37:139728+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2580078738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2580078738},{"type":"v1","addr":"172.21.15.64:6835","nonce":2580078738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2333415004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2333415004},{"type":"v1","addr":"172.21.15.64:6837","nonce":2333415004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/3030618395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":3030618395},{"type":"v1","addr":"172.21.15.64:6839","nonce":3030618395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3333057625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3333057625},{"type":"v1","addr":"172.21.15.64:6841","nonce":3333057625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2991933266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2991933266},{"type":"v1","addr":"172.21.15.195:6833","nonce":2991933266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1138960346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1138960346},{"type":"v1","addr":"172.21.15.115:6833","nonce":1138960346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:26:59.647657+0000","modified":"2024-09-12T03:27:36.865338+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24481,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/332737916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":332737916},{"type":"v1","addr":"172.21.15.115:6835","nonce":332737916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/2531045583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2531045583},{"type":"v1","addr":"172.21.15.195:6835","nonce":2531045583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/71647401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":71647401},{"type":"v1","addr":"172.21.15.195:6837","nonce":71647401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.115:6837/2915155705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2915155705},{"type":"v1","addr":"172.21.15.115:6837","nonce":2915155705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.195:6839/2574695134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":2574695134},{"type":"v1","addr":"172.21.15.195:6839","nonce":2574695134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6839/2238031796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2238031796},{"type":"v1","addr":"172.21.15.115:6839","nonce":2238031796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24470,24481,24500]},"id":1}]} 2024-09-12T03:28:37.346 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 24 2024-09-12T03:28:37.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.347+0000 7ff145342640 1 -- 172.21.15.64:0/1922352440 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff1280a2c10 msgr2=0x7ff1280a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:37.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.347+0000 7ff145342640 1 --2- 172.21.15.64:0/1922352440 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff1280a2c10 0x7ff1280a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff1400ffe30 tx=0x7ff134005e50 comp rx=0 tx=0).stop 2024-09-12T03:28:37.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.347+0000 7ff145342640 1 -- 172.21.15.64:0/1922352440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff140101de0 msgr2=0x7ff140198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:37.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.347+0000 7ff145342640 1 --2- 172.21.15.64:0/1922352440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff140101de0 0x7ff140198380 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff130005940 tx=0x7ff130004270 comp rx=0 tx=0).stop 2024-09-12T03:28:37.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.347+0000 7ff145342640 1 -- 172.21.15.64:0/1922352440 shutdown_connections 2024-09-12T03:28:37.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.347+0000 7ff145342640 1 --2- 172.21.15.64:0/1922352440 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff1280a2c10 0x7ff1280a50d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:37.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.347+0000 7ff145342640 1 --2- 172.21.15.64:0/1922352440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff140106f10 0x7ff14019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:37.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.347+0000 7ff145342640 1 --2- 172.21.15.64:0/1922352440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff140102790 0x7ff1401988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:37.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.347+0000 7ff145342640 1 --2- 172.21.15.64:0/1922352440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff140101de0 0x7ff140198380 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:37.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.347+0000 7ff145342640 1 -- 172.21.15.64:0/1922352440 >> 172.21.15.64:0/1922352440 conn(0x7ff1400fd8c0 msgr2=0x7ff1400fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:37.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.348+0000 7ff145342640 1 -- 172.21.15.64:0/1922352440 shutdown_connections 2024-09-12T03:28:37.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:37.348+0000 7ff145342640 1 -- 172.21.15.64:0/1922352440 wait complete. 2024-09-12T03:28:37.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:36 smithi115 ceph-mon[34801]: pgmap v334: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:28:38.079 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:37 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/1922352440' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:28:38.112 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:28:38.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:37 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/1922352440' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:28:38.375 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:38.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:37 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/1922352440' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:28:38.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:38.882+0000 7f063b912640 1 -- 172.21.15.64:0/2608427160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0634101de0 msgr2=0x7f06341021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:38.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:38.882+0000 7f063b912640 1 --2- 172.21.15.64:0/2608427160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0634101de0 0x7f06341021c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0628009900 tx=0x7f062802a410 comp rx=0 tx=0).stop 2024-09-12T03:28:38.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:38.882+0000 7f063b912640 1 -- 172.21.15.64:0/2608427160 shutdown_connections 2024-09-12T03:28:38.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:38.882+0000 7f063b912640 1 --2- 172.21.15.64:0/2608427160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0634106f10 0x7f063410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:38.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:38.882+0000 7f063b912640 1 --2- 172.21.15.64:0/2608427160 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0634102790 0x7f06341067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:38.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:38.882+0000 7f063b912640 1 --2- 172.21.15.64:0/2608427160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0634101de0 0x7f06341021c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:38.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:38.882+0000 7f063b912640 1 -- 172.21.15.64:0/2608427160 >> 172.21.15.64:0/2608427160 conn(0x7f06340fd8c0 msgr2=0x7f06340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:38.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:38.882+0000 7f063b912640 1 -- 172.21.15.64:0/2608427160 shutdown_connections 2024-09-12T03:28:38.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:38.882+0000 7f063b912640 1 -- 172.21.15.64:0/2608427160 wait complete. 2024-09-12T03:28:38.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:38.882+0000 7f063b912640 1 Processor -- start 2024-09-12T03:28:38.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:38.883+0000 7f063b912640 1 -- start start 2024-09-12T03:28:38.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:38.883+0000 7f063b912640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0634101de0 0x7f06340744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:38.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:38.883+0000 7f063b912640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0634102790 0x7f0634074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:38.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:38.883+0000 7f063b912640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0634106f10 0x7f0634074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:38.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:38.883+0000 7f063b912640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0634111e40 con 0x7f0634102790 2024-09-12T03:28:38.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:38.883+0000 7f063b912640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0634111cc0 con 0x7f0634106f10 2024-09-12T03:28:38.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:38.883+0000 7f063b912640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0634111fc0 con 0x7f0634101de0 2024-09-12T03:28:38.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:38.883+0000 7f0638ead640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0634102790 0x7f0634074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:38.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:38.883+0000 7f0638ead640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0634102790 0x7f0634074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41828/0 (socket says 172.21.15.64:41828) 2024-09-12T03:28:38.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:38.883+0000 7f0638ead640 1 -- 172.21.15.64:0/2834669089 learned_addr learned my addr 172.21.15.64:0/2834669089 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:38.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:38.883+0000 7f06396ae640 1 --2- 172.21.15.64:0/2834669089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0634101de0 0x7f06340744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:38.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:38.883+0000 7f0639eaf640 1 --2- 172.21.15.64:0/2834669089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0634106f10 0x7f0634074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:38.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:38.883+0000 7f0638ead640 1 -- 172.21.15.64:0/2834669089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0634101de0 msgr2=0x7f06340744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:38.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:38.883+0000 7f0638ead640 1 --2- 172.21.15.64:0/2834669089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0634101de0 0x7f06340744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:38.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:38.883+0000 7f0638ead640 1 -- 172.21.15.64:0/2834669089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0634106f10 msgr2=0x7f0634074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:38.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:38.883+0000 7f0638ead640 1 --2- 172.21.15.64:0/2834669089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0634106f10 0x7f0634074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:38.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:38.883+0000 7f0638ead640 1 -- 172.21.15.64:0/2834669089 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06341a6860 con 0x7f0634102790 2024-09-12T03:28:38.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:38.884+0000 7f0638ead640 1 --2- 172.21.15.64:0/2834669089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0634102790 0x7f0634074a30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f063000ba50 tx=0x7f063000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:38.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:38.884+0000 7f06267fc640 1 -- 172.21.15.64:0/2834669089 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0630019070 con 0x7f0634102790 2024-09-12T03:28:38.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:38.884+0000 7f06267fc640 1 -- 172.21.15.64:0/2834669089 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f063000cbe0 con 0x7f0634102790 2024-09-12T03:28:38.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:38.884+0000 7f06267fc640 1 -- 172.21.15.64:0/2834669089 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06300075d0 con 0x7f0634102790 2024-09-12T03:28:38.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:38.884+0000 7f063b912640 1 -- 172.21.15.64:0/2834669089 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06341a6b50 con 0x7f0634102790 2024-09-12T03:28:38.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:38.884+0000 7f063b912640 1 -- 172.21.15.64:0/2834669089 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06341a7040 con 0x7f0634102790 2024-09-12T03:28:38.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:38.885+0000 7f063b912640 1 -- 172.21.15.64:0/2834669089 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06341021c0 con 0x7f0634102790 2024-09-12T03:28:38.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:38.886+0000 7f06267fc640 1 -- 172.21.15.64:0/2834669089 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f063001c070 con 0x7f0634102790 2024-09-12T03:28:38.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:38.887+0000 7f06267fc640 1 --2- 172.21.15.64:0/2834669089 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f06200a2c10 0x7f06200a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:38.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:38.887+0000 7f06396ae640 1 --2- 172.21.15.64:0/2834669089 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f06200a2c10 0x7f06200a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:38.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:38.887+0000 7f06267fc640 1 -- 172.21.15.64:0/2834669089 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f06300ce9d0 con 0x7f0634102790 2024-09-12T03:28:38.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:38.887+0000 7f06396ae640 1 --2- 172.21.15.64:0/2834669089 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f06200a2c10 0x7f06200a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0628031440 tx=0x7f0628005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:38.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:38.891+0000 7f06267fc640 1 -- 172.21.15.64:0/2834669089 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f063009af30 con 0x7f0634102790 2024-09-12T03:28:39.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:39.070+0000 7f063b912640 1 -- 172.21.15.64:0/2834669089 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f063410a0f0 con 0x7f0634102790 2024-09-12T03:28:39.071 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:38 smithi064 ceph-mon[30952]: pgmap v335: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:28:39.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:39.072+0000 7f06267fc640 1 -- 172.21.15.64:0/2834669089 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24680 (secure 0 0 0) 0x7f063009f230 con 0x7f0634102790 2024-09-12T03:28:39.073 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:28:39.073 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":90,"fsid":"b0812460-70b5-11ef-bcea-c7b262605968","created":"2024-09-12T03:19:15.900322+0000","modified":"2024-09-12T03:28:08.835864+0000","last_up_change":"2024-09-12T03:26:06.122971+0000","last_in_change":"2024-09-12T03:25:43.130689+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:22:21.915870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:26:55.826348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:26:57.984319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T03:27:04.460292+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"326271c9-1f28-4ce6-b3be-fcc57479d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6803","nonce":1307832590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6805","nonce":1307832590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6809","nonce":1307832590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6807","nonce":1307832590}]},"public_addr":"172.21.15.64:6803/1307832590","cluster_addr":"172.21.15.64:6805/1307832590","heartbeat_back_addr":"172.21.15.64:6809/1307832590","heartbeat_front_addr":"172.21.15.64:6807/1307832590","state":["exists","up"]},{"osd":1,"uuid":"7dca0c28-c234-423c-936e-2351d63f918e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6801","nonce":3123728622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6803","nonce":3123728622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6807","nonce":3123728622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6805","nonce":3123728622}]},"public_addr":"172.21.15.115:6801/3123728622","cluster_addr":"172.21.15.115:6803/3123728622","heartbeat_back_addr":"172.21.15.115:6807/3123728622","heartbeat_front_addr":"172.21.15.115:6805/3123728622","state":["exists","up"]},{"osd":2,"uuid":"4b3a3c7a-aec6-455a-9a9a-1ac9cc900f24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6801","nonce":1664125451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6803","nonce":1664125451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6807","nonce":1664125451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6805","nonce":1664125451}]},"public_addr":"172.21.15.195:6801/1664125451","cluster_addr":"172.21.15.195:6803/1664125451","heartbeat_back_addr":"172.21.15.195:6807/1664125451","heartbeat_front_addr":"172.21.15.195:6805/1664125451","state":["exists","up"]},{"osd":3,"uuid":"8768c339-2c1e-4294-8b78-3c2adab6750f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6811","nonce":3935517185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6813","nonce":3935517185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6817","nonce":3935517185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6815","nonce":3935517185}]},"public_addr":"172.21.15.64:6811/3935517185","cluster_addr":"172.21.15.64:6813/3935517185","heartbeat_back_addr":"172.21.15.64:6817/3935517185","heartbeat_front_addr":"172.21.15.64:6815/3935517185","state":["exists","up"]},{"osd":4,"uuid":"123aac47-a8ae-47c7-93a7-0dd293356322","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6809","nonce":446665227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6811","nonce":446665227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6815","nonce":446665227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6813","nonce":446665227}]},"public_addr":"172.21.15.115:6809/446665227","cluster_addr":"172.21.15.115:6811/446665227","heartbeat_back_addr":"172.21.15.115:6815/446665227","heartbeat_front_addr":"172.21.15.115:6813/446665227","state":["exists","up"]},{"osd":5,"uuid":"c90bcaed-e7a8-4852-8677-b01dc73ea44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6809","nonce":1187293668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6811","nonce":1187293668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6815","nonce":1187293668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6813","nonce":1187293668}]},"public_addr":"172.21.15.195:6809/1187293668","cluster_addr":"172.21.15.195:6811/1187293668","heartbeat_back_addr":"172.21.15.195:6815/1187293668","heartbeat_front_addr":"172.21.15.195:6813/1187293668","state":["exists","up"]},{"osd":6,"uuid":"fd292547-6695-4889-8867-e74318bbb00f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6819","nonce":560343274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6821","nonce":560343274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6825","nonce":560343274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6823","nonce":560343274}]},"public_addr":"172.21.15.64:6819/560343274","cluster_addr":"172.21.15.64:6821/560343274","heartbeat_back_addr":"172.21.15.64:6825/560343274","heartbeat_front_addr":"172.21.15.64:6823/560343274","state":["exists","up"]},{"osd":7,"uuid":"ad7c4151-cd98-4113-879b-ca2b7c83c7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6817","nonce":3075734174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6819","nonce":3075734174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6823","nonce":3075734174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6821","nonce":3075734174}]},"public_addr":"172.21.15.115:6817/3075734174","cluster_addr":"172.21.15.115:6819/3075734174","heartbeat_back_addr":"172.21.15.115:6823/3075734174","heartbeat_front_addr":"172.21.15.115:6821/3075734174","state":["exists","up"]},{"osd":8,"uuid":"6de50049-c28f-4f21-b123-05e55bf59127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6817","nonce":2184742494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6819","nonce":2184742494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6823","nonce":2184742494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6821","nonce":2184742494}]},"public_addr":"172.21.15.195:6817/2184742494","cluster_addr":"172.21.15.195:6819/2184742494","heartbeat_back_addr":"172.21.15.195:6823/2184742494","heartbeat_front_addr":"172.21.15.195:6821/2184742494","state":["exists","up"]},{"osd":9,"uuid":"36ab792c-58f2-4beb-9024-7f3d1e3785dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6827","nonce":561728112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6829","nonce":561728112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6833","nonce":561728112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6831","nonce":561728112}]},"public_addr":"172.21.15.64:6827/561728112","cluster_addr":"172.21.15.64:6829/561728112","heartbeat_back_addr":"172.21.15.64:6833/561728112","heartbeat_front_addr":"172.21.15.64:6831/561728112","state":["exists","up"]},{"osd":10,"uuid":"d1f7881c-ec3f-466e-9cea-1d5cb441d970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6825","nonce":2977887627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6827","nonce":2977887627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6831","nonce":2977887627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6829","nonce":2977887627}]},"public_addr":"172.21.15.115:6825/2977887627","cluster_addr":"172.21.15.115:6827/2977887627","heartbeat_back_addr":"172.21.15.115:6831/2977887627","heartbeat_front_addr":"172.21.15.115:6829/2977887627","state":["exists","up"]},{"osd":11,"uuid":"7c46c028-22c7-450f-9927-2144a7a09e9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6825","nonce":591599352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6827","nonce":591599352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6831","nonce":591599352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6829","nonce":591599352}]},"public_addr":"172.21.15.195:6825/591599352","cluster_addr":"172.21.15.195:6827/591599352","heartbeat_back_addr":"172.21.15.195:6831/591599352","heartbeat_front_addr":"172.21.15.195:6829/591599352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:29.151454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:54.308909+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:18.680995+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:44.687548+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:10.166131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:35.026132+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:24:00.242966+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:24:24.600848+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:24:49.563060+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:25:15.025730+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:25:39.830995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:26:04.394276+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6801/68603379":"2024-09-13T03:20:07.411102+0000","172.21.15.64:0/3827202979":"2024-09-13T03:20:07.411102+0000","172.21.15.64:6800/68603379":"2024-09-13T03:20:07.411102+0000","172.21.15.64:0/1217136221":"2024-09-13T03:20:07.411102+0000","172.21.15.64:0/3079679964":"2024-09-13T03:19:36.839431+0000","172.21.15.64:6801/3186640984":"2024-09-13T03:19:36.839431+0000","172.21.15.64:0/3334629354":"2024-09-13T03:20:07.411102+0000","172.21.15.64:6800/3186640984":"2024-09-13T03:19:36.839431+0000","172.21.15.64:0/3564418115":"2024-09-13T03:19:36.839431+0000","172.21.15.64:0/686137382":"2024-09-13T03:19:36.839431+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:28:39.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:39.074+0000 7f063b912640 1 -- 172.21.15.64:0/2834669089 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f06200a2c10 msgr2=0x7f06200a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:39.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:39.074+0000 7f063b912640 1 --2- 172.21.15.64:0/2834669089 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f06200a2c10 0x7f06200a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0628031440 tx=0x7f0628005e80 comp rx=0 tx=0).stop 2024-09-12T03:28:39.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:39.074+0000 7f063b912640 1 -- 172.21.15.64:0/2834669089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0634102790 msgr2=0x7f0634074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:39.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:39.074+0000 7f063b912640 1 --2- 172.21.15.64:0/2834669089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0634102790 0x7f0634074a30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f063000ba50 tx=0x7f063000bf10 comp rx=0 tx=0).stop 2024-09-12T03:28:39.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:39.074+0000 7f063b912640 1 -- 172.21.15.64:0/2834669089 shutdown_connections 2024-09-12T03:28:39.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:39.074+0000 7f063b912640 1 --2- 172.21.15.64:0/2834669089 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f06200a2c10 0x7f06200a50d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:39.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:39.074+0000 7f063b912640 1 --2- 172.21.15.64:0/2834669089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0634106f10 0x7f0634074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:39.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:39.074+0000 7f063b912640 1 --2- 172.21.15.64:0/2834669089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0634102790 0x7f0634074a30 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:39.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:39.074+0000 7f063b912640 1 --2- 172.21.15.64:0/2834669089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0634101de0 0x7f06340744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:39.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:39.074+0000 7f063b912640 1 -- 172.21.15.64:0/2834669089 >> 172.21.15.64:0/2834669089 conn(0x7f06340fd8c0 msgr2=0x7f06340fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:39.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:39.074+0000 7f063b912640 1 -- 172.21.15.64:0/2834669089 shutdown_connections 2024-09-12T03:28:39.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:39.074+0000 7f063b912640 1 -- 172.21.15.64:0/2834669089 wait complete. 2024-09-12T03:28:39.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:38 smithi195 ceph-mon[34786]: pgmap v335: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:28:39.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:38 smithi115 ceph-mon[34801]: pgmap v335: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:28:39.750 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:28:40.006 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:40.021 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:39 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/2834669089' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:28:40.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:39 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/2834669089' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:28:40.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:39 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/2834669089' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:28:40.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.522+0000 7f5865152640 1 -- 172.21.15.64:0/154471692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5860101420 msgr2=0x7f586010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:40.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.522+0000 7f5865152640 1 --2- 172.21.15.64:0/154471692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5860101420 0x7f586010d950 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5858009930 tx=0x7f585802a420 comp rx=0 tx=0).stop 2024-09-12T03:28:40.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.522+0000 7f5865152640 1 -- 172.21.15.64:0/154471692 shutdown_connections 2024-09-12T03:28:40.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.522+0000 7f5865152640 1 --2- 172.21.15.64:0/154471692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5860101420 0x7f586010d950 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:40.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.522+0000 7f5865152640 1 --2- 172.21.15.64:0/154471692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5860100a80 0x7f5860100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:40.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.522+0000 7f5865152640 1 --2- 172.21.15.64:0/154471692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5860106a80 0x7f5860106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:40.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.522+0000 7f5865152640 1 -- 172.21.15.64:0/154471692 >> 172.21.15.64:0/154471692 conn(0x7f58600fc7a0 msgr2=0x7f58600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:40.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.522+0000 7f5865152640 1 -- 172.21.15.64:0/154471692 shutdown_connections 2024-09-12T03:28:40.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.522+0000 7f5865152640 1 -- 172.21.15.64:0/154471692 wait complete. 2024-09-12T03:28:40.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.523+0000 7f5865152640 1 Processor -- start 2024-09-12T03:28:40.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.523+0000 7f5865152640 1 -- start start 2024-09-12T03:28:40.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.524+0000 7f5865152640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5860100a80 0x7f5860102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:40.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.524+0000 7f5865152640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5860101420 0x7f5860102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:40.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.524+0000 7f5865152640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5860106a80 0x7f58601030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:40.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.524+0000 7f5865152640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5860110d30 con 0x7f5860100a80 2024-09-12T03:28:40.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.524+0000 7f5865152640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5860110bb0 con 0x7f5860106a80 2024-09-12T03:28:40.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.524+0000 7f5865152640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5860110eb0 con 0x7f5860101420 2024-09-12T03:28:40.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.524+0000 7f585ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5860100a80 0x7f5860102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:40.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.524+0000 7f585ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5860100a80 0x7f5860102650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41834/0 (socket says 172.21.15.64:41834) 2024-09-12T03:28:40.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.524+0000 7f585ed9d640 1 -- 172.21.15.64:0/3920543485 learned_addr learned my addr 172.21.15.64:0/3920543485 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:40.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.524+0000 7f585e59c640 1 --2- 172.21.15.64:0/3920543485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5860101420 0x7f5860102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:40.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.524+0000 7f585f59e640 1 --2- 172.21.15.64:0/3920543485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5860106a80 0x7f58601030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:40.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.524+0000 7f585ed9d640 1 -- 172.21.15.64:0/3920543485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5860101420 msgr2=0x7f5860102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:40.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.524+0000 7f585ed9d640 1 --2- 172.21.15.64:0/3920543485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5860101420 0x7f5860102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:40.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.524+0000 7f585ed9d640 1 -- 172.21.15.64:0/3920543485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5860106a80 msgr2=0x7f58601030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:40.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.524+0000 7f585ed9d640 1 --2- 172.21.15.64:0/3920543485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5860106a80 0x7f58601030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:40.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.524+0000 7f585ed9d640 1 -- 172.21.15.64:0/3920543485 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58601a6790 con 0x7f5860100a80 2024-09-12T03:28:40.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.524+0000 7f585ed9d640 1 --2- 172.21.15.64:0/3920543485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5860100a80 0x7f5860102650 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f585000cce0 tx=0x7f585000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:40.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.524+0000 7f5857fff640 1 -- 172.21.15.64:0/3920543485 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585000f650 con 0x7f5860100a80 2024-09-12T03:28:40.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.524+0000 7f5857fff640 1 -- 172.21.15.64:0/3920543485 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f585000fca0 con 0x7f5860100a80 2024-09-12T03:28:40.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.524+0000 7f5865152640 1 -- 172.21.15.64:0/3920543485 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58601a6a80 con 0x7f5860100a80 2024-09-12T03:28:40.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.525+0000 7f5857fff640 1 -- 172.21.15.64:0/3920543485 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5850018b90 con 0x7f5860100a80 2024-09-12T03:28:40.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.525+0000 7f5865152640 1 -- 172.21.15.64:0/3920543485 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58601a6fc0 con 0x7f5860100a80 2024-09-12T03:28:40.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.526+0000 7f5865152640 1 -- 172.21.15.64:0/3920543485 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5860100ee0 con 0x7f5860100a80 2024-09-12T03:28:40.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.527+0000 7f5857fff640 1 -- 172.21.15.64:0/3920543485 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f585000f7f0 con 0x7f5860100a80 2024-09-12T03:28:40.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.528+0000 7f5857fff640 1 --2- 172.21.15.64:0/3920543485 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f584c0a2c10 0x7f584c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:40.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.528+0000 7f585e59c640 1 --2- 172.21.15.64:0/3920543485 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f584c0a2c10 0x7f584c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:40.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.528+0000 7f5857fff640 1 -- 172.21.15.64:0/3920543485 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f58500cfc50 con 0x7f5860100a80 2024-09-12T03:28:40.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.528+0000 7f585e59c640 1 --2- 172.21.15.64:0/3920543485 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f584c0a2c10 0x7f584c0a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f58600feab0 tx=0x7f584800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:40.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.531+0000 7f5857fff640 1 -- 172.21.15.64:0/3920543485 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5850014030 con 0x7f5860100a80 2024-09-12T03:28:40.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.740+0000 7f5865152640 1 -- 172.21.15.64:0/3920543485 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5860103b80 con 0x7f5860100a80 2024-09-12T03:28:40.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.741+0000 7f5857fff640 1 -- 172.21.15.64:0/3920543485 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11921 (secure 0 0 0) 0x7f585009c780 con 0x7f5860100a80 2024-09-12T03:28:40.741 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:28:40.741 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":24,"btime":"2024-09-12T03:27:37:139728+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2580078738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2580078738},{"type":"v1","addr":"172.21.15.64:6835","nonce":2580078738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2333415004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2333415004},{"type":"v1","addr":"172.21.15.64:6837","nonce":2333415004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/3030618395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":3030618395},{"type":"v1","addr":"172.21.15.64:6839","nonce":3030618395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3333057625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3333057625},{"type":"v1","addr":"172.21.15.64:6841","nonce":3333057625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2991933266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2991933266},{"type":"v1","addr":"172.21.15.195:6833","nonce":2991933266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1138960346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1138960346},{"type":"v1","addr":"172.21.15.115:6833","nonce":1138960346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:26:59.647657+0000","modified":"2024-09-12T03:27:36.865338+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24481,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/332737916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":332737916},{"type":"v1","addr":"172.21.15.115:6835","nonce":332737916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/2531045583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2531045583},{"type":"v1","addr":"172.21.15.195:6835","nonce":2531045583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/71647401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":71647401},{"type":"v1","addr":"172.21.15.195:6837","nonce":71647401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.115:6837/2915155705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2915155705},{"type":"v1","addr":"172.21.15.115:6837","nonce":2915155705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.195:6839/2574695134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":2574695134},{"type":"v1","addr":"172.21.15.195:6839","nonce":2574695134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6839/2238031796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2238031796},{"type":"v1","addr":"172.21.15.115:6839","nonce":2238031796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24470,24481,24500]},"id":1}]} 2024-09-12T03:28:40.742 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 24 2024-09-12T03:28:40.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.743+0000 7f5865152640 1 -- 172.21.15.64:0/3920543485 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f584c0a2c10 msgr2=0x7f584c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:40.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.743+0000 7f5865152640 1 --2- 172.21.15.64:0/3920543485 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f584c0a2c10 0x7f584c0a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f58600feab0 tx=0x7f584800b390 comp rx=0 tx=0).stop 2024-09-12T03:28:40.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.743+0000 7f5865152640 1 -- 172.21.15.64:0/3920543485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5860100a80 msgr2=0x7f5860102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:40.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.743+0000 7f5865152640 1 --2- 172.21.15.64:0/3920543485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5860100a80 0x7f5860102650 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f585000cce0 tx=0x7f585000a620 comp rx=0 tx=0).stop 2024-09-12T03:28:40.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.743+0000 7f5865152640 1 -- 172.21.15.64:0/3920543485 shutdown_connections 2024-09-12T03:28:40.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.743+0000 7f5865152640 1 --2- 172.21.15.64:0/3920543485 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f584c0a2c10 0x7f584c0a50d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:40.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.743+0000 7f5865152640 1 --2- 172.21.15.64:0/3920543485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5860106a80 0x7f58601030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:40.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.743+0000 7f5865152640 1 --2- 172.21.15.64:0/3920543485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5860101420 0x7f5860102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:40.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.743+0000 7f5865152640 1 --2- 172.21.15.64:0/3920543485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5860100a80 0x7f5860102650 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:40.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.743+0000 7f5865152640 1 -- 172.21.15.64:0/3920543485 >> 172.21.15.64:0/3920543485 conn(0x7f58600fc7a0 msgr2=0x7f586006a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:40.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.744+0000 7f5865152640 1 -- 172.21.15.64:0/3920543485 shutdown_connections 2024-09-12T03:28:40.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:40.744+0000 7f5865152640 1 -- 172.21.15.64:0/3920543485 wait complete. 2024-09-12T03:28:41.021 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:40 smithi064 ceph-mon[30952]: pgmap v336: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:28:41.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:40 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/3920543485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:28:41.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:40 smithi195 ceph-mon[34786]: pgmap v336: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:28:41.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:40 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/3920543485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:28:41.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:40 smithi115 ceph-mon[34801]: pgmap v336: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:28:41.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:40 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/3920543485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:28:41.453 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-12T03:26:59.647657+0000', 'modified': '2024-09-12T03:27:36.865338+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24500, 'mds_1': 24481, 'mds_2': 24470}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24449': {'gid': 24449, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.115:6835/332737916', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6834', 'nonce': 332737916}, {'type': 'v1', 'addr': '172.21.15.115:6835', 'nonce': 332737916}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.195:6835/2531045583', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6834', 'nonce': 2531045583}, {'type': 'v1', 'addr': '172.21.15.195:6835', 'nonce': 2531045583}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.195:6837/71647401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6836', 'nonce': 71647401}, {'type': 'v1', 'addr': '172.21.15.195:6837', 'nonce': 71647401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.115:6837/2915155705', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6836', 'nonce': 2915155705}, {'type': 'v1', 'addr': '172.21.15.115:6837', 'nonce': 2915155705}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.195:6839/2574695134', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6838', 'nonce': 2574695134}, {'type': 'v1', 'addr': '172.21.15.195:6839', 'nonce': 2574695134}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24500': {'gid': 24500, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.115:6839/2238031796', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6838', 'nonce': 2238031796}, {'type': 'v1', 'addr': '172.21.15.115:6839', 'nonce': 2238031796}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24500, 'qdb_cluster': [24470, 24481, 24500]} 2024-09-12T03:28:41.454 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-12T03:28:41.454 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-12T03:28:41.716 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:42.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.234+0000 7f0b2650c640 1 -- 172.21.15.64:0/1677334096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b2010e480 msgr2=0x7f0b20110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:42.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.234+0000 7f0b2650c640 1 --2- 172.21.15.64:0/1677334096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b2010e480 0x7f0b20110870 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0b18009ca0 tx=0x7f0b1802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:28:42.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.234+0000 7f0b2650c640 1 -- 172.21.15.64:0/1677334096 shutdown_connections 2024-09-12T03:28:42.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.234+0000 7f0b2650c640 1 --2- 172.21.15.64:0/1677334096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b2010e480 0x7f0b20110870 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:42.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.234+0000 7f0b2650c640 1 --2- 172.21.15.64:0/1677334096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b20101280 0x7f0b2010df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:42.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.234+0000 7f0b2650c640 1 --2- 172.21.15.64:0/1677334096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b201008d0 0x7f0b20100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:42.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.234+0000 7f0b2650c640 1 -- 172.21.15.64:0/1677334096 >> 172.21.15.64:0/1677334096 conn(0x7f0b200fc7a0 msgr2=0x7f0b200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:42.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.234+0000 7f0b2650c640 1 -- 172.21.15.64:0/1677334096 shutdown_connections 2024-09-12T03:28:42.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.234+0000 7f0b2650c640 1 -- 172.21.15.64:0/1677334096 wait complete. 2024-09-12T03:28:42.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.235+0000 7f0b2650c640 1 Processor -- start 2024-09-12T03:28:42.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.235+0000 7f0b2650c640 1 -- start start 2024-09-12T03:28:42.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.235+0000 7f0b2650c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b201008d0 0x7f0b2019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:42.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.235+0000 7f0b2650c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b20101280 0x7f0b2019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:42.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.235+0000 7f0b2650c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b2010e480 0x7f0b2019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:42.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.235+0000 7f0b2650c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0b20114080 con 0x7f0b20101280 2024-09-12T03:28:42.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.235+0000 7f0b2650c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0b20113f00 con 0x7f0b201008d0 2024-09-12T03:28:42.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.235+0000 7f0b2650c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0b20114200 con 0x7f0b2010e480 2024-09-12T03:28:42.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.236+0000 7f0b1f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b20101280 0x7f0b2019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:42.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.236+0000 7f0b1f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b20101280 0x7f0b2019aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41860/0 (socket says 172.21.15.64:41860) 2024-09-12T03:28:42.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.236+0000 7f0b1f7fe640 1 -- 172.21.15.64:0/462752377 learned_addr learned my addr 172.21.15.64:0/462752377 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:42.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.236+0000 7f0b1ffff640 1 --2- 172.21.15.64:0/462752377 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b201008d0 0x7f0b2019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:42.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.236+0000 7f0b24aa9640 1 --2- 172.21.15.64:0/462752377 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b2010e480 0x7f0b2019ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:42.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.236+0000 7f0b1f7fe640 1 -- 172.21.15.64:0/462752377 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b2010e480 msgr2=0x7f0b2019ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:42.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.236+0000 7f0b1f7fe640 1 --2- 172.21.15.64:0/462752377 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b2010e480 0x7f0b2019ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:42.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.236+0000 7f0b1f7fe640 1 -- 172.21.15.64:0/462752377 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b201008d0 msgr2=0x7f0b2019a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:42.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.236+0000 7f0b1f7fe640 1 --2- 172.21.15.64:0/462752377 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b201008d0 0x7f0b2019a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:42.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.236+0000 7f0b1f7fe640 1 -- 172.21.15.64:0/462752377 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b2019f470 con 0x7f0b20101280 2024-09-12T03:28:42.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.236+0000 7f0b1f7fe640 1 --2- 172.21.15.64:0/462752377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b20101280 0x7f0b2019aa00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0b1400c970 tx=0x7f0b1400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:42.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.236+0000 7f0b1d7fa640 1 -- 172.21.15.64:0/462752377 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b14007bb0 con 0x7f0b20101280 2024-09-12T03:28:42.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.236+0000 7f0b1d7fa640 1 -- 172.21.15.64:0/462752377 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b14007d50 con 0x7f0b20101280 2024-09-12T03:28:42.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.236+0000 7f0b2650c640 1 -- 172.21.15.64:0/462752377 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b2019f760 con 0x7f0b20101280 2024-09-12T03:28:42.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.236+0000 7f0b1d7fa640 1 -- 172.21.15.64:0/462752377 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b14004e50 con 0x7f0b20101280 2024-09-12T03:28:42.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.236+0000 7f0b2650c640 1 -- 172.21.15.64:0/462752377 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b201a7040 con 0x7f0b20101280 2024-09-12T03:28:42.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.239+0000 7f0b1d7fa640 1 -- 172.21.15.64:0/462752377 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0b14005020 con 0x7f0b20101280 2024-09-12T03:28:42.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.239+0000 7f0b288e5740 1 Processor -- start 2024-09-12T03:28:42.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.239+0000 7f0b288e5740 1 -- start start 2024-09-12T03:28:42.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.239+0000 7f0b288e5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a0907f9d0 0x562a0907fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:42.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.239+0000 7f0b288e5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a090802f0 0x562a0908a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:42.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.239+0000 7f0b288e5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a0908abb0 0x562a0908cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:42.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.239+0000 7f0b288e5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562a08fe6090 con 0x562a0907f9d0 2024-09-12T03:28:42.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.239+0000 7f0b288e5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562a08f8da70 con 0x562a090802f0 2024-09-12T03:28:42.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.239+0000 7f0b288e5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562a08fe2050 con 0x562a0908abb0 2024-09-12T03:28:42.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.239+0000 7f0b1d7fa640 1 --2- 172.21.15.64:0/462752377 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f0b080a2e30 0x7f0b080a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:42.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.240+0000 7f0b1f7fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a0908abb0 0x562a0908cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:42.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.240+0000 7f0b24aa9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a0907f9d0 0x562a0907fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:42.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.240+0000 7f0b1ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a090802f0 0x562a0908a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:42.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.240+0000 7f0b1f7fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a0908abb0 0x562a0908cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.64:52662/0 (socket says 172.21.15.64:52662) 2024-09-12T03:28:42.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.240+0000 7f0b24aa9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a0907f9d0 0x562a0907fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41870/0 (socket says 172.21.15.64:41870) 2024-09-12T03:28:42.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.240+0000 7f0b1ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a090802f0 0x562a0908a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.64:52224/0 (socket says 172.21.15.64:52224) 2024-09-12T03:28:42.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.240+0000 7f0b1f7fe640 1 -- 172.21.15.64:0/3549843668 learned_addr learned my addr 172.21.15.64:0/3549843668 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:42.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.240+0000 7f0b24aa9640 1 --2- 172.21.15.64:0/462752377 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f0b080a2e30 0x7f0b080a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:42.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.240+0000 7f0b1d7fa640 1 -- 172.21.15.64:0/462752377 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f0b140cf180 con 0x7f0b20101280 2024-09-12T03:28:42.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.240+0000 7f0b1f7fe640 1 -- 172.21.15.64:0/3549843668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a090802f0 msgr2=0x562a0908a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:42.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.240+0000 7f0b1f7fe640 1 --2- 172.21.15.64:0/3549843668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a090802f0 0x562a0908a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:42.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.240+0000 7f0b1f7fe640 1 -- 172.21.15.64:0/3549843668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a0907f9d0 msgr2=0x562a0907fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:42.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.240+0000 7f0b1f7fe640 1 --2- 172.21.15.64:0/3549843668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a0907f9d0 0x562a0907fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:42.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.240+0000 7f0b1f7fe640 1 -- 172.21.15.64:0/3549843668 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a08f9a130 con 0x562a0908abb0 2024-09-12T03:28:42.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.240+0000 7f0b1ffff640 1 --2- 172.21.15.64:0/3549843668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a090802f0 0x562a0908a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:42.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.240+0000 7f0b24aa9640 1 --2- 172.21.15.64:0/3549843668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a0907f9d0 0x562a0907fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:42.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.240+0000 7f0b1f7fe640 1 --2- 172.21.15.64:0/3549843668 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a0908abb0 0x562a0908cfa0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0b1409b4b0 tx=0x7f0b1400cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b28470755fdabc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:42.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.240+0000 7f0b24aa9640 1 --2- 172.21.15.64:0/462752377 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f0b080a2e30 0x7f0b080a52f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0b18006010 tx=0x7f0b180095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:42.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.241+0000 7f0b067fc640 1 -- 172.21.15.64:0/3549843668 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1401d030 con 0x562a0908abb0 2024-09-12T03:28:42.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.241+0000 7f0b067fc640 1 -- 172.21.15.64:0/3549843668 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b1409bc20 con 0x562a0908abb0 2024-09-12T03:28:42.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.241+0000 7f0b067fc640 1 -- 172.21.15.64:0/3549843668 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b14005560 con 0x562a0908abb0 2024-09-12T03:28:42.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.241+0000 7f0b288e5740 1 -- 172.21.15.64:0/3549843668 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a0908abb0 msgr2=0x562a0908cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:42.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.241+0000 7f0b288e5740 1 --2- 172.21.15.64:0/3549843668 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a0908abb0 0x562a0908cfa0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0b1409b4b0 tx=0x7f0b1400cf60 comp rx=0 tx=0).stop 2024-09-12T03:28:42.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.241+0000 7f0b288e5740 1 -- 172.21.15.64:0/3549843668 shutdown_connections 2024-09-12T03:28:42.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.241+0000 7f0b288e5740 1 --2- 172.21.15.64:0/3549843668 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a0908abb0 0x562a0908cfa0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:42.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.241+0000 7f0b288e5740 1 --2- 172.21.15.64:0/3549843668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a090802f0 0x562a0908a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:42.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.241+0000 7f0b288e5740 1 --2- 172.21.15.64:0/3549843668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a0907f9d0 0x562a0907fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:42.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.241+0000 7f0b288e5740 1 -- 172.21.15.64:0/3549843668 >> 172.21.15.64:0/3549843668 conn(0x562a0907bf60 msgr2=0x562a0907c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:42.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.241+0000 7f0b288e5740 1 -- 172.21.15.64:0/3549843668 shutdown_connections 2024-09-12T03:28:42.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.241+0000 7f0b288e5740 1 -- 172.21.15.64:0/3549843668 wait complete. 2024-09-12T03:28:42.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.242+0000 7f0b288e5740 1 -- start start 2024-09-12T03:28:42.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.242+0000 7f0b288e5740 1 Processor -- start 2024-09-12T03:28:42.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.242+0000 7f0b288e5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a0907f9d0 0x562a0908a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:42.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.242+0000 7f0b288e5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a090802f0 0x562a090877f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:42.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.242+0000 7f0b288e5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a0908abb0 0x562a09087d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:42.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.242+0000 7f0b288e5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562a0908d780 con 0x562a090802f0 2024-09-12T03:28:42.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.242+0000 7f0b288e5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562a08fa0f20 con 0x562a0908abb0 2024-09-12T03:28:42.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.242+0000 7f0b288e5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562a0908d900 con 0x562a0907f9d0 2024-09-12T03:28:42.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.242+0000 7f0b288e5740 10 client.0 _ll_register_callbacks cb 0x562a08f6a030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:42.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.243+0000 7f0b1f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a090802f0 0x562a090877f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:42.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.243+0000 7f0b1ffff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a0907f9d0 0x562a0908a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:42.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.243+0000 7f0b1f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a090802f0 0x562a090877f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41876/0 (socket says 172.21.15.64:41876) 2024-09-12T03:28:42.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.243+0000 7f0b1f7fe640 1 -- 172.21.15.64:0/386347645 learned_addr learned my addr 172.21.15.64:0/386347645 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:42.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.243+0000 7f0b24aa9640 1 --2- 172.21.15.64:0/386347645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a0908abb0 0x562a09087d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:42.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.243+0000 7f0b1f7fe640 1 -- 172.21.15.64:0/386347645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a0907f9d0 msgr2=0x562a0908a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:42.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.243+0000 7f0b1f7fe640 1 --2- 172.21.15.64:0/386347645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a0907f9d0 0x562a0908a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:42.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.243+0000 7f0b1f7fe640 1 -- 172.21.15.64:0/386347645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a0908abb0 msgr2=0x562a09087d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:42.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.243+0000 7f0b1f7fe640 1 --2- 172.21.15.64:0/386347645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a0908abb0 0x562a09087d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:42.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.243+0000 7f0b1f7fe640 1 -- 172.21.15.64:0/386347645 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a08d572b0 con 0x562a090802f0 2024-09-12T03:28:42.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.243+0000 7f0b1f7fe640 1 --2- 172.21.15.64:0/386347645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a090802f0 0x562a090877f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0b140cc850 tx=0x7f0b1409b370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e30c893f0bf8077 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:42.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.243+0000 7f0aff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:28:42.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.243+0000 7f0aff7fe640 1 -- 172.21.15.64:0/386347645 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1401d040 con 0x562a090802f0 2024-09-12T03:28:42.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.243+0000 7f0aff7fe640 1 -- 172.21.15.64:0/386347645 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f0b14099820 con 0x562a090802f0 2024-09-12T03:28:42.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.243+0000 7f0b288e5740 1 -- 172.21.15.64:0/386347645 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b1409bc20 con 0x562a090802f0 2024-09-12T03:28:42.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.243+0000 7f0aff7fe640 1 -- 172.21.15.64:0/386347645 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0b1409ae80 con 0x562a090802f0 2024-09-12T03:28:42.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.243+0000 7f0b288e5740 10 client.15063 fetch_fsmap learned FSMap version 24 2024-09-12T03:28:42.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.243+0000 7f0b288e5740 1 -- 172.21.15.64:0/386347645 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562a090990f0 con 0x562a090802f0 2024-09-12T03:28:42.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.244+0000 7f0aff7fe640 1 -- 172.21.15.64:0/386347645 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b1409bc20 con 0x562a090802f0 2024-09-12T03:28:42.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.244+0000 7f0aff7fe640 1 -- 172.21.15.64:0/386347645 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f0b140a3d00 con 0x562a090802f0 2024-09-12T03:28:42.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.244+0000 7f0b288e5740 10 client.15063 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T03:28:42.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.244+0000 7f0b288e5740 10 client.15063 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-12T03:28:42.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.244+0000 7f0b288e5740 20 client.15063 populate_metadata read hostname 'smithi064' 2024-09-12T03:28:42.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.244+0000 7f0b288e5740 1 --2- 172.21.15.64:0/386347645 >> [v2:172.21.15.115:6836/2915155705,v1:172.21.15.115:6837/2915155705] conn(0x562a0909a440 0x562a0909c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:42.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.244+0000 7f0b288e5740 4 client.15063 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:28:42.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.244+0000 7f0b288e5740 1 -- 172.21.15.64:0/386347645 --> [v2:172.21.15.115:6836/2915155705,v1:172.21.15.115:6837/2915155705] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a08ff3600 con 0x562a0909a440 2024-09-12T03:28:42.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.244+0000 7f0b1ffff640 1 --2- 172.21.15.64:0/386347645 >> [v2:172.21.15.115:6836/2915155705,v1:172.21.15.115:6837/2915155705] conn(0x562a0909a440 0x562a0909c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:42.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.245+0000 7f0aff7fe640 10 client.15063 ms_handle_connect on v2:172.21.15.115:6836/2915155705 2024-09-12T03:28:42.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.245+0000 7f0b1ffff640 1 --2- 172.21.15.64:0/386347645 >> [v2:172.21.15.115:6836/2915155705,v1:172.21.15.115:6837/2915155705] conn(0x562a0909a440 0x562a0909c560 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3334572695ba9c73 server_cookie=b3f34a4d835be8ad in_seq=0 out_seq=0 2024-09-12T03:28:42.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.245+0000 7f0aff7fe640 1 -- 172.21.15.64:0/386347645 <== mds.2 v2:172.21.15.115:6836/2915155705 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562a08ff3600 con 0x562a0909a440 2024-09-12T03:28:42.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.245+0000 7f0aff7fe640 10 client.15063 handle_command_reply: tid=0 2024-09-12T03:28:42.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.245+0000 7f0b288e5740 1 client.15063 shutdown 2024-09-12T03:28:42.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.246+0000 7f0b288e5740 1 -- 172.21.15.64:0/386347645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a090802f0 msgr2=0x562a090877f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:42.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.246+0000 7f0b288e5740 1 --2- 172.21.15.64:0/386347645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a090802f0 0x562a090877f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0b140cc850 tx=0x7f0b1409b370 comp rx=0 tx=0).stop 2024-09-12T03:28:42.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.246+0000 7f0b288e5740 1 -- 172.21.15.64:0/386347645 shutdown_connections 2024-09-12T03:28:42.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.246+0000 7f0b288e5740 1 --2- 172.21.15.64:0/386347645 >> [v2:172.21.15.115:6836/2915155705,v1:172.21.15.115:6837/2915155705] conn(0x562a0909a440 0x562a0909c560 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:42.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.246+0000 7f0b288e5740 1 --2- 172.21.15.64:0/386347645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a0908abb0 0x562a09087d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:42.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.246+0000 7f0b288e5740 1 --2- 172.21.15.64:0/386347645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a090802f0 0x562a090877f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:42.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.246+0000 7f0b288e5740 1 --2- 172.21.15.64:0/386347645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a0907f9d0 0x562a0908a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:42.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.246+0000 7f0b288e5740 1 -- 172.21.15.64:0/386347645 >> 172.21.15.64:0/386347645 conn(0x562a0907bf60 msgr2=0x562a09081c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:42.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.246+0000 7f0aff7fe640 0 client.15063 ms_handle_reset on v2:172.21.15.115:6836/2915155705 2024-09-12T03:28:42.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.246+0000 7f0b288e5740 1 -- 172.21.15.64:0/386347645 shutdown_connections 2024-09-12T03:28:42.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.246+0000 7f0b288e5740 1 -- 172.21.15.64:0/386347645 wait complete. 2024-09-12T03:28:42.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.246+0000 7f0b288e5740 20 client.15063 trim_cache size 0 max 16384 2024-09-12T03:28:42.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.264+0000 7f0b288e5740 1 Processor -- start 2024-09-12T03:28:42.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.264+0000 7f0b288e5740 1 -- start start 2024-09-12T03:28:42.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.264+0000 7f0b288e5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a0907f9d0 0x562a0909aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:42.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.264+0000 7f0b288e5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a090802f0 0x562a0909be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:42.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.264+0000 7f0b288e5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a0908abb0 0x562a09085f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:42.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.264+0000 7f0b288e5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562a0909a8a0 con 0x562a0908abb0 2024-09-12T03:28:42.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.264+0000 7f0b288e5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562a0909af70 con 0x562a090802f0 2024-09-12T03:28:42.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.264+0000 7f0b288e5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562a0909b640 con 0x562a0907f9d0 2024-09-12T03:28:42.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.264+0000 7f0b24aa9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a0908abb0 0x562a09085f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:42.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.264+0000 7f0b24aa9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a0908abb0 0x562a09085f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41892/0 (socket says 172.21.15.64:41892) 2024-09-12T03:28:42.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.264+0000 7f0b24aa9640 1 -- 172.21.15.64:0/528034844 learned_addr learned my addr 172.21.15.64:0/528034844 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:42.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.264+0000 7f0b1f7fe640 1 --2- 172.21.15.64:0/528034844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a090802f0 0x562a0909be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:42.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.264+0000 7f0b1ffff640 1 --2- 172.21.15.64:0/528034844 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a0907f9d0 0x562a0909aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:42.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.264+0000 7f0b24aa9640 1 -- 172.21.15.64:0/528034844 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a0907f9d0 msgr2=0x562a0909aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:42.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.264+0000 7f0b24aa9640 1 --2- 172.21.15.64:0/528034844 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a0907f9d0 0x562a0909aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:42.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.264+0000 7f0b24aa9640 1 -- 172.21.15.64:0/528034844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a090802f0 msgr2=0x562a0909be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:42.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.264+0000 7f0b24aa9640 1 --2- 172.21.15.64:0/528034844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a090802f0 0x562a0909be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:42.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.264+0000 7f0b24aa9640 1 -- 172.21.15.64:0/528034844 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a09080120 con 0x562a0908abb0 2024-09-12T03:28:42.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.264+0000 7f0b1ffff640 1 --2- 172.21.15.64:0/528034844 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a0907f9d0 0x562a0909aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:28:42.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.265+0000 7f0b24aa9640 1 --2- 172.21.15.64:0/528034844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a0908abb0 0x562a09085f90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0b18009ca0 tx=0x7f0b18043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fff59d68ac90140a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:42.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.265+0000 7f0b05ffb640 1 -- 172.21.15.64:0/528034844 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18047040 con 0x562a0908abb0 2024-09-12T03:28:42.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.265+0000 7f0b05ffb640 1 -- 172.21.15.64:0/528034844 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b18036040 con 0x562a0908abb0 2024-09-12T03:28:42.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.265+0000 7f0b05ffb640 1 -- 172.21.15.64:0/528034844 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18030920 con 0x562a0908abb0 2024-09-12T03:28:42.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.265+0000 7f0b288e5740 1 -- 172.21.15.64:0/528034844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a0908abb0 msgr2=0x562a09085f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:42.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.265+0000 7f0b288e5740 1 --2- 172.21.15.64:0/528034844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a0908abb0 0x562a09085f90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0b18009ca0 tx=0x7f0b18043000 comp rx=0 tx=0).stop 2024-09-12T03:28:42.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.265+0000 7f0b288e5740 1 -- 172.21.15.64:0/528034844 shutdown_connections 2024-09-12T03:28:42.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.265+0000 7f0b288e5740 1 --2- 172.21.15.64:0/528034844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a0908abb0 0x562a09085f90 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:42.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.265+0000 7f0b288e5740 1 --2- 172.21.15.64:0/528034844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a090802f0 0x562a0909be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:42.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.265+0000 7f0b288e5740 1 --2- 172.21.15.64:0/528034844 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a0907f9d0 0x562a0909aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:42.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.265+0000 7f0b288e5740 1 -- 172.21.15.64:0/528034844 >> 172.21.15.64:0/528034844 conn(0x562a0907bf60 msgr2=0x562a09082de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:42.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.266+0000 7f0b288e5740 1 -- 172.21.15.64:0/528034844 shutdown_connections 2024-09-12T03:28:42.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.266+0000 7f0b288e5740 1 -- 172.21.15.64:0/528034844 wait complete. 2024-09-12T03:28:42.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.266+0000 7f0b288e5740 1 -- start start 2024-09-12T03:28:42.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.266+0000 7f0b288e5740 1 Processor -- start 2024-09-12T03:28:42.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.266+0000 7f0b288e5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a0907f9d0 0x562a0909dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:42.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.266+0000 7f0b288e5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a090802f0 0x562a0909e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:42.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.266+0000 7f0b288e5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a0908abb0 0x562a091037f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:42.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.266+0000 7f0b288e5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562a0909c3e0 con 0x562a0907f9d0 2024-09-12T03:28:42.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.266+0000 7f0b288e5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562a0909bd10 con 0x562a0908abb0 2024-09-12T03:28:42.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.266+0000 7f0b288e5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562a0909cab0 con 0x562a090802f0 2024-09-12T03:28:42.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.266+0000 7f0b288e5740 10 client.0 _ll_register_callbacks cb 0x7f0b140cf070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:42.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.267+0000 7f0b1ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a0907f9d0 0x562a0909dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:42.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.267+0000 7f0b1ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a0907f9d0 0x562a0909dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41902/0 (socket says 172.21.15.64:41902) 2024-09-12T03:28:42.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.267+0000 7f0b1ffff640 1 -- 172.21.15.64:0/3597504955 learned_addr learned my addr 172.21.15.64:0/3597504955 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:42.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.267+0000 7f0b24aa9640 1 --2- 172.21.15.64:0/3597504955 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a0908abb0 0x562a091037f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:42.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.267+0000 7f0b1f7fe640 1 --2- 172.21.15.64:0/3597504955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a090802f0 0x562a0909e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:42.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.267+0000 7f0b1ffff640 1 -- 172.21.15.64:0/3597504955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a090802f0 msgr2=0x562a0909e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:42.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.267+0000 7f0b1ffff640 1 --2- 172.21.15.64:0/3597504955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a090802f0 0x562a0909e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:42.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.267+0000 7f0b1ffff640 1 -- 172.21.15.64:0/3597504955 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a0908abb0 msgr2=0x562a091037f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:42.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.267+0000 7f0b1ffff640 1 --2- 172.21.15.64:0/3597504955 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a0908abb0 0x562a091037f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:42.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.267+0000 7f0b1ffff640 1 -- 172.21.15.64:0/3597504955 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a09103d30 con 0x562a0907f9d0 2024-09-12T03:28:42.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.267+0000 7f0b24aa9640 1 --2- 172.21.15.64:0/3597504955 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a0908abb0 0x562a091037f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:28:42.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.267+0000 7f0b1ffff640 1 --2- 172.21.15.64:0/3597504955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a0907f9d0 0x562a0909dce0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0b10010430 tx=0x7f0b100108f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb3eeade526393ab server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:42.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.267+0000 7f0aff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:28:42.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.267+0000 7f0aff7fe640 1 -- 172.21.15.64:0/3597504955 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1002b060 con 0x562a0907f9d0 2024-09-12T03:28:42.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.267+0000 7f0aff7fe640 1 -- 172.21.15.64:0/3597504955 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f0b1001e8d0 con 0x562a0907f9d0 2024-09-12T03:28:42.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.267+0000 7f0b288e5740 1 -- 172.21.15.64:0/3597504955 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b18036040 con 0x562a0907f9d0 2024-09-12T03:28:42.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.267+0000 7f0aff7fe640 1 -- 172.21.15.64:0/3597504955 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0b10032050 con 0x562a0907f9d0 2024-09-12T03:28:42.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.267+0000 7f0b288e5740 10 client.15069 fetch_fsmap learned FSMap version 24 2024-09-12T03:28:42.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.268+0000 7f0b288e5740 1 -- 172.21.15.64:0/3597504955 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562a09104ec0 con 0x562a0907f9d0 2024-09-12T03:28:42.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.268+0000 7f0aff7fe640 1 -- 172.21.15.64:0/3597504955 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b18036040 con 0x562a0907f9d0 2024-09-12T03:28:42.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.268+0000 7f0aff7fe640 1 -- 172.21.15.64:0/3597504955 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f0b1001d390 con 0x562a0907f9d0 2024-09-12T03:28:42.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.268+0000 7f0b288e5740 10 client.15069 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T03:28:42.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.268+0000 7f0b288e5740 10 client.15069 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-12T03:28:42.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.268+0000 7f0b288e5740 20 client.15069 populate_metadata read hostname 'smithi064' 2024-09-12T03:28:42.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.268+0000 7f0b288e5740 1 --2- 172.21.15.64:0/3597504955 >> [v2:172.21.15.115:6836/2915155705,v1:172.21.15.115:6837/2915155705] conn(0x562a09109b00 0x562a09108810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:42.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.268+0000 7f0b288e5740 4 client.15069 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:28:42.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.268+0000 7f0b288e5740 1 -- 172.21.15.64:0/3597504955 --> [v2:172.21.15.115:6836/2915155705,v1:172.21.15.115:6837/2915155705] -- command(tid 0: {"prefix": "status"}) -- 0x562a08fedb10 con 0x562a09109b00 2024-09-12T03:28:42.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.268+0000 7f0b1f7fe640 1 --2- 172.21.15.64:0/3597504955 >> [v2:172.21.15.115:6836/2915155705,v1:172.21.15.115:6837/2915155705] conn(0x562a09109b00 0x562a09108810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:42.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.269+0000 7f0b1f7fe640 1 --2- 172.21.15.64:0/3597504955 >> [v2:172.21.15.115:6836/2915155705,v1:172.21.15.115:6837/2915155705] conn(0x562a09109b00 0x562a09108810 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3ebf63b7935abeda server_cookie=7763f1a4278cc9c3 in_seq=0 out_seq=0 2024-09-12T03:28:42.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.269+0000 7f0aff7fe640 10 client.15069 ms_handle_connect on v2:172.21.15.115:6836/2915155705 2024-09-12T03:28:42.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.269+0000 7f0aff7fe640 1 -- 172.21.15.64:0/3597504955 <== mds.2 v2:172.21.15.115:6836/2915155705 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x562a08fedb10 con 0x562a09109b00 2024-09-12T03:28:42.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.269+0000 7f0aff7fe640 10 client.15069 handle_command_reply: tid=0 2024-09-12T03:28:42.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.269+0000 7f0b288e5740 1 client.15069 shutdown 2024-09-12T03:28:42.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.269+0000 7f0b288e5740 1 -- 172.21.15.64:0/3597504955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a0907f9d0 msgr2=0x562a0909dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:42.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.269+0000 7f0b288e5740 1 --2- 172.21.15.64:0/3597504955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a0907f9d0 0x562a0909dce0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0b10010430 tx=0x7f0b100108f0 comp rx=0 tx=0).stop 2024-09-12T03:28:42.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.269+0000 7f0b288e5740 1 -- 172.21.15.64:0/3597504955 shutdown_connections 2024-09-12T03:28:42.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.269+0000 7f0b288e5740 1 --2- 172.21.15.64:0/3597504955 >> [v2:172.21.15.115:6836/2915155705,v1:172.21.15.115:6837/2915155705] conn(0x562a09109b00 0x562a09108810 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:42.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.269+0000 7f0b288e5740 1 --2- 172.21.15.64:0/3597504955 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a0908abb0 0x562a091037f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:42.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.269+0000 7f0aff7fe640 0 client.15069 ms_handle_reset on v2:172.21.15.115:6836/2915155705 2024-09-12T03:28:42.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.270+0000 7f0b288e5740 1 --2- 172.21.15.64:0/3597504955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a090802f0 0x562a0909e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:42.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.270+0000 7f0b288e5740 1 --2- 172.21.15.64:0/3597504955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a0907f9d0 0x562a0909dce0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:42.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.270+0000 7f0b288e5740 1 -- 172.21.15.64:0/3597504955 >> 172.21.15.64:0/3597504955 conn(0x562a0907bf60 msgr2=0x562a0907a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:42.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.270+0000 7f0b288e5740 1 -- 172.21.15.64:0/3597504955 shutdown_connections 2024-09-12T03:28:42.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.270+0000 7f0b288e5740 1 -- 172.21.15.64:0/3597504955 wait complete. 2024-09-12T03:28:42.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.270+0000 7f0b288e5740 20 client.15069 trim_cache size 0 max 16384 2024-09-12T03:28:42.274 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T03:28:42.274 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "b0812460-70b5-11ef-bcea-c7b262605968", 2024-09-12T03:28:42.275 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 2, 2024-09-12T03:28:42.275 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24470, 2024-09-12T03:28:42.275 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-09-12T03:28:42.275 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-09-12T03:28:42.275 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-09-12T03:28:42.275 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 68.136287545000002, 2024-09-12T03:28:42.275 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 24, 2024-09-12T03:28:42.275 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 90, 2024-09-12T03:28:42.275 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 88, 2024-09-12T03:28:42.275 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 126.984879337 2024-09-12T03:28:42.275 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T03:28:42.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.270+0000 7f0b2650c640 1 -- 172.21.15.64:0/462752377 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f0b080a2e30 msgr2=0x7f0b080a52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:42.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.270+0000 7f0b2650c640 1 --2- 172.21.15.64:0/462752377 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f0b080a2e30 0x7f0b080a52f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0b18006010 tx=0x7f0b180095c0 comp rx=0 tx=0).stop 2024-09-12T03:28:42.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.270+0000 7f0b2650c640 1 -- 172.21.15.64:0/462752377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b20101280 msgr2=0x7f0b2019aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:42.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.270+0000 7f0b2650c640 1 --2- 172.21.15.64:0/462752377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b20101280 0x7f0b2019aa00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0b1400c970 tx=0x7f0b1400cd70 comp rx=0 tx=0).stop 2024-09-12T03:28:42.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.270+0000 7f0b2650c640 1 -- 172.21.15.64:0/462752377 shutdown_connections 2024-09-12T03:28:42.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.270+0000 7f0b2650c640 1 --2- 172.21.15.64:0/462752377 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f0b080a2e30 0x7f0b080a52f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:42.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.270+0000 7f0b2650c640 1 --2- 172.21.15.64:0/462752377 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b2010e480 0x7f0b2019ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:42.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.270+0000 7f0b2650c640 1 --2- 172.21.15.64:0/462752377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b20101280 0x7f0b2019aa00 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:42.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.270+0000 7f0b2650c640 1 --2- 172.21.15.64:0/462752377 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b201008d0 0x7f0b2019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:42.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.270+0000 7f0b2650c640 1 -- 172.21.15.64:0/462752377 >> 172.21.15.64:0/462752377 conn(0x7f0b200fc7a0 msgr2=0x7f0b200feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:42.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.270+0000 7f0b2650c640 1 -- 172.21.15.64:0/462752377 shutdown_connections 2024-09-12T03:28:42.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:42.270+0000 7f0b2650c640 1 -- 172.21.15.64:0/462752377 wait complete. 2024-09-12T03:28:43.019 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-12T03:28:43.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:42 smithi195 ceph-mon[34786]: pgmap v337: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:28:43.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:42 smithi064 ceph-mon[30952]: pgmap v337: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:28:43.275 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:43.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:42 smithi115 ceph-mon[34801]: pgmap v337: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:28:43.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.769+0000 7f86004cf640 1 -- 172.21.15.64:0/4124850836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85f8106a60 msgr2=0x7f85f8106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:43.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.769+0000 7f86004cf640 1 --2- 172.21.15.64:0/4124850836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85f8106a60 0x7f85f8106e40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f85f4009930 tx=0x7f85f402a420 comp rx=0 tx=0).stop 2024-09-12T03:28:43.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.769+0000 7f86004cf640 1 -- 172.21.15.64:0/4124850836 shutdown_connections 2024-09-12T03:28:43.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.769+0000 7f86004cf640 1 --2- 172.21.15.64:0/4124850836 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85f8101400 0x7f85f810d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:43.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.769+0000 7f86004cf640 1 --2- 172.21.15.64:0/4124850836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85f8100a60 0x7f85f8100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:43.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.769+0000 7f86004cf640 1 --2- 172.21.15.64:0/4124850836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85f8106a60 0x7f85f8106e40 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:43.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.769+0000 7f86004cf640 1 -- 172.21.15.64:0/4124850836 >> 172.21.15.64:0/4124850836 conn(0x7f85f80fc780 msgr2=0x7f85f80feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:43.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.769+0000 7f86004cf640 1 -- 172.21.15.64:0/4124850836 shutdown_connections 2024-09-12T03:28:43.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.769+0000 7f86004cf640 1 -- 172.21.15.64:0/4124850836 wait complete. 2024-09-12T03:28:43.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.770+0000 7f86004cf640 1 Processor -- start 2024-09-12T03:28:43.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.770+0000 7f86004cf640 1 -- start start 2024-09-12T03:28:43.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.770+0000 7f86004cf640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85f8100a60 0x7f85f81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:43.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.770+0000 7f86004cf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85f8101400 0x7f85f8198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:43.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.770+0000 7f86004cf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85f8106a60 0x7f85f819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:43.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.770+0000 7f86004cf640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f85f8110d10 con 0x7f85f8101400 2024-09-12T03:28:43.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.770+0000 7f86004cf640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f85f8110b90 con 0x7f85f8106a60 2024-09-12T03:28:43.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.770+0000 7f86004cf640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f85f8110e90 con 0x7f85f8100a60 2024-09-12T03:28:43.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.770+0000 7f85fda6a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85f8101400 0x7f85f8198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:43.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.770+0000 7f85fda6a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85f8101400 0x7f85f8198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41924/0 (socket says 172.21.15.64:41924) 2024-09-12T03:28:43.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.770+0000 7f85fda6a640 1 -- 172.21.15.64:0/1402295894 learned_addr learned my addr 172.21.15.64:0/1402295894 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:43.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.770+0000 7f85fea6c640 1 --2- 172.21.15.64:0/1402295894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85f8106a60 0x7f85f819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:43.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.770+0000 7f85fe26b640 1 --2- 172.21.15.64:0/1402295894 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85f8100a60 0x7f85f81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:43.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.771+0000 7f85fda6a640 1 -- 172.21.15.64:0/1402295894 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85f8100a60 msgr2=0x7f85f81983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:43.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.771+0000 7f85fda6a640 1 --2- 172.21.15.64:0/1402295894 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85f8100a60 0x7f85f81983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:43.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.771+0000 7f85fda6a640 1 -- 172.21.15.64:0/1402295894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85f8106a60 msgr2=0x7f85f819ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:43.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.771+0000 7f85fda6a640 1 --2- 172.21.15.64:0/1402295894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85f8106a60 0x7f85f819ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:43.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.771+0000 7f85fda6a640 1 -- 172.21.15.64:0/1402295894 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85f819d390 con 0x7f85f8101400 2024-09-12T03:28:43.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.771+0000 7f85fe26b640 1 --2- 172.21.15.64:0/1402295894 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85f8100a60 0x7f85f81983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:28:43.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.771+0000 7f85fda6a640 1 --2- 172.21.15.64:0/1402295894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85f8101400 0x7f85f8198920 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f85ec00e9f0 tx=0x7f85ec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:43.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.771+0000 7f85eb7fe640 1 -- 172.21.15.64:0/1402295894 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec00cd80 con 0x7f85f8101400 2024-09-12T03:28:43.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.771+0000 7f85eb7fe640 1 -- 172.21.15.64:0/1402295894 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85ec0054d0 con 0x7f85f8101400 2024-09-12T03:28:43.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.771+0000 7f85eb7fe640 1 -- 172.21.15.64:0/1402295894 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec010490 con 0x7f85f8101400 2024-09-12T03:28:43.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.771+0000 7f86004cf640 1 -- 172.21.15.64:0/1402295894 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85f819d680 con 0x7f85f8101400 2024-09-12T03:28:43.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.771+0000 7f86004cf640 1 -- 172.21.15.64:0/1402295894 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85f81a4f60 con 0x7f85f8101400 2024-09-12T03:28:43.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.773+0000 7f85eb7fe640 1 -- 172.21.15.64:0/1402295894 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f85ec005020 con 0x7f85f8101400 2024-09-12T03:28:43.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.774+0000 7f86028a8740 1 Processor -- start 2024-09-12T03:28:43.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.774+0000 7f86028a8740 1 -- start start 2024-09-12T03:28:43.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.774+0000 7f86028a8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c991af890 0x555c991afc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:43.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.774+0000 7f86028a8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c991b01b0 0x555c991bea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:43.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.774+0000 7f86028a8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c991befc0 0x555c991c13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:43.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.774+0000 7f86028a8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555c99116090 con 0x555c991befc0 2024-09-12T03:28:43.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.774+0000 7f86028a8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555c990bda70 con 0x555c991b01b0 2024-09-12T03:28:43.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.774+0000 7f86028a8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555c99112050 con 0x555c991af890 2024-09-12T03:28:43.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.774+0000 7f85fda6a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c991befc0 0x555c991c13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:43.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.774+0000 7f85eb7fe640 1 --2- 172.21.15.64:0/1402295894 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f85e40a2c10 0x7f85e40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:43.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.774+0000 7f85fe26b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c991b01b0 0x555c991bea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:43.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.774+0000 7f85fda6a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c991befc0 0x555c991c13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41934/0 (socket says 172.21.15.64:41934) 2024-09-12T03:28:43.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.774+0000 7f85fda6a640 1 -- 172.21.15.64:0/1982263130 learned_addr learned my addr 172.21.15.64:0/1982263130 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:43.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.774+0000 7f85fea6c640 1 --2- 172.21.15.64:0/1982263130 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c991af890 0x555c991afc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:43.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.774+0000 7f85fea6c640 1 --2- 172.21.15.64:0/1402295894 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f85e40a2c10 0x7f85e40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:43.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.775+0000 7f85eb7fe640 1 -- 172.21.15.64:0/1402295894 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f85ec0cf940 con 0x7f85f8101400 2024-09-12T03:28:43.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.775+0000 7f85fe26b640 1 -- 172.21.15.64:0/1982263130 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c991af890 msgr2=0x555c991afc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:43.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.775+0000 7f85fe26b640 1 --2- 172.21.15.64:0/1982263130 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c991af890 0x555c991afc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:43.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.775+0000 7f85fe26b640 1 -- 172.21.15.64:0/1982263130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c991befc0 msgr2=0x555c991c13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:43.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.775+0000 7f85fe26b640 1 --2- 172.21.15.64:0/1982263130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c991befc0 0x555c991c13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:43.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.775+0000 7f85fe26b640 1 -- 172.21.15.64:0/1982263130 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c990ca130 con 0x555c991b01b0 2024-09-12T03:28:43.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.775+0000 7f85fea6c640 1 --2- 172.21.15.64:0/1402295894 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f85e40a2c10 0x7f85e40a50d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f85f0009fd0 tx=0x7f85f00096a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:43.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.775+0000 7f85fe26b640 1 --2- 172.21.15.64:0/1982263130 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c991b01b0 0x555c991bea80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f85f4000c00 tx=0x7f85f4032c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f45e12ccc6e8655 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:43.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.775+0000 7f85e8ff9640 1 -- 172.21.15.64:0/1982263130 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f403f070 con 0x555c991b01b0 2024-09-12T03:28:43.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.775+0000 7f85e8ff9640 1 -- 172.21.15.64:0/1982263130 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85f4004030 con 0x555c991b01b0 2024-09-12T03:28:43.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.775+0000 7f85e8ff9640 1 -- 172.21.15.64:0/1982263130 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f4035070 con 0x555c991b01b0 2024-09-12T03:28:43.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.776+0000 7f86028a8740 1 -- 172.21.15.64:0/1982263130 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c991b01b0 msgr2=0x555c991bea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:43.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.776+0000 7f86028a8740 1 --2- 172.21.15.64:0/1982263130 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c991b01b0 0x555c991bea80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f85f4000c00 tx=0x7f85f4032c10 comp rx=0 tx=0).stop 2024-09-12T03:28:43.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.776+0000 7f86028a8740 1 -- 172.21.15.64:0/1982263130 shutdown_connections 2024-09-12T03:28:43.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.776+0000 7f86028a8740 1 --2- 172.21.15.64:0/1982263130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c991befc0 0x555c991c13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:43.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.776+0000 7f86028a8740 1 --2- 172.21.15.64:0/1982263130 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c991b01b0 0x555c991bea80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:43.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.776+0000 7f86028a8740 1 --2- 172.21.15.64:0/1982263130 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c991af890 0x555c991afc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:43.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.776+0000 7f86028a8740 1 -- 172.21.15.64:0/1982263130 >> 172.21.15.64:0/1982263130 conn(0x555c991abf60 msgr2=0x555c991ac340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:43.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.776+0000 7f86028a8740 1 -- 172.21.15.64:0/1982263130 shutdown_connections 2024-09-12T03:28:43.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.776+0000 7f86028a8740 1 -- 172.21.15.64:0/1982263130 wait complete. 2024-09-12T03:28:43.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.776+0000 7f86028a8740 1 -- start start 2024-09-12T03:28:43.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.777+0000 7f86028a8740 1 Processor -- start 2024-09-12T03:28:43.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.777+0000 7f86028a8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c991af890 0x555c991be810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:43.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.777+0000 7f86028a8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c991b01b0 0x555c991bbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:43.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.777+0000 7f86028a8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c991befc0 0x555c991bc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:43.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.777+0000 7f86028a8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555c991c1b30 con 0x555c991b01b0 2024-09-12T03:28:43.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.777+0000 7f86028a8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555c990d0f20 con 0x555c991af890 2024-09-12T03:28:43.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.777+0000 7f86028a8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555c991c1cb0 con 0x555c991befc0 2024-09-12T03:28:43.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.777+0000 7f86028a8740 10 client.0 _ll_register_callbacks cb 0x555c9909a030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:43.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.777+0000 7f85fda6a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c991b01b0 0x555c991bbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:43.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.777+0000 7f85fda6a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c991b01b0 0x555c991bbb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41936/0 (socket says 172.21.15.64:41936) 2024-09-12T03:28:43.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.777+0000 7f85fda6a640 1 -- 172.21.15.64:0/1470822733 learned_addr learned my addr 172.21.15.64:0/1470822733 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:43.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.777+0000 7f85fe26b640 1 --2- 172.21.15.64:0/1470822733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c991af890 0x555c991be810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:43.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.777+0000 7f85fda6a640 1 -- 172.21.15.64:0/1470822733 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c991befc0 msgr2=0x555c991bc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:43.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.777+0000 7f85fda6a640 1 --2- 172.21.15.64:0/1470822733 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c991befc0 0x555c991bc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:43.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.777+0000 7f85fda6a640 1 -- 172.21.15.64:0/1470822733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c991af890 msgr2=0x555c991be810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:43.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.777+0000 7f85fda6a640 1 --2- 172.21.15.64:0/1470822733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c991af890 0x555c991be810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:43.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.777+0000 7f85fda6a640 1 -- 172.21.15.64:0/1470822733 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c98e872b0 con 0x555c991b01b0 2024-09-12T03:28:43.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.777+0000 7f85fda6a640 1 --2- 172.21.15.64:0/1470822733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c991b01b0 0x555c991bbb60 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f85ec0a4420 tx=0x7f85ec09b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aa5f82d21fb6fd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:43.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.777+0000 7f85e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:28:43.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.778+0000 7f85e1ffb640 1 -- 172.21.15.64:0/1470822733 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec09bce0 con 0x555c991b01b0 2024-09-12T03:28:43.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.778+0000 7f85e1ffb640 1 -- 172.21.15.64:0/1470822733 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f85ec0a44c0 con 0x555c991b01b0 2024-09-12T03:28:43.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.778+0000 7f86028a8740 1 -- 172.21.15.64:0/1470822733 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85f4004030 con 0x555c991b01b0 2024-09-12T03:28:43.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.778+0000 7f85e1ffb640 1 -- 172.21.15.64:0/1470822733 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f85ec005680 con 0x555c991b01b0 2024-09-12T03:28:43.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.778+0000 7f86028a8740 10 client.15081 fetch_fsmap learned FSMap version 24 2024-09-12T03:28:43.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.778+0000 7f86028a8740 1 -- 172.21.15.64:0/1470822733 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555c991cd4a0 con 0x555c991b01b0 2024-09-12T03:28:43.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.778+0000 7f85e1ffb640 1 -- 172.21.15.64:0/1470822733 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85f4004030 con 0x555c991b01b0 2024-09-12T03:28:43.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.778+0000 7f85e1ffb640 1 -- 172.21.15.64:0/1470822733 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f85ec062020 con 0x555c991b01b0 2024-09-12T03:28:43.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.778+0000 7f86028a8740 10 client.15081 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T03:28:43.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.778+0000 7f86028a8740 10 client.15081 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T03:28:43.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.778+0000 7f86028a8740 20 client.15081 populate_metadata read hostname 'smithi064' 2024-09-12T03:28:43.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.778+0000 7f86028a8740 1 --2- 172.21.15.64:0/1470822733 >> [v2:172.21.15.195:6838/2574695134,v1:172.21.15.195:6839/2574695134] conn(0x555c991cfac0 0x555c991d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:43.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.778+0000 7f86028a8740 4 client.15081 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:28:43.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.778+0000 7f86028a8740 1 -- 172.21.15.64:0/1470822733 --> [v2:172.21.15.195:6838/2574695134,v1:172.21.15.195:6839/2574695134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c99123600 con 0x555c991cfac0 2024-09-12T03:28:43.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.779+0000 7f85fe26b640 1 --2- 172.21.15.64:0/1470822733 >> [v2:172.21.15.195:6838/2574695134,v1:172.21.15.195:6839/2574695134] conn(0x555c991cfac0 0x555c991d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:43.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.779+0000 7f85fe26b640 1 --2- 172.21.15.64:0/1470822733 >> [v2:172.21.15.195:6838/2574695134,v1:172.21.15.195:6839/2574695134] conn(0x555c991cfac0 0x555c991d0f30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f5f8815b440b0564 server_cookie=2fb9dd96e81f06e7 in_seq=0 out_seq=0 2024-09-12T03:28:43.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.779+0000 7f85e1ffb640 10 client.15081 ms_handle_connect on v2:172.21.15.195:6838/2574695134 2024-09-12T03:28:43.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.780+0000 7f85e1ffb640 1 -- 172.21.15.64:0/1470822733 <== mds.1 v2:172.21.15.195:6838/2574695134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555c99123600 con 0x555c991cfac0 2024-09-12T03:28:43.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.780+0000 7f85e1ffb640 10 client.15081 handle_command_reply: tid=0 2024-09-12T03:28:43.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.780+0000 7f86028a8740 1 client.15081 shutdown 2024-09-12T03:28:43.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.780+0000 7f86028a8740 1 -- 172.21.15.64:0/1470822733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c991b01b0 msgr2=0x555c991bbb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:43.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.780+0000 7f86028a8740 1 --2- 172.21.15.64:0/1470822733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c991b01b0 0x555c991bbb60 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f85ec0a4420 tx=0x7f85ec09b700 comp rx=0 tx=0).stop 2024-09-12T03:28:43.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.780+0000 7f86028a8740 1 -- 172.21.15.64:0/1470822733 shutdown_connections 2024-09-12T03:28:43.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.780+0000 7f86028a8740 1 --2- 172.21.15.64:0/1470822733 >> [v2:172.21.15.195:6838/2574695134,v1:172.21.15.195:6839/2574695134] conn(0x555c991cfac0 0x555c991d0f30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:43.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.780+0000 7f86028a8740 1 --2- 172.21.15.64:0/1470822733 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c991befc0 0x555c991bc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:43.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.780+0000 7f86028a8740 1 --2- 172.21.15.64:0/1470822733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c991b01b0 0x555c991bbb60 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:43.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.780+0000 7f86028a8740 1 --2- 172.21.15.64:0/1470822733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c991af890 0x555c991be810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:43.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.780+0000 7f85e1ffb640 0 client.15081 ms_handle_reset on v2:172.21.15.195:6838/2574695134 2024-09-12T03:28:43.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.780+0000 7f86028a8740 1 -- 172.21.15.64:0/1470822733 >> 172.21.15.64:0/1470822733 conn(0x555c991abf60 msgr2=0x555c991b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:43.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.780+0000 7f86028a8740 1 -- 172.21.15.64:0/1470822733 shutdown_connections 2024-09-12T03:28:43.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.780+0000 7f86028a8740 1 -- 172.21.15.64:0/1470822733 wait complete. 2024-09-12T03:28:43.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.780+0000 7f86028a8740 20 client.15081 trim_cache size 0 max 16384 2024-09-12T03:28:43.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.798+0000 7f86028a8740 1 Processor -- start 2024-09-12T03:28:43.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.798+0000 7f86028a8740 1 -- start start 2024-09-12T03:28:43.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.798+0000 7f86028a8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c991af890 0x555c991cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:43.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.798+0000 7f86028a8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c991b01b0 0x555c991d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:43.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.798+0000 7f86028a8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c991befc0 0x555c991a6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:43.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.798+0000 7f86028a8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555c991cf270 con 0x555c991b01b0 2024-09-12T03:28:43.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.798+0000 7f86028a8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555c991cf940 con 0x555c991befc0 2024-09-12T03:28:43.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.798+0000 7f86028a8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555c991d0010 con 0x555c991af890 2024-09-12T03:28:43.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.798+0000 7f85fda6a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c991b01b0 0x555c991d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:43.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.798+0000 7f85fda6a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c991b01b0 0x555c991d0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41952/0 (socket says 172.21.15.64:41952) 2024-09-12T03:28:43.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.798+0000 7f85fda6a640 1 -- 172.21.15.64:0/1607575740 learned_addr learned my addr 172.21.15.64:0/1607575740 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:43.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.799+0000 7f85fe26b640 1 --2- 172.21.15.64:0/1607575740 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c991af890 0x555c991cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:43.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.799+0000 7f85fda6a640 1 -- 172.21.15.64:0/1607575740 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c991af890 msgr2=0x555c991cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:43.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.799+0000 7f85fda6a640 1 --2- 172.21.15.64:0/1607575740 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c991af890 0x555c991cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:43.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.799+0000 7f85fda6a640 1 -- 172.21.15.64:0/1607575740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c991befc0 msgr2=0x555c991a6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:43.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.799+0000 7f85fda6a640 1 --2- 172.21.15.64:0/1607575740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c991befc0 0x555c991a6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:43.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.799+0000 7f85fda6a640 1 -- 172.21.15.64:0/1607575740 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c991affe0 con 0x555c991b01b0 2024-09-12T03:28:43.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.799+0000 7f85fe26b640 1 --2- 172.21.15.64:0/1607575740 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c991af890 0x555c991cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:28:43.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.799+0000 7f85fda6a640 1 --2- 172.21.15.64:0/1607575740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c991b01b0 0x555c991d0f30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f85ec098660 tx=0x7f85ec09b770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=360636ad4bc7515e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:43.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.799+0000 7f85e3fff640 1 -- 172.21.15.64:0/1607575740 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec097db0 con 0x555c991b01b0 2024-09-12T03:28:43.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.799+0000 7f85e3fff640 1 -- 172.21.15.64:0/1607575740 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85ec09c3e0 con 0x555c991b01b0 2024-09-12T03:28:43.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.799+0000 7f85e3fff640 1 -- 172.21.15.64:0/1607575740 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec0a4950 con 0x555c991b01b0 2024-09-12T03:28:43.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.799+0000 7f86028a8740 1 -- 172.21.15.64:0/1607575740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c991b01b0 msgr2=0x555c991d0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:43.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.799+0000 7f86028a8740 1 --2- 172.21.15.64:0/1607575740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c991b01b0 0x555c991d0f30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f85ec098660 tx=0x7f85ec09b770 comp rx=0 tx=0).stop 2024-09-12T03:28:43.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.800+0000 7f86028a8740 1 -- 172.21.15.64:0/1607575740 shutdown_connections 2024-09-12T03:28:43.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.800+0000 7f86028a8740 1 --2- 172.21.15.64:0/1607575740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c991befc0 0x555c991a6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:43.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.800+0000 7f86028a8740 1 --2- 172.21.15.64:0/1607575740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c991b01b0 0x555c991d0f30 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:43.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.800+0000 7f86028a8740 1 --2- 172.21.15.64:0/1607575740 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c991af890 0x555c991cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:43.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.800+0000 7f86028a8740 1 -- 172.21.15.64:0/1607575740 >> 172.21.15.64:0/1607575740 conn(0x555c991abf60 msgr2=0x555c991ac340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:43.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.800+0000 7f86028a8740 1 -- 172.21.15.64:0/1607575740 shutdown_connections 2024-09-12T03:28:43.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.800+0000 7f86028a8740 1 -- 172.21.15.64:0/1607575740 wait complete. 2024-09-12T03:28:43.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.800+0000 7f86028a8740 1 -- start start 2024-09-12T03:28:43.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.800+0000 7f86028a8740 1 Processor -- start 2024-09-12T03:28:43.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.800+0000 7f86028a8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c991af890 0x555c991d9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:43.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.800+0000 7f86028a8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c991b01b0 0x555c991b8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:43.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.800+0000 7f86028a8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c991befc0 0x555c991b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:43.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.800+0000 7f86028a8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555c991d0db0 con 0x555c991af890 2024-09-12T03:28:43.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.800+0000 7f86028a8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555c991d06e0 con 0x555c991b01b0 2024-09-12T03:28:43.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.800+0000 7f86028a8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555c991d1480 con 0x555c991befc0 2024-09-12T03:28:43.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.800+0000 7f86028a8740 10 client.0 _ll_register_callbacks cb 0x7f85ec009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:43.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.800+0000 7f85fe26b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c991af890 0x555c991d9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:43.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.801+0000 7f85fe26b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c991af890 0x555c991d9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41956/0 (socket says 172.21.15.64:41956) 2024-09-12T03:28:43.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.801+0000 7f85fe26b640 1 -- 172.21.15.64:0/3096643580 learned_addr learned my addr 172.21.15.64:0/3096643580 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:43.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.801+0000 7f85fda6a640 1 --2- 172.21.15.64:0/3096643580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c991b01b0 0x555c991b8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:43.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.801+0000 7f85fea6c640 1 --2- 172.21.15.64:0/3096643580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c991befc0 0x555c991b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:43.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.801+0000 7f85fe26b640 1 -- 172.21.15.64:0/3096643580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c991befc0 msgr2=0x555c991b8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:43.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.801+0000 7f85fe26b640 1 --2- 172.21.15.64:0/3096643580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c991befc0 0x555c991b8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:43.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.801+0000 7f85fe26b640 1 -- 172.21.15.64:0/3096643580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c991b01b0 msgr2=0x555c991b8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:43.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.801+0000 7f85fe26b640 1 --2- 172.21.15.64:0/3096643580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c991b01b0 0x555c991b8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:43.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.801+0000 7f85fe26b640 1 -- 172.21.15.64:0/3096643580 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c991b90b0 con 0x555c991af890 2024-09-12T03:28:43.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.801+0000 7f85fda6a640 1 --2- 172.21.15.64:0/3096643580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c991b01b0 0x555c991b8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:28:43.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.801+0000 7f85fea6c640 1 --2- 172.21.15.64:0/3096643580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c991befc0 0x555c991b8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:28:43.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.801+0000 7f85fe26b640 1 --2- 172.21.15.64:0/3096643580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c991af890 0x555c991d9610 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f85f402a460 tx=0x7f85f402aa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd0a51ed8cf59a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:43.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.801+0000 7f85e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:28:43.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.801+0000 7f85e1ffb640 1 -- 172.21.15.64:0/3096643580 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f4050070 con 0x555c991af890 2024-09-12T03:28:43.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.801+0000 7f85e1ffb640 1 -- 172.21.15.64:0/3096643580 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f85f40051c0 con 0x555c991af890 2024-09-12T03:28:43.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.801+0000 7f86028a8740 1 -- 172.21.15.64:0/3096643580 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85ec09c3e0 con 0x555c991af890 2024-09-12T03:28:43.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.801+0000 7f85e1ffb640 1 -- 172.21.15.64:0/3096643580 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f85f405c050 con 0x555c991af890 2024-09-12T03:28:43.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.801+0000 7f86028a8740 10 client.15087 fetch_fsmap learned FSMap version 24 2024-09-12T03:28:43.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.801+0000 7f86028a8740 1 -- 172.21.15.64:0/3096643580 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555c991d7e80 con 0x555c991af890 2024-09-12T03:28:43.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.802+0000 7f85e1ffb640 1 -- 172.21.15.64:0/3096643580 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85ec09c3e0 con 0x555c991af890 2024-09-12T03:28:43.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.802+0000 7f85e1ffb640 1 -- 172.21.15.64:0/3096643580 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f85f40046c0 con 0x555c991af890 2024-09-12T03:28:43.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.802+0000 7f86028a8740 10 client.15087 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T03:28:43.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.802+0000 7f86028a8740 10 client.15087 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T03:28:43.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.802+0000 7f86028a8740 20 client.15087 populate_metadata read hostname 'smithi064' 2024-09-12T03:28:43.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.802+0000 7f86028a8740 1 --2- 172.21.15.64:0/3096643580 >> [v2:172.21.15.195:6838/2574695134,v1:172.21.15.195:6839/2574695134] conn(0x555c9923e5f0 0x555c9923d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:43.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.802+0000 7f86028a8740 4 client.15087 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:28:43.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.802+0000 7f86028a8740 1 -- 172.21.15.64:0/3096643580 --> [v2:172.21.15.195:6838/2574695134,v1:172.21.15.195:6839/2574695134] -- command(tid 0: {"prefix": "status"}) -- 0x555c9911db10 con 0x555c9923e5f0 2024-09-12T03:28:43.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.802+0000 7f85fda6a640 1 --2- 172.21.15.64:0/3096643580 >> [v2:172.21.15.195:6838/2574695134,v1:172.21.15.195:6839/2574695134] conn(0x555c9923e5f0 0x555c9923d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:43.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.803+0000 7f85e1ffb640 10 client.15087 ms_handle_connect on v2:172.21.15.195:6838/2574695134 2024-09-12T03:28:43.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.803+0000 7f85fda6a640 1 --2- 172.21.15.64:0/3096643580 >> [v2:172.21.15.195:6838/2574695134,v1:172.21.15.195:6839/2574695134] conn(0x555c9923e5f0 0x555c9923d300 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e79146bcc734a7a7 server_cookie=6126d834f5df0892 in_seq=0 out_seq=0 2024-09-12T03:28:43.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.803+0000 7f85e1ffb640 1 -- 172.21.15.64:0/3096643580 <== mds.1 v2:172.21.15.195:6838/2574695134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x555c9911db10 con 0x555c9923e5f0 2024-09-12T03:28:43.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.803+0000 7f85e1ffb640 10 client.15087 handle_command_reply: tid=0 2024-09-12T03:28:43.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.803+0000 7f86028a8740 1 client.15087 shutdown 2024-09-12T03:28:43.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.803+0000 7f86028a8740 1 -- 172.21.15.64:0/3096643580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c991af890 msgr2=0x555c991d9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:43.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.803+0000 7f86028a8740 1 --2- 172.21.15.64:0/3096643580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c991af890 0x555c991d9610 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f85f402a460 tx=0x7f85f402aa60 comp rx=0 tx=0).stop 2024-09-12T03:28:43.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.803+0000 7f86028a8740 1 -- 172.21.15.64:0/3096643580 shutdown_connections 2024-09-12T03:28:43.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.803+0000 7f86028a8740 1 --2- 172.21.15.64:0/3096643580 >> [v2:172.21.15.195:6838/2574695134,v1:172.21.15.195:6839/2574695134] conn(0x555c9923e5f0 0x555c9923d300 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:43.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.803+0000 7f86028a8740 1 --2- 172.21.15.64:0/3096643580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c991befc0 0x555c991b8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:43.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.803+0000 7f86028a8740 1 --2- 172.21.15.64:0/3096643580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c991b01b0 0x555c991b8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:43.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.803+0000 7f86028a8740 1 --2- 172.21.15.64:0/3096643580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c991af890 0x555c991d9610 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:43.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.803+0000 7f86028a8740 1 -- 172.21.15.64:0/3096643580 >> 172.21.15.64:0/3096643580 conn(0x555c991abf60 msgr2=0x555c991a6ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:43.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.803+0000 7f85e1ffb640 0 client.15087 ms_handle_reset on v2:172.21.15.195:6838/2574695134 2024-09-12T03:28:43.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.803+0000 7f86028a8740 1 -- 172.21.15.64:0/3096643580 shutdown_connections 2024-09-12T03:28:43.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.803+0000 7f86028a8740 1 -- 172.21.15.64:0/3096643580 wait complete. 2024-09-12T03:28:43.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.803+0000 7f86028a8740 20 client.15087 trim_cache size 0 max 16384 2024-09-12T03:28:43.820 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T03:28:43.820 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "b0812460-70b5-11ef-bcea-c7b262605968", 2024-09-12T03:28:43.820 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 1, 2024-09-12T03:28:43.821 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24481, 2024-09-12T03:28:43.821 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-09-12T03:28:43.821 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-09-12T03:28:43.821 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-09-12T03:28:43.821 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 70.679801694000005, 2024-09-12T03:28:43.821 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 24, 2024-09-12T03:28:43.821 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 90, 2024-09-12T03:28:43.821 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 88, 2024-09-12T03:28:43.821 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 125.92972473099999 2024-09-12T03:28:43.821 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T03:28:43.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.804+0000 7f86004cf640 1 -- 172.21.15.64:0/1402295894 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f85e40a2c10 msgr2=0x7f85e40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:43.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.804+0000 7f86004cf640 1 --2- 172.21.15.64:0/1402295894 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f85e40a2c10 0x7f85e40a50d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f85f0009fd0 tx=0x7f85f00096a0 comp rx=0 tx=0).stop 2024-09-12T03:28:43.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.804+0000 7f86004cf640 1 -- 172.21.15.64:0/1402295894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85f8101400 msgr2=0x7f85f8198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:43.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.804+0000 7f86004cf640 1 --2- 172.21.15.64:0/1402295894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85f8101400 0x7f85f8198920 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f85ec00e9f0 tx=0x7f85ec00edf0 comp rx=0 tx=0).stop 2024-09-12T03:28:43.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.804+0000 7f86004cf640 1 -- 172.21.15.64:0/1402295894 shutdown_connections 2024-09-12T03:28:43.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.804+0000 7f86004cf640 1 --2- 172.21.15.64:0/1402295894 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f85e40a2c10 0x7f85e40a50d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:43.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.804+0000 7f86004cf640 1 --2- 172.21.15.64:0/1402295894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85f8106a60 0x7f85f819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:43.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.804+0000 7f86004cf640 1 --2- 172.21.15.64:0/1402295894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85f8101400 0x7f85f8198920 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:43.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.804+0000 7f86004cf640 1 --2- 172.21.15.64:0/1402295894 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85f8100a60 0x7f85f81983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:43.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.804+0000 7f86004cf640 1 -- 172.21.15.64:0/1402295894 >> 172.21.15.64:0/1402295894 conn(0x7f85f80fc780 msgr2=0x7f85f806ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:43.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.804+0000 7f86004cf640 1 -- 172.21.15.64:0/1402295894 shutdown_connections 2024-09-12T03:28:43.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:43.804+0000 7f86004cf640 1 -- 172.21.15.64:0/1402295894 wait complete. 2024-09-12T03:28:44.602 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-12T03:28:44.859 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:45.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:44 smithi195 ceph-mon[34786]: pgmap v338: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:28:45.235 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:44 smithi064 ceph-mon[30952]: pgmap v338: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:28:45.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.319+0000 7f9bdfe9d640 1 -- 172.21.15.64:0/3532173935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bd8106f10 msgr2=0x7f9bd810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:45.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.319+0000 7f9bdfe9d640 1 --2- 172.21.15.64:0/3532173935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bd8106f10 0x7f9bd810ea90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0009900 tx=0x7f9bd002a410 comp rx=0 tx=0).stop 2024-09-12T03:28:45.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.319+0000 7f9bdfe9d640 1 -- 172.21.15.64:0/3532173935 shutdown_connections 2024-09-12T03:28:45.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.319+0000 7f9bdfe9d640 1 --2- 172.21.15.64:0/3532173935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bd8106f10 0x7f9bd810ea90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:45.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.319+0000 7f9bdfe9d640 1 --2- 172.21.15.64:0/3532173935 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9bd8102790 0x7f9bd81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:45.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.319+0000 7f9bdfe9d640 1 --2- 172.21.15.64:0/3532173935 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9bd8101de0 0x7f9bd81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:45.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.319+0000 7f9bdfe9d640 1 -- 172.21.15.64:0/3532173935 >> 172.21.15.64:0/3532173935 conn(0x7f9bd80fd8c0 msgr2=0x7f9bd80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:45.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.319+0000 7f9bdfe9d640 1 -- 172.21.15.64:0/3532173935 shutdown_connections 2024-09-12T03:28:45.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.319+0000 7f9bdfe9d640 1 -- 172.21.15.64:0/3532173935 wait complete. 2024-09-12T03:28:45.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.320+0000 7f9bdfe9d640 1 Processor -- start 2024-09-12T03:28:45.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.320+0000 7f9bdfe9d640 1 -- start start 2024-09-12T03:28:45.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.320+0000 7f9bdfe9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9bd8101de0 0x7f9bd8198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:45.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.320+0000 7f9bdfe9d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9bd8102790 0x7f9bd81987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:45.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.320+0000 7f9bdfe9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bd8106f10 0x7f9bd819cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:45.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.320+0000 7f9bdfe9d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd8111e20 con 0x7f9bd8106f10 2024-09-12T03:28:45.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.320+0000 7f9bdfe9d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd8111ca0 con 0x7f9bd8101de0 2024-09-12T03:28:45.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.320+0000 7f9bdfe9d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd8111fa0 con 0x7f9bd8102790 2024-09-12T03:28:45.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.320+0000 7f9bde43a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bd8106f10 0x7f9bd819cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:45.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.320+0000 7f9bde43a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bd8106f10 0x7f9bd819cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41976/0 (socket says 172.21.15.64:41976) 2024-09-12T03:28:45.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.320+0000 7f9bde43a640 1 -- 172.21.15.64:0/391168590 learned_addr learned my addr 172.21.15.64:0/391168590 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:45.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.320+0000 7f9bdd438640 1 --2- 172.21.15.64:0/391168590 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9bd8102790 0x7f9bd81987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:45.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.321+0000 7f9bddc39640 1 --2- 172.21.15.64:0/391168590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9bd8101de0 0x7f9bd8198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:45.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.321+0000 7f9bde43a640 1 -- 172.21.15.64:0/391168590 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9bd8102790 msgr2=0x7f9bd81987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:45.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.321+0000 7f9bde43a640 1 --2- 172.21.15.64:0/391168590 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9bd8102790 0x7f9bd81987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:45.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.321+0000 7f9bde43a640 1 -- 172.21.15.64:0/391168590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9bd8101de0 msgr2=0x7f9bd8198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:45.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.321+0000 7f9bde43a640 1 --2- 172.21.15.64:0/391168590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9bd8101de0 0x7f9bd8198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:45.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.321+0000 7f9bde43a640 1 -- 172.21.15.64:0/391168590 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd819d210 con 0x7f9bd8106f10 2024-09-12T03:28:45.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.321+0000 7f9bde43a640 1 --2- 172.21.15.64:0/391168590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bd8106f10 0x7f9bd819cb30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0009230 tx=0x7f9bd0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:45.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.321+0000 7f9bd6ffd640 1 -- 172.21.15.64:0/391168590 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd003f030 con 0x7f9bd8106f10 2024-09-12T03:28:45.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.321+0000 7f9bd6ffd640 1 -- 172.21.15.64:0/391168590 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9bd0003aa0 con 0x7f9bd8106f10 2024-09-12T03:28:45.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.321+0000 7f9bd6ffd640 1 -- 172.21.15.64:0/391168590 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd0035070 con 0x7f9bd8106f10 2024-09-12T03:28:45.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.321+0000 7f9bdfe9d640 1 -- 172.21.15.64:0/391168590 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bd819d4a0 con 0x7f9bd8106f10 2024-09-12T03:28:45.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.321+0000 7f9bdfe9d640 1 -- 172.21.15.64:0/391168590 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bd819d930 con 0x7f9bd8106f10 2024-09-12T03:28:45.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.323+0000 7f9bd6ffd640 1 -- 172.21.15.64:0/391168590 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9bd0042030 con 0x7f9bd8106f10 2024-09-12T03:28:45.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.324+0000 7f9be2276740 1 Processor -- start 2024-09-12T03:28:45.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.324+0000 7f9be2276740 1 -- start start 2024-09-12T03:28:45.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.324+0000 7f9be2276740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a555077890 0x55a555077c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:45.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.324+0000 7f9be2276740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5550781b0 0x55a555086a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:45.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.324+0000 7f9be2276740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a555086fc0 0x55a5550893b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:45.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.324+0000 7f9be2276740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a554fde090 con 0x55a555077890 2024-09-12T03:28:45.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.324+0000 7f9be2276740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a554f85a70 con 0x55a5550781b0 2024-09-12T03:28:45.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.324+0000 7f9be2276740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a554fda050 con 0x55a555086fc0 2024-09-12T03:28:45.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.324+0000 7f9bd6ffd640 1 --2- 172.21.15.64:0/391168590 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f9bc00a2c90 0x7f9bc00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:45.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.324+0000 7f9bdd438640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a555077890 0x55a555077c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:45.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.324+0000 7f9bdd438640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a555077890 0x55a555077c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41986/0 (socket says 172.21.15.64:41986) 2024-09-12T03:28:45.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.324+0000 7f9bdd438640 1 -- 172.21.15.64:0/349739211 learned_addr learned my addr 172.21.15.64:0/349739211 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:45.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.325+0000 7f9bde43a640 1 --2- 172.21.15.64:0/391168590 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f9bc00a2c90 0x7f9bc00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:45.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.325+0000 7f9bddc39640 1 --2- 172.21.15.64:0/349739211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5550781b0 0x55a555086a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:45.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.325+0000 7f9bdd438640 1 --2- 172.21.15.64:0/349739211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a555086fc0 0x55a5550893b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:45.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.325+0000 7f9bd6ffd640 1 -- 172.21.15.64:0/391168590 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f9bd00efed0 con 0x7f9bd8106f10 2024-09-12T03:28:45.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.325+0000 7f9bdd438640 1 -- 172.21.15.64:0/349739211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a555086fc0 msgr2=0x55a5550893b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:45.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.325+0000 7f9bdd438640 1 --2- 172.21.15.64:0/349739211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a555086fc0 0x55a5550893b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:45.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.325+0000 7f9bdd438640 1 -- 172.21.15.64:0/349739211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5550781b0 msgr2=0x55a555086a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:45.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.325+0000 7f9bdd438640 1 --2- 172.21.15.64:0/349739211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5550781b0 0x55a555086a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:45.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.325+0000 7f9bdd438640 1 -- 172.21.15.64:0/349739211 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a554f92130 con 0x55a555077890 2024-09-12T03:28:45.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.325+0000 7f9bdd438640 1 --2- 172.21.15.64:0/349739211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a555077890 0x55a555077c70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc014850 tx=0x7f9bcc009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ab922a37645bc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:45.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.325+0000 7f9bde43a640 1 --2- 172.21.15.64:0/391168590 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f9bc00a2c90 0x7f9bc00a5150 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9bd00b86d0 tx=0x7f9bd0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:45.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.325+0000 7f9bbffff640 1 -- 172.21.15.64:0/349739211 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc01a030 con 0x55a555077890 2024-09-12T03:28:45.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.325+0000 7f9bbffff640 1 -- 172.21.15.64:0/349739211 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9bcc0120d0 con 0x55a555077890 2024-09-12T03:28:45.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.325+0000 7f9bbffff640 1 -- 172.21.15.64:0/349739211 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc012d50 con 0x55a555077890 2024-09-12T03:28:45.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.326+0000 7f9be2276740 1 -- 172.21.15.64:0/349739211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a555077890 msgr2=0x55a555077c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:45.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.326+0000 7f9be2276740 1 --2- 172.21.15.64:0/349739211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a555077890 0x55a555077c70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc014850 tx=0x7f9bcc009b30 comp rx=0 tx=0).stop 2024-09-12T03:28:45.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.326+0000 7f9be2276740 1 -- 172.21.15.64:0/349739211 shutdown_connections 2024-09-12T03:28:45.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.326+0000 7f9be2276740 1 --2- 172.21.15.64:0/349739211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a555086fc0 0x55a5550893b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:45.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.326+0000 7f9be2276740 1 --2- 172.21.15.64:0/349739211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a5550781b0 0x55a555086a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:45.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.326+0000 7f9be2276740 1 --2- 172.21.15.64:0/349739211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a555077890 0x55a555077c70 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:45.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.326+0000 7f9be2276740 1 -- 172.21.15.64:0/349739211 >> 172.21.15.64:0/349739211 conn(0x55a555073f60 msgr2=0x55a555074340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:45.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.326+0000 7f9be2276740 1 -- 172.21.15.64:0/349739211 shutdown_connections 2024-09-12T03:28:45.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.326+0000 7f9be2276740 1 -- 172.21.15.64:0/349739211 wait complete. 2024-09-12T03:28:45.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.327+0000 7f9be2276740 1 -- start start 2024-09-12T03:28:45.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.327+0000 7f9be2276740 1 Processor -- start 2024-09-12T03:28:45.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.327+0000 7f9be2276740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a555077890 0x55a555086870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:45.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.327+0000 7f9be2276740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a5550781b0 0x55a555083bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:45.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.327+0000 7f9be2276740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a555086fc0 0x55a555084100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:45.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.327+0000 7f9be2276740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a555089b30 con 0x55a555077890 2024-09-12T03:28:45.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.327+0000 7f9be2276740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a554f98f20 con 0x55a555086fc0 2024-09-12T03:28:45.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.327+0000 7f9be2276740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a555089cb0 con 0x55a5550781b0 2024-09-12T03:28:45.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.327+0000 7f9be2276740 10 client.0 _ll_register_callbacks cb 0x55a554f62030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:45.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.327+0000 7f9bdd438640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a555077890 0x55a555086870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:45.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.327+0000 7f9bdd438640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a555077890 0x55a555086870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41992/0 (socket says 172.21.15.64:41992) 2024-09-12T03:28:45.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.327+0000 7f9bdd438640 1 -- 172.21.15.64:0/1360253085 learned_addr learned my addr 172.21.15.64:0/1360253085 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:45.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.328+0000 7f9bdd438640 1 -- 172.21.15.64:0/1360253085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a5550781b0 msgr2=0x55a555083bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:45.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.328+0000 7f9bddc39640 1 --2- 172.21.15.64:0/1360253085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a5550781b0 0x55a555083bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:45.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.328+0000 7f9bdd438640 1 --2- 172.21.15.64:0/1360253085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a5550781b0 0x55a555083bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:45.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.328+0000 7f9bdd438640 1 -- 172.21.15.64:0/1360253085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a555086fc0 msgr2=0x55a555084100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:45.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.328+0000 7f9bdd438640 1 --2- 172.21.15.64:0/1360253085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a555086fc0 0x55a555084100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:45.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.328+0000 7f9bdd438640 1 -- 172.21.15.64:0/1360253085 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a554d4f2b0 con 0x55a555077890 2024-09-12T03:28:45.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.328+0000 7f9bddc39640 1 --2- 172.21.15.64:0/1360253085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a5550781b0 0x55a555083bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:28:45.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.328+0000 7f9bdd438640 1 --2- 172.21.15.64:0/1360253085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a555077890 0x55a555086870 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc012ea0 tx=0x7f9bcc012ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f23b7782737cf0e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:45.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.328+0000 7f9bbd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:28:45.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.328+0000 7f9bbd7fa640 1 -- 172.21.15.64:0/1360253085 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc014240 con 0x55a555077890 2024-09-12T03:28:45.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.328+0000 7f9be2276740 1 -- 172.21.15.64:0/1360253085 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bcc0120d0 con 0x55a555077890 2024-09-12T03:28:45.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.328+0000 7f9bbd7fa640 1 -- 172.21.15.64:0/1360253085 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f9bcc0143e0 con 0x55a555077890 2024-09-12T03:28:45.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.328+0000 7f9bbd7fa640 1 -- 172.21.15.64:0/1360253085 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9bcc031050 con 0x55a555077890 2024-09-12T03:28:45.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.328+0000 7f9be2276740 10 client.15099 fetch_fsmap learned FSMap version 24 2024-09-12T03:28:45.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.328+0000 7f9be2276740 1 -- 172.21.15.64:0/1360253085 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a555095490 con 0x55a555077890 2024-09-12T03:28:45.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.328+0000 7f9bbd7fa640 1 -- 172.21.15.64:0/1360253085 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9bcc0120d0 con 0x55a555077890 2024-09-12T03:28:45.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.329+0000 7f9bbd7fa640 1 -- 172.21.15.64:0/1360253085 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f9bcc00f040 con 0x55a555077890 2024-09-12T03:28:45.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.329+0000 7f9be2276740 10 client.15099 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T03:28:45.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.329+0000 7f9be2276740 10 client.15099 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T03:28:45.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.329+0000 7f9be2276740 20 client.15099 populate_metadata read hostname 'smithi064' 2024-09-12T03:28:45.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.329+0000 7f9be2276740 1 --2- 172.21.15.64:0/1360253085 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55a5550976d0 0x55a555098b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:45.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.329+0000 7f9be2276740 4 client.15099 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:28:45.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.329+0000 7f9be2276740 1 -- 172.21.15.64:0/1360253085 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a554feb600 con 0x55a5550976d0 2024-09-12T03:28:45.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.329+0000 7f9bddc39640 1 --2- 172.21.15.64:0/1360253085 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55a5550976d0 0x55a555098b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:45.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.329+0000 7f9bddc39640 1 --2- 172.21.15.64:0/1360253085 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55a5550976d0 0x55a555098b40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=417c9c59b2084824 server_cookie=2651010b06f398c8 in_seq=0 out_seq=0 2024-09-12T03:28:45.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.329+0000 7f9bbd7fa640 10 client.15099 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:28:45.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.330+0000 7f9bbd7fa640 1 -- 172.21.15.64:0/1360253085 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a554feb600 con 0x55a5550976d0 2024-09-12T03:28:45.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.330+0000 7f9bbd7fa640 10 client.15099 handle_command_reply: tid=0 2024-09-12T03:28:45.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.330+0000 7f9be2276740 1 client.15099 shutdown 2024-09-12T03:28:45.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.330+0000 7f9be2276740 1 -- 172.21.15.64:0/1360253085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a555077890 msgr2=0x55a555086870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:45.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.330+0000 7f9be2276740 1 --2- 172.21.15.64:0/1360253085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a555077890 0x55a555086870 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc012ea0 tx=0x7f9bcc012ed0 comp rx=0 tx=0).stop 2024-09-12T03:28:45.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.330+0000 7f9be2276740 1 -- 172.21.15.64:0/1360253085 shutdown_connections 2024-09-12T03:28:45.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.330+0000 7f9be2276740 1 --2- 172.21.15.64:0/1360253085 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55a5550976d0 0x55a555098b40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:45.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.330+0000 7f9be2276740 1 --2- 172.21.15.64:0/1360253085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a555086fc0 0x55a555084100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:45.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.330+0000 7f9be2276740 1 --2- 172.21.15.64:0/1360253085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a5550781b0 0x55a555083bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:45.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.330+0000 7f9bbd7fa640 0 client.15099 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:28:45.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.330+0000 7f9be2276740 1 --2- 172.21.15.64:0/1360253085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a555077890 0x55a555086870 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:45.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.330+0000 7f9be2276740 1 -- 172.21.15.64:0/1360253085 >> 172.21.15.64:0/1360253085 conn(0x55a555073f60 msgr2=0x55a555079cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:45.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.330+0000 7f9be2276740 1 -- 172.21.15.64:0/1360253085 shutdown_connections 2024-09-12T03:28:45.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.330+0000 7f9be2276740 1 -- 172.21.15.64:0/1360253085 wait complete. 2024-09-12T03:28:45.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.331+0000 7f9be2276740 20 client.15099 trim_cache size 0 max 16384 2024-09-12T03:28:45.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.348+0000 7f9be2276740 1 Processor -- start 2024-09-12T03:28:45.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.348+0000 7f9be2276740 1 -- start start 2024-09-12T03:28:45.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.348+0000 7f9be2276740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a555077890 0x55a555098b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:45.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.349+0000 7f9be2276740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a5550781b0 0x55a5550a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:45.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.349+0000 7f9be2276740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a555086fc0 0x55a5550a1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:45.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.349+0000 7f9be2276740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a555096e80 con 0x55a555077890 2024-09-12T03:28:45.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.349+0000 7f9be2276740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a555097550 con 0x55a555086fc0 2024-09-12T03:28:45.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.349+0000 7f9be2276740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a555097c20 con 0x55a5550781b0 2024-09-12T03:28:45.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.349+0000 7f9bdd438640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a555077890 0x55a555098b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:45.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.349+0000 7f9bdd438640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a555077890 0x55a555098b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41996/0 (socket says 172.21.15.64:41996) 2024-09-12T03:28:45.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.349+0000 7f9bdd438640 1 -- 172.21.15.64:0/4637924 learned_addr learned my addr 172.21.15.64:0/4637924 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:45.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.349+0000 7f9bddc39640 1 --2- 172.21.15.64:0/4637924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a5550781b0 0x55a5550a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:45.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.349+0000 7f9bdd438640 1 -- 172.21.15.64:0/4637924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a5550781b0 msgr2=0x55a5550a0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:45.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.349+0000 7f9bdd438640 1 --2- 172.21.15.64:0/4637924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a5550781b0 0x55a5550a0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:45.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.349+0000 7f9bdd438640 1 -- 172.21.15.64:0/4637924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a555086fc0 msgr2=0x55a5550a1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:45.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.349+0000 7f9bdd438640 1 --2- 172.21.15.64:0/4637924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a555086fc0 0x55a5550a1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:45.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.349+0000 7f9bdd438640 1 -- 172.21.15.64:0/4637924 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a555077fe0 con 0x55a555077890 2024-09-12T03:28:45.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.349+0000 7f9bdd438640 1 --2- 172.21.15.64:0/4637924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a555077890 0x55a555098b40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc014850 tx=0x7f9bcc009a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec62fdecfc91721e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:45.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.349+0000 7f9bbf7fe640 1 -- 172.21.15.64:0/4637924 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc01a040 con 0x55a555077890 2024-09-12T03:28:45.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.349+0000 7f9bbf7fe640 1 -- 172.21.15.64:0/4637924 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9bcc024030 con 0x55a555077890 2024-09-12T03:28:45.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.349+0000 7f9bbf7fe640 1 -- 172.21.15.64:0/4637924 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc0296a0 con 0x55a555077890 2024-09-12T03:28:45.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.350+0000 7f9be2276740 1 -- 172.21.15.64:0/4637924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a555077890 msgr2=0x55a555098b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:45.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.350+0000 7f9be2276740 1 --2- 172.21.15.64:0/4637924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a555077890 0x55a555098b40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc014850 tx=0x7f9bcc009a80 comp rx=0 tx=0).stop 2024-09-12T03:28:45.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.350+0000 7f9be2276740 1 -- 172.21.15.64:0/4637924 shutdown_connections 2024-09-12T03:28:45.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.350+0000 7f9be2276740 1 --2- 172.21.15.64:0/4637924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a555086fc0 0x55a5550a1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:45.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.350+0000 7f9be2276740 1 --2- 172.21.15.64:0/4637924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a5550781b0 0x55a5550a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:45.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.350+0000 7f9be2276740 1 --2- 172.21.15.64:0/4637924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a555077890 0x55a555098b40 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:45.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.350+0000 7f9be2276740 1 -- 172.21.15.64:0/4637924 >> 172.21.15.64:0/4637924 conn(0x55a555073f60 msgr2=0x55a55506ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:45.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.350+0000 7f9be2276740 1 -- 172.21.15.64:0/4637924 shutdown_connections 2024-09-12T03:28:45.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.350+0000 7f9be2276740 1 -- 172.21.15.64:0/4637924 wait complete. 2024-09-12T03:28:45.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.350+0000 7f9be2276740 1 -- start start 2024-09-12T03:28:45.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.351+0000 7f9be2276740 1 Processor -- start 2024-09-12T03:28:45.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.351+0000 7f9be2276740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a555077890 0x55a555099db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:45.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.351+0000 7f9be2276740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5550781b0 0x55a55509a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:45.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.351+0000 7f9be2276740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a555086fc0 0x55a55509a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:45.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.351+0000 7f9be2276740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a5550989c0 con 0x55a5550781b0 2024-09-12T03:28:45.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.351+0000 7f9be2276740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a5550982f0 con 0x55a555077890 2024-09-12T03:28:45.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.351+0000 7f9be2276740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a555099090 con 0x55a555086fc0 2024-09-12T03:28:45.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.351+0000 7f9be2276740 10 client.0 _ll_register_callbacks cb 0x7f9bcc014730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:45.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.351+0000 7f9bddc39640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5550781b0 0x55a55509a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:45.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.351+0000 7f9bddc39640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5550781b0 0x55a55509a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42008/0 (socket says 172.21.15.64:42008) 2024-09-12T03:28:45.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.351+0000 7f9bddc39640 1 -- 172.21.15.64:0/1078644229 learned_addr learned my addr 172.21.15.64:0/1078644229 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:45.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.351+0000 7f9bdd438640 1 --2- 172.21.15.64:0/1078644229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a555077890 0x55a555099db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:45.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.351+0000 7f9bdd438640 1 --2- 172.21.15.64:0/1078644229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a555086fc0 0x55a55509a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:45.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.351+0000 7f9bddc39640 1 -- 172.21.15.64:0/1078644229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a555086fc0 msgr2=0x55a55509a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:45.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.351+0000 7f9bddc39640 1 --2- 172.21.15.64:0/1078644229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a555086fc0 0x55a55509a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:45.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.351+0000 7f9bddc39640 1 -- 172.21.15.64:0/1078644229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a555077890 msgr2=0x55a555099db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:45.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.351+0000 7f9bddc39640 1 --2- 172.21.15.64:0/1078644229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a555077890 0x55a555099db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:45.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.352+0000 7f9bddc39640 1 -- 172.21.15.64:0/1078644229 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5550fffd0 con 0x55a5550781b0 2024-09-12T03:28:45.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.352+0000 7f9bdd438640 1 --2- 172.21.15.64:0/1078644229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a555086fc0 0x55a55509a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:28:45.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.352+0000 7f9bddc39640 1 --2- 172.21.15.64:0/1078644229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5550781b0 0x55a55509a2f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8004700 tx=0x7f9bc800f8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6270c160278db2f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:45.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.352+0000 7f9bbd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:28:45.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.352+0000 7f9bbd7fa640 1 -- 172.21.15.64:0/1078644229 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8005220 con 0x55a5550781b0 2024-09-12T03:28:45.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.352+0000 7f9bbd7fa640 1 -- 172.21.15.64:0/1078644229 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f9bc80053c0 con 0x55a5550781b0 2024-09-12T03:28:45.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.352+0000 7f9be2276740 1 -- 172.21.15.64:0/1078644229 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bcc024030 con 0x55a5550781b0 2024-09-12T03:28:45.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.352+0000 7f9bbd7fa640 1 -- 172.21.15.64:0/1078644229 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9bc802c050 con 0x55a5550781b0 2024-09-12T03:28:45.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.352+0000 7f9be2276740 10 client.15105 fetch_fsmap learned FSMap version 24 2024-09-12T03:28:45.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.352+0000 7f9be2276740 1 -- 172.21.15.64:0/1078644229 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a5551012d0 con 0x55a5550781b0 2024-09-12T03:28:45.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.352+0000 7f9bbd7fa640 1 -- 172.21.15.64:0/1078644229 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9bcc024030 con 0x55a5550781b0 2024-09-12T03:28:45.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.353+0000 7f9bbd7fa640 1 -- 172.21.15.64:0/1078644229 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f9bc8004bb0 con 0x55a5550781b0 2024-09-12T03:28:45.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.353+0000 7f9be2276740 10 client.15105 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T03:28:45.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.353+0000 7f9be2276740 10 client.15105 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T03:28:45.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.353+0000 7f9be2276740 20 client.15105 populate_metadata read hostname 'smithi064' 2024-09-12T03:28:45.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.353+0000 7f9be2276740 1 --2- 172.21.15.64:0/1078644229 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55a555105df0 0x55a555104b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:45.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.353+0000 7f9be2276740 4 client.15105 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:28:45.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.353+0000 7f9be2276740 1 -- 172.21.15.64:0/1078644229 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "status"}) -- 0x55a554fe5b10 con 0x55a555105df0 2024-09-12T03:28:45.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.353+0000 7f9bdd438640 1 --2- 172.21.15.64:0/1078644229 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55a555105df0 0x55a555104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:45.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.353+0000 7f9bdd438640 1 --2- 172.21.15.64:0/1078644229 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55a555105df0 0x55a555104b00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=670836abfc5b83a6 server_cookie=43e5af802934873e in_seq=0 out_seq=0 2024-09-12T03:28:45.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.353+0000 7f9bbd7fa640 10 client.15105 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:28:45.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.354+0000 7f9bbd7fa640 1 -- 172.21.15.64:0/1078644229 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a554fe5b10 con 0x55a555105df0 2024-09-12T03:28:45.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.354+0000 7f9bbd7fa640 10 client.15105 handle_command_reply: tid=0 2024-09-12T03:28:45.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.354+0000 7f9be2276740 1 client.15105 shutdown 2024-09-12T03:28:45.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.354+0000 7f9be2276740 1 -- 172.21.15.64:0/1078644229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5550781b0 msgr2=0x55a55509a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:45.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.354+0000 7f9be2276740 1 --2- 172.21.15.64:0/1078644229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5550781b0 0x55a55509a2f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8004700 tx=0x7f9bc800f8b0 comp rx=0 tx=0).stop 2024-09-12T03:28:45.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.354+0000 7f9be2276740 1 -- 172.21.15.64:0/1078644229 shutdown_connections 2024-09-12T03:28:45.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.354+0000 7f9be2276740 1 --2- 172.21.15.64:0/1078644229 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55a555105df0 0x55a555104b00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:45.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.354+0000 7f9be2276740 1 --2- 172.21.15.64:0/1078644229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a555086fc0 0x55a55509a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:45.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.354+0000 7f9bbd7fa640 0 client.15105 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:28:45.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.354+0000 7f9be2276740 1 --2- 172.21.15.64:0/1078644229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5550781b0 0x55a55509a2f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:45.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.354+0000 7f9be2276740 1 --2- 172.21.15.64:0/1078644229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a555077890 0x55a555099db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:45.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.354+0000 7f9be2276740 1 -- 172.21.15.64:0/1078644229 >> 172.21.15.64:0/1078644229 conn(0x55a555073f60 msgr2=0x55a55507ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:45.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.354+0000 7f9be2276740 1 -- 172.21.15.64:0/1078644229 shutdown_connections 2024-09-12T03:28:45.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.354+0000 7f9be2276740 1 -- 172.21.15.64:0/1078644229 wait complete. 2024-09-12T03:28:45.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.354+0000 7f9be2276740 20 client.15105 trim_cache size 0 max 16384 2024-09-12T03:28:45.361 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T03:28:45.361 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "b0812460-70b5-11ef-bcea-c7b262605968", 2024-09-12T03:28:45.361 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2024-09-12T03:28:45.361 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24500, 2024-09-12T03:28:45.361 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-09-12T03:28:45.361 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-09-12T03:28:45.361 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-09-12T03:28:45.361 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 105.352884624, 2024-09-12T03:28:45.361 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 24, 2024-09-12T03:28:45.361 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 90, 2024-09-12T03:28:45.362 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 88, 2024-09-12T03:28:45.362 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 117.27133322500001 2024-09-12T03:28:45.362 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T03:28:45.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.355+0000 7f9bdfe9d640 1 -- 172.21.15.64:0/391168590 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f9bc00a2c90 msgr2=0x7f9bc00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:45.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.355+0000 7f9bdfe9d640 1 --2- 172.21.15.64:0/391168590 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f9bc00a2c90 0x7f9bc00a5150 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9bd00b86d0 tx=0x7f9bd0009260 comp rx=0 tx=0).stop 2024-09-12T03:28:45.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.355+0000 7f9bdfe9d640 1 -- 172.21.15.64:0/391168590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bd8106f10 msgr2=0x7f9bd819cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:45.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.355+0000 7f9bdfe9d640 1 --2- 172.21.15.64:0/391168590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bd8106f10 0x7f9bd819cb30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0009230 tx=0x7f9bd0004270 comp rx=0 tx=0).stop 2024-09-12T03:28:45.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.355+0000 7f9bdfe9d640 1 -- 172.21.15.64:0/391168590 shutdown_connections 2024-09-12T03:28:45.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.355+0000 7f9bdfe9d640 1 --2- 172.21.15.64:0/391168590 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f9bc00a2c90 0x7f9bc00a5150 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:45.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.355+0000 7f9bdfe9d640 1 --2- 172.21.15.64:0/391168590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bd8106f10 0x7f9bd819cb30 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:45.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.355+0000 7f9bdfe9d640 1 --2- 172.21.15.64:0/391168590 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9bd8102790 0x7f9bd81987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:45.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.355+0000 7f9bdfe9d640 1 --2- 172.21.15.64:0/391168590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9bd8101de0 0x7f9bd8198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:45.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.355+0000 7f9bdfe9d640 1 -- 172.21.15.64:0/391168590 >> 172.21.15.64:0/391168590 conn(0x7f9bd80fd8c0 msgr2=0x7f9bd80fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:45.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.355+0000 7f9bdfe9d640 1 -- 172.21.15.64:0/391168590 shutdown_connections 2024-09-12T03:28:45.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:45.355+0000 7f9bdfe9d640 1 -- 172.21.15.64:0/391168590 wait complete. 2024-09-12T03:28:45.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:44 smithi115 ceph-mon[34801]: pgmap v338: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:28:46.146 INFO:tasks.cephadm:Setup complete, yielding 2024-09-12T03:28:46.146 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-12T03:28:46.157 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi064.front.sepia.ceph.com 2024-09-12T03:28:46.157 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-12T03:28:46.414 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:46.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:46.962+0000 7f66325b9640 1 -- 172.21.15.64:0/2044515791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f662c101d00 msgr2=0x7f662c102180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:46.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:46.962+0000 7f66325b9640 1 --2- 172.21.15.64:0/2044515791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f662c101d00 0x7f662c102180 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6620009900 tx=0x7f662002a3f0 comp rx=0 tx=0).stop 2024-09-12T03:28:46.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:46.962+0000 7f66325b9640 1 -- 172.21.15.64:0/2044515791 shutdown_connections 2024-09-12T03:28:46.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:46.962+0000 7f66325b9640 1 --2- 172.21.15.64:0/2044515791 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f662c1026c0 0x7f662c108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:46.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:46.962+0000 7f66325b9640 1 --2- 172.21.15.64:0/2044515791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f662c101d00 0x7f662c102180 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:46.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:46.962+0000 7f66325b9640 1 --2- 172.21.15.64:0/2044515791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f662c100b00 0x7f662c100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:46.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:46.962+0000 7f66325b9640 1 -- 172.21.15.64:0/2044515791 >> 172.21.15.64:0/2044515791 conn(0x7f662c0fc290 msgr2=0x7f662c0fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:46.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:46.962+0000 7f66325b9640 1 -- 172.21.15.64:0/2044515791 shutdown_connections 2024-09-12T03:28:46.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:46.962+0000 7f66325b9640 1 -- 172.21.15.64:0/2044515791 wait complete. 2024-09-12T03:28:46.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:46.963+0000 7f66325b9640 1 Processor -- start 2024-09-12T03:28:46.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:46.963+0000 7f66325b9640 1 -- start start 2024-09-12T03:28:46.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:46.963+0000 7f66325b9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f662c100b00 0x7f662c196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:46.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:46.963+0000 7f66325b9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f662c101d00 0x7f662c197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:46.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:46.963+0000 7f66325b9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f662c1026c0 0x7f662c19e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:46.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:46.963+0000 7f66325b9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f662c10c610 con 0x7f662c101d00 2024-09-12T03:28:46.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:46.963+0000 7f66325b9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f662c10c490 con 0x7f662c1026c0 2024-09-12T03:28:46.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:46.963+0000 7f66325b9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f662c10c790 con 0x7f662c100b00 2024-09-12T03:28:46.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:46.963+0000 7f662b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f662c101d00 0x7f662c197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:46.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:46.963+0000 7f662b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f662c101d00 0x7f662c197220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42020/0 (socket says 172.21.15.64:42020) 2024-09-12T03:28:46.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:46.963+0000 7f662b7fe640 1 -- 172.21.15.64:0/1459708358 learned_addr learned my addr 172.21.15.64:0/1459708358 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:46.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:46.964+0000 7f662bfff640 1 --2- 172.21.15.64:0/1459708358 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f662c100b00 0x7f662c196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:46.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:46.964+0000 7f6630b56640 1 --2- 172.21.15.64:0/1459708358 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f662c1026c0 0x7f662c19e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:46.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:46.964+0000 7f662b7fe640 1 -- 172.21.15.64:0/1459708358 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f662c100b00 msgr2=0x7f662c196ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:46.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:46.964+0000 7f662b7fe640 1 --2- 172.21.15.64:0/1459708358 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f662c100b00 0x7f662c196ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:46.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:46.964+0000 7f662b7fe640 1 -- 172.21.15.64:0/1459708358 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f662c1026c0 msgr2=0x7f662c19e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:46.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:46.964+0000 7f662b7fe640 1 --2- 172.21.15.64:0/1459708358 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f662c1026c0 0x7f662c19e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:46.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:46.964+0000 7f662b7fe640 1 -- 172.21.15.64:0/1459708358 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f662c19e900 con 0x7f662c101d00 2024-09-12T03:28:46.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:46.964+0000 7f662bfff640 1 --2- 172.21.15.64:0/1459708358 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f662c100b00 0x7f662c196ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:28:46.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:46.964+0000 7f662b7fe640 1 --2- 172.21.15.64:0/1459708358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f662c101d00 0x7f662c197220 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6620004770 tx=0x7f66200042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:46.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:46.964+0000 7f66297fa640 1 -- 172.21.15.64:0/1459708358 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662003f030 con 0x7f662c101d00 2024-09-12T03:28:46.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:46.964+0000 7f66325b9640 1 -- 172.21.15.64:0/1459708358 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f662c19eb90 con 0x7f662c101d00 2024-09-12T03:28:46.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:46.964+0000 7f66297fa640 1 -- 172.21.15.64:0/1459708358 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6620003cb0 con 0x7f662c101d00 2024-09-12T03:28:46.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:46.964+0000 7f66297fa640 1 -- 172.21.15.64:0/1459708358 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6620035070 con 0x7f662c101d00 2024-09-12T03:28:46.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:46.964+0000 7f66325b9640 1 -- 172.21.15.64:0/1459708358 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f662c19f070 con 0x7f662c101d00 2024-09-12T03:28:46.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:46.965+0000 7f66325b9640 1 -- 172.21.15.64:0/1459708358 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f662c107f50 con 0x7f662c101d00 2024-09-12T03:28:46.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:46.966+0000 7f66297fa640 1 -- 172.21.15.64:0/1459708358 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6620042030 con 0x7f662c101d00 2024-09-12T03:28:46.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:46.967+0000 7f66297fa640 1 --2- 172.21.15.64:0/1459708358 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f66140a2c10 0x7f66140a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:46.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:46.967+0000 7f66297fa640 1 -- 172.21.15.64:0/1459708358 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f66200efec0 con 0x7f662c101d00 2024-09-12T03:28:46.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:46.967+0000 7f662bfff640 1 --2- 172.21.15.64:0/1459708358 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f66140a2c10 0x7f66140a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:46.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:46.968+0000 7f662bfff640 1 --2- 172.21.15.64:0/1459708358 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f66140a2c10 0x7f66140a50d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f661c0058c0 tx=0x7f661c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:46.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:46.971+0000 7f66297fa640 1 -- 172.21.15.64:0/1459708358 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f66200bc960 con 0x7f662c101d00 2024-09-12T03:28:47.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:47.165+0000 7f66325b9640 1 -- 172.21.15.64:0/1459708358 --> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f662c062630 con 0x7f66140a2c10 2024-09-12T03:28:47.165 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:46 smithi064 ceph-mon[30952]: pgmap v339: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:28:47.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:47.166+0000 7f66297fa640 1 -- 172.21.15.64:0/1459708358 <== mgr.14150 v2:172.21.15.64:6800/759931848 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f662c062630 con 0x7f66140a2c10 2024-09-12T03:28:47.166 INFO:teuthology.orchestra.run.smithi064.stdout:Backend: cephadm 2024-09-12T03:28:47.166 INFO:teuthology.orchestra.run.smithi064.stdout:Available: Yes 2024-09-12T03:28:47.166 INFO:teuthology.orchestra.run.smithi064.stdout:Paused: No 2024-09-12T03:28:47.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:47.169+0000 7f66325b9640 1 -- 172.21.15.64:0/1459708358 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f66140a2c10 msgr2=0x7f66140a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:47.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:47.169+0000 7f66325b9640 1 --2- 172.21.15.64:0/1459708358 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f66140a2c10 0x7f66140a50d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f661c0058c0 tx=0x7f661c005850 comp rx=0 tx=0).stop 2024-09-12T03:28:47.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:47.169+0000 7f66325b9640 1 -- 172.21.15.64:0/1459708358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f662c101d00 msgr2=0x7f662c197220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:47.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:47.169+0000 7f66325b9640 1 --2- 172.21.15.64:0/1459708358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f662c101d00 0x7f662c197220 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6620004770 tx=0x7f66200042f0 comp rx=0 tx=0).stop 2024-09-12T03:28:47.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:47.169+0000 7f66325b9640 1 -- 172.21.15.64:0/1459708358 shutdown_connections 2024-09-12T03:28:47.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:47.169+0000 7f66325b9640 1 --2- 172.21.15.64:0/1459708358 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f66140a2c10 0x7f66140a50d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:47.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:47.169+0000 7f66325b9640 1 --2- 172.21.15.64:0/1459708358 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f662c1026c0 0x7f662c19e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:47.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:47.169+0000 7f66325b9640 1 --2- 172.21.15.64:0/1459708358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f662c101d00 0x7f662c197220 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:47.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:47.169+0000 7f66325b9640 1 --2- 172.21.15.64:0/1459708358 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f662c100b00 0x7f662c196ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:47.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:47.169+0000 7f66325b9640 1 -- 172.21.15.64:0/1459708358 >> 172.21.15.64:0/1459708358 conn(0x7f662c0fc290 msgr2=0x7f662c105120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:47.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:47.169+0000 7f66325b9640 1 -- 172.21.15.64:0/1459708358 shutdown_connections 2024-09-12T03:28:47.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:47.169+0000 7f66325b9640 1 -- 172.21.15.64:0/1459708358 wait complete. 2024-09-12T03:28:47.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:46 smithi195 ceph-mon[34786]: pgmap v339: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:28:47.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:46 smithi115 ceph-mon[34801]: pgmap v339: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:28:47.963 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-12T03:28:48.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:47 smithi195 ceph-mon[34786]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:28:48.228 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:48.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:47 smithi064 ceph-mon[30952]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:28:48.293 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:47 smithi115 ceph-mon[34801]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:28:48.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.728+0000 7f211b229640 1 -- 172.21.15.64:0/1035081562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2114107900 msgr2=0x7f2114109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:48.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.728+0000 7f211b229640 1 --2- 172.21.15.64:0/1035081562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2114107900 0x7f2114109d50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f210c009ca0 tx=0x7f210c02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:28:48.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.728+0000 7f211b229640 1 -- 172.21.15.64:0/1035081562 shutdown_connections 2024-09-12T03:28:48.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.728+0000 7f211b229640 1 --2- 172.21.15.64:0/1035081562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2114107900 0x7f2114109d50 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:48.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.728+0000 7f211b229640 1 --2- 172.21.15.64:0/1035081562 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2114104fd0 0x7f21141073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:48.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.728+0000 7f211b229640 1 --2- 172.21.15.64:0/1035081562 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21141026a0 0x7f2114104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:48.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.728+0000 7f211b229640 1 -- 172.21.15.64:0/1035081562 >> 172.21.15.64:0/1035081562 conn(0x7f21140fc2a0 msgr2=0x7f21140fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:48.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.728+0000 7f211b229640 1 -- 172.21.15.64:0/1035081562 shutdown_connections 2024-09-12T03:28:48.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.728+0000 7f211b229640 1 -- 172.21.15.64:0/1035081562 wait complete. 2024-09-12T03:28:48.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.729+0000 7f211b229640 1 Processor -- start 2024-09-12T03:28:48.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.729+0000 7f211b229640 1 -- start start 2024-09-12T03:28:48.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.729+0000 7f211b229640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21141026a0 0x7f2114197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:48.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.729+0000 7f211b229640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2114104fd0 0x7f21141983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:48.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.729+0000 7f211b229640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2114107900 0x7f211419f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:48.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.729+0000 7f211b229640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f211410d7c0 con 0x7f2114104fd0 2024-09-12T03:28:48.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.729+0000 7f211b229640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f211410d640 con 0x7f2114107900 2024-09-12T03:28:48.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.729+0000 7f211b229640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f211410d940 con 0x7f21141026a0 2024-09-12T03:28:48.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.729+0000 7f2113fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2114104fd0 0x7f21141983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:48.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.730+0000 7f2113fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2114104fd0 0x7f21141983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59986/0 (socket says 172.21.15.64:59986) 2024-09-12T03:28:48.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.730+0000 7f2113fff640 1 -- 172.21.15.64:0/2904901909 learned_addr learned my addr 172.21.15.64:0/2904901909 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:48.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.730+0000 7f21197c6640 1 --2- 172.21.15.64:0/2904901909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2114107900 0x7f211419f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:48.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.730+0000 7f2118fc5640 1 --2- 172.21.15.64:0/2904901909 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21141026a0 0x7f2114197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:48.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.730+0000 7f2113fff640 1 -- 172.21.15.64:0/2904901909 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21141026a0 msgr2=0x7f2114197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:48.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.730+0000 7f2113fff640 1 --2- 172.21.15.64:0/2904901909 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21141026a0 0x7f2114197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:48.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.730+0000 7f2113fff640 1 -- 172.21.15.64:0/2904901909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2114107900 msgr2=0x7f211419f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:48.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.730+0000 7f2113fff640 1 --2- 172.21.15.64:0/2904901909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2114107900 0x7f211419f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:48.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.730+0000 7f2113fff640 1 -- 172.21.15.64:0/2904901909 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f211419fab0 con 0x7f2114104fd0 2024-09-12T03:28:48.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.730+0000 7f2113fff640 1 --2- 172.21.15.64:0/2904901909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2114104fd0 0x7f21141983c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f210000b450 tx=0x7f210000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:48.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.730+0000 7f2111ffb640 1 -- 172.21.15.64:0/2904901909 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2100015020 con 0x7f2114104fd0 2024-09-12T03:28:48.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.730+0000 7f2111ffb640 1 -- 172.21.15.64:0/2904901909 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2100011d40 con 0x7f2114104fd0 2024-09-12T03:28:48.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.730+0000 7f211b229640 1 -- 172.21.15.64:0/2904901909 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f211419fda0 con 0x7f2114104fd0 2024-09-12T03:28:48.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.730+0000 7f2111ffb640 1 -- 172.21.15.64:0/2904901909 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210000f340 con 0x7f2114104fd0 2024-09-12T03:28:48.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.731+0000 7f211b229640 1 -- 172.21.15.64:0/2904901909 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21141a0260 con 0x7f2114104fd0 2024-09-12T03:28:48.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.731+0000 7f211b229640 1 -- 172.21.15.64:0/2904901909 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21140fdd90 con 0x7f2114104fd0 2024-09-12T03:28:48.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.732+0000 7f2111ffb640 1 -- 172.21.15.64:0/2904901909 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2100018020 con 0x7f2114104fd0 2024-09-12T03:28:48.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.733+0000 7f2111ffb640 1 --2- 172.21.15.64:0/2904901909 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f21040a2cc0 0x7f21040a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:48.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.733+0000 7f2111ffb640 1 -- 172.21.15.64:0/2904901909 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f21000cee00 con 0x7f2114104fd0 2024-09-12T03:28:48.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.733+0000 7f2118fc5640 1 --2- 172.21.15.64:0/2904901909 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f21040a2cc0 0x7f21040a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:48.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.734+0000 7f2118fc5640 1 --2- 172.21.15.64:0/2904901909 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f21040a2cc0 0x7f21040a5180 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2108005fd0 tx=0x7f2108005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:48.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.737+0000 7f2111ffb640 1 -- 172.21.15.64:0/2904901909 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f210001f020 con 0x7f2114104fd0 2024-09-12T03:28:48.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.923+0000 7f211b229640 1 -- 172.21.15.64:0/2904901909 --> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f2114062630 con 0x7f21040a2cc0 2024-09-12T03:28:48.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.936+0000 7f2111ffb640 1 -- 172.21.15.64:0/2904901909 <== mgr.14150 v2:172.21.15.64:6800/759931848 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4061 (secure 0 0 0) 0x7f2114062630 con 0x7f21040a2cc0 2024-09-12T03:28:48.936 INFO:teuthology.orchestra.run.smithi064.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-12T03:28:48.936 INFO:teuthology.orchestra.run.smithi064.stdout:mds.a smithi064 running (2m) 53s ago 2m 16.6M - 19.1.1-298-gb53b6d3e 54467abc55f8 74a579f1e1e3 2024-09-12T03:28:48.936 INFO:teuthology.orchestra.run.smithi064.stdout:mds.b smithi115 running (2m) 54s ago 2m 26.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 66e746a88369 2024-09-12T03:28:48.937 INFO:teuthology.orchestra.run.smithi064.stdout:mds.c smithi195 running (2m) 54s ago 2m 16.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 1c5adae5c26d 2024-09-12T03:28:48.937 INFO:teuthology.orchestra.run.smithi064.stdout:mds.d smithi064 running (2m) 53s ago 2m 13.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 379a334559f5 2024-09-12T03:28:48.937 INFO:teuthology.orchestra.run.smithi064.stdout:mds.e smithi115 running (2m) 54s ago 2m 17.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 e75ad4f34575 2024-09-12T03:28:48.937 INFO:teuthology.orchestra.run.smithi064.stdout:mds.f smithi195 running (2m) 54s ago 2m 17.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 ccaea7d954ed 2024-09-12T03:28:48.937 INFO:teuthology.orchestra.run.smithi064.stdout:mds.g smithi064 running (2m) 53s ago 2m 14.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 30355fb6af3a 2024-09-12T03:28:48.937 INFO:teuthology.orchestra.run.smithi064.stdout:mds.h smithi115 running (2m) 54s ago 2m 19.7M - 19.1.1-298-gb53b6d3e 54467abc55f8 d0ac6da922dc 2024-09-12T03:28:48.937 INFO:teuthology.orchestra.run.smithi064.stdout:mds.i smithi195 running (2m) 54s ago 2m 19.8M - 19.1.1-298-gb53b6d3e 54467abc55f8 f47840627fc8 2024-09-12T03:28:48.937 INFO:teuthology.orchestra.run.smithi064.stdout:mds.j smithi064 running (2m) 53s ago 2m 14.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 38ce48cd536e 2024-09-12T03:28:48.937 INFO:teuthology.orchestra.run.smithi064.stdout:mds.k smithi115 running (2m) 54s ago 2m 16.7M - 19.1.1-298-gb53b6d3e 54467abc55f8 cf7a19fc47bb 2024-09-12T03:28:48.937 INFO:teuthology.orchestra.run.smithi064.stdout:mds.l smithi195 running (2m) 54s ago 2m 14.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 bdd5cfba81e9 2024-09-12T03:28:48.937 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.x smithi064 *:9283,8765 running (9m) 53s ago 9m 509M - 19.1.1-298-gb53b6d3e 54467abc55f8 7310212332e2 2024-09-12T03:28:48.937 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.y smithi115 *:8443,8765 running (7m) 54s ago 7m 442M - 19.1.1-298-gb53b6d3e 54467abc55f8 4d9efb933425 2024-09-12T03:28:48.937 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.z smithi195 *:8443,8765 running (7m) 54s ago 7m 441M - 19.1.1-298-gb53b6d3e 54467abc55f8 b3e449520773 2024-09-12T03:28:48.938 INFO:teuthology.orchestra.run.smithi064.stdout:mon.a smithi064 running (9m) 53s ago 9m 49.7M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 9aa76b73f7af 2024-09-12T03:28:48.938 INFO:teuthology.orchestra.run.smithi064.stdout:mon.b smithi115 running (7m) 54s ago 7m 39.3M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 a962e723c8e9 2024-09-12T03:28:48.938 INFO:teuthology.orchestra.run.smithi064.stdout:mon.c smithi195 running (8m) 54s ago 8m 42.6M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 7ddb2cba258e 2024-09-12T03:28:48.938 INFO:teuthology.orchestra.run.smithi064.stdout:osd.0 smithi064 running (7m) 53s ago 7m 56.5M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 f528e6db87f7 2024-09-12T03:28:48.938 INFO:teuthology.orchestra.run.smithi064.stdout:osd.1 smithi115 running (7m) 54s ago 7m 46.7M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 e388777c79dc 2024-09-12T03:28:48.938 INFO:teuthology.orchestra.run.smithi064.stdout:osd.2 smithi195 running (6m) 54s ago 6m 57.7M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 3a77ee82f58c 2024-09-12T03:28:48.938 INFO:teuthology.orchestra.run.smithi064.stdout:osd.3 smithi064 running (6m) 53s ago 6m 55.2M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 93bf05989d26 2024-09-12T03:28:48.938 INFO:teuthology.orchestra.run.smithi064.stdout:osd.4 smithi115 running (5m) 54s ago 5m 53.8M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 ff76fa4e1f1f 2024-09-12T03:28:48.938 INFO:teuthology.orchestra.run.smithi064.stdout:osd.5 smithi195 running (5m) 54s ago 5m 45.7M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 653991c61592 2024-09-12T03:28:48.938 INFO:teuthology.orchestra.run.smithi064.stdout:osd.6 smithi064 running (4m) 53s ago 4m 55.2M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 cc985890eb3e 2024-09-12T03:28:48.938 INFO:teuthology.orchestra.run.smithi064.stdout:osd.7 smithi115 running (4m) 54s ago 4m 43.6M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 827da419ab67 2024-09-12T03:28:48.938 INFO:teuthology.orchestra.run.smithi064.stdout:osd.8 smithi195 running (4m) 54s ago 4m 44.0M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 260c29ce5ea8 2024-09-12T03:28:48.938 INFO:teuthology.orchestra.run.smithi064.stdout:osd.9 smithi064 running (3m) 53s ago 3m 43.3M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 fd367fb13c75 2024-09-12T03:28:48.938 INFO:teuthology.orchestra.run.smithi064.stdout:osd.10 smithi115 running (3m) 54s ago 3m 54.0M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 c411a2636075 2024-09-12T03:28:48.939 INFO:teuthology.orchestra.run.smithi064.stdout:osd.11 smithi195 running (2m) 54s ago 2m 40.4M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 60de4099a240 2024-09-12T03:28:48.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.939+0000 7f211b229640 1 -- 172.21.15.64:0/2904901909 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f21040a2cc0 msgr2=0x7f21040a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:48.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.939+0000 7f211b229640 1 --2- 172.21.15.64:0/2904901909 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f21040a2cc0 0x7f21040a5180 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2108005fd0 tx=0x7f2108005850 comp rx=0 tx=0).stop 2024-09-12T03:28:48.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.939+0000 7f211b229640 1 -- 172.21.15.64:0/2904901909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2114104fd0 msgr2=0x7f21141983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:48.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.939+0000 7f211b229640 1 --2- 172.21.15.64:0/2904901909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2114104fd0 0x7f21141983c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f210000b450 tx=0x7f210000b910 comp rx=0 tx=0).stop 2024-09-12T03:28:48.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.939+0000 7f211b229640 1 -- 172.21.15.64:0/2904901909 shutdown_connections 2024-09-12T03:28:48.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.939+0000 7f211b229640 1 --2- 172.21.15.64:0/2904901909 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f21040a2cc0 0x7f21040a5180 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:48.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.939+0000 7f211b229640 1 --2- 172.21.15.64:0/2904901909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2114107900 0x7f211419f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:48.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.939+0000 7f211b229640 1 --2- 172.21.15.64:0/2904901909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2114104fd0 0x7f21141983c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:48.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.939+0000 7f211b229640 1 --2- 172.21.15.64:0/2904901909 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21141026a0 0x7f2114197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:48.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.939+0000 7f211b229640 1 -- 172.21.15.64:0/2904901909 >> 172.21.15.64:0/2904901909 conn(0x7f21140fc2a0 msgr2=0x7f2114103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:48.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.939+0000 7f211b229640 1 -- 172.21.15.64:0/2904901909 shutdown_connections 2024-09-12T03:28:48.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:48.939+0000 7f211b229640 1 -- 172.21.15.64:0/2904901909 wait complete. 2024-09-12T03:28:49.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:48 smithi195 ceph-mon[34786]: pgmap v340: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:28:49.271 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:48 smithi064 ceph-mon[30952]: pgmap v340: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:28:49.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:48 smithi115 ceph-mon[34801]: pgmap v340: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:28:49.768 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-12T03:28:50.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:49 smithi064 ceph-mon[30952]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:28:50.074 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:50.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:49 smithi195 ceph-mon[34786]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:28:50.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:49 smithi115 ceph-mon[34801]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:28:50.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.572+0000 7f6346fbc640 1 -- 172.21.15.64:0/463943302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6340107940 msgr2=0x7f6340109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:50.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.572+0000 7f6346fbc640 1 --2- 172.21.15.64:0/463943302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6340107940 0x7f6340109d30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6338009ca0 tx=0x7f633802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:28:50.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.573+0000 7f6346fbc640 1 -- 172.21.15.64:0/463943302 shutdown_connections 2024-09-12T03:28:50.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.573+0000 7f6346fbc640 1 --2- 172.21.15.64:0/463943302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6340107940 0x7f6340109d30 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:50.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.573+0000 7f6346fbc640 1 --2- 172.21.15.64:0/463943302 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6340105010 0x7f6340107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:50.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.573+0000 7f6346fbc640 1 --2- 172.21.15.64:0/463943302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f63401026e0 0x7f6340104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:50.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.573+0000 7f6346fbc640 1 -- 172.21.15.64:0/463943302 >> 172.21.15.64:0/463943302 conn(0x7f63400fc280 msgr2=0x7f63400fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:50.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.573+0000 7f6346fbc640 1 -- 172.21.15.64:0/463943302 shutdown_connections 2024-09-12T03:28:50.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.573+0000 7f6346fbc640 1 -- 172.21.15.64:0/463943302 wait complete. 2024-09-12T03:28:50.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.573+0000 7f6346fbc640 1 Processor -- start 2024-09-12T03:28:50.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.573+0000 7f6346fbc640 1 -- start start 2024-09-12T03:28:50.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.574+0000 7f6346fbc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f63401026e0 0x7f6340197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:50.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.574+0000 7f6346fbc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6340105010 0x7f63401982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:50.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.574+0000 7f6346fbc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6340107940 0x7f634019f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:50.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.574+0000 7f6346fbc640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f634010d710 con 0x7f6340105010 2024-09-12T03:28:50.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.574+0000 7f6346fbc640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f634010d590 con 0x7f63401026e0 2024-09-12T03:28:50.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.574+0000 7f6346fbc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f634010d890 con 0x7f6340107940 2024-09-12T03:28:50.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.574+0000 7f633ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6340105010 0x7f63401982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:50.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.574+0000 7f633ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6340105010 0x7f63401982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60002/0 (socket says 172.21.15.64:60002) 2024-09-12T03:28:50.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.574+0000 7f633ffff640 1 -- 172.21.15.64:0/1695765802 learned_addr learned my addr 172.21.15.64:0/1695765802 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:50.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.574+0000 7f6345559640 1 --2- 172.21.15.64:0/1695765802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6340107940 0x7f634019f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:50.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.574+0000 7f6344d58640 1 --2- 172.21.15.64:0/1695765802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f63401026e0 0x7f6340197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:50.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.574+0000 7f633ffff640 1 -- 172.21.15.64:0/1695765802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6340107940 msgr2=0x7f634019f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:50.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.574+0000 7f633ffff640 1 --2- 172.21.15.64:0/1695765802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6340107940 0x7f634019f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:50.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.574+0000 7f633ffff640 1 -- 172.21.15.64:0/1695765802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f63401026e0 msgr2=0x7f6340197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:50.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.574+0000 7f633ffff640 1 --2- 172.21.15.64:0/1695765802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f63401026e0 0x7f6340197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:50.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.574+0000 7f633ffff640 1 -- 172.21.15.64:0/1695765802 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f634019f990 con 0x7f6340105010 2024-09-12T03:28:50.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.574+0000 7f6344d58640 1 --2- 172.21.15.64:0/1695765802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f63401026e0 0x7f6340197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:28:50.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.574+0000 7f633ffff640 1 --2- 172.21.15.64:0/1695765802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6340105010 0x7f63401982a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f633400c970 tx=0x7f633400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:50.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.575+0000 7f633dffb640 1 -- 172.21.15.64:0/1695765802 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6334014070 con 0x7f6340105010 2024-09-12T03:28:50.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.575+0000 7f633dffb640 1 -- 172.21.15.64:0/1695765802 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6334004440 con 0x7f6340105010 2024-09-12T03:28:50.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.575+0000 7f6346fbc640 1 -- 172.21.15.64:0/1695765802 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f634019fc80 con 0x7f6340105010 2024-09-12T03:28:50.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.575+0000 7f633dffb640 1 -- 172.21.15.64:0/1695765802 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6334003e50 con 0x7f6340105010 2024-09-12T03:28:50.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.575+0000 7f6346fbc640 1 -- 172.21.15.64:0/1695765802 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63401a01c0 con 0x7f6340105010 2024-09-12T03:28:50.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.576+0000 7f6346fbc640 1 -- 172.21.15.64:0/1695765802 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63400fdd70 con 0x7f6340105010 2024-09-12T03:28:50.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.577+0000 7f633dffb640 1 -- 172.21.15.64:0/1695765802 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6334017070 con 0x7f6340105010 2024-09-12T03:28:50.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.577+0000 7f633dffb640 1 --2- 172.21.15.64:0/1695765802 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f63280a2de0 0x7f63280a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:50.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.578+0000 7f6344d58640 1 --2- 172.21.15.64:0/1695765802 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f63280a2de0 0x7f63280a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:50.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.578+0000 7f633dffb640 1 -- 172.21.15.64:0/1695765802 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f63340cf090 con 0x7f6340105010 2024-09-12T03:28:50.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.578+0000 7f6344d58640 1 --2- 172.21.15.64:0/1695765802 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f63280a2de0 0x7f63280a52a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f63300058c0 tx=0x7f6330005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:50.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.581+0000 7f633dffb640 1 -- 172.21.15.64:0/1695765802 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f633401e030 con 0x7f6340105010 2024-09-12T03:28:50.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.767+0000 7f6346fbc640 1 -- 172.21.15.64:0/1695765802 --> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f6340062630 con 0x7f63280a2de0 2024-09-12T03:28:50.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.773+0000 7f633dffb640 1 -- 172.21.15.64:0/1695765802 <== mgr.14150 v2:172.21.15.64:6800/759931848 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f6340062630 con 0x7f63280a2de0 2024-09-12T03:28:50.773 INFO:teuthology.orchestra.run.smithi064.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-12T03:28:50.773 INFO:teuthology.orchestra.run.smithi064.stdout:mds.all 12/12 56s ago 2m smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi115=b;smithi115=e;smithi115=h;smithi115=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l;count:12 2024-09-12T03:28:50.773 INFO:teuthology.orchestra.run.smithi064.stdout:mgr 3/3 56s ago 7m smithi064=x;smithi115=y;smithi195=z;count:3 2024-09-12T03:28:50.773 INFO:teuthology.orchestra.run.smithi064.stdout:mon 3/3 56s ago 8m smithi064:172.21.15.64=a;smithi115:172.21.15.115=b;smithi195:172.21.15.195=c;count:3 2024-09-12T03:28:50.774 INFO:teuthology.orchestra.run.smithi064.stdout:osd 12 56s ago - 2024-09-12T03:28:50.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.775+0000 7f6346fbc640 1 -- 172.21.15.64:0/1695765802 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f63280a2de0 msgr2=0x7f63280a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:50.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.775+0000 7f6346fbc640 1 --2- 172.21.15.64:0/1695765802 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f63280a2de0 0x7f63280a52a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f63300058c0 tx=0x7f6330005850 comp rx=0 tx=0).stop 2024-09-12T03:28:50.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.775+0000 7f6346fbc640 1 -- 172.21.15.64:0/1695765802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6340105010 msgr2=0x7f63401982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:50.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.775+0000 7f6346fbc640 1 --2- 172.21.15.64:0/1695765802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6340105010 0x7f63401982a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f633400c970 tx=0x7f633400ce30 comp rx=0 tx=0).stop 2024-09-12T03:28:50.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.776+0000 7f6346fbc640 1 -- 172.21.15.64:0/1695765802 shutdown_connections 2024-09-12T03:28:50.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.776+0000 7f6346fbc640 1 --2- 172.21.15.64:0/1695765802 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f63280a2de0 0x7f63280a52a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:50.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.776+0000 7f6346fbc640 1 --2- 172.21.15.64:0/1695765802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6340107940 0x7f634019f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:50.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.776+0000 7f6346fbc640 1 --2- 172.21.15.64:0/1695765802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6340105010 0x7f63401982a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:50.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.776+0000 7f6346fbc640 1 --2- 172.21.15.64:0/1695765802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f63401026e0 0x7f6340197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:50.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.776+0000 7f6346fbc640 1 -- 172.21.15.64:0/1695765802 >> 172.21.15.64:0/1695765802 conn(0x7f63400fc280 msgr2=0x7f6340102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:50.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.776+0000 7f6346fbc640 1 -- 172.21.15.64:0/1695765802 shutdown_connections 2024-09-12T03:28:50.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:50.776+0000 7f6346fbc640 1 -- 172.21.15.64:0/1695765802 wait complete. 2024-09-12T03:28:51.021 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:50 smithi064 ceph-mon[30952]: pgmap v341: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:28:51.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:50 smithi195 ceph-mon[34786]: pgmap v341: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:28:51.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:50 smithi115 ceph-mon[34801]: pgmap v341: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:28:51.505 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-12T03:28:51.761 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:52.147 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:51 smithi064 ceph-mon[30952]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:28:52.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.231+0000 7fbf27753640 1 -- 172.21.15.64:0/812555250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf20100b10 msgr2=0x7fbf20100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:52.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.231+0000 7fbf27753640 1 --2- 172.21.15.64:0/812555250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf20100b10 0x7fbf20100f10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009900 tx=0x7fbf1c02a410 comp rx=0 tx=0).stop 2024-09-12T03:28:52.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.231+0000 7fbf27753640 1 -- 172.21.15.64:0/812555250 shutdown_connections 2024-09-12T03:28:52.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.231+0000 7fbf27753640 1 --2- 172.21.15.64:0/812555250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf201026d0 0x7fbf20108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.231+0000 7fbf27753640 1 --2- 172.21.15.64:0/812555250 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf20101d10 0x7fbf20102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.231+0000 7fbf27753640 1 --2- 172.21.15.64:0/812555250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf20100b10 0x7fbf20100f10 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.231+0000 7fbf27753640 1 -- 172.21.15.64:0/812555250 >> 172.21.15.64:0/812555250 conn(0x7fbf200fc2a0 msgr2=0x7fbf200fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:52.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.231+0000 7fbf27753640 1 -- 172.21.15.64:0/812555250 shutdown_connections 2024-09-12T03:28:52.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.231+0000 7fbf27753640 1 -- 172.21.15.64:0/812555250 wait complete. 2024-09-12T03:28:52.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.232+0000 7fbf27753640 1 Processor -- start 2024-09-12T03:28:52.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.232+0000 7fbf27753640 1 -- start start 2024-09-12T03:28:52.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.232+0000 7fbf27753640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf20100b10 0x7fbf20197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:52.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.232+0000 7fbf27753640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf20101d10 0x7fbf201983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:52.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.232+0000 7fbf27753640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf201026d0 0x7fbf2019f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:52.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.232+0000 7fbf27753640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2010c640 con 0x7fbf201026d0 2024-09-12T03:28:52.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.232+0000 7fbf27753640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2010c4c0 con 0x7fbf20101d10 2024-09-12T03:28:52.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.232+0000 7fbf27753640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2010c7c0 con 0x7fbf20100b10 2024-09-12T03:28:52.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.232+0000 7fbf25cf0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf201026d0 0x7fbf2019f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:52.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.232+0000 7fbf25cf0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf201026d0 0x7fbf2019f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60024/0 (socket says 172.21.15.64:60024) 2024-09-12T03:28:52.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.232+0000 7fbf25cf0640 1 -- 172.21.15.64:0/2852960359 learned_addr learned my addr 172.21.15.64:0/2852960359 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:52.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.232+0000 7fbf254ef640 1 --2- 172.21.15.64:0/2852960359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf20100b10 0x7fbf20197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:52.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.232+0000 7fbf24cee640 1 --2- 172.21.15.64:0/2852960359 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf20101d10 0x7fbf201983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:52.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.232+0000 7fbf25cf0640 1 -- 172.21.15.64:0/2852960359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf20100b10 msgr2=0x7fbf20197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:52.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.232+0000 7fbf25cf0640 1 --2- 172.21.15.64:0/2852960359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf20100b10 0x7fbf20197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.232+0000 7fbf25cf0640 1 -- 172.21.15.64:0/2852960359 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf20101d10 msgr2=0x7fbf201983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:52.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.232+0000 7fbf25cf0640 1 --2- 172.21.15.64:0/2852960359 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf20101d10 0x7fbf201983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.232+0000 7fbf25cf0640 1 -- 172.21.15.64:0/2852960359 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf2019fa90 con 0x7fbf201026d0 2024-09-12T03:28:52.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.232+0000 7fbf25cf0640 1 --2- 172.21.15.64:0/2852960359 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf201026d0 0x7fbf2019f420 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbf1800eb10 tx=0x7fbf1800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:52.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.233+0000 7fbf127fc640 1 -- 172.21.15.64:0/2852960359 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf18004b20 con 0x7fbf201026d0 2024-09-12T03:28:52.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.233+0000 7fbf127fc640 1 -- 172.21.15.64:0/2852960359 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf18004cc0 con 0x7fbf201026d0 2024-09-12T03:28:52.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.233+0000 7fbf127fc640 1 -- 172.21.15.64:0/2852960359 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf18010690 con 0x7fbf201026d0 2024-09-12T03:28:52.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.233+0000 7fbf27753640 1 -- 172.21.15.64:0/2852960359 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf2019fd80 con 0x7fbf201026d0 2024-09-12T03:28:52.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.233+0000 7fbf27753640 1 -- 172.21.15.64:0/2852960359 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf201a0210 con 0x7fbf201026d0 2024-09-12T03:28:52.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.234+0000 7fbf27753640 1 -- 172.21.15.64:0/2852960359 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf201a04a0 con 0x7fbf201026d0 2024-09-12T03:28:52.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.235+0000 7fbf127fc640 1 -- 172.21.15.64:0/2852960359 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fbf18005020 con 0x7fbf201026d0 2024-09-12T03:28:52.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.235+0000 7fbf127fc640 1 --2- 172.21.15.64:0/2852960359 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fbf0c0a2c40 0x7fbf0c0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:52.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.236+0000 7fbf254ef640 1 --2- 172.21.15.64:0/2852960359 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fbf0c0a2c40 0x7fbf0c0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:52.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.236+0000 7fbf127fc640 1 -- 172.21.15.64:0/2852960359 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fbf180cfc10 con 0x7fbf201026d0 2024-09-12T03:28:52.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.236+0000 7fbf254ef640 1 --2- 172.21.15.64:0/2852960359 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fbf0c0a2c40 0x7fbf0c0a5100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c000c00 tx=0x7fbf1c0314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:52.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.240+0000 7fbf127fc640 1 -- 172.21.15.64:0/2852960359 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbf1809c6b0 con 0x7fbf201026d0 2024-09-12T03:28:52.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:51 smithi115 ceph-mon[34801]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:28:52.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.423+0000 7fbf27753640 1 -- 172.21.15.64:0/2852960359 --> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fbf2006aea0 con 0x7fbf0c0a2c40 2024-09-12T03:28:52.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.425+0000 7fbf127fc640 1 -- 172.21.15.64:0/2852960359 <== mgr.14150 v2:172.21.15.64:6800/759931848 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fbf2006aea0 con 0x7fbf0c0a2c40 2024-09-12T03:28:52.425 INFO:teuthology.orchestra.run.smithi064.stdout:HOST ADDR LABELS STATUS 2024-09-12T03:28:52.425 INFO:teuthology.orchestra.run.smithi064.stdout:smithi064 172.21.15.64 2024-09-12T03:28:52.425 INFO:teuthology.orchestra.run.smithi064.stdout:smithi115 172.21.15.115 2024-09-12T03:28:52.425 INFO:teuthology.orchestra.run.smithi064.stdout:smithi195 172.21.15.195 2024-09-12T03:28:52.425 INFO:teuthology.orchestra.run.smithi064.stdout:3 hosts in cluster 2024-09-12T03:28:52.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.427+0000 7fbf27753640 1 -- 172.21.15.64:0/2852960359 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fbf0c0a2c40 msgr2=0x7fbf0c0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:52.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.427+0000 7fbf27753640 1 --2- 172.21.15.64:0/2852960359 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fbf0c0a2c40 0x7fbf0c0a5100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c000c00 tx=0x7fbf1c0314a0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.427+0000 7fbf27753640 1 -- 172.21.15.64:0/2852960359 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf201026d0 msgr2=0x7fbf2019f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:52.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.427+0000 7fbf27753640 1 --2- 172.21.15.64:0/2852960359 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf201026d0 0x7fbf2019f420 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbf1800eb10 tx=0x7fbf1800c2d0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.427+0000 7fbf27753640 1 -- 172.21.15.64:0/2852960359 shutdown_connections 2024-09-12T03:28:52.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.427+0000 7fbf27753640 1 --2- 172.21.15.64:0/2852960359 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fbf0c0a2c40 0x7fbf0c0a5100 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.427+0000 7fbf27753640 1 --2- 172.21.15.64:0/2852960359 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf201026d0 0x7fbf2019f420 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.427+0000 7fbf27753640 1 --2- 172.21.15.64:0/2852960359 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf20101d10 0x7fbf201983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.427+0000 7fbf27753640 1 --2- 172.21.15.64:0/2852960359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf20100b10 0x7fbf20197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:52.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.427+0000 7fbf27753640 1 -- 172.21.15.64:0/2852960359 >> 172.21.15.64:0/2852960359 conn(0x7fbf200fc2a0 msgr2=0x7fbf2006a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:52.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.428+0000 7fbf27753640 1 -- 172.21.15.64:0/2852960359 shutdown_connections 2024-09-12T03:28:52.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:52.428+0000 7fbf27753640 1 -- 172.21.15.64:0/2852960359 wait complete. 2024-09-12T03:28:52.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:51 smithi195 ceph-mon[34786]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:28:53.164 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:52 smithi064 ceph-mon[30952]: pgmap v342: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:28:53.164 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:52 smithi064 ceph-mon[30952]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:28:53.165 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-12T03:28:53.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:52 smithi115 ceph-mon[34801]: pgmap v342: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:28:53.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:52 smithi115 ceph-mon[34801]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:28:53.422 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:53.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:52 smithi195 ceph-mon[34786]: pgmap v342: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:28:53.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:52 smithi195 ceph-mon[34786]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:28:53.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:53.883+0000 7f9176e14640 1 -- 172.21.15.64:0/3115270340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9170100ad0 msgr2=0x7f9170100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:53.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:53.883+0000 7f9176e14640 1 --2- 172.21.15.64:0/3115270340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9170100ad0 0x7f9170100ed0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9168009ca0 tx=0x7f916802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:28:53.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:53.883+0000 7f9176e14640 1 -- 172.21.15.64:0/3115270340 shutdown_connections 2024-09-12T03:28:53.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:53.883+0000 7f9176e14640 1 --2- 172.21.15.64:0/3115270340 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9170102690 0x7f9170108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:53.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:53.883+0000 7f9176e14640 1 --2- 172.21.15.64:0/3115270340 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9170101cd0 0x7f9170102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:53.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:53.883+0000 7f9176e14640 1 --2- 172.21.15.64:0/3115270340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9170100ad0 0x7f9170100ed0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:53.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:53.883+0000 7f9176e14640 1 -- 172.21.15.64:0/3115270340 >> 172.21.15.64:0/3115270340 conn(0x7f91700fc280 msgr2=0x7f91700fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:53.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:53.883+0000 7f9176e14640 1 -- 172.21.15.64:0/3115270340 shutdown_connections 2024-09-12T03:28:53.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:53.883+0000 7f9176e14640 1 -- 172.21.15.64:0/3115270340 wait complete. 2024-09-12T03:28:53.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:53.883+0000 7f9176e14640 1 Processor -- start 2024-09-12T03:28:53.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:53.883+0000 7f9176e14640 1 -- start start 2024-09-12T03:28:53.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:53.884+0000 7f9176e14640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9170100ad0 0x7f9170197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:53.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:53.884+0000 7f9176e14640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9170101cd0 0x7f9170198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:53.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:53.884+0000 7f9176e14640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9170102690 0x7f917019f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:53.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:53.884+0000 7f9176e14640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f917010d790 con 0x7f9170102690 2024-09-12T03:28:53.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:53.884+0000 7f9176e14640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f917010d610 con 0x7f9170100ad0 2024-09-12T03:28:53.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:53.884+0000 7f9176e14640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f917010d910 con 0x7f9170101cd0 2024-09-12T03:28:53.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:53.884+0000 7f91753b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9170102690 0x7f917019f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:53.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:53.884+0000 7f91753b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9170102690 0x7f917019f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60044/0 (socket says 172.21.15.64:60044) 2024-09-12T03:28:53.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:53.884+0000 7f91753b1640 1 -- 172.21.15.64:0/1649986113 learned_addr learned my addr 172.21.15.64:0/1649986113 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:53.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:53.884+0000 7f9174bb0640 1 --2- 172.21.15.64:0/1649986113 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9170100ad0 0x7f9170197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:53.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:53.884+0000 7f916ffff640 1 --2- 172.21.15.64:0/1649986113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9170101cd0 0x7f9170198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:53.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:53.884+0000 7f91753b1640 1 -- 172.21.15.64:0/1649986113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9170101cd0 msgr2=0x7f9170198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:53.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:53.884+0000 7f91753b1640 1 --2- 172.21.15.64:0/1649986113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9170101cd0 0x7f9170198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:53.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:53.884+0000 7f91753b1640 1 -- 172.21.15.64:0/1649986113 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9170100ad0 msgr2=0x7f9170197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:53.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:53.884+0000 7f91753b1640 1 --2- 172.21.15.64:0/1649986113 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9170100ad0 0x7f9170197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:53.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:53.884+0000 7f91753b1640 1 -- 172.21.15.64:0/1649986113 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f917019fa30 con 0x7f9170102690 2024-09-12T03:28:53.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:53.885+0000 7f91753b1640 1 --2- 172.21.15.64:0/1649986113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9170102690 0x7f917019f3c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f916000d750 tx=0x7f916000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:53.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:53.885+0000 7f916dffb640 1 -- 172.21.15.64:0/1649986113 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9160017070 con 0x7f9170102690 2024-09-12T03:28:53.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:53.885+0000 7f916dffb640 1 -- 172.21.15.64:0/1649986113 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f91600054d0 con 0x7f9170102690 2024-09-12T03:28:53.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:53.885+0000 7f9176e14640 1 -- 172.21.15.64:0/1649986113 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f917019fd20 con 0x7f9170102690 2024-09-12T03:28:53.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:53.885+0000 7f916dffb640 1 -- 172.21.15.64:0/1649986113 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9160004c60 con 0x7f9170102690 2024-09-12T03:28:53.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:53.885+0000 7f9176e14640 1 -- 172.21.15.64:0/1649986113 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91701a0250 con 0x7f9170102690 2024-09-12T03:28:53.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:53.886+0000 7f9176e14640 1 -- 172.21.15.64:0/1649986113 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9170107f70 con 0x7f9170102690 2024-09-12T03:28:53.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:53.887+0000 7f916dffb640 1 -- 172.21.15.64:0/1649986113 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9160004e00 con 0x7f9170102690 2024-09-12T03:28:53.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:53.888+0000 7f916dffb640 1 --2- 172.21.15.64:0/1649986113 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f91580a2cc0 0x7f91580a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:53.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:53.888+0000 7f9174bb0640 1 --2- 172.21.15.64:0/1649986113 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f91580a2cc0 0x7f91580a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:53.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:53.888+0000 7f916dffb640 1 -- 172.21.15.64:0/1649986113 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f91600ced60 con 0x7f9170102690 2024-09-12T03:28:53.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:53.888+0000 7f9174bb0640 1 --2- 172.21.15.64:0/1649986113 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f91580a2cc0 0x7f91580a5180 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9168000f80 tx=0x7f9168031000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:53.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:53.892+0000 7f916dffb640 1 -- 172.21.15.64:0/1649986113 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f916009b1b0 con 0x7f9170102690 2024-09-12T03:28:54.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:54.077+0000 7f9176e14640 1 -- 172.21.15.64:0/1649986113 --> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f9170062630 con 0x7f91580a2cc0 2024-09-12T03:28:54.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:54.079+0000 7f916dffb640 1 -- 172.21.15.64:0/1649986113 <== mgr.14150 v2:172.21.15.64:6800/759931848 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f9170062630 con 0x7f91580a2cc0 2024-09-12T03:28:54.079 INFO:teuthology.orchestra.run.smithi064.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-12T03:28:54.079 INFO:teuthology.orchestra.run.smithi064.stdout:smithi064 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT534400C2400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-12T03:28:54.079 INFO:teuthology.orchestra.run.smithi064.stdout:smithi115 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400UU400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-12T03:28:54.080 INFO:teuthology.orchestra.run.smithi064.stdout:smithi195 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300FS480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-09-12T03:28:54.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:54.081+0000 7f9176e14640 1 -- 172.21.15.64:0/1649986113 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f91580a2cc0 msgr2=0x7f91580a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:54.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:54.081+0000 7f9176e14640 1 --2- 172.21.15.64:0/1649986113 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f91580a2cc0 0x7f91580a5180 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9168000f80 tx=0x7f9168031000 comp rx=0 tx=0).stop 2024-09-12T03:28:54.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:54.081+0000 7f9176e14640 1 -- 172.21.15.64:0/1649986113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9170102690 msgr2=0x7f917019f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:54.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:54.081+0000 7f9176e14640 1 --2- 172.21.15.64:0/1649986113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9170102690 0x7f917019f3c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f916000d750 tx=0x7f916000dc10 comp rx=0 tx=0).stop 2024-09-12T03:28:54.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:54.082+0000 7f9176e14640 1 -- 172.21.15.64:0/1649986113 shutdown_connections 2024-09-12T03:28:54.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:54.082+0000 7f9176e14640 1 --2- 172.21.15.64:0/1649986113 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f91580a2cc0 0x7f91580a5180 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:54.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:54.082+0000 7f9176e14640 1 --2- 172.21.15.64:0/1649986113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9170102690 0x7f917019f3c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:54.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:54.082+0000 7f9176e14640 1 --2- 172.21.15.64:0/1649986113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9170101cd0 0x7f9170198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:54.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:54.082+0000 7f9176e14640 1 --2- 172.21.15.64:0/1649986113 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9170100ad0 0x7f9170197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:54.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:54.082+0000 7f9176e14640 1 -- 172.21.15.64:0/1649986113 >> 172.21.15.64:0/1649986113 conn(0x7f91700fc280 msgr2=0x7f9170105140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:54.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:54.082+0000 7f9176e14640 1 -- 172.21.15.64:0/1649986113 shutdown_connections 2024-09-12T03:28:54.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:54.082+0000 7f9176e14640 1 -- 172.21.15.64:0/1649986113 wait complete. 2024-09-12T03:28:54.887 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-12T03:28:54.896 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi064.front.sepia.ceph.com 2024-09-12T03:28:54.897 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-12T03:28:55.159 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:55.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:54 smithi064 ceph-mon[30952]: pgmap v343: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:28:55.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:54 smithi064 ceph-mon[30952]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:28:55.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:54 smithi115 ceph-mon[34801]: pgmap v343: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:28:55.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:54 smithi115 ceph-mon[34801]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:28:55.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:54 smithi195 ceph-mon[34786]: pgmap v343: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:28:55.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:54 smithi195 ceph-mon[34786]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:28:55.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.649+0000 7f7023794640 1 -- 172.21.15.64:0/2526300481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f701c1026b0 msgr2=0x7f701c108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:55.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.649+0000 7f7023794640 1 --2- 172.21.15.64:0/2526300481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f701c1026b0 0x7f701c108f40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7014009ca0 tx=0x7f701402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:28:55.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.649+0000 7f7023794640 1 -- 172.21.15.64:0/2526300481 shutdown_connections 2024-09-12T03:28:55.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.649+0000 7f7023794640 1 --2- 172.21.15.64:0/2526300481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f701c1026b0 0x7f701c108f40 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:55.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.649+0000 7f7023794640 1 --2- 172.21.15.64:0/2526300481 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f701c101cf0 0x7f701c102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:55.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.649+0000 7f7023794640 1 --2- 172.21.15.64:0/2526300481 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f701c100af0 0x7f701c100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:55.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.649+0000 7f7023794640 1 -- 172.21.15.64:0/2526300481 >> 172.21.15.64:0/2526300481 conn(0x7f701c0fc2a0 msgr2=0x7f701c0fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:55.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.649+0000 7f7023794640 1 -- 172.21.15.64:0/2526300481 shutdown_connections 2024-09-12T03:28:55.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.649+0000 7f7023794640 1 -- 172.21.15.64:0/2526300481 wait complete. 2024-09-12T03:28:55.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.649+0000 7f7023794640 1 Processor -- start 2024-09-12T03:28:55.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.649+0000 7f7023794640 1 -- start start 2024-09-12T03:28:55.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.650+0000 7f7023794640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f701c100af0 0x7f701c06b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:55.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.650+0000 7f7023794640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f701c101cf0 0x7f701c19cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:55.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.650+0000 7f7023794640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f701c1026b0 0x7f701c19f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:55.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.650+0000 7f7023794640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f701c10c620 con 0x7f701c1026b0 2024-09-12T03:28:55.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.650+0000 7f7023794640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f701c10c4a0 con 0x7f701c100af0 2024-09-12T03:28:55.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.650+0000 7f7023794640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f701c10c7a0 con 0x7f701c101cf0 2024-09-12T03:28:55.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.650+0000 7f7021d31640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f701c1026b0 0x7f701c19f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:55.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.650+0000 7f7021d31640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f701c1026b0 0x7f701c19f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60060/0 (socket says 172.21.15.64:60060) 2024-09-12T03:28:55.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.650+0000 7f7021d31640 1 -- 172.21.15.64:0/754666990 learned_addr learned my addr 172.21.15.64:0/754666990 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:55.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.650+0000 7f7020d2f640 1 --2- 172.21.15.64:0/754666990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f701c101cf0 0x7f701c19cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:55.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.650+0000 7f7021530640 1 --2- 172.21.15.64:0/754666990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f701c100af0 0x7f701c06b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:55.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.650+0000 7f7021d31640 1 -- 172.21.15.64:0/754666990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f701c101cf0 msgr2=0x7f701c19cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:55.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.650+0000 7f7021d31640 1 --2- 172.21.15.64:0/754666990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f701c101cf0 0x7f701c19cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:55.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.650+0000 7f7021d31640 1 -- 172.21.15.64:0/754666990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f701c100af0 msgr2=0x7f701c06b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:55.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.650+0000 7f7021d31640 1 --2- 172.21.15.64:0/754666990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f701c100af0 0x7f701c06b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:55.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.650+0000 7f7021d31640 1 -- 172.21.15.64:0/754666990 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f701c19fa70 con 0x7f701c1026b0 2024-09-12T03:28:55.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.650+0000 7f7021530640 1 --2- 172.21.15.64:0/754666990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f701c100af0 0x7f701c06b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:28:55.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.650+0000 7f7021d31640 1 --2- 172.21.15.64:0/754666990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f701c1026b0 0x7f701c19f160 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7014009dc0 tx=0x7f7014003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:55.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.650+0000 7f70127fc640 1 -- 172.21.15.64:0/754666990 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7014032ec0 con 0x7f701c1026b0 2024-09-12T03:28:55.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.650+0000 7f7023794640 1 -- 172.21.15.64:0/754666990 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f701c19fca0 con 0x7f701c1026b0 2024-09-12T03:28:55.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.650+0000 7f70127fc640 1 -- 172.21.15.64:0/754666990 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70140044f0 con 0x7f701c1026b0 2024-09-12T03:28:55.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.650+0000 7f70127fc640 1 -- 172.21.15.64:0/754666990 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7014030d40 con 0x7f701c1026b0 2024-09-12T03:28:55.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.650+0000 7f7023794640 1 -- 172.21.15.64:0/754666990 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f701c1a01e0 con 0x7f701c1026b0 2024-09-12T03:28:55.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.651+0000 7f7023794640 1 -- 172.21.15.64:0/754666990 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f701c100f20 con 0x7f701c1026b0 2024-09-12T03:28:55.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.653+0000 7f70127fc640 1 -- 172.21.15.64:0/754666990 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7014004040 con 0x7f701c1026b0 2024-09-12T03:28:55.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.653+0000 7f70127fc640 1 --2- 172.21.15.64:0/754666990 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f70080a2db0 0x7f70080a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:55.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.653+0000 7f7021530640 1 --2- 172.21.15.64:0/754666990 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f70080a2db0 0x7f70080a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:55.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.653+0000 7f70127fc640 1 -- 172.21.15.64:0/754666990 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f70140f0ae0 con 0x7f701c1026b0 2024-09-12T03:28:55.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.654+0000 7f7021530640 1 --2- 172.21.15.64:0/754666990 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f70080a2db0 0x7f70080a5270 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f701c101b50 tx=0x7f7018005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:55.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.658+0000 7f70127fc640 1 -- 172.21.15.64:0/754666990 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f701c100f20 con 0x7f701c1026b0 2024-09-12T03:28:55.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.868+0000 7f7023794640 1 -- 172.21.15.64:0/754666990 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f701c02d840 con 0x7f701c1026b0 2024-09-12T03:28:55.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.869+0000 7f70127fc640 1 -- 172.21.15.64:0/754666990 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3094 (secure 0 0 0) 0x7f70140bd690 con 0x7f701c1026b0 2024-09-12T03:28:55.869 INFO:teuthology.orchestra.run.smithi064.stdout:e24 2024-09-12T03:28:55.869 INFO:teuthology.orchestra.run.smithi064.stdout:btime 2024-09-12T03:27:37:139728+0000 2024-09-12T03:28:55.869 INFO:teuthology.orchestra.run.smithi064.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T03:28:55.869 INFO:teuthology.orchestra.run.smithi064.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T03:28:55.869 INFO:teuthology.orchestra.run.smithi064.stdout:legacy client fscid: 1 2024-09-12T03:28:55.869 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:28:55.870 INFO:teuthology.orchestra.run.smithi064.stdout:Filesystem 'cephfs' (1) 2024-09-12T03:28:55.870 INFO:teuthology.orchestra.run.smithi064.stdout:fs_name cephfs 2024-09-12T03:28:55.870 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 24 2024-09-12T03:28:55.870 INFO:teuthology.orchestra.run.smithi064.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-12T03:28:55.870 INFO:teuthology.orchestra.run.smithi064.stdout:created 2024-09-12T03:26:59.647657+0000 2024-09-12T03:28:55.870 INFO:teuthology.orchestra.run.smithi064.stdout:modified 2024-09-12T03:27:36.865338+0000 2024-09-12T03:28:55.870 INFO:teuthology.orchestra.run.smithi064.stdout:tableserver 0 2024-09-12T03:28:55.870 INFO:teuthology.orchestra.run.smithi064.stdout:root 0 2024-09-12T03:28:55.870 INFO:teuthology.orchestra.run.smithi064.stdout:session_timeout 300 2024-09-12T03:28:55.870 INFO:teuthology.orchestra.run.smithi064.stdout:session_autoclose 300 2024-09-12T03:28:55.870 INFO:teuthology.orchestra.run.smithi064.stdout:max_file_size 1099511627776 2024-09-12T03:28:55.870 INFO:teuthology.orchestra.run.smithi064.stdout:max_xattr_size 65536 2024-09-12T03:28:55.870 INFO:teuthology.orchestra.run.smithi064.stdout:required_client_features {} 2024-09-12T03:28:55.870 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure 0 2024-09-12T03:28:55.871 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure_osd_epoch 0 2024-09-12T03:28:55.871 INFO:teuthology.orchestra.run.smithi064.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T03:28:55.871 INFO:teuthology.orchestra.run.smithi064.stdout:max_mds 3 2024-09-12T03:28:55.871 INFO:teuthology.orchestra.run.smithi064.stdout:in 0,1,2 2024-09-12T03:28:55.871 INFO:teuthology.orchestra.run.smithi064.stdout:up {0=24500,1=24481,2=24470} 2024-09-12T03:28:55.871 INFO:teuthology.orchestra.run.smithi064.stdout:failed 2024-09-12T03:28:55.871 INFO:teuthology.orchestra.run.smithi064.stdout:damaged 2024-09-12T03:28:55.871 INFO:teuthology.orchestra.run.smithi064.stdout:stopped 2024-09-12T03:28:55.871 INFO:teuthology.orchestra.run.smithi064.stdout:data_pools [3,4] 2024-09-12T03:28:55.871 INFO:teuthology.orchestra.run.smithi064.stdout:metadata_pool 2 2024-09-12T03:28:55.871 INFO:teuthology.orchestra.run.smithi064.stdout:inline_data disabled 2024-09-12T03:28:55.871 INFO:teuthology.orchestra.run.smithi064.stdout:balancer 2024-09-12T03:28:55.871 INFO:teuthology.orchestra.run.smithi064.stdout:bal_rank_mask -1 2024-09-12T03:28:55.871 INFO:teuthology.orchestra.run.smithi064.stdout:standby_count_wanted 0 2024-09-12T03:28:55.872 INFO:teuthology.orchestra.run.smithi064.stdout:qdb_cluster leader: 24500 members: 24470,24481,24500 2024-09-12T03:28:55.872 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.b{0:24500} state up:active seq 4 addr [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:28:55.872 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.c{0:24463} state up:standby-replay seq 1 addr [v2:172.21.15.195:6836/71647401,v1:172.21.15.195:6837/71647401] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:28:55.872 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.i{1:24481} state up:active seq 15 addr [v2:172.21.15.195:6838/2574695134,v1:172.21.15.195:6839/2574695134] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:28:55.872 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.f{1:24457} state up:standby-replay seq 1 addr [v2:172.21.15.195:6834/2531045583,v1:172.21.15.195:6835/2531045583] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:28:55.872 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.h{2:24470} state up:active seq 16 addr [v2:172.21.15.115:6836/2915155705,v1:172.21.15.115:6837/2915155705] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:28:55.872 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.e{2:24449} state up:standby-replay seq 1 addr [v2:172.21.15.115:6834/332737916,v1:172.21.15.115:6835/332737916] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:28:55.872 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:28:55.872 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:28:55.872 INFO:teuthology.orchestra.run.smithi064.stdout:Standby daemons: 2024-09-12T03:28:55.872 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:28:55.872 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.j{-1:14529} state up:standby seq 1 addr [v2:172.21.15.64:6834/2580078738,v1:172.21.15.64:6835/2580078738] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:28:55.872 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.d{-1:14541} state up:standby seq 1 addr [v2:172.21.15.64:6836/2333415004,v1:172.21.15.64:6837/2333415004] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:28:55.873 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.g{-1:14595} state up:standby seq 1 addr [v2:172.21.15.64:6838/3030618395,v1:172.21.15.64:6839/3030618395] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:28:55.873 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.a{-1:14607} state up:standby seq 1 addr [v2:172.21.15.64:6840/3333057625,v1:172.21.15.64:6841/3333057625] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:28:55.873 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.l{-1:24418} state up:standby seq 1 addr [v2:172.21.15.195:6832/2991933266,v1:172.21.15.195:6833/2991933266] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:28:55.873 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.k{-1:24422} state up:standby seq 1 addr [v2:172.21.15.115:6832/1138960346,v1:172.21.15.115:6833/1138960346] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:28:55.873 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 24 2024-09-12T03:28:55.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.872+0000 7f7023794640 1 -- 172.21.15.64:0/754666990 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f70080a2db0 msgr2=0x7f70080a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:55.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.872+0000 7f7023794640 1 --2- 172.21.15.64:0/754666990 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f70080a2db0 0x7f70080a5270 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f701c101b50 tx=0x7f7018005d30 comp rx=0 tx=0).stop 2024-09-12T03:28:55.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.872+0000 7f7023794640 1 -- 172.21.15.64:0/754666990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f701c1026b0 msgr2=0x7f701c19f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:55.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.872+0000 7f7023794640 1 --2- 172.21.15.64:0/754666990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f701c1026b0 0x7f701c19f160 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7014009dc0 tx=0x7f7014003ba0 comp rx=0 tx=0).stop 2024-09-12T03:28:55.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.872+0000 7f7023794640 1 -- 172.21.15.64:0/754666990 shutdown_connections 2024-09-12T03:28:55.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.872+0000 7f7023794640 1 --2- 172.21.15.64:0/754666990 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f70080a2db0 0x7f70080a5270 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:55.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.872+0000 7f7023794640 1 --2- 172.21.15.64:0/754666990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f701c1026b0 0x7f701c19f160 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:55.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.872+0000 7f7023794640 1 --2- 172.21.15.64:0/754666990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f701c101cf0 0x7f701c19cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:55.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.872+0000 7f7023794640 1 --2- 172.21.15.64:0/754666990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f701c100af0 0x7f701c06b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:55.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.872+0000 7f7023794640 1 -- 172.21.15.64:0/754666990 >> 172.21.15.64:0/754666990 conn(0x7f701c0fc2a0 msgr2=0x7f701c105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:55.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.872+0000 7f7023794640 1 -- 172.21.15.64:0/754666990 shutdown_connections 2024-09-12T03:28:55.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:55.872+0000 7f7023794640 1 -- 172.21.15.64:0/754666990 wait complete. 2024-09-12T03:28:56.246 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:55 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/754666990' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T03:28:56.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:55 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/754666990' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T03:28:56.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:55 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/754666990' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T03:28:56.685 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-12T03:28:57.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:56 smithi064 ceph-mon[30952]: pgmap v344: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:28:57.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:56 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:28:57.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:56 smithi115 ceph-mon[34801]: pgmap v344: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:28:57.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:56 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:28:57.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:56 smithi195 ceph-mon[34786]: pgmap v344: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:28:57.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:56 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:28:57.590 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:58.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.074+0000 7f2f38936640 1 -- 172.21.15.64:0/508963917 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f34102690 msgr2=0x7f2f34108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:58.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.074+0000 7f2f38936640 1 --2- 172.21.15.64:0/508963917 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f34102690 0x7f2f34108f20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c009ca0 tx=0x7f2f2c02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.074+0000 7f2f38936640 1 -- 172.21.15.64:0/508963917 shutdown_connections 2024-09-12T03:28:58.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.074+0000 7f2f38936640 1 --2- 172.21.15.64:0/508963917 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f34102690 0x7f2f34108f20 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.074+0000 7f2f38936640 1 --2- 172.21.15.64:0/508963917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f34101cd0 0x7f2f34102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.074+0000 7f2f38936640 1 --2- 172.21.15.64:0/508963917 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f34100ad0 0x7f2f34100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.074+0000 7f2f38936640 1 -- 172.21.15.64:0/508963917 >> 172.21.15.64:0/508963917 conn(0x7f2f340fc280 msgr2=0x7f2f340fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:58.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.074+0000 7f2f38936640 1 -- 172.21.15.64:0/508963917 shutdown_connections 2024-09-12T03:28:58.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.074+0000 7f2f38936640 1 -- 172.21.15.64:0/508963917 wait complete. 2024-09-12T03:28:58.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.075+0000 7f2f38936640 1 Processor -- start 2024-09-12T03:28:58.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.075+0000 7f2f38936640 1 -- start start 2024-09-12T03:28:58.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.075+0000 7f2f38936640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f34100ad0 0x7f2f34197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:58.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.075+0000 7f2f38936640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f34101cd0 0x7f2f341982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:58.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.075+0000 7f2f38936640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f34102690 0x7f2f3419f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:58.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.075+0000 7f2f38936640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3410d730 con 0x7f2f34101cd0 2024-09-12T03:28:58.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.075+0000 7f2f38936640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3410d5b0 con 0x7f2f34100ad0 2024-09-12T03:28:58.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.075+0000 7f2f38936640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3410d8b0 con 0x7f2f34102690 2024-09-12T03:28:58.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.075+0000 7f2f31d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f34101cd0 0x7f2f341982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:58.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.075+0000 7f2f31d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f34101cd0 0x7f2f341982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35184/0 (socket says 172.21.15.64:35184) 2024-09-12T03:28:58.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.075+0000 7f2f31d9b640 1 -- 172.21.15.64:0/3629211396 learned_addr learned my addr 172.21.15.64:0/3629211396 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:58.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.075+0000 7f2f3259c640 1 --2- 172.21.15.64:0/3629211396 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f34100ad0 0x7f2f34197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:58.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.075+0000 7f2f32d9d640 1 --2- 172.21.15.64:0/3629211396 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f34102690 0x7f2f3419f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:58.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.076+0000 7f2f31d9b640 1 -- 172.21.15.64:0/3629211396 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f34102690 msgr2=0x7f2f3419f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:58.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.076+0000 7f2f31d9b640 1 --2- 172.21.15.64:0/3629211396 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f34102690 0x7f2f3419f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.076+0000 7f2f31d9b640 1 -- 172.21.15.64:0/3629211396 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f34100ad0 msgr2=0x7f2f34197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:58.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.076+0000 7f2f31d9b640 1 --2- 172.21.15.64:0/3629211396 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f34100ad0 0x7f2f34197da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.076+0000 7f2f31d9b640 1 -- 172.21.15.64:0/3629211396 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f3419f9d0 con 0x7f2f34101cd0 2024-09-12T03:28:58.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.076+0000 7f2f31d9b640 1 --2- 172.21.15.64:0/3629211396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f34101cd0 0x7f2f341982e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2f2800e9f0 tx=0x7f2f2800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:58.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.076+0000 7f2f237fe640 1 -- 172.21.15.64:0/3629211396 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2800cd80 con 0x7f2f34101cd0 2024-09-12T03:28:58.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.076+0000 7f2f38936640 1 -- 172.21.15.64:0/3629211396 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f3419fcc0 con 0x7f2f34101cd0 2024-09-12T03:28:58.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.076+0000 7f2f237fe640 1 -- 172.21.15.64:0/3629211396 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f280108e0 con 0x7f2f34101cd0 2024-09-12T03:28:58.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.076+0000 7f2f237fe640 1 -- 172.21.15.64:0/3629211396 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f28018920 con 0x7f2f34101cd0 2024-09-12T03:28:58.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.076+0000 7f2f38936640 1 -- 172.21.15.64:0/3629211396 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f341a01f0 con 0x7f2f34101cd0 2024-09-12T03:28:58.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.077+0000 7f2f38936640 1 -- 172.21.15.64:0/3629211396 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f34107ec0 con 0x7f2f34101cd0 2024-09-12T03:28:58.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.078+0000 7f2f237fe640 1 -- 172.21.15.64:0/3629211396 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2f28010a90 con 0x7f2f34101cd0 2024-09-12T03:28:58.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.079+0000 7f2f237fe640 1 --2- 172.21.15.64:0/3629211396 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f2f1c0a2c40 0x7f2f1c0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:58.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.080+0000 7f2f3259c640 1 --2- 172.21.15.64:0/3629211396 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f2f1c0a2c40 0x7f2f1c0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:58.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.080+0000 7f2f237fe640 1 -- 172.21.15.64:0/3629211396 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f2f280cfce0 con 0x7f2f34101cd0 2024-09-12T03:28:58.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.080+0000 7f2f3259c640 1 --2- 172.21.15.64:0/3629211396 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f2f1c0a2c40 0x7f2f1c0a5100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2f34101b30 tx=0x7f2f24005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:58.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.083+0000 7f2f237fe640 1 -- 172.21.15.64:0/3629211396 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2f2809c0d0 con 0x7f2f34101cd0 2024-09-12T03:28:58.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.273+0000 7f2f38936640 1 -- 172.21.15.64:0/3629211396 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f2f3402d960 con 0x7f2f34101cd0 2024-09-12T03:28:58.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.275+0000 7f2f237fe640 1 -- 172.21.15.64:0/3629211396 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v90) ==== 56+0+5595 (secure 0 0 0) 0x7f2f280a03d0 con 0x7f2f34101cd0 2024-09-12T03:28:58.275 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 90 2024-09-12T03:28:58.275 INFO:teuthology.orchestra.run.smithi064.stdout:fsid b0812460-70b5-11ef-bcea-c7b262605968 2024-09-12T03:28:58.275 INFO:teuthology.orchestra.run.smithi064.stdout:created 2024-09-12T03:19:15.900322+0000 2024-09-12T03:28:58.275 INFO:teuthology.orchestra.run.smithi064.stdout:modified 2024-09-12T03:28:08.835864+0000 2024-09-12T03:28:58.275 INFO:teuthology.orchestra.run.smithi064.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-12T03:28:58.275 INFO:teuthology.orchestra.run.smithi064.stdout:crush_version 27 2024-09-12T03:28:58.275 INFO:teuthology.orchestra.run.smithi064.stdout:full_ratio 0.95 2024-09-12T03:28:58.276 INFO:teuthology.orchestra.run.smithi064.stdout:backfillfull_ratio 0.9 2024-09-12T03:28:58.276 INFO:teuthology.orchestra.run.smithi064.stdout:nearfull_ratio 0.85 2024-09-12T03:28:58.276 INFO:teuthology.orchestra.run.smithi064.stdout:require_min_compat_client luminous 2024-09-12T03:28:58.276 INFO:teuthology.orchestra.run.smithi064.stdout:min_compat_client luminous 2024-09-12T03:28:58.276 INFO:teuthology.orchestra.run.smithi064.stdout:require_osd_release squid 2024-09-12T03:28:58.276 INFO:teuthology.orchestra.run.smithi064.stdout:stretch_mode_enabled false 2024-09-12T03:28:58.276 INFO:teuthology.orchestra.run.smithi064.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-12T03:28:58.276 INFO:teuthology.orchestra.run.smithi064.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-09-12T03:28:58.276 INFO:teuthology.orchestra.run.smithi064.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-12T03:28:58.276 INFO:teuthology.orchestra.run.smithi064.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 88 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-12T03:28:58.276 INFO:teuthology.orchestra.run.smithi064.stdout:max_osd 12 2024-09-12T03:28:58.277 INFO:teuthology.orchestra.run.smithi064.stdout:osd.0 up in weight 1 up_from 8 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6802/1307832590,v1:172.21.15.64:6803/1307832590] [v2:172.21.15.64:6804/1307832590,v1:172.21.15.64:6805/1307832590] exists,up 326271c9-1f28-4ce6-b3be-fcc57479d8b7 2024-09-12T03:28:58.277 INFO:teuthology.orchestra.run.smithi064.stdout:osd.1 up in weight 1 up_from 13 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6800/3123728622,v1:172.21.15.115:6801/3123728622] [v2:172.21.15.115:6802/3123728622,v1:172.21.15.115:6803/3123728622] exists,up 7dca0c28-c234-423c-936e-2351d63f918e 2024-09-12T03:28:58.277 INFO:teuthology.orchestra.run.smithi064.stdout:osd.2 up in weight 1 up_from 18 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.195:6800/1664125451,v1:172.21.15.195:6801/1664125451] [v2:172.21.15.195:6802/1664125451,v1:172.21.15.195:6803/1664125451] exists,up 4b3a3c7a-aec6-455a-9a9a-1ac9cc900f24 2024-09-12T03:28:58.277 INFO:teuthology.orchestra.run.smithi064.stdout:osd.3 up in weight 1 up_from 24 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6810/3935517185,v1:172.21.15.64:6811/3935517185] [v2:172.21.15.64:6812/3935517185,v1:172.21.15.64:6813/3935517185] exists,up 8768c339-2c1e-4294-8b78-3c2adab6750f 2024-09-12T03:28:58.277 INFO:teuthology.orchestra.run.smithi064.stdout:osd.4 up in weight 1 up_from 32 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6808/446665227,v1:172.21.15.115:6809/446665227] [v2:172.21.15.115:6810/446665227,v1:172.21.15.115:6811/446665227] exists,up 123aac47-a8ae-47c7-93a7-0dd293356322 2024-09-12T03:28:58.277 INFO:teuthology.orchestra.run.smithi064.stdout:osd.5 up in weight 1 up_from 36 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.195:6808/1187293668,v1:172.21.15.195:6809/1187293668] [v2:172.21.15.195:6810/1187293668,v1:172.21.15.195:6811/1187293668] exists,up c90bcaed-e7a8-4852-8677-b01dc73ea44e 2024-09-12T03:28:58.277 INFO:teuthology.orchestra.run.smithi064.stdout:osd.6 up in weight 1 up_from 43 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6818/560343274,v1:172.21.15.64:6819/560343274] [v2:172.21.15.64:6820/560343274,v1:172.21.15.64:6821/560343274] exists,up fd292547-6695-4889-8867-e74318bbb00f 2024-09-12T03:28:58.277 INFO:teuthology.orchestra.run.smithi064.stdout:osd.7 up in weight 1 up_from 48 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6816/3075734174,v1:172.21.15.115:6817/3075734174] [v2:172.21.15.115:6818/3075734174,v1:172.21.15.115:6819/3075734174] exists,up ad7c4151-cd98-4113-879b-ca2b7c83c7cc 2024-09-12T03:28:58.277 INFO:teuthology.orchestra.run.smithi064.stdout:osd.8 up in weight 1 up_from 55 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.195:6816/2184742494,v1:172.21.15.195:6817/2184742494] [v2:172.21.15.195:6818/2184742494,v1:172.21.15.195:6819/2184742494] exists,up 6de50049-c28f-4f21-b123-05e55bf59127 2024-09-12T03:28:58.277 INFO:teuthology.orchestra.run.smithi064.stdout:osd.9 up in weight 1 up_from 63 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6826/561728112,v1:172.21.15.64:6827/561728112] [v2:172.21.15.64:6828/561728112,v1:172.21.15.64:6829/561728112] exists,up 36ab792c-58f2-4beb-9024-7f3d1e3785dd 2024-09-12T03:28:58.277 INFO:teuthology.orchestra.run.smithi064.stdout:osd.10 up in weight 1 up_from 68 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6824/2977887627,v1:172.21.15.115:6825/2977887627] [v2:172.21.15.115:6826/2977887627,v1:172.21.15.115:6827/2977887627] exists,up d1f7881c-ec3f-466e-9cea-1d5cb441d970 2024-09-12T03:28:58.277 INFO:teuthology.orchestra.run.smithi064.stdout:osd.11 up in weight 1 up_from 76 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.195:6824/591599352,v1:172.21.15.195:6825/591599352] [v2:172.21.15.195:6826/591599352,v1:172.21.15.195:6827/591599352] exists,up 7c46c028-22c7-450f-9927-2144a7a09e9e 2024-09-12T03:28:58.277 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 3.18 [3,9] 2024-09-12T03:28:58.278 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 3.21 [4,8] 2024-09-12T03:28:58.278 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 3.28 [5,8] 2024-09-12T03:28:58.278 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 3.31 [4,7] 2024-09-12T03:28:58.278 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 3.34 [4,9] 2024-09-12T03:28:58.278 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 3.39 [5,6] 2024-09-12T03:28:58.278 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 4.2 [9,1] 2024-09-12T03:28:58.278 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 4.6 [5,8] 2024-09-12T03:28:58.278 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 4.13 [9,10] 2024-09-12T03:28:58.278 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 4.1c [9,8] 2024-09-12T03:28:58.278 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:6801/68603379 expires 2024-09-13T03:20:07.411102+0000 2024-09-12T03:28:58.278 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:0/3827202979 expires 2024-09-13T03:20:07.411102+0000 2024-09-12T03:28:58.278 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:6800/68603379 expires 2024-09-13T03:20:07.411102+0000 2024-09-12T03:28:58.278 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:0/1217136221 expires 2024-09-13T03:20:07.411102+0000 2024-09-12T03:28:58.278 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:0/3079679964 expires 2024-09-13T03:19:36.839431+0000 2024-09-12T03:28:58.279 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:6801/3186640984 expires 2024-09-13T03:19:36.839431+0000 2024-09-12T03:28:58.279 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:0/3334629354 expires 2024-09-13T03:20:07.411102+0000 2024-09-12T03:28:58.279 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:6800/3186640984 expires 2024-09-13T03:19:36.839431+0000 2024-09-12T03:28:58.279 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:0/3564418115 expires 2024-09-13T03:19:36.839431+0000 2024-09-12T03:28:58.279 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:0/686137382 expires 2024-09-13T03:19:36.839431+0000 2024-09-12T03:28:58.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.277+0000 7f2f38936640 1 -- 172.21.15.64:0/3629211396 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f2f1c0a2c40 msgr2=0x7f2f1c0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:58.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.277+0000 7f2f38936640 1 --2- 172.21.15.64:0/3629211396 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f2f1c0a2c40 0x7f2f1c0a5100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2f34101b30 tx=0x7f2f24005f90 comp rx=0 tx=0).stop 2024-09-12T03:28:58.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.277+0000 7f2f38936640 1 -- 172.21.15.64:0/3629211396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f34101cd0 msgr2=0x7f2f341982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:58.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.277+0000 7f2f38936640 1 --2- 172.21.15.64:0/3629211396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f34101cd0 0x7f2f341982e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2f2800e9f0 tx=0x7f2f2800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.277+0000 7f2f38936640 1 -- 172.21.15.64:0/3629211396 shutdown_connections 2024-09-12T03:28:58.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.277+0000 7f2f38936640 1 --2- 172.21.15.64:0/3629211396 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f2f1c0a2c40 0x7f2f1c0a5100 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.277+0000 7f2f38936640 1 --2- 172.21.15.64:0/3629211396 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f34102690 0x7f2f3419f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.277+0000 7f2f38936640 1 --2- 172.21.15.64:0/3629211396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f34101cd0 0x7f2f341982e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.277+0000 7f2f38936640 1 --2- 172.21.15.64:0/3629211396 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f34100ad0 0x7f2f34197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.277+0000 7f2f38936640 1 -- 172.21.15.64:0/3629211396 >> 172.21.15.64:0/3629211396 conn(0x7f2f340fc280 msgr2=0x7f2f34105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:58.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.277+0000 7f2f38936640 1 -- 172.21.15.64:0/3629211396 shutdown_connections 2024-09-12T03:28:58.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:58.277+0000 7f2f38936640 1 -- 172.21.15.64:0/3629211396 wait complete. 2024-09-12T03:28:59.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:58 smithi064 ceph-mon[30952]: pgmap v345: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:28:59.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:58 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:28:59.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:58 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:28:59.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:58 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:28:59.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:58 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:28:59.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:28:58 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/3629211396' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T03:28:59.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:58 smithi115 ceph-mon[34801]: pgmap v345: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:28:59.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:58 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:28:59.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:58 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:28:59.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:58 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:28:59.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:58 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:28:59.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:28:58 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/3629211396' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T03:28:59.177 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-12T03:28:59.190 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:28:59.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:58 smithi195 ceph-mon[34786]: pgmap v345: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:28:59.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:58 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:28:59.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:58 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:28:59.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:58 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:28:59.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:58 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:28:59.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:28:58 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/3629211396' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T03:28:59.466 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:59.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:59.970+0000 7fa9dc7c5640 1 -- 172.21.15.64:0/2846843813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9d4102570 msgr2=0x7fa9d41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:59.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:59.970+0000 7fa9dc7c5640 1 --2- 172.21.15.64:0/2846843813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9d4102570 0x7fa9d41029b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4009900 tx=0x7fa9c402a480 comp rx=0 tx=0).stop 2024-09-12T03:28:59.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:59.970+0000 7fa9dc7c5640 1 -- 172.21.15.64:0/2846843813 shutdown_connections 2024-09-12T03:28:59.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:59.970+0000 7fa9dc7c5640 1 --2- 172.21.15.64:0/2846843813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9d4102ef0 0x7fa9d410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:59.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:59.970+0000 7fa9dc7c5640 1 --2- 172.21.15.64:0/2846843813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9d4102570 0x7fa9d41029b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:59.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:59.970+0000 7fa9dc7c5640 1 --2- 172.21.15.64:0/2846843813 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9d4100e20 0x7fa9d4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:59.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:59.970+0000 7fa9dc7c5640 1 -- 172.21.15.64:0/2846843813 >> 172.21.15.64:0/2846843813 conn(0x7fa9d40fc7d0 msgr2=0x7fa9d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:59.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:59.970+0000 7fa9dc7c5640 1 -- 172.21.15.64:0/2846843813 shutdown_connections 2024-09-12T03:28:59.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:59.970+0000 7fa9dc7c5640 1 -- 172.21.15.64:0/2846843813 wait complete. 2024-09-12T03:28:59.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:59.971+0000 7fa9dc7c5640 1 Processor -- start 2024-09-12T03:28:59.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:59.971+0000 7fa9dc7c5640 1 -- start start 2024-09-12T03:28:59.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:59.971+0000 7fa9dc7c5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9d4100e20 0x7fa9d4197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:59.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:59.971+0000 7fa9dc7c5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9d4102570 0x7fa9d4197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:59.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:59.971+0000 7fa9dc7c5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9d4102ef0 0x7fa9d419bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:59.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:59.971+0000 7fa9dc7c5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d4110e00 con 0x7fa9d4102ef0 2024-09-12T03:28:59.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:59.971+0000 7fa9dc7c5640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d4110c80 con 0x7fa9d4100e20 2024-09-12T03:28:59.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:59.971+0000 7fa9dc7c5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d4110f80 con 0x7fa9d4102570 2024-09-12T03:28:59.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:59.971+0000 7fa9dad62640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9d4102ef0 0x7fa9d419bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:59.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:59.971+0000 7fa9dad62640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9d4102ef0 0x7fa9d419bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35206/0 (socket says 172.21.15.64:35206) 2024-09-12T03:28:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:59.971+0000 7fa9dad62640 1 -- 172.21.15.64:0/1108014264 learned_addr learned my addr 172.21.15.64:0/1108014264 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:28:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:59.971+0000 7fa9da561640 1 --2- 172.21.15.64:0/1108014264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9d4100e20 0x7fa9d4197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:59.971+0000 7fa9dad62640 1 -- 172.21.15.64:0/1108014264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9d4102570 msgr2=0x7fa9d4197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:59.971+0000 7fa9d9d60640 1 --2- 172.21.15.64:0/1108014264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9d4102570 0x7fa9d4197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:59.971+0000 7fa9dad62640 1 --2- 172.21.15.64:0/1108014264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9d4102570 0x7fa9d4197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:59.971+0000 7fa9dad62640 1 -- 172.21.15.64:0/1108014264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9d4100e20 msgr2=0x7fa9d4197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:59.971+0000 7fa9dad62640 1 --2- 172.21.15.64:0/1108014264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9d4100e20 0x7fa9d4197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:59.971+0000 7fa9dad62640 1 -- 172.21.15.64:0/1108014264 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d419c2b0 con 0x7fa9d4102ef0 2024-09-12T03:28:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:59.971+0000 7fa9d9d60640 1 --2- 172.21.15.64:0/1108014264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9d4102570 0x7fa9d4197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:28:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:59.972+0000 7fa9dad62640 1 --2- 172.21.15.64:0/1108014264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9d4102ef0 0x7fa9d419bbd0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00d750 tx=0x7fa9cc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:59.972+0000 7fa9cb7fe640 1 -- 172.21.15.64:0/1108014264 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc017070 con 0x7fa9d4102ef0 2024-09-12T03:28:59.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:59.972+0000 7fa9cb7fe640 1 -- 172.21.15.64:0/1108014264 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9cc0054d0 con 0x7fa9d4102ef0 2024-09-12T03:28:59.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:59.972+0000 7fa9dc7c5640 1 -- 172.21.15.64:0/1108014264 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9d419c5a0 con 0x7fa9d4102ef0 2024-09-12T03:28:59.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:59.972+0000 7fa9cb7fe640 1 -- 172.21.15.64:0/1108014264 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc004c60 con 0x7fa9d4102ef0 2024-09-12T03:28:59.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:59.972+0000 7fa9dc7c5640 1 -- 172.21.15.64:0/1108014264 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9d41a3e80 con 0x7fa9d4102ef0 2024-09-12T03:28:59.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:59.973+0000 7fa9dc7c5640 1 -- 172.21.15.64:0/1108014264 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9d4101200 con 0x7fa9d4102ef0 2024-09-12T03:28:59.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:59.974+0000 7fa9cb7fe640 1 -- 172.21.15.64:0/1108014264 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa9cc004e00 con 0x7fa9d4102ef0 2024-09-12T03:28:59.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:59.975+0000 7fa9cb7fe640 1 --2- 172.21.15.64:0/1108014264 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fa9c00a2c90 0x7fa9c00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:59.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:59.975+0000 7fa9da561640 1 --2- 172.21.15.64:0/1108014264 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fa9c00a2c90 0x7fa9c00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:59.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:59.975+0000 7fa9cb7fe640 1 -- 172.21.15.64:0/1108014264 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fa9cc0cee10 con 0x7fa9d4102ef0 2024-09-12T03:28:59.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:59.975+0000 7fa9da561640 1 --2- 172.21.15.64:0/1108014264 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fa9c00a2c90 0x7fa9c00a5150 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0005fd0 tx=0x7fa9d0009760 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:59.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:28:59.979+0000 7fa9cb7fe640 1 -- 172.21.15.64:0/1108014264 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa9cc09b8b0 con 0x7fa9d4102ef0 2024-09-12T03:29:00.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:00.194+0000 7fa9dc7c5640 1 -- 172.21.15.64:0/1108014264 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9d4109090 con 0x7fa9d4102ef0 2024-09-12T03:29:00.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:00.195+0000 7fa9cb7fe640 1 -- 172.21.15.64:0/1108014264 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11921 (secure 0 0 0) 0x7fa9cc09b1b0 con 0x7fa9d4102ef0 2024-09-12T03:29:00.195 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:29:00.196 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":24,"btime":"2024-09-12T03:27:37:139728+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2580078738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2580078738},{"type":"v1","addr":"172.21.15.64:6835","nonce":2580078738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2333415004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2333415004},{"type":"v1","addr":"172.21.15.64:6837","nonce":2333415004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/3030618395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":3030618395},{"type":"v1","addr":"172.21.15.64:6839","nonce":3030618395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3333057625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3333057625},{"type":"v1","addr":"172.21.15.64:6841","nonce":3333057625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2991933266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2991933266},{"type":"v1","addr":"172.21.15.195:6833","nonce":2991933266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1138960346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1138960346},{"type":"v1","addr":"172.21.15.115:6833","nonce":1138960346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:26:59.647657+0000","modified":"2024-09-12T03:27:36.865338+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24481,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/332737916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":332737916},{"type":"v1","addr":"172.21.15.115:6835","nonce":332737916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/2531045583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2531045583},{"type":"v1","addr":"172.21.15.195:6835","nonce":2531045583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/71647401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":71647401},{"type":"v1","addr":"172.21.15.195:6837","nonce":71647401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.115:6837/2915155705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2915155705},{"type":"v1","addr":"172.21.15.115:6837","nonce":2915155705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.195:6839/2574695134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":2574695134},{"type":"v1","addr":"172.21.15.195:6839","nonce":2574695134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6839/2238031796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2238031796},{"type":"v1","addr":"172.21.15.115:6839","nonce":2238031796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24470,24481,24500]},"id":1}]} 2024-09-12T03:29:00.196 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 24 2024-09-12T03:29:00.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:00.197+0000 7fa9dc7c5640 1 -- 172.21.15.64:0/1108014264 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fa9c00a2c90 msgr2=0x7fa9c00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:00.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:00.197+0000 7fa9dc7c5640 1 --2- 172.21.15.64:0/1108014264 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fa9c00a2c90 0x7fa9c00a5150 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0005fd0 tx=0x7fa9d0009760 comp rx=0 tx=0).stop 2024-09-12T03:29:00.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:00.197+0000 7fa9dc7c5640 1 -- 172.21.15.64:0/1108014264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9d4102ef0 msgr2=0x7fa9d419bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:00.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:00.197+0000 7fa9dc7c5640 1 --2- 172.21.15.64:0/1108014264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9d4102ef0 0x7fa9d419bbd0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00d750 tx=0x7fa9cc00dc10 comp rx=0 tx=0).stop 2024-09-12T03:29:00.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:00.197+0000 7fa9dc7c5640 1 -- 172.21.15.64:0/1108014264 shutdown_connections 2024-09-12T03:29:00.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:00.197+0000 7fa9dc7c5640 1 --2- 172.21.15.64:0/1108014264 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fa9c00a2c90 0x7fa9c00a5150 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:00.197+0000 7fa9dc7c5640 1 --2- 172.21.15.64:0/1108014264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9d4102ef0 0x7fa9d419bbd0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:00.197+0000 7fa9dc7c5640 1 --2- 172.21.15.64:0/1108014264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9d4102570 0x7fa9d4197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:00.197+0000 7fa9dc7c5640 1 --2- 172.21.15.64:0/1108014264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9d4100e20 0x7fa9d4197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:00.197+0000 7fa9dc7c5640 1 -- 172.21.15.64:0/1108014264 >> 172.21.15.64:0/1108014264 conn(0x7fa9d40fc7d0 msgr2=0x7fa9d40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:00.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:00.197+0000 7fa9dc7c5640 1 -- 172.21.15.64:0/1108014264 shutdown_connections 2024-09-12T03:29:00.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:00.197+0000 7fa9dc7c5640 1 -- 172.21.15.64:0/1108014264 wait complete. 2024-09-12T03:29:00.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:00 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:29:00.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:00 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:29:00.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:00 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:29:00.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:00 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:29:00.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:00 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:29:00.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:00 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/1108014264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:29:00.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:00 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:29:00.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:00 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:29:00.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:00 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:29:00.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:00 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:29:00.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:00 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:29:00.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:00 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/1108014264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:29:00.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:00 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:29:00.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:00 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:29:00.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:00 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:29:00.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:00 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:29:00.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:00 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' 2024-09-12T03:29:00.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:00 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/1108014264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:29:01.035 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:29:01.297 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:29:01.576 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:01 smithi064 ceph-mon[30952]: pgmap v346: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:29:01.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:01 smithi115 ceph-mon[34801]: pgmap v346: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:29:01.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:01 smithi195 ceph-mon[34786]: pgmap v346: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:29:01.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.777+0000 7f8c7cbc1640 1 -- 172.21.15.64:0/840303862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c78102da0 msgr2=0x7f8c78103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:01.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.777+0000 7f8c7cbc1640 1 --2- 172.21.15.64:0/840303862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c78102da0 0x7f8c78103180 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8c68009900 tx=0x7f8c6802a480 comp rx=0 tx=0).stop 2024-09-12T03:29:01.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.777+0000 7f8c7cbc1640 1 -- 172.21.15.64:0/840303862 shutdown_connections 2024-09-12T03:29:01.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.777+0000 7f8c7cbc1640 1 --2- 172.21.15.64:0/840303862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8c7810c180 0x7f8c7810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:01.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.777+0000 7f8c7cbc1640 1 --2- 172.21.15.64:0/840303862 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c781036c0 0x7f8c7810bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:01.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.777+0000 7f8c7cbc1640 1 --2- 172.21.15.64:0/840303862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c78102da0 0x7f8c78103180 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:01.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.777+0000 7f8c7cbc1640 1 -- 172.21.15.64:0/840303862 >> 172.21.15.64:0/840303862 conn(0x7f8c780fc7a0 msgr2=0x7f8c780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:01.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.777+0000 7f8c7cbc1640 1 -- 172.21.15.64:0/840303862 shutdown_connections 2024-09-12T03:29:01.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.777+0000 7f8c7cbc1640 1 -- 172.21.15.64:0/840303862 wait complete. 2024-09-12T03:29:01.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.778+0000 7f8c7cbc1640 1 Processor -- start 2024-09-12T03:29:01.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.778+0000 7f8c7cbc1640 1 -- start start 2024-09-12T03:29:01.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.778+0000 7f8c7cbc1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8c78102da0 0x7f8c78198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:01.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.778+0000 7f8c7cbc1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c781036c0 0x7f8c781988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:01.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.778+0000 7f8c7cbc1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c7810c180 0x7f8c7819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:01.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.778+0000 7f8c7cbc1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8c78111e10 con 0x7f8c7810c180 2024-09-12T03:29:01.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.778+0000 7f8c7cbc1640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8c78111c90 con 0x7f8c781036c0 2024-09-12T03:29:01.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.778+0000 7f8c7cbc1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8c78111f90 con 0x7f8c78102da0 2024-09-12T03:29:01.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.779+0000 7f8c76d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c7810c180 0x7f8c7819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:01.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.779+0000 7f8c76d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c7810c180 0x7f8c7819cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35220/0 (socket says 172.21.15.64:35220) 2024-09-12T03:29:01.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.779+0000 7f8c76d9d640 1 -- 172.21.15.64:0/1738959741 learned_addr learned my addr 172.21.15.64:0/1738959741 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:01.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.779+0000 7f8c75d9b640 1 --2- 172.21.15.64:0/1738959741 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c781036c0 0x7f8c781988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:01.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.779+0000 7f8c7659c640 1 --2- 172.21.15.64:0/1738959741 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8c78102da0 0x7f8c78198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:01.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.779+0000 7f8c76d9d640 1 -- 172.21.15.64:0/1738959741 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8c78102da0 msgr2=0x7f8c78198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:01.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.779+0000 7f8c76d9d640 1 --2- 172.21.15.64:0/1738959741 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8c78102da0 0x7f8c78198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:01.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.779+0000 7f8c76d9d640 1 -- 172.21.15.64:0/1738959741 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c781036c0 msgr2=0x7f8c781988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:01.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.779+0000 7f8c76d9d640 1 --2- 172.21.15.64:0/1738959741 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c781036c0 0x7f8c781988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:01.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.779+0000 7f8c76d9d640 1 -- 172.21.15.64:0/1738959741 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c7819d340 con 0x7f8c7810c180 2024-09-12T03:29:01.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.779+0000 7f8c76d9d640 1 --2- 172.21.15.64:0/1738959741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c7810c180 0x7f8c7819cc60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8c7000eaa0 tx=0x7f8c7000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:01.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.779+0000 7f8c677fe640 1 -- 172.21.15.64:0/1738959741 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7000cdb0 con 0x7f8c7810c180 2024-09-12T03:29:01.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.779+0000 7f8c7cbc1640 1 -- 172.21.15.64:0/1738959741 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c7819d630 con 0x7f8c7810c180 2024-09-12T03:29:01.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.779+0000 7f8c677fe640 1 -- 172.21.15.64:0/1738959741 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c70005020 con 0x7f8c7810c180 2024-09-12T03:29:01.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.779+0000 7f8c677fe640 1 -- 172.21.15.64:0/1738959741 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c700116d0 con 0x7f8c7810c180 2024-09-12T03:29:01.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.779+0000 7f8c7cbc1640 1 -- 172.21.15.64:0/1738959741 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c781a4f10 con 0x7f8c7810c180 2024-09-12T03:29:01.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.780+0000 7f8c7cbc1640 1 -- 172.21.15.64:0/1738959741 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c78103180 con 0x7f8c7810c180 2024-09-12T03:29:01.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.782+0000 7f8c677fe640 1 -- 172.21.15.64:0/1738959741 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8c700051d0 con 0x7f8c7810c180 2024-09-12T03:29:01.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.782+0000 7f8c677fe640 1 --2- 172.21.15.64:0/1738959741 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f8c600a2c10 0x7f8c600a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:01.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.782+0000 7f8c7659c640 1 --2- 172.21.15.64:0/1738959741 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f8c600a2c10 0x7f8c600a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:01.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.783+0000 7f8c677fe640 1 -- 172.21.15.64:0/1738959741 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f8c700cf600 con 0x7f8c7810c180 2024-09-12T03:29:01.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.783+0000 7f8c7659c640 1 --2- 172.21.15.64:0/1738959741 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f8c600a2c10 0x7f8c600a50d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8c68031440 tx=0x7f8c68005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:01.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.787+0000 7f8c677fe640 1 -- 172.21.15.64:0/1738959741 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8c70016030 con 0x7f8c7810c180 2024-09-12T03:29:01.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.999+0000 7f8c7cbc1640 1 -- 172.21.15.64:0/1738959741 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c78107280 con 0x7f8c7810c180 2024-09-12T03:29:01.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:01.999+0000 7f8c677fe640 1 -- 172.21.15.64:0/1738959741 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11921 (secure 0 0 0) 0x7f8c7009ba50 con 0x7f8c7810c180 2024-09-12T03:29:02.000 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:29:02.000 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":24,"btime":"2024-09-12T03:27:37:139728+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2580078738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2580078738},{"type":"v1","addr":"172.21.15.64:6835","nonce":2580078738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2333415004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2333415004},{"type":"v1","addr":"172.21.15.64:6837","nonce":2333415004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/3030618395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":3030618395},{"type":"v1","addr":"172.21.15.64:6839","nonce":3030618395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3333057625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3333057625},{"type":"v1","addr":"172.21.15.64:6841","nonce":3333057625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2991933266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2991933266},{"type":"v1","addr":"172.21.15.195:6833","nonce":2991933266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1138960346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1138960346},{"type":"v1","addr":"172.21.15.115:6833","nonce":1138960346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:26:59.647657+0000","modified":"2024-09-12T03:27:36.865338+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24481,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/332737916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":332737916},{"type":"v1","addr":"172.21.15.115:6835","nonce":332737916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/2531045583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2531045583},{"type":"v1","addr":"172.21.15.195:6835","nonce":2531045583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/71647401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":71647401},{"type":"v1","addr":"172.21.15.195:6837","nonce":71647401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.115:6837/2915155705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2915155705},{"type":"v1","addr":"172.21.15.115:6837","nonce":2915155705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.195:6839/2574695134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":2574695134},{"type":"v1","addr":"172.21.15.195:6839","nonce":2574695134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6839/2238031796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2238031796},{"type":"v1","addr":"172.21.15.115:6839","nonce":2238031796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24470,24481,24500]},"id":1}]} 2024-09-12T03:29:02.000 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 24 2024-09-12T03:29:02.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:02.001+0000 7f8c7cbc1640 1 -- 172.21.15.64:0/1738959741 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f8c600a2c10 msgr2=0x7f8c600a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:02.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:02.001+0000 7f8c7cbc1640 1 --2- 172.21.15.64:0/1738959741 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f8c600a2c10 0x7f8c600a50d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8c68031440 tx=0x7f8c68005b50 comp rx=0 tx=0).stop 2024-09-12T03:29:02.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:02.001+0000 7f8c7cbc1640 1 -- 172.21.15.64:0/1738959741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c7810c180 msgr2=0x7f8c7819cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:02.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:02.001+0000 7f8c7cbc1640 1 --2- 172.21.15.64:0/1738959741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c7810c180 0x7f8c7819cc60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8c7000eaa0 tx=0x7f8c7000ef60 comp rx=0 tx=0).stop 2024-09-12T03:29:02.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:02.001+0000 7f8c7cbc1640 1 -- 172.21.15.64:0/1738959741 shutdown_connections 2024-09-12T03:29:02.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:02.001+0000 7f8c7cbc1640 1 --2- 172.21.15.64:0/1738959741 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f8c600a2c10 0x7f8c600a50d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:02.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:02.001+0000 7f8c7cbc1640 1 --2- 172.21.15.64:0/1738959741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c7810c180 0x7f8c7819cc60 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:02.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:02.001+0000 7f8c7cbc1640 1 --2- 172.21.15.64:0/1738959741 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c781036c0 0x7f8c781988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:02.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:02.001+0000 7f8c7cbc1640 1 --2- 172.21.15.64:0/1738959741 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8c78102da0 0x7f8c78198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:02.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:02.001+0000 7f8c7cbc1640 1 -- 172.21.15.64:0/1738959741 >> 172.21.15.64:0/1738959741 conn(0x7f8c780fc7a0 msgr2=0x7f8c780fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:02.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:02.001+0000 7f8c7cbc1640 1 -- 172.21.15.64:0/1738959741 shutdown_connections 2024-09-12T03:29:02.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:02.002+0000 7f8c7cbc1640 1 -- 172.21.15.64:0/1738959741 wait complete. 2024-09-12T03:29:02.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:02 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/1738959741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:29:02.692 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:02 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/1738959741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:29:02.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:02 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/1738959741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:29:02.725 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:29:02.987 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:29:03.379 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:03 smithi064 ceph-mon[30952]: pgmap v347: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:29:03.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.461+0000 7f967da44640 1 -- 172.21.15.64:0/4148475510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f967810c170 msgr2=0x7f967810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:03.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.461+0000 7f967da44640 1 --2- 172.21.15.64:0/4148475510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f967810c170 0x7f967810e610 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9670009ca0 tx=0x7f967002a950 comp rx=0 tx=0).stop 2024-09-12T03:29:03.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.461+0000 7f967da44640 1 -- 172.21.15.64:0/4148475510 shutdown_connections 2024-09-12T03:29:03.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.461+0000 7f967da44640 1 --2- 172.21.15.64:0/4148475510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f967810c170 0x7f967810e610 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:03.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.461+0000 7f967da44640 1 --2- 172.21.15.64:0/4148475510 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9678103660 0x7f967810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:03.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.461+0000 7f967da44640 1 --2- 172.21.15.64:0/4148475510 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9678102d40 0x7f9678103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:03.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.461+0000 7f967da44640 1 -- 172.21.15.64:0/4148475510 >> 172.21.15.64:0/4148475510 conn(0x7f96780fc7a0 msgr2=0x7f96780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:03.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.462+0000 7f967da44640 1 -- 172.21.15.64:0/4148475510 shutdown_connections 2024-09-12T03:29:03.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.462+0000 7f967da44640 1 -- 172.21.15.64:0/4148475510 wait complete. 2024-09-12T03:29:03.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.462+0000 7f967da44640 1 Processor -- start 2024-09-12T03:29:03.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.462+0000 7f967da44640 1 -- start start 2024-09-12T03:29:03.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.462+0000 7f967da44640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9678102d40 0x7f96781009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:03.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.463+0000 7f967da44640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9678103660 0x7f9678100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:03.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.463+0000 7f967da44640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f967810c170 0x7f9678101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:03.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.463+0000 7f967da44640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9678111e60 con 0x7f9678103660 2024-09-12T03:29:03.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.463+0000 7f967da44640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9678111ce0 con 0x7f9678102d40 2024-09-12T03:29:03.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.463+0000 7f967da44640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9678111fe0 con 0x7f967810c170 2024-09-12T03:29:03.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.463+0000 7f966ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9678103660 0x7f9678100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:03.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.463+0000 7f966ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9678103660 0x7f9678100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35236/0 (socket says 172.21.15.64:35236) 2024-09-12T03:29:03.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.463+0000 7f966ffff640 1 -- 172.21.15.64:0/2854721963 learned_addr learned my addr 172.21.15.64:0/2854721963 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:03.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.463+0000 7f9676ffd640 1 --2- 172.21.15.64:0/2854721963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9678102d40 0x7f96781009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:03.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.463+0000 7f96777fe640 1 --2- 172.21.15.64:0/2854721963 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f967810c170 0x7f9678101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:03.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.463+0000 7f966ffff640 1 -- 172.21.15.64:0/2854721963 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f967810c170 msgr2=0x7f9678101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:03.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.463+0000 7f966ffff640 1 --2- 172.21.15.64:0/2854721963 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f967810c170 0x7f9678101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:03.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.463+0000 7f966ffff640 1 -- 172.21.15.64:0/2854721963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9678102d40 msgr2=0x7f96781009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:03.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.463+0000 7f966ffff640 1 --2- 172.21.15.64:0/2854721963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9678102d40 0x7f96781009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:03.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.463+0000 7f966ffff640 1 -- 172.21.15.64:0/2854721963 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96781a6840 con 0x7f9678103660 2024-09-12T03:29:03.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.463+0000 7f966ffff640 1 --2- 172.21.15.64:0/2854721963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9678103660 0x7f9678100ee0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f966000b3f0 tx=0x7f966000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:03.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.463+0000 7f967ca42640 1 -- 172.21.15.64:0/2854721963 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9660015020 con 0x7f9678103660 2024-09-12T03:29:03.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.463+0000 7f967da44640 1 -- 172.21.15.64:0/2854721963 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96781a6b30 con 0x7f9678103660 2024-09-12T03:29:03.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.463+0000 7f967ca42640 1 -- 172.21.15.64:0/2854721963 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9660011ce0 con 0x7f9678103660 2024-09-12T03:29:03.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.463+0000 7f967ca42640 1 -- 172.21.15.64:0/2854721963 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9660010d80 con 0x7f9678103660 2024-09-12T03:29:03.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.463+0000 7f967da44640 1 -- 172.21.15.64:0/2854721963 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96781a7070 con 0x7f9678103660 2024-09-12T03:29:03.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.464+0000 7f967da44640 1 -- 172.21.15.64:0/2854721963 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9678103120 con 0x7f9678103660 2024-09-12T03:29:03.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.466+0000 7f967ca42640 1 -- 172.21.15.64:0/2854721963 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9660018020 con 0x7f9678103660 2024-09-12T03:29:03.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.467+0000 7f967ca42640 1 --2- 172.21.15.64:0/2854721963 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f96640a2c10 0x7f96640a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:03.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.467+0000 7f9676ffd640 1 --2- 172.21.15.64:0/2854721963 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f96640a2c10 0x7f96640a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:03.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.467+0000 7f967ca42640 1 -- 172.21.15.64:0/2854721963 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f96600cf1c0 con 0x7f9678103660 2024-09-12T03:29:03.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.467+0000 7f9676ffd640 1 --2- 172.21.15.64:0/2854721963 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f96640a2c10 0x7f96640a50d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f966800bbe0 tx=0x7f966800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:03.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.470+0000 7f967ca42640 1 -- 172.21.15.64:0/2854721963 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f966001f020 con 0x7f9678103660 2024-09-12T03:29:03.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:03 smithi115 ceph-mon[34801]: pgmap v347: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:29:03.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.648+0000 7f967da44640 1 -- 172.21.15.64:0/2854721963 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f967802d7c0 con 0x7f9678103660 2024-09-12T03:29:03.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.650+0000 7f967ca42640 1 -- 172.21.15.64:0/2854721963 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24680 (secure 0 0 0) 0x7f966009bd70 con 0x7f9678103660 2024-09-12T03:29:03.650 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:29:03.651 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":90,"fsid":"b0812460-70b5-11ef-bcea-c7b262605968","created":"2024-09-12T03:19:15.900322+0000","modified":"2024-09-12T03:28:08.835864+0000","last_up_change":"2024-09-12T03:26:06.122971+0000","last_in_change":"2024-09-12T03:25:43.130689+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:22:21.915870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:26:55.826348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:26:57.984319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T03:27:04.460292+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"326271c9-1f28-4ce6-b3be-fcc57479d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6803","nonce":1307832590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6805","nonce":1307832590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6809","nonce":1307832590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6807","nonce":1307832590}]},"public_addr":"172.21.15.64:6803/1307832590","cluster_addr":"172.21.15.64:6805/1307832590","heartbeat_back_addr":"172.21.15.64:6809/1307832590","heartbeat_front_addr":"172.21.15.64:6807/1307832590","state":["exists","up"]},{"osd":1,"uuid":"7dca0c28-c234-423c-936e-2351d63f918e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6801","nonce":3123728622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6803","nonce":3123728622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6807","nonce":3123728622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6805","nonce":3123728622}]},"public_addr":"172.21.15.115:6801/3123728622","cluster_addr":"172.21.15.115:6803/3123728622","heartbeat_back_addr":"172.21.15.115:6807/3123728622","heartbeat_front_addr":"172.21.15.115:6805/3123728622","state":["exists","up"]},{"osd":2,"uuid":"4b3a3c7a-aec6-455a-9a9a-1ac9cc900f24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6801","nonce":1664125451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6803","nonce":1664125451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6807","nonce":1664125451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6805","nonce":1664125451}]},"public_addr":"172.21.15.195:6801/1664125451","cluster_addr":"172.21.15.195:6803/1664125451","heartbeat_back_addr":"172.21.15.195:6807/1664125451","heartbeat_front_addr":"172.21.15.195:6805/1664125451","state":["exists","up"]},{"osd":3,"uuid":"8768c339-2c1e-4294-8b78-3c2adab6750f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6811","nonce":3935517185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6813","nonce":3935517185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6817","nonce":3935517185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6815","nonce":3935517185}]},"public_addr":"172.21.15.64:6811/3935517185","cluster_addr":"172.21.15.64:6813/3935517185","heartbeat_back_addr":"172.21.15.64:6817/3935517185","heartbeat_front_addr":"172.21.15.64:6815/3935517185","state":["exists","up"]},{"osd":4,"uuid":"123aac47-a8ae-47c7-93a7-0dd293356322","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6809","nonce":446665227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6811","nonce":446665227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6815","nonce":446665227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6813","nonce":446665227}]},"public_addr":"172.21.15.115:6809/446665227","cluster_addr":"172.21.15.115:6811/446665227","heartbeat_back_addr":"172.21.15.115:6815/446665227","heartbeat_front_addr":"172.21.15.115:6813/446665227","state":["exists","up"]},{"osd":5,"uuid":"c90bcaed-e7a8-4852-8677-b01dc73ea44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6809","nonce":1187293668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6811","nonce":1187293668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6815","nonce":1187293668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6813","nonce":1187293668}]},"public_addr":"172.21.15.195:6809/1187293668","cluster_addr":"172.21.15.195:6811/1187293668","heartbeat_back_addr":"172.21.15.195:6815/1187293668","heartbeat_front_addr":"172.21.15.195:6813/1187293668","state":["exists","up"]},{"osd":6,"uuid":"fd292547-6695-4889-8867-e74318bbb00f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6819","nonce":560343274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6821","nonce":560343274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6825","nonce":560343274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6823","nonce":560343274}]},"public_addr":"172.21.15.64:6819/560343274","cluster_addr":"172.21.15.64:6821/560343274","heartbeat_back_addr":"172.21.15.64:6825/560343274","heartbeat_front_addr":"172.21.15.64:6823/560343274","state":["exists","up"]},{"osd":7,"uuid":"ad7c4151-cd98-4113-879b-ca2b7c83c7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6817","nonce":3075734174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6819","nonce":3075734174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6823","nonce":3075734174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6821","nonce":3075734174}]},"public_addr":"172.21.15.115:6817/3075734174","cluster_addr":"172.21.15.115:6819/3075734174","heartbeat_back_addr":"172.21.15.115:6823/3075734174","heartbeat_front_addr":"172.21.15.115:6821/3075734174","state":["exists","up"]},{"osd":8,"uuid":"6de50049-c28f-4f21-b123-05e55bf59127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6817","nonce":2184742494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6819","nonce":2184742494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6823","nonce":2184742494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6821","nonce":2184742494}]},"public_addr":"172.21.15.195:6817/2184742494","cluster_addr":"172.21.15.195:6819/2184742494","heartbeat_back_addr":"172.21.15.195:6823/2184742494","heartbeat_front_addr":"172.21.15.195:6821/2184742494","state":["exists","up"]},{"osd":9,"uuid":"36ab792c-58f2-4beb-9024-7f3d1e3785dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6827","nonce":561728112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6829","nonce":561728112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6833","nonce":561728112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6831","nonce":561728112}]},"public_addr":"172.21.15.64:6827/561728112","cluster_addr":"172.21.15.64:6829/561728112","heartbeat_back_addr":"172.21.15.64:6833/561728112","heartbeat_front_addr":"172.21.15.64:6831/561728112","state":["exists","up"]},{"osd":10,"uuid":"d1f7881c-ec3f-466e-9cea-1d5cb441d970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6825","nonce":2977887627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6827","nonce":2977887627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6831","nonce":2977887627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6829","nonce":2977887627}]},"public_addr":"172.21.15.115:6825/2977887627","cluster_addr":"172.21.15.115:6827/2977887627","heartbeat_back_addr":"172.21.15.115:6831/2977887627","heartbeat_front_addr":"172.21.15.115:6829/2977887627","state":["exists","up"]},{"osd":11,"uuid":"7c46c028-22c7-450f-9927-2144a7a09e9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6825","nonce":591599352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6827","nonce":591599352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6831","nonce":591599352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6829","nonce":591599352}]},"public_addr":"172.21.15.195:6825/591599352","cluster_addr":"172.21.15.195:6827/591599352","heartbeat_back_addr":"172.21.15.195:6831/591599352","heartbeat_front_addr":"172.21.15.195:6829/591599352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:29.151454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:54.308909+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:18.680995+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:44.687548+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:10.166131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:35.026132+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:24:00.242966+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:24:24.600848+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:24:49.563060+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:25:15.025730+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:25:39.830995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:26:04.394276+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6801/68603379":"2024-09-13T03:20:07.411102+0000","172.21.15.64:0/3827202979":"2024-09-13T03:20:07.411102+0000","172.21.15.64:6800/68603379":"2024-09-13T03:20:07.411102+0000","172.21.15.64:0/1217136221":"2024-09-13T03:20:07.411102+0000","172.21.15.64:0/3079679964":"2024-09-13T03:19:36.839431+0000","172.21.15.64:6801/3186640984":"2024-09-13T03:19:36.839431+0000","172.21.15.64:0/3334629354":"2024-09-13T03:20:07.411102+0000","172.21.15.64:6800/3186640984":"2024-09-13T03:19:36.839431+0000","172.21.15.64:0/3564418115":"2024-09-13T03:19:36.839431+0000","172.21.15.64:0/686137382":"2024-09-13T03:19:36.839431+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:29:03.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.652+0000 7f967da44640 1 -- 172.21.15.64:0/2854721963 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f96640a2c10 msgr2=0x7f96640a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:03.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.652+0000 7f967da44640 1 --2- 172.21.15.64:0/2854721963 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f96640a2c10 0x7f96640a50d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f966800bbe0 tx=0x7f966800b390 comp rx=0 tx=0).stop 2024-09-12T03:29:03.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.652+0000 7f967da44640 1 -- 172.21.15.64:0/2854721963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9678103660 msgr2=0x7f9678100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:03.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.652+0000 7f967da44640 1 --2- 172.21.15.64:0/2854721963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9678103660 0x7f9678100ee0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f966000b3f0 tx=0x7f966000b8b0 comp rx=0 tx=0).stop 2024-09-12T03:29:03.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.652+0000 7f967da44640 1 -- 172.21.15.64:0/2854721963 shutdown_connections 2024-09-12T03:29:03.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.652+0000 7f967da44640 1 --2- 172.21.15.64:0/2854721963 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f96640a2c10 0x7f96640a50d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:03.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.652+0000 7f967da44640 1 --2- 172.21.15.64:0/2854721963 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f967810c170 0x7f9678101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:03.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.652+0000 7f967da44640 1 --2- 172.21.15.64:0/2854721963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9678103660 0x7f9678100ee0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:03.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.652+0000 7f967da44640 1 --2- 172.21.15.64:0/2854721963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9678102d40 0x7f96781009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:03.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.652+0000 7f967da44640 1 -- 172.21.15.64:0/2854721963 >> 172.21.15.64:0/2854721963 conn(0x7f96780fc7a0 msgr2=0x7f96780fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:03.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.652+0000 7f967da44640 1 -- 172.21.15.64:0/2854721963 shutdown_connections 2024-09-12T03:29:03.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:03.652+0000 7f967da44640 1 -- 172.21.15.64:0/2854721963 wait complete. 2024-09-12T03:29:03.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:03 smithi195 ceph-mon[34786]: pgmap v347: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:29:04.433 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:04 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/2854721963' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:29:04.465 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-12T03:26:59.647657+0000', 'modified': '2024-09-12T03:27:36.865338+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24500, 'mds_1': 24481, 'mds_2': 24470}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24449': {'gid': 24449, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.115:6835/332737916', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6834', 'nonce': 332737916}, {'type': 'v1', 'addr': '172.21.15.115:6835', 'nonce': 332737916}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.195:6835/2531045583', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6834', 'nonce': 2531045583}, {'type': 'v1', 'addr': '172.21.15.195:6835', 'nonce': 2531045583}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.195:6837/71647401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6836', 'nonce': 71647401}, {'type': 'v1', 'addr': '172.21.15.195:6837', 'nonce': 71647401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.115:6837/2915155705', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6836', 'nonce': 2915155705}, {'type': 'v1', 'addr': '172.21.15.115:6837', 'nonce': 2915155705}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.195:6839/2574695134', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6838', 'nonce': 2574695134}, {'type': 'v1', 'addr': '172.21.15.195:6839', 'nonce': 2574695134}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24500': {'gid': 24500, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.115:6839/2238031796', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6838', 'nonce': 2238031796}, {'type': 'v1', 'addr': '172.21.15.115:6839', 'nonce': 2238031796}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24500, 'qdb_cluster': [24470, 24481, 24500]} 2024-09-12T03:29:04.466 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-12T03:29:04.466 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-12T03:29:04.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:04 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/2854721963' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:29:04.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:04 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/2854721963' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:29:04.725 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:29:05.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.200+0000 7f0a777ca640 1 -- 172.21.15.64:0/569976467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a70102d30 msgr2=0x7f0a70103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:05.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.200+0000 7f0a777ca640 1 --2- 172.21.15.64:0/569976467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a70102d30 0x7f0a70103110 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c009930 tx=0x7f0a6c02a420 comp rx=0 tx=0).stop 2024-09-12T03:29:05.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.200+0000 7f0a777ca640 1 -- 172.21.15.64:0/569976467 shutdown_connections 2024-09-12T03:29:05.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.200+0000 7f0a777ca640 1 --2- 172.21.15.64:0/569976467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a701082a0 0x7f0a7010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.200+0000 7f0a777ca640 1 --2- 172.21.15.64:0/569976467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a701036e0 0x7f0a70107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.200+0000 7f0a777ca640 1 --2- 172.21.15.64:0/569976467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a70102d30 0x7f0a70103110 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.200+0000 7f0a777ca640 1 -- 172.21.15.64:0/569976467 >> 172.21.15.64:0/569976467 conn(0x7f0a700fc7d0 msgr2=0x7f0a700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:05.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.200+0000 7f0a777ca640 1 -- 172.21.15.64:0/569976467 shutdown_connections 2024-09-12T03:29:05.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.200+0000 7f0a777ca640 1 -- 172.21.15.64:0/569976467 wait complete. 2024-09-12T03:29:05.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.201+0000 7f0a777ca640 1 Processor -- start 2024-09-12T03:29:05.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.201+0000 7f0a777ca640 1 -- start start 2024-09-12T03:29:05.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.201+0000 7f0a777ca640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a70102d30 0x7f0a70198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:05.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.201+0000 7f0a777ca640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a701036e0 0x7f0a70198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:05.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.201+0000 7f0a777ca640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a701082a0 0x7f0a7019ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:05.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.201+0000 7f0a777ca640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0a70111f50 con 0x7f0a70102d30 2024-09-12T03:29:05.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.201+0000 7f0a777ca640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0a70111dd0 con 0x7f0a701082a0 2024-09-12T03:29:05.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.201+0000 7f0a777ca640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0a701120d0 con 0x7f0a701036e0 2024-09-12T03:29:05.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.201+0000 7f0a75566640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a70102d30 0x7f0a70198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:05.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.201+0000 7f0a75566640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a70102d30 0x7f0a70198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35262/0 (socket says 172.21.15.64:35262) 2024-09-12T03:29:05.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.201+0000 7f0a75566640 1 -- 172.21.15.64:0/457618284 learned_addr learned my addr 172.21.15.64:0/457618284 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:05.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.202+0000 7f0a75d67640 1 --2- 172.21.15.64:0/457618284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a701082a0 0x7f0a7019ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:05.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.202+0000 7f0a74d65640 1 --2- 172.21.15.64:0/457618284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a701036e0 0x7f0a70198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:05.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.202+0000 7f0a75566640 1 -- 172.21.15.64:0/457618284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a701036e0 msgr2=0x7f0a70198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:05.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.202+0000 7f0a75566640 1 --2- 172.21.15.64:0/457618284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a701036e0 0x7f0a70198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.202+0000 7f0a75566640 1 -- 172.21.15.64:0/457618284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a701082a0 msgr2=0x7f0a7019ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:05.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.202+0000 7f0a75566640 1 --2- 172.21.15.64:0/457618284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a701082a0 0x7f0a7019ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.202+0000 7f0a75566640 1 -- 172.21.15.64:0/457618284 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a7019d4e0 con 0x7f0a70102d30 2024-09-12T03:29:05.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.202+0000 7f0a75566640 1 --2- 172.21.15.64:0/457618284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a70102d30 0x7f0a70198530 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c004770 tx=0x7f0a6c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:05.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.202+0000 7f0a667fc640 1 -- 172.21.15.64:0/457618284 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c03f070 con 0x7f0a70102d30 2024-09-12T03:29:05.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.202+0000 7f0a777ca640 1 -- 172.21.15.64:0/457618284 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a7019d770 con 0x7f0a70102d30 2024-09-12T03:29:05.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.202+0000 7f0a667fc640 1 -- 172.21.15.64:0/457618284 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a6c03b460 con 0x7f0a70102d30 2024-09-12T03:29:05.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.202+0000 7f0a667fc640 1 -- 172.21.15.64:0/457618284 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c035070 con 0x7f0a70102d30 2024-09-12T03:29:05.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.202+0000 7f0a777ca640 1 -- 172.21.15.64:0/457618284 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a701a50b0 con 0x7f0a70102d30 2024-09-12T03:29:05.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.205+0000 7f0a79ba3740 1 Processor -- start 2024-09-12T03:29:05.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.206+0000 7f0a79ba3740 1 -- start start 2024-09-12T03:29:05.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.206+0000 7f0a667fc640 1 -- 172.21.15.64:0/457618284 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0a6c03b610 con 0x7f0a70102d30 2024-09-12T03:29:05.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.206+0000 7f0a79ba3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ef65998d0 0x564ef6599cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:05.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.206+0000 7f0a79ba3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ef659a1f0 0x564ef65a8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:05.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.206+0000 7f0a79ba3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef65a8fc0 0x564ef65ab3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:05.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.206+0000 7f0a79ba3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564ef6500090 con 0x564ef65a8fc0 2024-09-12T03:29:05.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.206+0000 7f0a79ba3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564ef64a7a70 con 0x564ef659a1f0 2024-09-12T03:29:05.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.206+0000 7f0a79ba3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564ef64fc050 con 0x564ef65998d0 2024-09-12T03:29:05.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.206+0000 7f0a74d65640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef65a8fc0 0x564ef65ab3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:05.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.206+0000 7f0a75d67640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ef65998d0 0x564ef6599cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:05.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.206+0000 7f0a74d65640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef65a8fc0 0x564ef65ab3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35268/0 (socket says 172.21.15.64:35268) 2024-09-12T03:29:05.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.206+0000 7f0a74d65640 1 -- 172.21.15.64:0/572165566 learned_addr learned my addr 172.21.15.64:0/572165566 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:05.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.206+0000 7f0a75566640 1 --2- 172.21.15.64:0/572165566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ef659a1f0 0x564ef65a8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:05.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.207+0000 7f0a75d67640 1 -- 172.21.15.64:0/572165566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ef659a1f0 msgr2=0x564ef65a8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:05.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.207+0000 7f0a75d67640 1 --2- 172.21.15.64:0/572165566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ef659a1f0 0x564ef65a8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.207+0000 7f0a75d67640 1 -- 172.21.15.64:0/572165566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef65a8fc0 msgr2=0x564ef65ab3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:05.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.207+0000 7f0a75d67640 1 --2- 172.21.15.64:0/572165566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef65a8fc0 0x564ef65ab3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.207+0000 7f0a75d67640 1 -- 172.21.15.64:0/572165566 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ef64b4130 con 0x564ef65998d0 2024-09-12T03:29:05.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.207+0000 7f0a667fc640 1 --2- 172.21.15.64:0/457618284 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f0a5c0a2c10 0x7f0a5c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:05.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.207+0000 7f0a75d67640 1 --2- 172.21.15.64:0/572165566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ef65998d0 0x564ef6599cb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0a6800ad50 tx=0x7f0a6800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8fe07387723d15f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:05.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.207+0000 7f0a75566640 1 --2- 172.21.15.64:0/457618284 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f0a5c0a2c10 0x7f0a5c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:05.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.207+0000 7f0a667fc640 1 -- 172.21.15.64:0/457618284 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f0a6c0ef810 con 0x7f0a70102d30 2024-09-12T03:29:05.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.207+0000 7f0a5b7fe640 1 -- 172.21.15.64:0/572165566 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a68003b70 con 0x564ef65998d0 2024-09-12T03:29:05.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.207+0000 7f0a5b7fe640 1 -- 172.21.15.64:0/572165566 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a6800ae40 con 0x564ef65998d0 2024-09-12T03:29:05.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.207+0000 7f0a5b7fe640 1 -- 172.21.15.64:0/572165566 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a680109c0 con 0x564ef65998d0 2024-09-12T03:29:05.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.208+0000 7f0a79ba3740 1 -- 172.21.15.64:0/572165566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ef65998d0 msgr2=0x564ef6599cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:05.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.208+0000 7f0a75566640 1 --2- 172.21.15.64:0/457618284 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f0a5c0a2c10 0x7f0a5c0a50d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x564ef659c5c0 tx=0x7f0a6c0f5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:05.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.208+0000 7f0a79ba3740 1 --2- 172.21.15.64:0/572165566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ef65998d0 0x564ef6599cb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0a6800ad50 tx=0x7f0a6800f790 comp rx=0 tx=0).stop 2024-09-12T03:29:05.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.208+0000 7f0a79ba3740 1 -- 172.21.15.64:0/572165566 shutdown_connections 2024-09-12T03:29:05.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.208+0000 7f0a79ba3740 1 --2- 172.21.15.64:0/572165566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef65a8fc0 0x564ef65ab3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.208+0000 7f0a79ba3740 1 --2- 172.21.15.64:0/572165566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ef659a1f0 0x564ef65a8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.208+0000 7f0a79ba3740 1 --2- 172.21.15.64:0/572165566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ef65998d0 0x564ef6599cb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.208+0000 7f0a79ba3740 1 -- 172.21.15.64:0/572165566 >> 172.21.15.64:0/572165566 conn(0x564ef6595f60 msgr2=0x564ef6596340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:05.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.208+0000 7f0a79ba3740 1 -- 172.21.15.64:0/572165566 shutdown_connections 2024-09-12T03:29:05.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.208+0000 7f0a79ba3740 1 -- 172.21.15.64:0/572165566 wait complete. 2024-09-12T03:29:05.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.208+0000 7f0a79ba3740 1 -- start start 2024-09-12T03:29:05.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.208+0000 7f0a79ba3740 1 Processor -- start 2024-09-12T03:29:05.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.209+0000 7f0a79ba3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ef65998d0 0x564ef65a87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:05.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.209+0000 7f0a79ba3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef659a1f0 0x564ef65a5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:05.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.209+0000 7f0a79ba3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ef65a8fc0 0x564ef65a6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:05.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.209+0000 7f0a79ba3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564ef65abbf0 con 0x564ef659a1f0 2024-09-12T03:29:05.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.209+0000 7f0a79ba3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564ef64baf20 con 0x564ef65998d0 2024-09-12T03:29:05.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.209+0000 7f0a79ba3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564ef65abd70 con 0x564ef65a8fc0 2024-09-12T03:29:05.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.209+0000 7f0a79ba3740 10 client.0 _ll_register_callbacks cb 0x564ef6484030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:05.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.209+0000 7f0a74d65640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef659a1f0 0x564ef65a5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:05.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.209+0000 7f0a74d65640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef659a1f0 0x564ef65a5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35272/0 (socket says 172.21.15.64:35272) 2024-09-12T03:29:05.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.209+0000 7f0a74d65640 1 -- 172.21.15.64:0/719670968 learned_addr learned my addr 172.21.15.64:0/719670968 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:05.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.209+0000 7f0a75d67640 1 --2- 172.21.15.64:0/719670968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ef65998d0 0x564ef65a87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:05.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.209+0000 7f0a75d67640 1 --2- 172.21.15.64:0/719670968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ef65a8fc0 0x564ef65a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:05.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.209+0000 7f0a74d65640 1 -- 172.21.15.64:0/719670968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ef65a8fc0 msgr2=0x564ef65a6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:05.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.209+0000 7f0a74d65640 1 --2- 172.21.15.64:0/719670968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ef65a8fc0 0x564ef65a6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.209+0000 7f0a74d65640 1 -- 172.21.15.64:0/719670968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ef65998d0 msgr2=0x564ef65a87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:05.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.209+0000 7f0a74d65640 1 --2- 172.21.15.64:0/719670968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ef65998d0 0x564ef65a87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.209+0000 7f0a74d65640 1 -- 172.21.15.64:0/719670968 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ef62712b0 con 0x564ef659a1f0 2024-09-12T03:29:05.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.209+0000 7f0a75d67640 1 --2- 172.21.15.64:0/719670968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ef65a8fc0 0x564ef65a6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:29:05.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.210+0000 7f0a74d65640 1 --2- 172.21.15.64:0/719670968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef659a1f0 0x564ef65a5b30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0a6000baa0 tx=0x7f0a6000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52140c1ffc044fe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:05.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.210+0000 7f0a58ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:29:05.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.210+0000 7f0a58ff9640 1 -- 172.21.15.64:0/719670968 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a60003df0 con 0x564ef659a1f0 2024-09-12T03:29:05.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.210+0000 7f0a58ff9640 1 -- 172.21.15.64:0/719670968 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f0a60011440 con 0x564ef659a1f0 2024-09-12T03:29:05.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.210+0000 7f0a79ba3740 1 -- 172.21.15.64:0/719670968 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564ef650d600 con 0x564ef659a1f0 2024-09-12T03:29:05.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.210+0000 7f0a58ff9640 1 -- 172.21.15.64:0/719670968 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0a60011640 con 0x564ef659a1f0 2024-09-12T03:29:05.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.210+0000 7f0a79ba3740 10 client.15177 fetch_fsmap learned FSMap version 24 2024-09-12T03:29:05.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.210+0000 7f0a79ba3740 1 -- 172.21.15.64:0/719670968 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564ef65b7540 con 0x564ef659a1f0 2024-09-12T03:29:05.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.210+0000 7f0a58ff9640 1 -- 172.21.15.64:0/719670968 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x564ef650d600 con 0x564ef659a1f0 2024-09-12T03:29:05.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.210+0000 7f0a58ff9640 1 -- 172.21.15.64:0/719670968 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f0a6001b920 con 0x564ef659a1f0 2024-09-12T03:29:05.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.210+0000 7f0a79ba3740 10 client.15177 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T03:29:05.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.211+0000 7f0a79ba3740 10 client.15177 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-12T03:29:05.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.211+0000 7f0a79ba3740 20 client.15177 populate_metadata read hostname 'smithi064' 2024-09-12T03:29:05.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.211+0000 7f0a79ba3740 1 --2- 172.21.15.64:0/719670968 >> [v2:172.21.15.115:6836/2915155705,v1:172.21.15.115:6837/2915155705] conn(0x564ef65b9bc0 0x564ef65bb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:05.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.211+0000 7f0a79ba3740 4 client.15177 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:29:05.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.211+0000 7f0a79ba3740 1 -- 172.21.15.64:0/719670968 --> [v2:172.21.15.115:6836/2915155705,v1:172.21.15.115:6837/2915155705] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ef6507b10 con 0x564ef65b9bc0 2024-09-12T03:29:05.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.211+0000 7f0a75d67640 1 --2- 172.21.15.64:0/719670968 >> [v2:172.21.15.115:6836/2915155705,v1:172.21.15.115:6837/2915155705] conn(0x564ef65b9bc0 0x564ef65bb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:05.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.211+0000 7f0a58ff9640 10 client.15177 ms_handle_connect on v2:172.21.15.115:6836/2915155705 2024-09-12T03:29:05.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.211+0000 7f0a75d67640 1 --2- 172.21.15.64:0/719670968 >> [v2:172.21.15.115:6836/2915155705,v1:172.21.15.115:6837/2915155705] conn(0x564ef65b9bc0 0x564ef65bb030 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c2995b7dba2902e0 server_cookie=12ab51e8906ffb82 in_seq=0 out_seq=0 2024-09-12T03:29:05.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.212+0000 7f0a58ff9640 1 -- 172.21.15.64:0/719670968 <== mds.2 v2:172.21.15.115:6836/2915155705 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564ef6507b10 con 0x564ef65b9bc0 2024-09-12T03:29:05.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.212+0000 7f0a58ff9640 10 client.15177 handle_command_reply: tid=0 2024-09-12T03:29:05.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.212+0000 7f0a79ba3740 1 client.15177 shutdown 2024-09-12T03:29:05.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.212+0000 7f0a79ba3740 1 -- 172.21.15.64:0/719670968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef659a1f0 msgr2=0x564ef65a5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:05.275 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T03:29:05.275 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "b0812460-70b5-11ef-bcea-c7b262605968", 2024-09-12T03:29:05.275 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 2, 2024-09-12T03:29:05.275 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24470, 2024-09-12T03:29:05.275 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-09-12T03:29:05.275 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-09-12T03:29:05.275 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-09-12T03:29:05.275 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 91.102885327999999, 2024-09-12T03:29:05.275 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 24, 2024-09-12T03:29:05.275 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 90, 2024-09-12T03:29:05.276 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 88, 2024-09-12T03:29:05.276 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 149.951466557 2024-09-12T03:29:05.276 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T03:29:05.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.212+0000 7f0a79ba3740 1 --2- 172.21.15.64:0/719670968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef659a1f0 0x564ef65a5b30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0a6000baa0 tx=0x7f0a6000bf60 comp rx=0 tx=0).stop 2024-09-12T03:29:05.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.212+0000 7f0a79ba3740 1 -- 172.21.15.64:0/719670968 shutdown_connections 2024-09-12T03:29:05.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.212+0000 7f0a79ba3740 1 --2- 172.21.15.64:0/719670968 >> [v2:172.21.15.115:6836/2915155705,v1:172.21.15.115:6837/2915155705] conn(0x564ef65b9bc0 0x564ef65bb030 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.212+0000 7f0a79ba3740 1 --2- 172.21.15.64:0/719670968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ef65a8fc0 0x564ef65a6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.212+0000 7f0a79ba3740 1 --2- 172.21.15.64:0/719670968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef659a1f0 0x564ef65a5b30 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.212+0000 7f0a79ba3740 1 --2- 172.21.15.64:0/719670968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ef65998d0 0x564ef65a87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.212+0000 7f0a79ba3740 1 -- 172.21.15.64:0/719670968 >> 172.21.15.64:0/719670968 conn(0x564ef6595f60 msgr2=0x564ef659bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:05.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.212+0000 7f0a58ff9640 0 client.15177 ms_handle_reset on v2:172.21.15.115:6836/2915155705 2024-09-12T03:29:05.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.212+0000 7f0a79ba3740 1 -- 172.21.15.64:0/719670968 shutdown_connections 2024-09-12T03:29:05.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.212+0000 7f0a79ba3740 1 -- 172.21.15.64:0/719670968 wait complete. 2024-09-12T03:29:05.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.212+0000 7f0a79ba3740 20 client.15177 trim_cache size 0 max 16384 2024-09-12T03:29:05.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.230+0000 7f0a79ba3740 1 Processor -- start 2024-09-12T03:29:05.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.230+0000 7f0a79ba3740 1 -- start start 2024-09-12T03:29:05.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.231+0000 7f0a79ba3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ef65998d0 0x564ef6590ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:05.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.231+0000 7f0a79ba3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef659a1f0 0x564ef65a2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:05.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.231+0000 7f0a79ba3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ef65a8fc0 0x564ef65a3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:05.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.231+0000 7f0a79ba3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564ef65b9370 con 0x564ef659a1f0 2024-09-12T03:29:05.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.231+0000 7f0a79ba3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564ef65b9a40 con 0x564ef65a8fc0 2024-09-12T03:29:05.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.231+0000 7f0a79ba3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564ef65ba110 con 0x564ef65998d0 2024-09-12T03:29:05.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.231+0000 7f0a74d65640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef659a1f0 0x564ef65a2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:05.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.231+0000 7f0a74d65640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef659a1f0 0x564ef65a2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35284/0 (socket says 172.21.15.64:35284) 2024-09-12T03:29:05.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.231+0000 7f0a74d65640 1 -- 172.21.15.64:0/726693237 learned_addr learned my addr 172.21.15.64:0/726693237 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:05.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.231+0000 7f0a75d67640 1 --2- 172.21.15.64:0/726693237 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ef65998d0 0x564ef6590ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:05.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.231+0000 7f0a75d67640 1 --2- 172.21.15.64:0/726693237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ef65a8fc0 0x564ef65a3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:05.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.231+0000 7f0a74d65640 1 -- 172.21.15.64:0/726693237 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ef65998d0 msgr2=0x564ef6590ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:05.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.231+0000 7f0a74d65640 1 --2- 172.21.15.64:0/726693237 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ef65998d0 0x564ef6590ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.231+0000 7f0a74d65640 1 -- 172.21.15.64:0/726693237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ef65a8fc0 msgr2=0x564ef65a3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:05.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.231+0000 7f0a74d65640 1 --2- 172.21.15.64:0/726693237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ef65a8fc0 0x564ef65a3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.231+0000 7f0a74d65640 1 -- 172.21.15.64:0/726693237 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ef6595a00 con 0x564ef659a1f0 2024-09-12T03:29:05.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.231+0000 7f0a74d65640 1 --2- 172.21.15.64:0/726693237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef659a1f0 0x564ef65a2df0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0a6001bd60 tx=0x7f0a60005d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=decec56b36ce889c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:05.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.231+0000 7f0a5affd640 1 -- 172.21.15.64:0/726693237 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a60003df0 con 0x564ef659a1f0 2024-09-12T03:29:05.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.232+0000 7f0a5affd640 1 -- 172.21.15.64:0/726693237 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a60010440 con 0x564ef659a1f0 2024-09-12T03:29:05.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.232+0000 7f0a5affd640 1 -- 172.21.15.64:0/726693237 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a60011d00 con 0x564ef659a1f0 2024-09-12T03:29:05.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.232+0000 7f0a79ba3740 1 -- 172.21.15.64:0/726693237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef659a1f0 msgr2=0x564ef65a2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:05.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.232+0000 7f0a79ba3740 1 --2- 172.21.15.64:0/726693237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef659a1f0 0x564ef65a2df0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0a6001bd60 tx=0x7f0a60005d00 comp rx=0 tx=0).stop 2024-09-12T03:29:05.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.232+0000 7f0a79ba3740 1 -- 172.21.15.64:0/726693237 shutdown_connections 2024-09-12T03:29:05.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.232+0000 7f0a79ba3740 1 --2- 172.21.15.64:0/726693237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ef65a8fc0 0x564ef65a3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.232+0000 7f0a79ba3740 1 --2- 172.21.15.64:0/726693237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef659a1f0 0x564ef65a2df0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.232+0000 7f0a79ba3740 1 --2- 172.21.15.64:0/726693237 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ef65998d0 0x564ef6590ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.232+0000 7f0a79ba3740 1 -- 172.21.15.64:0/726693237 >> 172.21.15.64:0/726693237 conn(0x564ef6595f60 msgr2=0x564ef65a87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:05.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.232+0000 7f0a79ba3740 1 -- 172.21.15.64:0/726693237 shutdown_connections 2024-09-12T03:29:05.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.232+0000 7f0a79ba3740 1 -- 172.21.15.64:0/726693237 wait complete. 2024-09-12T03:29:05.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.233+0000 7f0a79ba3740 1 -- start start 2024-09-12T03:29:05.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.233+0000 7f0a79ba3740 1 Processor -- start 2024-09-12T03:29:05.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.233+0000 7f0a79ba3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef65998d0 0x564ef65a21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:05.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.233+0000 7f0a79ba3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ef659a1f0 0x564ef65a2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:05.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.233+0000 7f0a79ba3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ef65a8fc0 0x564ef65c2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:05.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.233+0000 7f0a79ba3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564ef65baeb0 con 0x564ef65998d0 2024-09-12T03:29:05.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.233+0000 7f0a79ba3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564ef65ba7e0 con 0x564ef659a1f0 2024-09-12T03:29:05.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.233+0000 7f0a79ba3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564ef65bb580 con 0x564ef65a8fc0 2024-09-12T03:29:05.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.233+0000 7f0a79ba3740 10 client.0 _ll_register_callbacks cb 0x7f0a60004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:05.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.233+0000 7f0a75d67640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef65998d0 0x564ef65a21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:05.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.233+0000 7f0a75d67640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef65998d0 0x564ef65a21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35288/0 (socket says 172.21.15.64:35288) 2024-09-12T03:29:05.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.233+0000 7f0a75d67640 1 -- 172.21.15.64:0/992474909 learned_addr learned my addr 172.21.15.64:0/992474909 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:05.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.234+0000 7f0a74d65640 1 --2- 172.21.15.64:0/992474909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ef659a1f0 0x564ef65a2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:05.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.234+0000 7f0a75d67640 1 -- 172.21.15.64:0/992474909 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ef65a8fc0 msgr2=0x564ef65c2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:05.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.234+0000 7f0a75d67640 1 --2- 172.21.15.64:0/992474909 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ef65a8fc0 0x564ef65c2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.234+0000 7f0a75d67640 1 -- 172.21.15.64:0/992474909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ef659a1f0 msgr2=0x564ef65a2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:05.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.234+0000 7f0a75d67640 1 --2- 172.21.15.64:0/992474909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ef659a1f0 0x564ef65a2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.234+0000 7f0a75d67640 1 -- 172.21.15.64:0/992474909 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ef65c27c0 con 0x564ef65998d0 2024-09-12T03:29:05.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.234+0000 7f0a74d65640 1 --2- 172.21.15.64:0/992474909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ef659a1f0 0x564ef65a2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:29:05.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.234+0000 7f0a75d67640 1 --2- 172.21.15.64:0/992474909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef65998d0 0x564ef65a21c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0a68010a40 tx=0x7f0a6800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=356796c4f2191e06 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:05.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.234+0000 7f0a58ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:29:05.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.234+0000 7f0a58ff9640 1 -- 172.21.15.64:0/992474909 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a68010b70 con 0x564ef65998d0 2024-09-12T03:29:05.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.234+0000 7f0a58ff9640 1 -- 172.21.15.64:0/992474909 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f0a68004a00 con 0x564ef65998d0 2024-09-12T03:29:05.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.234+0000 7f0a79ba3740 1 -- 172.21.15.64:0/992474909 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a60010440 con 0x564ef65998d0 2024-09-12T03:29:05.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.234+0000 7f0a58ff9640 1 -- 172.21.15.64:0/992474909 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0a68039050 con 0x564ef65998d0 2024-09-12T03:29:05.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.234+0000 7f0a79ba3740 10 client.15183 fetch_fsmap learned FSMap version 24 2024-09-12T03:29:05.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.234+0000 7f0a79ba3740 1 -- 172.21.15.64:0/992474909 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564ef65c3ac0 con 0x564ef65998d0 2024-09-12T03:29:05.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.234+0000 7f0a58ff9640 1 -- 172.21.15.64:0/992474909 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a60010440 con 0x564ef65998d0 2024-09-12T03:29:05.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.235+0000 7f0a58ff9640 1 -- 172.21.15.64:0/992474909 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f0a680057d0 con 0x564ef65998d0 2024-09-12T03:29:05.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.235+0000 7f0a79ba3740 10 client.15183 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T03:29:05.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.235+0000 7f0a79ba3740 10 client.15183 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-12T03:29:05.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.235+0000 7f0a79ba3740 20 client.15183 populate_metadata read hostname 'smithi064' 2024-09-12T03:29:05.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.235+0000 7f0a79ba3740 1 --2- 172.21.15.64:0/992474909 >> [v2:172.21.15.115:6836/2915155705,v1:172.21.15.115:6837/2915155705] conn(0x564ef6628580 0x564ef6627290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:05.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.235+0000 7f0a79ba3740 4 client.15183 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:29:05.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.235+0000 7f0a79ba3740 1 -- 172.21.15.64:0/992474909 --> [v2:172.21.15.115:6836/2915155705,v1:172.21.15.115:6837/2915155705] -- command(tid 0: {"prefix": "status"}) -- 0x564ef64840e0 con 0x564ef6628580 2024-09-12T03:29:05.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.235+0000 7f0a74d65640 1 --2- 172.21.15.64:0/992474909 >> [v2:172.21.15.115:6836/2915155705,v1:172.21.15.115:6837/2915155705] conn(0x564ef6628580 0x564ef6627290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:05.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.236+0000 7f0a74d65640 1 --2- 172.21.15.64:0/992474909 >> [v2:172.21.15.115:6836/2915155705,v1:172.21.15.115:6837/2915155705] conn(0x564ef6628580 0x564ef6627290 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5b33eefdf05073a server_cookie=e9eb420d80d1c27b in_seq=0 out_seq=0 2024-09-12T03:29:05.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.236+0000 7f0a58ff9640 10 client.15183 ms_handle_connect on v2:172.21.15.115:6836/2915155705 2024-09-12T03:29:05.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.236+0000 7f0a58ff9640 1 -- 172.21.15.64:0/992474909 <== mds.2 v2:172.21.15.115:6836/2915155705 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x564ef64840e0 con 0x564ef6628580 2024-09-12T03:29:05.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.236+0000 7f0a58ff9640 10 client.15183 handle_command_reply: tid=0 2024-09-12T03:29:05.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.236+0000 7f0a79ba3740 1 client.15183 shutdown 2024-09-12T03:29:05.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.236+0000 7f0a79ba3740 1 -- 172.21.15.64:0/992474909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef65998d0 msgr2=0x564ef65a21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:05.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.236+0000 7f0a79ba3740 1 --2- 172.21.15.64:0/992474909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef65998d0 0x564ef65a21c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0a68010a40 tx=0x7f0a6800a710 comp rx=0 tx=0).stop 2024-09-12T03:29:05.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.236+0000 7f0a79ba3740 1 -- 172.21.15.64:0/992474909 shutdown_connections 2024-09-12T03:29:05.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.236+0000 7f0a79ba3740 1 --2- 172.21.15.64:0/992474909 >> [v2:172.21.15.115:6836/2915155705,v1:172.21.15.115:6837/2915155705] conn(0x564ef6628580 0x564ef6627290 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.236+0000 7f0a79ba3740 1 --2- 172.21.15.64:0/992474909 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ef65a8fc0 0x564ef65c2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.236+0000 7f0a79ba3740 1 --2- 172.21.15.64:0/992474909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ef659a1f0 0x564ef65a2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.236+0000 7f0a79ba3740 1 --2- 172.21.15.64:0/992474909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ef65998d0 0x564ef65a21c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.236+0000 7f0a79ba3740 1 -- 172.21.15.64:0/992474909 >> 172.21.15.64:0/992474909 conn(0x564ef6595f60 msgr2=0x564ef6590ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:05.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.236+0000 7f0a58ff9640 0 client.15183 ms_handle_reset on v2:172.21.15.115:6836/2915155705 2024-09-12T03:29:05.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.236+0000 7f0a79ba3740 1 -- 172.21.15.64:0/992474909 shutdown_connections 2024-09-12T03:29:05.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.236+0000 7f0a79ba3740 1 -- 172.21.15.64:0/992474909 wait complete. 2024-09-12T03:29:05.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.236+0000 7f0a79ba3740 20 client.15183 trim_cache size 0 max 16384 2024-09-12T03:29:05.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.237+0000 7f0a777ca640 1 -- 172.21.15.64:0/457618284 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f0a5c0a2c10 msgr2=0x7f0a5c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:05.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.237+0000 7f0a777ca640 1 --2- 172.21.15.64:0/457618284 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f0a5c0a2c10 0x7f0a5c0a50d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x564ef659c5c0 tx=0x7f0a6c0f5530 comp rx=0 tx=0).stop 2024-09-12T03:29:05.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.237+0000 7f0a777ca640 1 -- 172.21.15.64:0/457618284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a70102d30 msgr2=0x7f0a70198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:05.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.237+0000 7f0a777ca640 1 --2- 172.21.15.64:0/457618284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a70102d30 0x7f0a70198530 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c004770 tx=0x7f0a6c0042f0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.237+0000 7f0a777ca640 1 -- 172.21.15.64:0/457618284 shutdown_connections 2024-09-12T03:29:05.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.237+0000 7f0a777ca640 1 --2- 172.21.15.64:0/457618284 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f0a5c0a2c10 0x7f0a5c0a50d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.237+0000 7f0a777ca640 1 --2- 172.21.15.64:0/457618284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a701082a0 0x7f0a7019ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.237+0000 7f0a777ca640 1 --2- 172.21.15.64:0/457618284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a701036e0 0x7f0a70198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.237+0000 7f0a777ca640 1 --2- 172.21.15.64:0/457618284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a70102d30 0x7f0a70198530 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.237+0000 7f0a777ca640 1 -- 172.21.15.64:0/457618284 >> 172.21.15.64:0/457618284 conn(0x7f0a700fc7d0 msgr2=0x7f0a70105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:05.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.237+0000 7f0a777ca640 1 -- 172.21.15.64:0/457618284 shutdown_connections 2024-09-12T03:29:05.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:05.237+0000 7f0a777ca640 1 -- 172.21.15.64:0/457618284 wait complete. 2024-09-12T03:29:05.390 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:05 smithi064 ceph-mon[30952]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:29:05.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:05 smithi115 ceph-mon[34801]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:29:05.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:05 smithi195 ceph-mon[34786]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:29:06.075 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-12T03:29:06.343 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:29:06.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.823+0000 7ff8bd86e640 1 -- 172.21.15.64:0/2190384157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8b8106f10 msgr2=0x7ff8b810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:06.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.823+0000 7ff8bd86e640 1 --2- 172.21.15.64:0/2190384157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8b8106f10 0x7ff8b810ea90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0009ca0 tx=0x7ff8b002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:29:06.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.823+0000 7ff8bd86e640 1 -- 172.21.15.64:0/2190384157 shutdown_connections 2024-09-12T03:29:06.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.823+0000 7ff8bd86e640 1 --2- 172.21.15.64:0/2190384157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8b8106f10 0x7ff8b810ea90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:06.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.823+0000 7ff8bd86e640 1 --2- 172.21.15.64:0/2190384157 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8b8102790 0x7ff8b81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:06.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.823+0000 7ff8bd86e640 1 --2- 172.21.15.64:0/2190384157 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8b8101de0 0x7ff8b81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:06.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.823+0000 7ff8bd86e640 1 -- 172.21.15.64:0/2190384157 >> 172.21.15.64:0/2190384157 conn(0x7ff8b80fd8c0 msgr2=0x7ff8b80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:06.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.823+0000 7ff8bd86e640 1 -- 172.21.15.64:0/2190384157 shutdown_connections 2024-09-12T03:29:06.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.823+0000 7ff8bd86e640 1 -- 172.21.15.64:0/2190384157 wait complete. 2024-09-12T03:29:06.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.824+0000 7ff8bd86e640 1 Processor -- start 2024-09-12T03:29:06.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.824+0000 7ff8bd86e640 1 -- start start 2024-09-12T03:29:06.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.824+0000 7ff8bd86e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8b8101de0 0x7ff8b8198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:06.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.824+0000 7ff8bd86e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8b8102790 0x7ff8b81988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:06.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.824+0000 7ff8bd86e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8b8106f10 0x7ff8b819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:06.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.824+0000 7ff8bd86e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b8111e20 con 0x7ff8b8101de0 2024-09-12T03:29:06.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.824+0000 7ff8bd86e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b8111ca0 con 0x7ff8b8106f10 2024-09-12T03:29:06.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.824+0000 7ff8bd86e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b8111fa0 con 0x7ff8b8102790 2024-09-12T03:29:06.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.824+0000 7ff8b6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8b8101de0 0x7ff8b8198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:06.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.824+0000 7ff8b6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8b8101de0 0x7ff8b8198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35312/0 (socket says 172.21.15.64:35312) 2024-09-12T03:29:06.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.824+0000 7ff8b6ffd640 1 -- 172.21.15.64:0/4012789598 learned_addr learned my addr 172.21.15.64:0/4012789598 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:06.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.825+0000 7ff8b67fc640 1 --2- 172.21.15.64:0/4012789598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8b8102790 0x7ff8b81988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:06.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.825+0000 7ff8b77fe640 1 --2- 172.21.15.64:0/4012789598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8b8106f10 0x7ff8b819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:06.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.825+0000 7ff8b6ffd640 1 -- 172.21.15.64:0/4012789598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8b8102790 msgr2=0x7ff8b81988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:06.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.825+0000 7ff8b6ffd640 1 --2- 172.21.15.64:0/4012789598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8b8102790 0x7ff8b81988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:06.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.825+0000 7ff8b6ffd640 1 -- 172.21.15.64:0/4012789598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8b8106f10 msgr2=0x7ff8b819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:06.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.825+0000 7ff8b6ffd640 1 --2- 172.21.15.64:0/4012789598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8b8106f10 0x7ff8b819cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:06.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.825+0000 7ff8b6ffd640 1 -- 172.21.15.64:0/4012789598 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b819d310 con 0x7ff8b8101de0 2024-09-12T03:29:06.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.825+0000 7ff8b6ffd640 1 --2- 172.21.15.64:0/4012789598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8b8101de0 0x7ff8b8198360 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff8a800c890 tx=0x7ff8a800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:06.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.825+0000 7ff8bc86c640 1 -- 172.21.15.64:0/4012789598 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a8004220 con 0x7ff8b8101de0 2024-09-12T03:29:06.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.825+0000 7ff8bd86e640 1 -- 172.21.15.64:0/4012789598 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8b819d600 con 0x7ff8b8101de0 2024-09-12T03:29:06.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.825+0000 7ff8bc86c640 1 -- 172.21.15.64:0/4012789598 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8a80043c0 con 0x7ff8b8101de0 2024-09-12T03:29:06.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.825+0000 7ff8bc86c640 1 -- 172.21.15.64:0/4012789598 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a8004a00 con 0x7ff8b8101de0 2024-09-12T03:29:06.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.825+0000 7ff8bd86e640 1 -- 172.21.15.64:0/4012789598 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8b81a4ee0 con 0x7ff8b8101de0 2024-09-12T03:29:06.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.827+0000 7ff8bc86c640 1 -- 172.21.15.64:0/4012789598 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff8a8004ba0 con 0x7ff8b8101de0 2024-09-12T03:29:06.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.827+0000 7ff8bfc47740 1 Processor -- start 2024-09-12T03:29:06.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.828+0000 7ff8bfc47740 1 -- start start 2024-09-12T03:29:06.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.828+0000 7ff8bfc47740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da182d88d0 0x55da182d8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:06.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.828+0000 7ff8bfc47740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da182d91f0 0x55da182e7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:06.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.828+0000 7ff8bfc47740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da182e7fc0 0x55da182ea3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:06.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.828+0000 7ff8bfc47740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55da1823f090 con 0x55da182e7fc0 2024-09-12T03:29:06.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.828+0000 7ff8bfc47740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55da181e6a70 con 0x55da182d88d0 2024-09-12T03:29:06.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.828+0000 7ff8bfc47740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55da1823b050 con 0x55da182d91f0 2024-09-12T03:29:06.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.828+0000 7ff8b67fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da182e7fc0 0x55da182ea3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:06.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.828+0000 7ff8b67fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da182e7fc0 0x55da182ea3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35326/0 (socket says 172.21.15.64:35326) 2024-09-12T03:29:06.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.828+0000 7ff8b67fc640 1 -- 172.21.15.64:0/776866841 learned_addr learned my addr 172.21.15.64:0/776866841 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:06.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.828+0000 7ff8bc86c640 1 --2- 172.21.15.64:0/4012789598 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff8a00a2c10 0x7ff8a00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:06.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.828+0000 7ff8b77fe640 1 --2- 172.21.15.64:0/776866841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da182d88d0 0x55da182d8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:06.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.828+0000 7ff8b6ffd640 1 --2- 172.21.15.64:0/776866841 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da182d91f0 0x55da182e7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:06.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.828+0000 7ff8b77fe640 1 --2- 172.21.15.64:0/4012789598 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff8a00a2c10 0x7ff8a00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:06.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.828+0000 7ff8b67fc640 1 -- 172.21.15.64:0/776866841 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da182d91f0 msgr2=0x55da182e7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:06.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.828+0000 7ff8bc86c640 1 -- 172.21.15.64:0/4012789598 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff8a80cf2a0 con 0x7ff8b8101de0 2024-09-12T03:29:06.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.828+0000 7ff8b67fc640 1 --2- 172.21.15.64:0/776866841 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da182d91f0 0x55da182e7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:06.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.828+0000 7ff8b67fc640 1 -- 172.21.15.64:0/776866841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da182d88d0 msgr2=0x55da182d8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:06.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.828+0000 7ff8b67fc640 1 --2- 172.21.15.64:0/776866841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da182d88d0 0x55da182d8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:06.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.828+0000 7ff8b67fc640 1 -- 172.21.15.64:0/776866841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da181f3130 con 0x55da182e7fc0 2024-09-12T03:29:06.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.828+0000 7ff8b6ffd640 1 --2- 172.21.15.64:0/776866841 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da182d91f0 0x55da182e7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:29:06.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.828+0000 7ff8b77fe640 1 --2- 172.21.15.64:0/776866841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da182d88d0 0x55da182d8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:29:06.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.829+0000 7ff8b67fc640 1 --2- 172.21.15.64:0/776866841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da182e7fc0 0x55da182ea3b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac00bc10 tx=0x7ff8ac00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aec54705787aff1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:06.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.829+0000 7ff89dffb640 1 -- 172.21.15.64:0/776866841 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac015070 con 0x55da182e7fc0 2024-09-12T03:29:06.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.829+0000 7ff8b77fe640 1 --2- 172.21.15.64:0/4012789598 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff8a00a2c10 0x7ff8a00a50d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0009430 tx=0x7ff8b00095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:06.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.829+0000 7ff89dffb640 1 -- 172.21.15.64:0/776866841 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8ac0118f0 con 0x55da182e7fc0 2024-09-12T03:29:06.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.829+0000 7ff89dffb640 1 -- 172.21.15.64:0/776866841 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac010730 con 0x55da182e7fc0 2024-09-12T03:29:06.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.829+0000 7ff8bfc47740 1 -- 172.21.15.64:0/776866841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da182e7fc0 msgr2=0x55da182ea3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:06.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.829+0000 7ff8bfc47740 1 --2- 172.21.15.64:0/776866841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da182e7fc0 0x55da182ea3b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac00bc10 tx=0x7ff8ac00f730 comp rx=0 tx=0).stop 2024-09-12T03:29:06.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.830+0000 7ff8bfc47740 1 -- 172.21.15.64:0/776866841 shutdown_connections 2024-09-12T03:29:06.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.830+0000 7ff8bfc47740 1 --2- 172.21.15.64:0/776866841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da182e7fc0 0x55da182ea3b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:06.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.830+0000 7ff8bfc47740 1 --2- 172.21.15.64:0/776866841 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da182d91f0 0x55da182e7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:06.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.830+0000 7ff8bfc47740 1 --2- 172.21.15.64:0/776866841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da182d88d0 0x55da182d8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:06.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.830+0000 7ff8bfc47740 1 -- 172.21.15.64:0/776866841 >> 172.21.15.64:0/776866841 conn(0x55da182d4f60 msgr2=0x55da182d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:06.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.830+0000 7ff8bfc47740 1 -- 172.21.15.64:0/776866841 shutdown_connections 2024-09-12T03:29:06.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.830+0000 7ff8bfc47740 1 -- 172.21.15.64:0/776866841 wait complete. 2024-09-12T03:29:06.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.830+0000 7ff8bfc47740 1 -- start start 2024-09-12T03:29:06.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.830+0000 7ff8bfc47740 1 Processor -- start 2024-09-12T03:29:06.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.830+0000 7ff8bfc47740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da182d88d0 0x55da182e7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:06.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.830+0000 7ff8bfc47740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da182d91f0 0x55da182e4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:06.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.830+0000 7ff8bfc47740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da182e7fc0 0x55da182e50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:06.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.830+0000 7ff8bfc47740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55da182eabf0 con 0x55da182d91f0 2024-09-12T03:29:06.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.830+0000 7ff8bfc47740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55da181f9f20 con 0x55da182e7fc0 2024-09-12T03:29:06.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.830+0000 7ff8bfc47740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55da182ead70 con 0x55da182d88d0 2024-09-12T03:29:06.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.830+0000 7ff8bfc47740 10 client.0 _ll_register_callbacks cb 0x55da181c3030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:06.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.831+0000 7ff8b67fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da182d91f0 0x55da182e4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:06.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.831+0000 7ff8b67fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da182d91f0 0x55da182e4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35338/0 (socket says 172.21.15.64:35338) 2024-09-12T03:29:06.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.831+0000 7ff8b67fc640 1 -- 172.21.15.64:0/1398343526 learned_addr learned my addr 172.21.15.64:0/1398343526 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:06.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.831+0000 7ff8b6ffd640 1 --2- 172.21.15.64:0/1398343526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da182d88d0 0x55da182e7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:06.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.831+0000 7ff8b77fe640 1 --2- 172.21.15.64:0/1398343526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da182e7fc0 0x55da182e50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:06.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.831+0000 7ff8b67fc640 1 -- 172.21.15.64:0/1398343526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da182d88d0 msgr2=0x55da182e7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:06.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.831+0000 7ff8b67fc640 1 --2- 172.21.15.64:0/1398343526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da182d88d0 0x55da182e7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:06.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.831+0000 7ff8b67fc640 1 -- 172.21.15.64:0/1398343526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da182e7fc0 msgr2=0x55da182e50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:06.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.831+0000 7ff8b67fc640 1 --2- 172.21.15.64:0/1398343526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da182e7fc0 0x55da182e50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:06.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.831+0000 7ff8b67fc640 1 -- 172.21.15.64:0/1398343526 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da17fb02b0 con 0x55da182d91f0 2024-09-12T03:29:06.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.831+0000 7ff8b67fc640 1 --2- 172.21.15.64:0/1398343526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da182d91f0 0x55da182e4bb0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac00b760 tx=0x7ff8ac011670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eea7541f9c6e368a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:06.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.831+0000 7ff896ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:29:06.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.831+0000 7ff896ffd640 1 -- 172.21.15.64:0/1398343526 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac015070 con 0x55da182d91f0 2024-09-12T03:29:06.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.831+0000 7ff896ffd640 1 -- 172.21.15.64:0/1398343526 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff8ac011cb0 con 0x55da182d91f0 2024-09-12T03:29:06.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.831+0000 7ff8bfc47740 1 -- 172.21.15.64:0/1398343526 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8ac0118f0 con 0x55da182d91f0 2024-09-12T03:29:06.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.831+0000 7ff896ffd640 1 -- 172.21.15.64:0/1398343526 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff8ac02e050 con 0x55da182d91f0 2024-09-12T03:29:06.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.831+0000 7ff8bfc47740 10 client.15195 fetch_fsmap learned FSMap version 24 2024-09-12T03:29:06.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.831+0000 7ff8bfc47740 1 -- 172.21.15.64:0/1398343526 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55da182f6550 con 0x55da182d91f0 2024-09-12T03:29:06.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.832+0000 7ff896ffd640 1 -- 172.21.15.64:0/1398343526 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8ac0118f0 con 0x55da182d91f0 2024-09-12T03:29:06.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.832+0000 7ff896ffd640 1 -- 172.21.15.64:0/1398343526 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7ff8ac004940 con 0x55da182d91f0 2024-09-12T03:29:06.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.832+0000 7ff8bfc47740 10 client.15195 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T03:29:06.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.832+0000 7ff8bfc47740 10 client.15195 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T03:29:06.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.832+0000 7ff8bfc47740 20 client.15195 populate_metadata read hostname 'smithi064' 2024-09-12T03:29:06.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.832+0000 7ff8bfc47740 1 --2- 172.21.15.64:0/1398343526 >> [v2:172.21.15.195:6838/2574695134,v1:172.21.15.195:6839/2574695134] conn(0x55da182f8850 0x55da182f9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:06.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.832+0000 7ff8bfc47740 4 client.15195 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:29:06.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.832+0000 7ff8bfc47740 1 -- 172.21.15.64:0/1398343526 --> [v2:172.21.15.195:6838/2574695134,v1:172.21.15.195:6839/2574695134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da1824c600 con 0x55da182f8850 2024-09-12T03:29:06.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.832+0000 7ff8b6ffd640 1 --2- 172.21.15.64:0/1398343526 >> [v2:172.21.15.195:6838/2574695134,v1:172.21.15.195:6839/2574695134] conn(0x55da182f8850 0x55da182f9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:06.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.833+0000 7ff8b6ffd640 1 --2- 172.21.15.64:0/1398343526 >> [v2:172.21.15.195:6838/2574695134,v1:172.21.15.195:6839/2574695134] conn(0x55da182f8850 0x55da182f9cc0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=67b95d4b56540b51 server_cookie=c927bf217c4f880 in_seq=0 out_seq=0 2024-09-12T03:29:06.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.833+0000 7ff896ffd640 10 client.15195 ms_handle_connect on v2:172.21.15.195:6838/2574695134 2024-09-12T03:29:06.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.833+0000 7ff896ffd640 1 -- 172.21.15.64:0/1398343526 <== mds.1 v2:172.21.15.195:6838/2574695134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55da1824c600 con 0x55da182f8850 2024-09-12T03:29:06.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.834+0000 7ff896ffd640 10 client.15195 handle_command_reply: tid=0 2024-09-12T03:29:06.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.834+0000 7ff8bfc47740 1 client.15195 shutdown 2024-09-12T03:29:06.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.834+0000 7ff8bfc47740 1 -- 172.21.15.64:0/1398343526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da182d91f0 msgr2=0x55da182e4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:06.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.834+0000 7ff8bfc47740 1 --2- 172.21.15.64:0/1398343526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da182d91f0 0x55da182e4bb0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac00b760 tx=0x7ff8ac011670 comp rx=0 tx=0).stop 2024-09-12T03:29:06.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.834+0000 7ff8bfc47740 1 -- 172.21.15.64:0/1398343526 shutdown_connections 2024-09-12T03:29:06.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.834+0000 7ff8bfc47740 1 --2- 172.21.15.64:0/1398343526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da182e7fc0 0x55da182e50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:06.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.834+0000 7ff8bfc47740 1 --2- 172.21.15.64:0/1398343526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da182d91f0 0x55da182e4bb0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:06.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.834+0000 7ff8bfc47740 1 --2- 172.21.15.64:0/1398343526 >> [v2:172.21.15.195:6838/2574695134,v1:172.21.15.195:6839/2574695134] conn(0x55da182f8850 0x55da182f9cc0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:06.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.834+0000 7ff8bfc47740 1 --2- 172.21.15.64:0/1398343526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da182d88d0 0x55da182e7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:06.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.834+0000 7ff8bfc47740 1 -- 172.21.15.64:0/1398343526 >> 172.21.15.64:0/1398343526 conn(0x55da182d4f60 msgr2=0x55da182dacc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:06.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.834+0000 7ff896ffd640 0 client.15195 ms_handle_reset on v2:172.21.15.195:6838/2574695134 2024-09-12T03:29:06.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.834+0000 7ff8bfc47740 1 -- 172.21.15.64:0/1398343526 shutdown_connections 2024-09-12T03:29:06.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.834+0000 7ff8bfc47740 1 -- 172.21.15.64:0/1398343526 wait complete. 2024-09-12T03:29:06.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.834+0000 7ff8bfc47740 20 client.15195 trim_cache size 0 max 16384 2024-09-12T03:29:06.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.852+0000 7ff8bfc47740 1 Processor -- start 2024-09-12T03:29:06.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.852+0000 7ff8bfc47740 1 -- start start 2024-09-12T03:29:06.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.852+0000 7ff8bfc47740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da182d88d0 0x55da182f9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:06.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.852+0000 7ff8bfc47740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da182d91f0 0x55da182e0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:06.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.852+0000 7ff8bfc47740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da182e7fc0 0x55da182e1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:06.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.852+0000 7ff8bfc47740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55da182f8000 con 0x55da182d91f0 2024-09-12T03:29:06.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.852+0000 7ff8bfc47740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55da182f86d0 con 0x55da182e7fc0 2024-09-12T03:29:06.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.852+0000 7ff8bfc47740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55da182f8da0 con 0x55da182d88d0 2024-09-12T03:29:06.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.852+0000 7ff8b67fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da182d91f0 0x55da182e0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:06.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.853+0000 7ff8b67fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da182d91f0 0x55da182e0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35342/0 (socket says 172.21.15.64:35342) 2024-09-12T03:29:06.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.853+0000 7ff8b67fc640 1 -- 172.21.15.64:0/3655276855 learned_addr learned my addr 172.21.15.64:0/3655276855 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:06.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.853+0000 7ff8b77fe640 1 --2- 172.21.15.64:0/3655276855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da182e7fc0 0x55da182e1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:06.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.853+0000 7ff8b6ffd640 1 --2- 172.21.15.64:0/3655276855 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da182d88d0 0x55da182f9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:06.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.853+0000 7ff8b67fc640 1 -- 172.21.15.64:0/3655276855 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da182d88d0 msgr2=0x55da182f9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:06.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.853+0000 7ff8b67fc640 1 --2- 172.21.15.64:0/3655276855 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da182d88d0 0x55da182f9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:06.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.853+0000 7ff8b67fc640 1 -- 172.21.15.64:0/3655276855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da182e7fc0 msgr2=0x55da182e1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:06.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.853+0000 7ff8b67fc640 1 --2- 172.21.15.64:0/3655276855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da182e7fc0 0x55da182e1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:06.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.853+0000 7ff8b67fc640 1 -- 172.21.15.64:0/3655276855 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da182d9020 con 0x55da182d91f0 2024-09-12T03:29:06.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.853+0000 7ff8b6ffd640 1 --2- 172.21.15.64:0/3655276855 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da182d88d0 0x55da182f9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:29:06.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.853+0000 7ff8b67fc640 1 --2- 172.21.15.64:0/3655276855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da182d91f0 0x55da182e0130 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac004e80 tx=0x7ff8ac00b3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6a41c0f45f33da server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:06.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.853+0000 7ff89d7fa640 1 -- 172.21.15.64:0/3655276855 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac015030 con 0x55da182d91f0 2024-09-12T03:29:06.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.853+0000 7ff89d7fa640 1 -- 172.21.15.64:0/3655276855 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8ac0103d0 con 0x55da182d91f0 2024-09-12T03:29:06.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.853+0000 7ff89d7fa640 1 -- 172.21.15.64:0/3655276855 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac0048e0 con 0x55da182d91f0 2024-09-12T03:29:06.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.854+0000 7ff8bfc47740 1 -- 172.21.15.64:0/3655276855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da182d91f0 msgr2=0x55da182e0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:06.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.854+0000 7ff8bfc47740 1 --2- 172.21.15.64:0/3655276855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da182d91f0 0x55da182e0130 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac004e80 tx=0x7ff8ac00b3a0 comp rx=0 tx=0).stop 2024-09-12T03:29:06.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.854+0000 7ff8bfc47740 1 -- 172.21.15.64:0/3655276855 shutdown_connections 2024-09-12T03:29:06.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.854+0000 7ff8bfc47740 1 --2- 172.21.15.64:0/3655276855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da182e7fc0 0x55da182e1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:06.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.854+0000 7ff8bfc47740 1 --2- 172.21.15.64:0/3655276855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da182d91f0 0x55da182e0130 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:06.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.854+0000 7ff8bfc47740 1 --2- 172.21.15.64:0/3655276855 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da182d88d0 0x55da182f9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:06.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.854+0000 7ff8bfc47740 1 -- 172.21.15.64:0/3655276855 >> 172.21.15.64:0/3655276855 conn(0x55da182d4f60 msgr2=0x55da182dfa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:06.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.854+0000 7ff8bfc47740 1 -- 172.21.15.64:0/3655276855 shutdown_connections 2024-09-12T03:29:06.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.854+0000 7ff8bfc47740 1 -- 172.21.15.64:0/3655276855 wait complete. 2024-09-12T03:29:06.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.854+0000 7ff8bfc47740 1 -- start start 2024-09-12T03:29:06.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.854+0000 7ff8bfc47740 1 Processor -- start 2024-09-12T03:29:06.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.855+0000 7ff8bfc47740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da182d88d0 0x55da18302100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:06.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.855+0000 7ff8bfc47740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da182d91f0 0x55da18302640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:06.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.855+0000 7ff8bfc47740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da182e7fc0 0x55da182fb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:06.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.855+0000 7ff8bfc47740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55da182f9b40 con 0x55da182d88d0 2024-09-12T03:29:06.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.855+0000 7ff8bfc47740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55da182f9470 con 0x55da182d91f0 2024-09-12T03:29:06.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.855+0000 7ff8bfc47740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55da182fa210 con 0x55da182e7fc0 2024-09-12T03:29:06.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.855+0000 7ff8bfc47740 10 client.0 _ll_register_callbacks cb 0x7ff8ac00ba60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:06.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.855+0000 7ff8b6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da182d88d0 0x55da18302100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:06.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.855+0000 7ff8b6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da182d88d0 0x55da18302100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35354/0 (socket says 172.21.15.64:35354) 2024-09-12T03:29:06.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.855+0000 7ff8b6ffd640 1 -- 172.21.15.64:0/3628131052 learned_addr learned my addr 172.21.15.64:0/3628131052 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:06.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.855+0000 7ff8b67fc640 1 --2- 172.21.15.64:0/3628131052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da182d91f0 0x55da18302640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:06.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.855+0000 7ff8b77fe640 1 --2- 172.21.15.64:0/3628131052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da182e7fc0 0x55da182fb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:06.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.855+0000 7ff8b6ffd640 1 -- 172.21.15.64:0/3628131052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da182e7fc0 msgr2=0x55da182fb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:06.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.855+0000 7ff8b6ffd640 1 --2- 172.21.15.64:0/3628131052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da182e7fc0 0x55da182fb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:06.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.855+0000 7ff8b6ffd640 1 -- 172.21.15.64:0/3628131052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da182d91f0 msgr2=0x55da18302640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:06.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.855+0000 7ff8b6ffd640 1 --2- 172.21.15.64:0/3628131052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da182d91f0 0x55da18302640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:06.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.855+0000 7ff8b6ffd640 1 -- 172.21.15.64:0/3628131052 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da182fbc90 con 0x55da182d88d0 2024-09-12T03:29:06.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.855+0000 7ff8b77fe640 1 --2- 172.21.15.64:0/3628131052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da182e7fc0 0x55da182fb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:29:06.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.855+0000 7ff8b6ffd640 1 --2- 172.21.15.64:0/3628131052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da182d88d0 0x55da18302100 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff8a809b340 tx=0x7ff8a806b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62cbfdec71e8d7cd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:06.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.855+0000 7ff896ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:29:06.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.856+0000 7ff896ffd640 1 -- 172.21.15.64:0/3628131052 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a801d040 con 0x55da182d88d0 2024-09-12T03:29:06.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.856+0000 7ff896ffd640 1 -- 172.21.15.64:0/3628131052 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff8a80050b0 con 0x55da182d88d0 2024-09-12T03:29:06.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.856+0000 7ff8bfc47740 1 -- 172.21.15.64:0/3628131052 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8ac0103d0 con 0x55da182d88d0 2024-09-12T03:29:06.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.856+0000 7ff896ffd640 1 -- 172.21.15.64:0/3628131052 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff8a8066720 con 0x55da182d88d0 2024-09-12T03:29:06.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.856+0000 7ff8bfc47740 10 client.15201 fetch_fsmap learned FSMap version 24 2024-09-12T03:29:06.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.856+0000 7ff8bfc47740 1 -- 172.21.15.64:0/3628131052 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55da18300690 con 0x55da182d88d0 2024-09-12T03:29:06.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.856+0000 7ff896ffd640 1 -- 172.21.15.64:0/3628131052 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8ac0103d0 con 0x55da182d88d0 2024-09-12T03:29:06.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.856+0000 7ff896ffd640 1 -- 172.21.15.64:0/3628131052 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7ff8a8097cb0 con 0x55da182d88d0 2024-09-12T03:29:06.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.856+0000 7ff8bfc47740 10 client.15201 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T03:29:06.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.856+0000 7ff8bfc47740 10 client.15201 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T03:29:06.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.856+0000 7ff8bfc47740 20 client.15201 populate_metadata read hostname 'smithi064' 2024-09-12T03:29:06.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.856+0000 7ff8bfc47740 1 --2- 172.21.15.64:0/3628131052 >> [v2:172.21.15.195:6838/2574695134,v1:172.21.15.195:6839/2574695134] conn(0x55da18367570 0x55da18366280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:06.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.856+0000 7ff8bfc47740 4 client.15201 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:29:06.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.856+0000 7ff8bfc47740 1 -- 172.21.15.64:0/3628131052 --> [v2:172.21.15.195:6838/2574695134,v1:172.21.15.195:6839/2574695134] -- command(tid 0: {"prefix": "status"}) -- 0x55da18246b10 con 0x55da18367570 2024-09-12T03:29:06.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.856+0000 7ff8b67fc640 1 --2- 172.21.15.64:0/3628131052 >> [v2:172.21.15.195:6838/2574695134,v1:172.21.15.195:6839/2574695134] conn(0x55da18367570 0x55da18366280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:06.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.857+0000 7ff8b67fc640 1 --2- 172.21.15.64:0/3628131052 >> [v2:172.21.15.195:6838/2574695134,v1:172.21.15.195:6839/2574695134] conn(0x55da18367570 0x55da18366280 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=abbc4e2e0072e0a5 server_cookie=274b65ea70f320e4 in_seq=0 out_seq=0 2024-09-12T03:29:06.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.857+0000 7ff896ffd640 10 client.15201 ms_handle_connect on v2:172.21.15.195:6838/2574695134 2024-09-12T03:29:06.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.857+0000 7ff896ffd640 1 -- 172.21.15.64:0/3628131052 <== mds.1 v2:172.21.15.195:6838/2574695134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55da18246b10 con 0x55da18367570 2024-09-12T03:29:06.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.857+0000 7ff896ffd640 10 client.15201 handle_command_reply: tid=0 2024-09-12T03:29:06.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.857+0000 7ff8bfc47740 1 client.15201 shutdown 2024-09-12T03:29:06.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.857+0000 7ff8bfc47740 1 -- 172.21.15.64:0/3628131052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da182d88d0 msgr2=0x55da18302100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:06.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.857+0000 7ff8bfc47740 1 --2- 172.21.15.64:0/3628131052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da182d88d0 0x55da18302100 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff8a809b340 tx=0x7ff8a806b4f0 comp rx=0 tx=0).stop 2024-09-12T03:29:06.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.857+0000 7ff8bfc47740 1 -- 172.21.15.64:0/3628131052 shutdown_connections 2024-09-12T03:29:06.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.857+0000 7ff8bfc47740 1 --2- 172.21.15.64:0/3628131052 >> [v2:172.21.15.195:6838/2574695134,v1:172.21.15.195:6839/2574695134] conn(0x55da18367570 0x55da18366280 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:06.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.857+0000 7ff8bfc47740 1 --2- 172.21.15.64:0/3628131052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da182e7fc0 0x55da182fb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:06.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.857+0000 7ff8bfc47740 1 --2- 172.21.15.64:0/3628131052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55da182d91f0 0x55da18302640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:06.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.857+0000 7ff896ffd640 0 client.15201 ms_handle_reset on v2:172.21.15.195:6838/2574695134 2024-09-12T03:29:06.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.857+0000 7ff8bfc47740 1 --2- 172.21.15.64:0/3628131052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da182d88d0 0x55da18302100 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:06.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.857+0000 7ff8bfc47740 1 -- 172.21.15.64:0/3628131052 >> 172.21.15.64:0/3628131052 conn(0x55da182d4f60 msgr2=0x55da182e0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:06.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.857+0000 7ff8bfc47740 1 -- 172.21.15.64:0/3628131052 shutdown_connections 2024-09-12T03:29:06.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.857+0000 7ff8bfc47740 1 -- 172.21.15.64:0/3628131052 wait complete. 2024-09-12T03:29:06.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.858+0000 7ff8bfc47740 20 client.15201 trim_cache size 0 max 16384 2024-09-12T03:29:06.869 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T03:29:06.869 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "b0812460-70b5-11ef-bcea-c7b262605968", 2024-09-12T03:29:06.869 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 1, 2024-09-12T03:29:06.870 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24481, 2024-09-12T03:29:06.870 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-09-12T03:29:06.870 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-09-12T03:29:06.870 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-09-12T03:29:06.870 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 93.735113182999996, 2024-09-12T03:29:06.870 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 24, 2024-09-12T03:29:06.870 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 90, 2024-09-12T03:29:06.870 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 88, 2024-09-12T03:29:06.870 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 148.98503572000001 2024-09-12T03:29:06.870 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T03:29:06.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.858+0000 7ff8bd86e640 1 -- 172.21.15.64:0/4012789598 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff8a00a2c10 msgr2=0x7ff8a00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:06.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.858+0000 7ff8bd86e640 1 --2- 172.21.15.64:0/4012789598 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff8a00a2c10 0x7ff8a00a50d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0009430 tx=0x7ff8b00095c0 comp rx=0 tx=0).stop 2024-09-12T03:29:06.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.858+0000 7ff8bd86e640 1 -- 172.21.15.64:0/4012789598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8b8101de0 msgr2=0x7ff8b8198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:06.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.858+0000 7ff8bd86e640 1 --2- 172.21.15.64:0/4012789598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8b8101de0 0x7ff8b8198360 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff8a800c890 tx=0x7ff8a800cd50 comp rx=0 tx=0).stop 2024-09-12T03:29:06.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.858+0000 7ff8bd86e640 1 -- 172.21.15.64:0/4012789598 shutdown_connections 2024-09-12T03:29:06.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.858+0000 7ff8bd86e640 1 --2- 172.21.15.64:0/4012789598 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff8a00a2c10 0x7ff8a00a50d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:06.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.858+0000 7ff8bd86e640 1 --2- 172.21.15.64:0/4012789598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8b8106f10 0x7ff8b819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:06.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.858+0000 7ff8bd86e640 1 --2- 172.21.15.64:0/4012789598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8b8102790 0x7ff8b81988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:06.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.858+0000 7ff8bd86e640 1 --2- 172.21.15.64:0/4012789598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8b8101de0 0x7ff8b8198360 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:06.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.858+0000 7ff8bd86e640 1 -- 172.21.15.64:0/4012789598 >> 172.21.15.64:0/4012789598 conn(0x7ff8b80fd8c0 msgr2=0x7ff8b80fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:06.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.858+0000 7ff8bd86e640 1 -- 172.21.15.64:0/4012789598 shutdown_connections 2024-09-12T03:29:06.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:06.858+0000 7ff8bd86e640 1 -- 172.21.15.64:0/4012789598 wait complete. 2024-09-12T03:29:07.605 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:07 smithi064 ceph-mon[30952]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:29:07.607 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-12T03:29:07.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:07 smithi115 ceph-mon[34801]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:29:07.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:07 smithi195 ceph-mon[34786]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:29:07.867 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:29:08.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.373+0000 7f7d3ab51640 1 -- 172.21.15.64:0/3391843943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d34106a80 msgr2=0x7f7d34106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:08.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.373+0000 7f7d3ab51640 1 --2- 172.21.15.64:0/3391843943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d34106a80 0x7f7d34106e60 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7d28009930 tx=0x7f7d2802a420 comp rx=0 tx=0).stop 2024-09-12T03:29:08.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.373+0000 7f7d3ab51640 1 -- 172.21.15.64:0/3391843943 shutdown_connections 2024-09-12T03:29:08.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.373+0000 7f7d3ab51640 1 --2- 172.21.15.64:0/3391843943 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d34101420 0x7f7d3410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:08.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.373+0000 7f7d3ab51640 1 --2- 172.21.15.64:0/3391843943 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d34100a80 0x7f7d34100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:08.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.373+0000 7f7d3ab51640 1 --2- 172.21.15.64:0/3391843943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d34106a80 0x7f7d34106e60 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:08.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.373+0000 7f7d3ab51640 1 -- 172.21.15.64:0/3391843943 >> 172.21.15.64:0/3391843943 conn(0x7f7d340fc7a0 msgr2=0x7f7d340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:08.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.374+0000 7f7d3ab51640 1 -- 172.21.15.64:0/3391843943 shutdown_connections 2024-09-12T03:29:08.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.374+0000 7f7d3ab51640 1 -- 172.21.15.64:0/3391843943 wait complete. 2024-09-12T03:29:08.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.374+0000 7f7d3ab51640 1 Processor -- start 2024-09-12T03:29:08.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.374+0000 7f7d3ab51640 1 -- start start 2024-09-12T03:29:08.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.374+0000 7f7d3ab51640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d34100a80 0x7f7d340fceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:08.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.374+0000 7f7d3ab51640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d34101420 0x7f7d340fd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:08.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.374+0000 7f7d3ab51640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d34106a80 0x7f7d340fd930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:08.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.374+0000 7f7d3ab51640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7d34110d70 con 0x7f7d34106a80 2024-09-12T03:29:08.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.374+0000 7f7d3ab51640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7d34110bf0 con 0x7f7d34100a80 2024-09-12T03:29:08.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.374+0000 7f7d3ab51640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7d34110ef0 con 0x7f7d34101420 2024-09-12T03:29:08.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.375+0000 7f7d390ee640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d34106a80 0x7f7d340fd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:08.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.375+0000 7f7d388ed640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d34100a80 0x7f7d340fceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:08.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.375+0000 7f7d390ee640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d34106a80 0x7f7d340fd930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36182/0 (socket says 172.21.15.64:36182) 2024-09-12T03:29:08.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.375+0000 7f7d390ee640 1 -- 172.21.15.64:0/207338832 learned_addr learned my addr 172.21.15.64:0/207338832 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:08.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.375+0000 7f7d33fff640 1 --2- 172.21.15.64:0/207338832 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d34101420 0x7f7d340fd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:08.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.375+0000 7f7d390ee640 1 -- 172.21.15.64:0/207338832 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d34101420 msgr2=0x7f7d340fd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:08.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.375+0000 7f7d390ee640 1 --2- 172.21.15.64:0/207338832 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d34101420 0x7f7d340fd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:08.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.375+0000 7f7d390ee640 1 -- 172.21.15.64:0/207338832 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d34100a80 msgr2=0x7f7d340fceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:08.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.375+0000 7f7d390ee640 1 --2- 172.21.15.64:0/207338832 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d34100a80 0x7f7d340fceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:08.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.375+0000 7f7d390ee640 1 -- 172.21.15.64:0/207338832 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d341a6900 con 0x7f7d34106a80 2024-09-12T03:29:08.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.375+0000 7f7d388ed640 1 --2- 172.21.15.64:0/207338832 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d34100a80 0x7f7d340fceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:08.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.375+0000 7f7d390ee640 1 --2- 172.21.15.64:0/207338832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d34106a80 0x7f7d340fd930 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00b970 tx=0x7f7d2c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:08.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.375+0000 7f7d31ffb640 1 -- 172.21.15.64:0/207338832 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c019070 con 0x7f7d34106a80 2024-09-12T03:29:08.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.375+0000 7f7d3ab51640 1 -- 172.21.15.64:0/207338832 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d341a6bf0 con 0x7f7d34106a80 2024-09-12T03:29:08.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.375+0000 7f7d31ffb640 1 -- 172.21.15.64:0/207338832 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d2c014440 con 0x7f7d34106a80 2024-09-12T03:29:08.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.375+0000 7f7d31ffb640 1 -- 172.21.15.64:0/207338832 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c0075d0 con 0x7f7d34106a80 2024-09-12T03:29:08.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.376+0000 7f7d3ab51640 1 -- 172.21.15.64:0/207338832 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d341a70e0 con 0x7f7d34106a80 2024-09-12T03:29:08.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.378+0000 7f7d31ffb640 1 -- 172.21.15.64:0/207338832 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7d2c01c070 con 0x7f7d34106a80 2024-09-12T03:29:08.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.378+0000 7f7d3cf2a740 1 Processor -- start 2024-09-12T03:29:08.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.378+0000 7f7d3cf2a740 1 -- start start 2024-09-12T03:29:08.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.379+0000 7f7d3cf2a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626289c99d0 0x5626289c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:08.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.379+0000 7f7d3cf2a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626289ca2f0 0x5626289d4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:08.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.379+0000 7f7d3cf2a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626289d4bb0 0x5626289d6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:08.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.379+0000 7f7d3cf2a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562628930090 con 0x5626289d4bb0 2024-09-12T03:29:08.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.379+0000 7f7d3cf2a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5626288d7a70 con 0x5626289c99d0 2024-09-12T03:29:08.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.379+0000 7f7d3cf2a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56262892c050 con 0x5626289ca2f0 2024-09-12T03:29:08.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.379+0000 7f7d31ffb640 1 --2- 172.21.15.64:0/207338832 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f7d240a2b80 0x7f7d240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:08.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.379+0000 7f7d33fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626289d4bb0 0x5626289d6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:08.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.379+0000 7f7d390ee640 1 --2- 172.21.15.64:0/207338832 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f7d240a2b80 0x7f7d240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:08.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.379+0000 7f7d33fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626289d4bb0 0x5626289d6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36198/0 (socket says 172.21.15.64:36198) 2024-09-12T03:29:08.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.379+0000 7f7d33fff640 1 -- 172.21.15.64:0/113336383 learned_addr learned my addr 172.21.15.64:0/113336383 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:08.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.379+0000 7f7d33fff640 1 --2- 172.21.15.64:0/113336383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626289c99d0 0x5626289c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:08.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.379+0000 7f7d388ed640 1 --2- 172.21.15.64:0/113336383 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626289ca2f0 0x5626289d4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:08.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.379+0000 7f7d31ffb640 1 -- 172.21.15.64:0/207338832 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9237+0+0 (secure 0 0 0) 0x7f7d2c0cea00 con 0x7f7d34106a80 2024-09-12T03:29:08.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.379+0000 7f7d33fff640 1 -- 172.21.15.64:0/113336383 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626289ca2f0 msgr2=0x5626289d4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:08.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.379+0000 7f7d33fff640 1 --2- 172.21.15.64:0/113336383 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626289ca2f0 0x5626289d4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:08.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.379+0000 7f7d33fff640 1 -- 172.21.15.64:0/113336383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626289c99d0 msgr2=0x5626289c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:08.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.379+0000 7f7d33fff640 1 --2- 172.21.15.64:0/113336383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626289c99d0 0x5626289c9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:08.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.379+0000 7f7d33fff640 1 -- 172.21.15.64:0/113336383 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626288e4130 con 0x5626289d4bb0 2024-09-12T03:29:08.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.379+0000 7f7d390ee640 1 --2- 172.21.15.64:0/207338832 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f7d240a2b80 0x7f7d240a5040 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c0978f0 tx=0x7f7d2c096aa0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:08.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.380+0000 7f7d33fff640 1 --2- 172.21.15.64:0/113336383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626289d4bb0 0x5626289d6fa0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7d200058c0 tx=0x7f7d2000b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea1504346b3dd6ec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:08.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.380+0000 7f7d1effd640 1 -- 172.21.15.64:0/113336383 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d20021950 con 0x5626289d4bb0 2024-09-12T03:29:08.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.380+0000 7f7d1effd640 1 -- 172.21.15.64:0/113336383 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d20005020 con 0x5626289d4bb0 2024-09-12T03:29:08.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.380+0000 7f7d1effd640 1 -- 172.21.15.64:0/113336383 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d20004a00 con 0x5626289d4bb0 2024-09-12T03:29:08.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.380+0000 7f7d3cf2a740 1 -- 172.21.15.64:0/113336383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626289d4bb0 msgr2=0x5626289d6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:08.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.380+0000 7f7d3cf2a740 1 --2- 172.21.15.64:0/113336383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626289d4bb0 0x5626289d6fa0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7d200058c0 tx=0x7f7d2000b4c0 comp rx=0 tx=0).stop 2024-09-12T03:29:08.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.380+0000 7f7d3cf2a740 1 -- 172.21.15.64:0/113336383 shutdown_connections 2024-09-12T03:29:08.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.380+0000 7f7d3cf2a740 1 --2- 172.21.15.64:0/113336383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626289d4bb0 0x5626289d6fa0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:08.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.380+0000 7f7d3cf2a740 1 --2- 172.21.15.64:0/113336383 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626289ca2f0 0x5626289d4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:08.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.380+0000 7f7d3cf2a740 1 --2- 172.21.15.64:0/113336383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626289c99d0 0x5626289c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:08.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.381+0000 7f7d3cf2a740 1 -- 172.21.15.64:0/113336383 >> 172.21.15.64:0/113336383 conn(0x5626289c5f60 msgr2=0x5626289c6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:08.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.381+0000 7f7d3cf2a740 1 -- 172.21.15.64:0/113336383 shutdown_connections 2024-09-12T03:29:08.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.381+0000 7f7d3cf2a740 1 -- 172.21.15.64:0/113336383 wait complete. 2024-09-12T03:29:08.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.381+0000 7f7d3cf2a740 1 -- start start 2024-09-12T03:29:08.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.381+0000 7f7d3cf2a740 1 Processor -- start 2024-09-12T03:29:08.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.381+0000 7f7d3cf2a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626289c99d0 0x5626289d43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:08.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.381+0000 7f7d3cf2a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626289ca2f0 0x5626289d1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:08.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.382+0000 7f7d3cf2a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626289d4bb0 0x5626289d1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:08.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.382+0000 7f7d3cf2a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5626289d7720 con 0x5626289d4bb0 2024-09-12T03:29:08.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.382+0000 7f7d3cf2a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5626288eaf20 con 0x5626289c99d0 2024-09-12T03:29:08.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.382+0000 7f7d3cf2a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5626289d78a0 con 0x5626289ca2f0 2024-09-12T03:29:08.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.382+0000 7f7d3cf2a740 10 client.0 _ll_register_callbacks cb 0x5626288b4030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:08.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.382+0000 7f7d33fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626289d4bb0 0x5626289d1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:08.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.382+0000 7f7d33fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626289d4bb0 0x5626289d1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36214/0 (socket says 172.21.15.64:36214) 2024-09-12T03:29:08.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.382+0000 7f7d33fff640 1 -- 172.21.15.64:0/1319972678 learned_addr learned my addr 172.21.15.64:0/1319972678 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:08.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.382+0000 7f7d33fff640 1 --2- 172.21.15.64:0/1319972678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626289c99d0 0x5626289d43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:08.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.382+0000 7f7d388ed640 1 --2- 172.21.15.64:0/1319972678 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626289ca2f0 0x5626289d1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:08.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.382+0000 7f7d33fff640 1 -- 172.21.15.64:0/1319972678 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626289ca2f0 msgr2=0x5626289d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:08.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.382+0000 7f7d33fff640 1 --2- 172.21.15.64:0/1319972678 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626289ca2f0 0x5626289d1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:08.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.382+0000 7f7d33fff640 1 -- 172.21.15.64:0/1319972678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626289c99d0 msgr2=0x5626289d43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:08.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.382+0000 7f7d33fff640 1 --2- 172.21.15.64:0/1319972678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626289c99d0 0x5626289d43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:08.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.382+0000 7f7d33fff640 1 -- 172.21.15.64:0/1319972678 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626286a12b0 con 0x5626289d4bb0 2024-09-12T03:29:08.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.382+0000 7f7d33fff640 1 --2- 172.21.15.64:0/1319972678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626289d4bb0 0x5626289d1c70 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7d20005790 tx=0x7f7d20014f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd579a548f4eeafb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:08.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.382+0000 7f7d13fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:29:08.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.383+0000 7f7d13fff640 1 -- 172.21.15.64:0/1319972678 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d20035060 con 0x5626289d4bb0 2024-09-12T03:29:08.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.383+0000 7f7d3cf2a740 1 -- 172.21.15.64:0/1319972678 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d20005020 con 0x5626289d4bb0 2024-09-12T03:29:08.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.383+0000 7f7d13fff640 1 -- 172.21.15.64:0/1319972678 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9237+0+0 (secure 0 0 0) 0x7f7d20021d60 con 0x5626289d4bb0 2024-09-12T03:29:08.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.383+0000 7f7d13fff640 1 -- 172.21.15.64:0/1319972678 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7d2000f700 con 0x5626289d4bb0 2024-09-12T03:29:08.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.383+0000 7f7d3cf2a740 10 client.15213 fetch_fsmap learned FSMap version 24 2024-09-12T03:29:08.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.383+0000 7f7d3cf2a740 1 -- 172.21.15.64:0/1319972678 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5626289e3000 con 0x5626289d4bb0 2024-09-12T03:29:08.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.383+0000 7f7d13fff640 1 -- 172.21.15.64:0/1319972678 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d20005020 con 0x5626289d4bb0 2024-09-12T03:29:08.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.383+0000 7f7d13fff640 1 -- 172.21.15.64:0/1319972678 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f7d20004ad0 con 0x5626289d4bb0 2024-09-12T03:29:08.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.383+0000 7f7d3cf2a740 10 client.15213 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T03:29:08.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.383+0000 7f7d3cf2a740 10 client.15213 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T03:29:08.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.383+0000 7f7d3cf2a740 20 client.15213 populate_metadata read hostname 'smithi064' 2024-09-12T03:29:08.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.383+0000 7f7d3cf2a740 1 --2- 172.21.15.64:0/1319972678 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5626289e4470 0x5626289e6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:08.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.383+0000 7f7d3cf2a740 4 client.15213 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:29:08.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.383+0000 7f7d3cf2a740 1 -- 172.21.15.64:0/1319972678 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56262893d600 con 0x5626289e4470 2024-09-12T03:29:08.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.384+0000 7f7d388ed640 1 --2- 172.21.15.64:0/1319972678 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5626289e4470 0x5626289e6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:08.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.384+0000 7f7d388ed640 1 --2- 172.21.15.64:0/1319972678 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5626289e4470 0x5626289e6660 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=473731125411b4d9 server_cookie=1842f1fb452dffbd in_seq=0 out_seq=0 2024-09-12T03:29:08.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.384+0000 7f7d13fff640 10 client.15213 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:29:08.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.385+0000 7f7d13fff640 1 -- 172.21.15.64:0/1319972678 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56262893d600 con 0x5626289e4470 2024-09-12T03:29:08.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.385+0000 7f7d13fff640 10 client.15213 handle_command_reply: tid=0 2024-09-12T03:29:08.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.385+0000 7f7d3cf2a740 1 client.15213 shutdown 2024-09-12T03:29:08.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.385+0000 7f7d3cf2a740 1 -- 172.21.15.64:0/1319972678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626289d4bb0 msgr2=0x5626289d1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:08.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.385+0000 7f7d3cf2a740 1 --2- 172.21.15.64:0/1319972678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626289d4bb0 0x5626289d1c70 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7d20005790 tx=0x7f7d20014f50 comp rx=0 tx=0).stop 2024-09-12T03:29:08.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.385+0000 7f7d3cf2a740 1 -- 172.21.15.64:0/1319972678 shutdown_connections 2024-09-12T03:29:08.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.385+0000 7f7d3cf2a740 1 --2- 172.21.15.64:0/1319972678 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5626289e4470 0x5626289e6660 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:08.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.385+0000 7f7d3cf2a740 1 --2- 172.21.15.64:0/1319972678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626289d4bb0 0x5626289d1c70 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:08.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.385+0000 7f7d3cf2a740 1 --2- 172.21.15.64:0/1319972678 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626289ca2f0 0x5626289d1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:08.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.385+0000 7f7d3cf2a740 1 --2- 172.21.15.64:0/1319972678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626289c99d0 0x5626289d43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:08.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.385+0000 7f7d3cf2a740 1 -- 172.21.15.64:0/1319972678 >> 172.21.15.64:0/1319972678 conn(0x5626289c5f60 msgr2=0x5626289cbc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:08.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.385+0000 7f7d13fff640 0 client.15213 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:29:08.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.385+0000 7f7d3cf2a740 1 -- 172.21.15.64:0/1319972678 shutdown_connections 2024-09-12T03:29:08.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.385+0000 7f7d3cf2a740 1 -- 172.21.15.64:0/1319972678 wait complete. 2024-09-12T03:29:08.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.385+0000 7f7d3cf2a740 20 client.15213 trim_cache size 0 max 16384 2024-09-12T03:29:08.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.403+0000 7f7d3cf2a740 1 Processor -- start 2024-09-12T03:29:08.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.403+0000 7f7d3cf2a740 1 -- start start 2024-09-12T03:29:08.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.403+0000 7f7d3cf2a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626289c99d0 0x5626289e4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:08.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.403+0000 7f7d3cf2a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626289ca2f0 0x5626289e58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:08.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.403+0000 7f7d3cf2a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626289d4bb0 0x5626289cab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:08.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.403+0000 7f7d3cf2a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5626289e49a0 con 0x5626289ca2f0 2024-09-12T03:29:08.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.403+0000 7f7d3cf2a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5626289e5070 con 0x5626289c99d0 2024-09-12T03:29:08.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.403+0000 7f7d3cf2a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5626289e5740 con 0x5626289d4bb0 2024-09-12T03:29:08.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.403+0000 7f7d388ed640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626289ca2f0 0x5626289e58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:08.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.403+0000 7f7d388ed640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626289ca2f0 0x5626289e58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36216/0 (socket says 172.21.15.64:36216) 2024-09-12T03:29:08.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.403+0000 7f7d388ed640 1 -- 172.21.15.64:0/295537144 learned_addr learned my addr 172.21.15.64:0/295537144 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:08.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.403+0000 7f7d33fff640 1 --2- 172.21.15.64:0/295537144 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626289c99d0 0x5626289e4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:08.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.403+0000 7f7d33fff640 1 --2- 172.21.15.64:0/295537144 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626289d4bb0 0x5626289cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:08.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.403+0000 7f7d388ed640 1 -- 172.21.15.64:0/295537144 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626289d4bb0 msgr2=0x5626289cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:08.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.403+0000 7f7d388ed640 1 --2- 172.21.15.64:0/295537144 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626289d4bb0 0x5626289cab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:08.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.403+0000 7f7d388ed640 1 -- 172.21.15.64:0/295537144 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626289c99d0 msgr2=0x5626289e4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:08.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.403+0000 7f7d388ed640 1 --2- 172.21.15.64:0/295537144 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626289c99d0 0x5626289e4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:08.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.403+0000 7f7d388ed640 1 -- 172.21.15.64:0/295537144 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626289d4000 con 0x5626289ca2f0 2024-09-12T03:29:08.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.404+0000 7f7d388ed640 1 --2- 172.21.15.64:0/295537144 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626289ca2f0 0x5626289e58c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7d28009900 tx=0x7f7d280328f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b8e91964be2821 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:08.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.404+0000 7f7d1e7fc640 1 -- 172.21.15.64:0/295537144 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d28035040 con 0x5626289ca2f0 2024-09-12T03:29:08.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.404+0000 7f7d1e7fc640 1 -- 172.21.15.64:0/295537144 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d280054d0 con 0x5626289ca2f0 2024-09-12T03:29:08.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.404+0000 7f7d1e7fc640 1 -- 172.21.15.64:0/295537144 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d28004c70 con 0x5626289ca2f0 2024-09-12T03:29:08.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.404+0000 7f7d3cf2a740 1 -- 172.21.15.64:0/295537144 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626289ca2f0 msgr2=0x5626289e58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:08.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.404+0000 7f7d3cf2a740 1 --2- 172.21.15.64:0/295537144 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626289ca2f0 0x5626289e58c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7d28009900 tx=0x7f7d280328f0 comp rx=0 tx=0).stop 2024-09-12T03:29:08.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.404+0000 7f7d3cf2a740 1 -- 172.21.15.64:0/295537144 shutdown_connections 2024-09-12T03:29:08.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.404+0000 7f7d3cf2a740 1 --2- 172.21.15.64:0/295537144 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626289d4bb0 0x5626289cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:08.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.404+0000 7f7d3cf2a740 1 --2- 172.21.15.64:0/295537144 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626289ca2f0 0x5626289e58c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:08.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.404+0000 7f7d3cf2a740 1 --2- 172.21.15.64:0/295537144 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626289c99d0 0x5626289e4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:08.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.404+0000 7f7d3cf2a740 1 -- 172.21.15.64:0/295537144 >> 172.21.15.64:0/295537144 conn(0x5626289c5f60 msgr2=0x5626289c6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:08.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.405+0000 7f7d3cf2a740 1 -- 172.21.15.64:0/295537144 shutdown_connections 2024-09-12T03:29:08.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.405+0000 7f7d3cf2a740 1 -- 172.21.15.64:0/295537144 wait complete. 2024-09-12T03:29:08.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.405+0000 7f7d3cf2a740 1 -- start start 2024-09-12T03:29:08.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.405+0000 7f7d3cf2a740 1 Processor -- start 2024-09-12T03:29:08.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.405+0000 7f7d3cf2a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626289c99d0 0x5626289e78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:08.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.405+0000 7f7d3cf2a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626289ca2f0 0x5626289e7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:08.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.405+0000 7f7d3cf2a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626289d4bb0 0x5626289e8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:08.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.405+0000 7f7d3cf2a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5626289e64e0 con 0x5626289d4bb0 2024-09-12T03:29:08.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.405+0000 7f7d3cf2a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5626289e5e10 con 0x5626289ca2f0 2024-09-12T03:29:08.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.405+0000 7f7d3cf2a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5626289e6bb0 con 0x5626289c99d0 2024-09-12T03:29:08.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.405+0000 7f7d3cf2a740 10 client.0 _ll_register_callbacks cb 0x7f7d2000bf00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:08.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.406+0000 7f7d33fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626289d4bb0 0x5626289e8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:08.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.406+0000 7f7d33fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626289d4bb0 0x5626289e8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36224/0 (socket says 172.21.15.64:36224) 2024-09-12T03:29:08.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.406+0000 7f7d33fff640 1 -- 172.21.15.64:0/241102361 learned_addr learned my addr 172.21.15.64:0/241102361 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:08.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.406+0000 7f7d388ed640 1 --2- 172.21.15.64:0/241102361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626289ca2f0 0x5626289e7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:08.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.406+0000 7f7d33fff640 1 --2- 172.21.15.64:0/241102361 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626289c99d0 0x5626289e78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:08.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.406+0000 7f7d33fff640 1 -- 172.21.15.64:0/241102361 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626289c99d0 msgr2=0x5626289e78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:08.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.406+0000 7f7d33fff640 1 --2- 172.21.15.64:0/241102361 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626289c99d0 0x5626289e78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:08.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.406+0000 7f7d33fff640 1 -- 172.21.15.64:0/241102361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626289ca2f0 msgr2=0x5626289e7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:08.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.406+0000 7f7d33fff640 1 --2- 172.21.15.64:0/241102361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626289ca2f0 0x5626289e7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:08.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.406+0000 7f7d33fff640 1 -- 172.21.15.64:0/241102361 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562628a4db10 con 0x5626289d4bb0 2024-09-12T03:29:08.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.406+0000 7f7d33fff640 1 --2- 172.21.15.64:0/241102361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626289d4bb0 0x5626289e8340 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7d20004ec0 tx=0x7f7d20014f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc5d5231540c78ed server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:08.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.406+0000 7f7d13fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:29:08.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.406+0000 7f7d13fff640 1 -- 172.21.15.64:0/241102361 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2000b960 con 0x5626289d4bb0 2024-09-12T03:29:08.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.406+0000 7f7d13fff640 1 -- 172.21.15.64:0/241102361 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9237+0+0 (secure 0 0 0) 0x7f7d2000b6a0 con 0x5626289d4bb0 2024-09-12T03:29:08.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.406+0000 7f7d3cf2a740 1 -- 172.21.15.64:0/241102361 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d280054d0 con 0x5626289d4bb0 2024-09-12T03:29:08.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.406+0000 7f7d13fff640 1 -- 172.21.15.64:0/241102361 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7d20035030 con 0x5626289d4bb0 2024-09-12T03:29:08.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.406+0000 7f7d3cf2a740 10 client.15219 fetch_fsmap learned FSMap version 24 2024-09-12T03:29:08.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.406+0000 7f7d3cf2a740 1 -- 172.21.15.64:0/241102361 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562628a4edf0 con 0x5626289d4bb0 2024-09-12T03:29:08.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.407+0000 7f7d13fff640 1 -- 172.21.15.64:0/241102361 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d280054d0 con 0x5626289d4bb0 2024-09-12T03:29:08.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.407+0000 7f7d13fff640 1 -- 172.21.15.64:0/241102361 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f7d20004660 con 0x5626289d4bb0 2024-09-12T03:29:08.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.407+0000 7f7d3cf2a740 10 client.15219 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T03:29:08.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.407+0000 7f7d3cf2a740 10 client.15219 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T03:29:08.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.407+0000 7f7d3cf2a740 20 client.15219 populate_metadata read hostname 'smithi064' 2024-09-12T03:29:08.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.407+0000 7f7d3cf2a740 1 --2- 172.21.15.64:0/241102361 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x562628a53930 0x562628a52640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:08.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.407+0000 7f7d3cf2a740 4 client.15219 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:29:08.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.407+0000 7f7d3cf2a740 1 -- 172.21.15.64:0/241102361 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "status"}) -- 0x562628937b10 con 0x562628a53930 2024-09-12T03:29:08.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.407+0000 7f7d388ed640 1 --2- 172.21.15.64:0/241102361 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x562628a53930 0x562628a52640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:08.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.407+0000 7f7d388ed640 1 --2- 172.21.15.64:0/241102361 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x562628a53930 0x562628a52640 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc9bba11753e3056 server_cookie=58cd748f4e0a81ab in_seq=0 out_seq=0 2024-09-12T03:29:08.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.407+0000 7f7d13fff640 10 client.15219 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:29:08.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.408+0000 7f7d13fff640 1 -- 172.21.15.64:0/241102361 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x562628937b10 con 0x562628a53930 2024-09-12T03:29:08.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.408+0000 7f7d13fff640 10 client.15219 handle_command_reply: tid=0 2024-09-12T03:29:08.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.408+0000 7f7d3cf2a740 1 client.15219 shutdown 2024-09-12T03:29:08.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.408+0000 7f7d3cf2a740 1 -- 172.21.15.64:0/241102361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626289d4bb0 msgr2=0x5626289e8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:08.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.408+0000 7f7d3cf2a740 1 --2- 172.21.15.64:0/241102361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626289d4bb0 0x5626289e8340 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7d20004ec0 tx=0x7f7d20014f50 comp rx=0 tx=0).stop 2024-09-12T03:29:08.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.408+0000 7f7d3cf2a740 1 -- 172.21.15.64:0/241102361 shutdown_connections 2024-09-12T03:29:08.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.408+0000 7f7d3cf2a740 1 --2- 172.21.15.64:0/241102361 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x562628a53930 0x562628a52640 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:08.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.408+0000 7f7d13fff640 0 client.15219 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:29:08.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.408+0000 7f7d3cf2a740 1 --2- 172.21.15.64:0/241102361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626289d4bb0 0x5626289e8340 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:08.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.408+0000 7f7d3cf2a740 1 --2- 172.21.15.64:0/241102361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5626289ca2f0 0x5626289e7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:08.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.408+0000 7f7d3cf2a740 1 --2- 172.21.15.64:0/241102361 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626289c99d0 0x5626289e78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:08.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.408+0000 7f7d3cf2a740 1 -- 172.21.15.64:0/241102361 >> 172.21.15.64:0/241102361 conn(0x5626289c5f60 msgr2=0x5626289c5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:08.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.408+0000 7f7d3cf2a740 1 -- 172.21.15.64:0/241102361 shutdown_connections 2024-09-12T03:29:08.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.408+0000 7f7d3cf2a740 1 -- 172.21.15.64:0/241102361 wait complete. 2024-09-12T03:29:08.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.408+0000 7f7d3cf2a740 20 client.15219 trim_cache size 0 max 16384 2024-09-12T03:29:08.414 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T03:29:08.415 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "b0812460-70b5-11ef-bcea-c7b262605968", 2024-09-12T03:29:08.415 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2024-09-12T03:29:08.415 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24500, 2024-09-12T03:29:08.415 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-09-12T03:29:08.415 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-09-12T03:29:08.415 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-09-12T03:29:08.415 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 128.40786712600001, 2024-09-12T03:29:08.415 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 24, 2024-09-12T03:29:08.415 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 91, 2024-09-12T03:29:08.415 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 88, 2024-09-12T03:29:08.415 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 140.326315053 2024-09-12T03:29:08.415 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T03:29:08.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.409+0000 7f7d3ab51640 1 -- 172.21.15.64:0/207338832 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f7d240a2b80 msgr2=0x7f7d240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:08.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.409+0000 7f7d3ab51640 1 --2- 172.21.15.64:0/207338832 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f7d240a2b80 0x7f7d240a5040 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c0978f0 tx=0x7f7d2c096aa0 comp rx=0 tx=0).stop 2024-09-12T03:29:08.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.409+0000 7f7d3ab51640 1 -- 172.21.15.64:0/207338832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d34106a80 msgr2=0x7f7d340fd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:08.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.409+0000 7f7d3ab51640 1 --2- 172.21.15.64:0/207338832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d34106a80 0x7f7d340fd930 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00b970 tx=0x7f7d2c00be30 comp rx=0 tx=0).stop 2024-09-12T03:29:08.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.409+0000 7f7d3ab51640 1 -- 172.21.15.64:0/207338832 shutdown_connections 2024-09-12T03:29:08.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.409+0000 7f7d3ab51640 1 --2- 172.21.15.64:0/207338832 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f7d240a2b80 0x7f7d240a5040 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:08.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.409+0000 7f7d3ab51640 1 --2- 172.21.15.64:0/207338832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d34106a80 0x7f7d340fd930 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:08.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.409+0000 7f7d3ab51640 1 --2- 172.21.15.64:0/207338832 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d34101420 0x7f7d340fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:08.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.409+0000 7f7d3ab51640 1 --2- 172.21.15.64:0/207338832 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d34100a80 0x7f7d340fceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:08.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.409+0000 7f7d3ab51640 1 -- 172.21.15.64:0/207338832 >> 172.21.15.64:0/207338832 conn(0x7f7d340fc7a0 msgr2=0x7f7d3406ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:08.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.409+0000 7f7d3ab51640 1 -- 172.21.15.64:0/207338832 shutdown_connections 2024-09-12T03:29:08.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:08.409+0000 7f7d3ab51640 1 -- 172.21.15.64:0/207338832 wait complete. 2024-09-12T03:29:08.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:08 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-09-12T03:29:08.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:08 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-12T03:29:08.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:08 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-12T03:29:08.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:08 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-09-12T03:29:08.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:08 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [1, 9]}]: dispatch 2024-09-12T03:29:08.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:08 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-09-12T03:29:08.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:08 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-12T03:29:08.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:08 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-12T03:29:08.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:08 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-09-12T03:29:08.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:08 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [1, 9]}]: dispatch 2024-09-12T03:29:08.721 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:08 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-09-12T03:29:08.721 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:08 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-12T03:29:08.721 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:08 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-12T03:29:08.721 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:08 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-09-12T03:29:08.721 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:08 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [1, 9]}]: dispatch 2024-09-12T03:29:09.146 INFO:teuthology.run_tasks:Running task sequential... 2024-09-12T03:29:09.160 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-12T03:29:09.160 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-12T03:29:09.161 INFO:teuthology.task.print:Enabling mgr modules 2024-09-12T03:29:09.161 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-09-12T03:29:09.162 INFO:teuthology.task.exec:Executing custom commands... 2024-09-12T03:29:09.162 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi064.front.sepia.ceph.com 2024-09-12T03:29:09.162 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-09-12T03:29:09.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:09.271+0000 7ff89a986640 1 -- 172.21.15.64:0/1449932491 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff894101c30 msgr2=0x7ff8941020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:09.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:09.271+0000 7ff89a986640 1 --2- 172.21.15.64:0/1449932491 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff894101c30 0x7ff8941020b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff888009900 tx=0x7ff88802a410 comp rx=0 tx=0).stop 2024-09-12T03:29:09.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:09.271+0000 7ff89a986640 1 -- 172.21.15.64:0/1449932491 shutdown_connections 2024-09-12T03:29:09.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:09.271+0000 7ff89a986640 1 --2- 172.21.15.64:0/1449932491 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8941025f0 0x7ff894108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:09.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:09.271+0000 7ff89a986640 1 --2- 172.21.15.64:0/1449932491 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff894101c30 0x7ff8941020b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:09.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:09.271+0000 7ff89a986640 1 --2- 172.21.15.64:0/1449932491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff894100a40 0x7ff894100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:09.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:09.271+0000 7ff89a986640 1 -- 172.21.15.64:0/1449932491 >> 172.21.15.64:0/1449932491 conn(0x7ff8940fc1e0 msgr2=0x7ff8940fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:09.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:09.271+0000 7ff89a986640 1 -- 172.21.15.64:0/1449932491 shutdown_connections 2024-09-12T03:29:09.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:09.271+0000 7ff89a986640 1 -- 172.21.15.64:0/1449932491 wait complete. 2024-09-12T03:29:09.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:09.272+0000 7ff89a986640 1 Processor -- start 2024-09-12T03:29:09.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:09.272+0000 7ff89a986640 1 -- start start 2024-09-12T03:29:09.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:09.272+0000 7ff89a986640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff894100a40 0x7ff894197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:09.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:09.272+0000 7ff89a986640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff894101c30 0x7ff894198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:09.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:09.272+0000 7ff89a986640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8941025f0 0x7ff89419f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:09.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:09.272+0000 7ff89a986640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff89410d720 con 0x7ff894100a40 2024-09-12T03:29:09.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:09.272+0000 7ff89a986640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff89410d5a0 con 0x7ff8941025f0 2024-09-12T03:29:09.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:09.272+0000 7ff89a986640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff89410d8a0 con 0x7ff894101c30 2024-09-12T03:29:09.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:09.272+0000 7ff893fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff894100a40 0x7ff894197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:09.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:09.272+0000 7ff893fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff894100a40 0x7ff894197cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36252/0 (socket says 172.21.15.64:36252) 2024-09-12T03:29:09.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:09.272+0000 7ff893fff640 1 -- 172.21.15.64:0/3150463515 learned_addr learned my addr 172.21.15.64:0/3150463515 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:09.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:09.272+0000 7ff893fff640 1 -- 172.21.15.64:0/3150463515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff894101c30 msgr2=0x7ff894198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:09.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:09.272+0000 7ff893fff640 1 --2- 172.21.15.64:0/3150463515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff894101c30 0x7ff894198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:09.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:09.272+0000 7ff893fff640 1 -- 172.21.15.64:0/3150463515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8941025f0 msgr2=0x7ff89419f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:09.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:09.272+0000 7ff893fff640 1 --2- 172.21.15.64:0/3150463515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8941025f0 0x7ff89419f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:09.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:09.272+0000 7ff893fff640 1 -- 172.21.15.64:0/3150463515 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff89419f900 con 0x7ff894100a40 2024-09-12T03:29:09.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:09.273+0000 7ff893fff640 1 --2- 172.21.15.64:0/3150463515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff894100a40 0x7ff894197cc0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff88400c970 tx=0x7ff88400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:09.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:09.273+0000 7ff8917fa640 1 -- 172.21.15.64:0/3150463515 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff884007c10 con 0x7ff894100a40 2024-09-12T03:29:09.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:09.273+0000 7ff8917fa640 1 -- 172.21.15.64:0/3150463515 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff884007db0 con 0x7ff894100a40 2024-09-12T03:29:09.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:09.273+0000 7ff89a986640 1 -- 172.21.15.64:0/3150463515 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff89419fbf0 con 0x7ff894100a40 2024-09-12T03:29:09.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:09.273+0000 7ff8917fa640 1 -- 172.21.15.64:0/3150463515 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88400f870 con 0x7ff894100a40 2024-09-12T03:29:09.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:09.273+0000 7ff89a986640 1 -- 172.21.15.64:0/3150463515 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8941a0190 con 0x7ff894100a40 2024-09-12T03:29:09.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:09.274+0000 7ff89a986640 1 -- 172.21.15.64:0/3150463515 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff894107d60 con 0x7ff894100a40 2024-09-12T03:29:09.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:09.275+0000 7ff8917fa640 1 -- 172.21.15.64:0/3150463515 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff88400fa10 con 0x7ff894100a40 2024-09-12T03:29:09.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:09.276+0000 7ff8917fa640 1 --2- 172.21.15.64:0/3150463515 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff87c0a2cb0 0x7ff87c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:09.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:09.276+0000 7ff8937fe640 1 --2- 172.21.15.64:0/3150463515 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff87c0a2cb0 0x7ff87c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:09.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:09.276+0000 7ff8917fa640 1 -- 172.21.15.64:0/3150463515 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9237+0+0 (secure 0 0 0) 0x7ff8840cf7f0 con 0x7ff894100a40 2024-09-12T03:29:09.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:09.276+0000 7ff8937fe640 1 --2- 172.21.15.64:0/3150463515 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff87c0a2cb0 0x7ff87c0a5170 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff888009230 tx=0x7ff8880314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:09.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:09.280+0000 7ff8917fa640 1 -- 172.21.15.64:0/3150463515 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff88409c3f0 con 0x7ff894100a40 2024-09-12T03:29:09.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:09.508+0000 7ff89a986640 1 -- 172.21.15.64:0/3150463515 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7ff894062ef0 con 0x7ff894100a40 2024-09-12T03:29:09.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:09 smithi064 ceph-mon[30952]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:29:09.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:09 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-12T03:29:09.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:09 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-12T03:29:09.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:09 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-12T03:29:09.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:09 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-09-12T03:29:09.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:09 smithi064 ceph-mon[30952]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [1, 9]}]': finished 2024-09-12T03:29:09.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:09 smithi064 ceph-mon[30952]: osdmap e91: 12 total, 12 up, 12 in 2024-09-12T03:29:09.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:09 smithi064 ceph-mon[30952]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-12T03:29:09.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:09 smithi115 ceph-mon[34801]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:29:09.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:09 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-12T03:29:09.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:09 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-12T03:29:09.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:09 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-12T03:29:09.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:09 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-09-12T03:29:09.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:09 smithi115 ceph-mon[34801]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [1, 9]}]': finished 2024-09-12T03:29:09.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:09 smithi115 ceph-mon[34801]: osdmap e91: 12 total, 12 up, 12 in 2024-09-12T03:29:09.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:09 smithi115 ceph-mon[34801]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-12T03:29:09.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:09 smithi195 ceph-mon[34786]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:29:09.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:09 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-12T03:29:09.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:09 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-12T03:29:09.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:09 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-12T03:29:09.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:09 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-09-12T03:29:09.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:09 smithi195 ceph-mon[34786]: from='mgr.14150 172.21.15.64:0/494675118' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [1, 9]}]': finished 2024-09-12T03:29:09.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:09 smithi195 ceph-mon[34786]: osdmap e91: 12 total, 12 up, 12 in 2024-09-12T03:29:09.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:09 smithi195 ceph-mon[34786]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-12T03:29:10.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:10.363+0000 7ff8917fa640 1 -- 172.21.15.64:0/3150463515 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7ff88409bcf0 con 0x7ff894100a40 2024-09-12T03:29:10.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:10.366+0000 7ff89a986640 1 -- 172.21.15.64:0/3150463515 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff87c0a2cb0 msgr2=0x7ff87c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:10.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:10.366+0000 7ff89a986640 1 --2- 172.21.15.64:0/3150463515 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff87c0a2cb0 0x7ff87c0a5170 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff888009230 tx=0x7ff8880314a0 comp rx=0 tx=0).stop 2024-09-12T03:29:10.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:10.366+0000 7ff89a986640 1 -- 172.21.15.64:0/3150463515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff894100a40 msgr2=0x7ff894197cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:10.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:10.366+0000 7ff89a986640 1 --2- 172.21.15.64:0/3150463515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff894100a40 0x7ff894197cc0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff88400c970 tx=0x7ff88400ce30 comp rx=0 tx=0).stop 2024-09-12T03:29:10.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:10.366+0000 7ff89a986640 1 -- 172.21.15.64:0/3150463515 shutdown_connections 2024-09-12T03:29:10.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:10.366+0000 7ff89a986640 1 --2- 172.21.15.64:0/3150463515 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7ff87c0a2cb0 0x7ff87c0a5170 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:10.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:10.366+0000 7ff89a986640 1 --2- 172.21.15.64:0/3150463515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8941025f0 0x7ff89419f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:10.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:10.366+0000 7ff89a986640 1 --2- 172.21.15.64:0/3150463515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff894101c30 0x7ff894198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:10.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:10.366+0000 7ff89a986640 1 --2- 172.21.15.64:0/3150463515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff894100a40 0x7ff894197cc0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:10.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:10.366+0000 7ff89a986640 1 -- 172.21.15.64:0/3150463515 >> 172.21.15.64:0/3150463515 conn(0x7ff8940fc1e0 msgr2=0x7ff894104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:10.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:10.366+0000 7ff89a986640 1 -- 172.21.15.64:0/3150463515 shutdown_connections 2024-09-12T03:29:10.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:10.367+0000 7ff89a986640 1 -- 172.21.15.64:0/3150463515 wait complete. 2024-09-12T03:29:10.386 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-12T03:29:10.397 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-12T03:29:10.397 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-09-12T03:29:10.397 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-09-12T03:29:10.397 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-12T03:29:10.397 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T03:29:10.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:29:10.398 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2024-09-12T03:29:10.416 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-12T03:29:10.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:29:10.417 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T03:29:10.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:29:10.486 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link delete ceph-brx 2024-09-12T03:29:10.597 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-12T03:29:10.598 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-12T03:29:10.598 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-12T03:29:10.611 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:10 smithi064 ceph-mon[30952]: osdmap e92: 12 total, 12 up, 12 in 2024-09-12T03:29:10.611 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:10 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/3150463515' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-12T03:29:10.612 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:10 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: ignoring --setuser ceph since I am not root 2024-09-12T03:29:10.612 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:10 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: ignoring --setgroup ceph since I am not root 2024-09-12T03:29:10.612 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:10 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:29:10.528+0000 7fa5bfb90040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T03:29:10.612 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:10 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:29:10.591+0000 7fa5bfb90040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T03:29:10.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:10 smithi115 ceph-mon[34801]: osdmap e92: 12 total, 12 up, 12 in 2024-09-12T03:29:10.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:10 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/3150463515' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-12T03:29:10.643 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:29:10 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: ignoring --setuser ceph since I am not root 2024-09-12T03:29:10.643 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:29:10 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: ignoring --setgroup ceph since I am not root 2024-09-12T03:29:10.643 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:29:10 smithi115 ceph-mgr[36172]: -- 172.21.15.115:0/74285976 <== mon.2 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560f904f30e0 con 0x560f904d1000 2024-09-12T03:29:10.643 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:29:10 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:29:10.518+0000 7faa05c55040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T03:29:10.643 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:29:10 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:29:10.575+0000 7faa05c55040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T03:29:10.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:10 smithi195 ceph-mon[34786]: osdmap e92: 12 total, 12 up, 12 in 2024-09-12T03:29:10.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:10 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/3150463515' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-12T03:29:10.706 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:29:10 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: ignoring --setuser ceph since I am not root 2024-09-12T03:29:10.707 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:29:10 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: ignoring --setgroup ceph since I am not root 2024-09-12T03:29:10.707 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:29:10 smithi195 ceph-mgr[36595]: -- 172.21.15.195:0/2044152584 <== mon.0 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56497889b0e0 con 0x564978879400 2024-09-12T03:29:10.707 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:29:10 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:29:10.519+0000 7f64eb5bc040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T03:29:10.707 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:29:10 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:29:10.576+0000 7f64eb5bc040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T03:29:10.865 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:29:10.987 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:10 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:29:10.752+0000 7fa5bfb90040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T03:29:10.987 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:10 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:29:10.828+0000 7fa5bfb90040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T03:29:11.073 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:29:10 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:29:10.722+0000 7faa05c55040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T03:29:11.074 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:29:10 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:29:10.786+0000 7faa05c55040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T03:29:11.074 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:29:10 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:29:10.933+0000 7faa05c55040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:29:11.074 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:29:11 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:29:11.073+0000 7faa05c55040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T03:29:11.078 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:29:10 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:29:10.724+0000 7f64eb5bc040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T03:29:11.078 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:29:10 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:29:10.790+0000 7f64eb5bc040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T03:29:11.078 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:29:10 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:29:10.937+0000 7f64eb5bc040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:29:11.078 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:29:11 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:29:11.078+0000 7f64eb5bc040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T03:29:11.272 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:10 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:29:10.989+0000 7fa5bfb90040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:29:11.272 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:11 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:29:11.141+0000 7fa5bfb90040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T03:29:11.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.361+0000 7f541dcd8640 1 -- 172.21.15.64:0/1851981007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54181082a0 msgr2=0x7f541810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:11.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.361+0000 7f541dcd8640 1 --2- 172.21.15.64:0/1851981007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54181082a0 0x7f541810ec40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5410009cd0 tx=0x7f541002a8e0 comp rx=0 tx=0).stop 2024-09-12T03:29:11.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.361+0000 7f541dcd8640 1 -- 172.21.15.64:0/1851981007 shutdown_connections 2024-09-12T03:29:11.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.361+0000 7f541dcd8640 1 --2- 172.21.15.64:0/1851981007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54181082a0 0x7f541810ec40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:11.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.361+0000 7f541dcd8640 1 --2- 172.21.15.64:0/1851981007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54181036e0 0x7f5418107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:11.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.361+0000 7f541dcd8640 1 --2- 172.21.15.64:0/1851981007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5418102d30 0x7f5418103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:11.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.361+0000 7f541dcd8640 1 -- 172.21.15.64:0/1851981007 >> 172.21.15.64:0/1851981007 conn(0x7f54180fc7d0 msgr2=0x7f54180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:11.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.361+0000 7f541dcd8640 1 -- 172.21.15.64:0/1851981007 shutdown_connections 2024-09-12T03:29:11.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.361+0000 7f541dcd8640 1 -- 172.21.15.64:0/1851981007 wait complete. 2024-09-12T03:29:11.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.362+0000 7f541dcd8640 1 Processor -- start 2024-09-12T03:29:11.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.362+0000 7f541dcd8640 1 -- start start 2024-09-12T03:29:11.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.362+0000 7f541dcd8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5418102d30 0x7f5418100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:11.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.362+0000 7f541dcd8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54181036e0 0x7f5418100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:11.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.362+0000 7f541dcd8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54181082a0 0x7f5418101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:11.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.362+0000 7f541dcd8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5418111f50 con 0x7f54181082a0 2024-09-12T03:29:11.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.362+0000 7f541dcd8640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5418111dd0 con 0x7f54181036e0 2024-09-12T03:29:11.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.362+0000 7f541dcd8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f54181120d0 con 0x7f5418102d30 2024-09-12T03:29:11.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.362+0000 7f5417fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54181082a0 0x7f5418101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:11.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.362+0000 7f5417fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54181082a0 0x7f5418101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36278/0 (socket says 172.21.15.64:36278) 2024-09-12T03:29:11.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.362+0000 7f5417fff640 1 -- 172.21.15.64:0/2109057526 learned_addr learned my addr 172.21.15.64:0/2109057526 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:11.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.362+0000 7f54177fe640 1 --2- 172.21.15.64:0/2109057526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5418102d30 0x7f5418100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:11.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.362+0000 7f5416ffd640 1 --2- 172.21.15.64:0/2109057526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54181036e0 0x7f5418100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:11.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.363+0000 7f5417fff640 1 -- 172.21.15.64:0/2109057526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5418102d30 msgr2=0x7f5418100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:11.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.363+0000 7f5417fff640 1 --2- 172.21.15.64:0/2109057526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5418102d30 0x7f5418100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:11.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.363+0000 7f5417fff640 1 -- 172.21.15.64:0/2109057526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54181036e0 msgr2=0x7f5418100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:11.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.363+0000 7f5417fff640 1 --2- 172.21.15.64:0/2109057526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54181036e0 0x7f5418100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:11.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.363+0000 7f5417fff640 1 -- 172.21.15.64:0/2109057526 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54181a6960 con 0x7f54181082a0 2024-09-12T03:29:11.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.363+0000 7f54177fe640 1 --2- 172.21.15.64:0/2109057526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5418102d30 0x7f5418100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:11.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.363+0000 7f5416ffd640 1 --2- 172.21.15.64:0/2109057526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54181036e0 0x7f5418100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:11.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.363+0000 7f5417fff640 1 --2- 172.21.15.64:0/2109057526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54181082a0 0x7f5418101410 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5410033040 tx=0x7f5410009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:11.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.363+0000 7f5414ff9640 1 -- 172.21.15.64:0/2109057526 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5410004040 con 0x7f54181082a0 2024-09-12T03:29:11.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.363+0000 7f5414ff9640 1 -- 172.21.15.64:0/2109057526 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5410004690 con 0x7f54181082a0 2024-09-12T03:29:11.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.363+0000 7f541dcd8640 1 -- 172.21.15.64:0/2109057526 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54181a6bf0 con 0x7f54181082a0 2024-09-12T03:29:11.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.363+0000 7f5414ff9640 1 -- 172.21.15.64:0/2109057526 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5410030da0 con 0x7f54181082a0 2024-09-12T03:29:11.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.363+0000 7f541dcd8640 1 -- 172.21.15.64:0/2109057526 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54181a7180 con 0x7f54181082a0 2024-09-12T03:29:11.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.364+0000 7f541dcd8640 1 -- 172.21.15.64:0/2109057526 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5418103110 con 0x7f54181082a0 2024-09-12T03:29:11.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.366+0000 7f5414ff9640 1 -- 172.21.15.64:0/2109057526 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f54100041e0 con 0x7f54181082a0 2024-09-12T03:29:11.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.367+0000 7f5414ff9640 1 --2- 172.21.15.64:0/2109057526 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f54040a2c50 0x7f54040a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:11.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.367+0000 7f54177fe640 1 -- 172.21.15.64:0/2109057526 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f54040a2c50 msgr2=0x7f54040a5110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/759931848 2024-09-12T03:29:11.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.367+0000 7f5414ff9640 1 -- 172.21.15.64:0/2109057526 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9237+0+0 (secure 0 0 0) 0x7f541002e030 con 0x7f54181082a0 2024-09-12T03:29:11.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.367+0000 7f54177fe640 1 --2- 172.21.15.64:0/2109057526 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f54040a2c50 0x7f54040a5110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T03:29:11.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.374+0000 7f5414ff9640 1 -- 172.21.15.64:0/2109057526 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f54100bd0b0 con 0x7f54181082a0 2024-09-12T03:29:11.543 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:11 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:29:11.478+0000 7fa5bfb90040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T03:29:11.543 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:11 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/3150463515' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-12T03:29:11.543 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:11 smithi064 ceph-mon[30952]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-12T03:29:11.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.568+0000 7f54177fe640 1 -- 172.21.15.64:0/2109057526 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f54040a2c50 msgr2=0x7f54040a5110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/759931848 2024-09-12T03:29:11.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.568+0000 7f54177fe640 1 --2- 172.21.15.64:0/2109057526 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f54040a2c50 0x7f54040a5110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T03:29:11.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.587+0000 7f541dcd8640 1 -- 172.21.15.64:0/2109057526 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5418101ec0 con 0x7f54181082a0 2024-09-12T03:29:11.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.588+0000 7f5414ff9640 1 -- 172.21.15.64:0/2109057526 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+88 (secure 0 0 0) 0x7f5418101ec0 con 0x7f54181082a0 2024-09-12T03:29:11.589 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-12T03:29:11.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.590+0000 7f541dcd8640 1 -- 172.21.15.64:0/2109057526 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f54040a2c50 msgr2=0x7f54040a5110 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:29:11.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.590+0000 7f541dcd8640 1 --2- 172.21.15.64:0/2109057526 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f54040a2c50 0x7f54040a5110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:11.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.590+0000 7f541dcd8640 1 -- 172.21.15.64:0/2109057526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54181082a0 msgr2=0x7f5418101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:11.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.590+0000 7f541dcd8640 1 --2- 172.21.15.64:0/2109057526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54181082a0 0x7f5418101410 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5410033040 tx=0x7f5410009360 comp rx=0 tx=0).stop 2024-09-12T03:29:11.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.590+0000 7f541dcd8640 1 -- 172.21.15.64:0/2109057526 shutdown_connections 2024-09-12T03:29:11.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.590+0000 7f541dcd8640 1 --2- 172.21.15.64:0/2109057526 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f54040a2c50 0x7f54040a5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:11.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.590+0000 7f541dcd8640 1 --2- 172.21.15.64:0/2109057526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54181082a0 0x7f5418101410 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:11.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.590+0000 7f541dcd8640 1 --2- 172.21.15.64:0/2109057526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54181036e0 0x7f5418100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:11.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.590+0000 7f541dcd8640 1 --2- 172.21.15.64:0/2109057526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5418102d30 0x7f5418100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:11.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.590+0000 7f541dcd8640 1 -- 172.21.15.64:0/2109057526 >> 172.21.15.64:0/2109057526 conn(0x7f54180fc7d0 msgr2=0x7f5418105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:11.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.590+0000 7f541dcd8640 1 -- 172.21.15.64:0/2109057526 shutdown_connections 2024-09-12T03:29:11.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:11.590+0000 7f541dcd8640 1 -- 172.21.15.64:0/2109057526 wait complete. 2024-09-12T03:29:11.632 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:11 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/3150463515' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-12T03:29:11.632 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:11 smithi115 ceph-mon[34801]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-12T03:29:11.632 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:29:11 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:29:11.386+0000 7faa05c55040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T03:29:11.632 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:29:11 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:29:11.448+0000 7faa05c55040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T03:29:11.635 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:11 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/3150463515' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-12T03:29:11.635 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:11 smithi195 ceph-mon[34786]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-12T03:29:11.636 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:29:11 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:29:11.392+0000 7f64eb5bc040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T03:29:11.636 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:29:11 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:29:11.452+0000 7f64eb5bc040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T03:29:11.636 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:29:11 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:29:11.635+0000 7f64eb5bc040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T03:29:11.892 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:29:11 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:29:11.632+0000 7faa05c55040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T03:29:11.973 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:11 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:29:11.542+0000 7fa5bfb90040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T03:29:11.974 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:11 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:29:11.740+0000 7fa5bfb90040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T03:29:12.329 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T03:29:12.329 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T03:29:12.329 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2024-09-12T03:29:12.329 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-12T03:29:12.329 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T03:29:12.329 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T03:29:12.329 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T03:29:12.329 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T03:29:12.329 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:29:12.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:29:12.330 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2024-09-12T03:29:12.386 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T03:29:12.386 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T03:29:12.386 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T03:29:12.386 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:29:12.386 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2024-09-12T03:29:12.387 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:29:12.387 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:29:12.387 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:29:12.387 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f0 2024-09-12T03:29:12.387 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:29:12.387 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:29:12.387 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f1 2024-09-12T03:29:12.387 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:29:12.387 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2024-09-12T03:29:12.387 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T03:29:12.387 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2024-09-12T03:29:12.387 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-12T03:29:12.388 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 42191sec preferred_lft 42191sec 2024-09-12T03:29:12.388 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link noprefixroute 2024-09-12T03:29:12.388 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:29:12.388 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-12T03:29:12.388 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:29:12.388 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-09-12T03:29:12.388 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link add name ceph-brx type bridge 2024-09-12T03:29:12.389 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip addr flush dev ceph-brx 2024-09-12T03:29:12.389 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set ceph-brx up 2024-09-12T03:29:12.389 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-12T03:29:12.389 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-09-12T03:29:12.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:29:12.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:29:12.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:29:12.534 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T03:29:12.556 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:12 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/2109057526' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T03:29:12.558 INFO:teuthology.orchestra.run.smithi064.stdout:1 2024-09-12T03:29:12.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:29:12.561 DEBUG:teuthology.orchestra.run.smithi064:> route 2024-09-12T03:29:12.618 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2024-09-12T03:29:12.618 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T03:29:12.618 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-12T03:29:12.618 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-12T03:29:12.618 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T03:29:12.619 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:29:12.619 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-09-12T03:29:12.619 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-12T03:29:12.619 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-12T03:29:12.619 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-12T03:29:12.619 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-09-12T03:29:12.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:12 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/2109057526' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T03:29:12.642 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:29:12 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:29:12.512+0000 7faa05c55040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T03:29:12.642 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:29:12 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:29:12.579+0000 7faa05c55040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T03:29:12.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:12 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/2109057526' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T03:29:12.706 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:29:12 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:29:12.521+0000 7f64eb5bc040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T03:29:12.706 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:29:12 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:29:12.589+0000 7f64eb5bc040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T03:29:12.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:29:12.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:29:12.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:29:12.769 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2024-09-12T03:29:12.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:29:12.785 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list-id 2024-09-12T03:29:12.842 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:29:12.842 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-09-12T03:29:12.842 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T03:29:12.842 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-12T03:29:12.842 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-09-12T03:29:12.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:29:12.941 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:12 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:29:12.694+0000 7fa5bfb90040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T03:29:12.942 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:12 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:29:12.782+0000 7fa5bfb90040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T03:29:12.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:29:12.965 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-12T03:29:12.965 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:29:12.965 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-09-12T03:29:12.965 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-12T03:29:12.966 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-12T03:29:12.966 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-12T03:29:12.966 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-12T03:29:12.966 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-12T03:29:12.966 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-09-12T03:29:13.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:29:13.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:29:13.089 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:29:13.089 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-09-12T03:29:13.089 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.0 up 2024-09-12T03:29:13.089 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set dev brx.0 master ceph-brx 2024-09-12T03:29:13.089 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-09-12T03:29:13.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:29:13.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:29:13.169 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-09-12T03:29:13.169 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:29:13.170 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-12T03:29:13.186 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:29:13.186 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:29:13.186 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-12T03:29:13.241 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2024-09-12T03:29:13.309 DEBUG:teuthology.orchestra.run.smithi064:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T03:29:13.365 INFO:teuthology.orchestra.run.smithi064.stdout:/proc 2024-09-12T03:29:13.366 INFO:teuthology.orchestra.run.smithi064.stdout:/sys 2024-09-12T03:29:13.366 INFO:teuthology.orchestra.run.smithi064.stdout:/dev 2024-09-12T03:29:13.366 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/security 2024-09-12T03:29:13.366 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/shm 2024-09-12T03:29:13.366 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/pts 2024-09-12T03:29:13.366 INFO:teuthology.orchestra.run.smithi064.stdout:/run 2024-09-12T03:29:13.366 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/cgroup 2024-09-12T03:29:13.366 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/pstore 2024-09-12T03:29:13.366 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/bpf 2024-09-12T03:29:13.366 INFO:teuthology.orchestra.run.smithi064.stdout:/ 2024-09-12T03:29:13.366 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/selinux 2024-09-12T03:29:13.367 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T03:29:13.367 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/mqueue 2024-09-12T03:29:13.367 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/hugepages 2024-09-12T03:29:13.367 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/debug 2024-09-12T03:29:13.367 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/tracing 2024-09-12T03:29:13.367 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/fuse/connections 2024-09-12T03:29:13.367 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/config 2024-09-12T03:29:13.367 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-sysctl.service 2024-09-12T03:29:13.367 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-12T03:29:13.367 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-12T03:29:13.367 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/nfs/rpc_pipefs 2024-09-12T03:29:13.367 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/1000 2024-09-12T03:29:13.367 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph 2024-09-12T03:29:13.368 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay 2024-09-12T03:29:13.368 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/ee33a1d8e4eaa1e034e23579453d7974193472acfcc3f767e0dd3d4df36a3590/merged 2024-09-12T03:29:13.368 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/659cd9feec649f7e9ef720891b9b8dee9cd5f2da02847139e453f34cea210194/merged 2024-09-12T03:29:13.368 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/0 2024-09-12T03:29:13.368 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T03:29:13.368 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/6657365ee9908bf70289e1e2b4eba2994c383e00f4e1271d1910290a4e352dfb/merged 2024-09-12T03:29:13.368 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/d63e2a8e350f2e9f6e0b5aaea9b095d7e0b824ac35353c9dfb67c8bbf5259935/merged 2024-09-12T03:29:13.368 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/d01078f963b9aca8da0d3791bbb79a913c48a122c7a98b85055d0e0b00bdae45/merged 2024-09-12T03:29:13.368 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/12ca51601f86349c78af34a3c17cb3ca77bb030d12b3c8f618a5a555c1466e7e/merged 2024-09-12T03:29:13.368 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/29a3719ab7c8a64396f80d3a1c6095fbc7965ffe2b1e0730f1b22eacf0ef7da5/merged 2024-09-12T03:29:13.368 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/7870e610b0a225b40a938f58f36210d45d2c00dfdaf3db8248049e82bc99279a/merged 2024-09-12T03:29:13.369 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/a27f9d3c9d12f78c7f07ff9d3dee29c8d26c31554950b6dc66444fc30bd57047/merged 2024-09-12T03:29:13.369 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/e71f1074293252655b8551c7acdc167b8003259d1051bf1071b7e92e0506ef0a/merged 2024-09-12T03:29:13.369 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns 2024-09-12T03:29:13.369 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T03:29:13.369 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T03:29:13.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:29:13.369 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2024-09-12T03:29:13.422 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-12T03:29:13.422 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/2158fa2b-94db-46f4-9312-ea8d5f733d5c --client_fs=cephfs) 2024-09-12T03:29:13.465 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2024-09-12T03:29:13.491 DEBUG:teuthology.orchestra.run.smithi064:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T03:29:13.547 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:29:13 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:29:13.289+0000 7faa05c55040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T03:29:13.547 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:29:13 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:29:13.404+0000 7faa05c55040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T03:29:13.549 INFO:teuthology.orchestra.run.smithi064.stdout:/proc 2024-09-12T03:29:13.549 INFO:teuthology.orchestra.run.smithi064.stdout:/sys 2024-09-12T03:29:13.549 INFO:teuthology.orchestra.run.smithi064.stdout:/dev 2024-09-12T03:29:13.549 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/security 2024-09-12T03:29:13.550 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/shm 2024-09-12T03:29:13.550 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/pts 2024-09-12T03:29:13.550 INFO:teuthology.orchestra.run.smithi064.stdout:/run 2024-09-12T03:29:13.550 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/cgroup 2024-09-12T03:29:13.550 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/pstore 2024-09-12T03:29:13.550 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/bpf 2024-09-12T03:29:13.550 INFO:teuthology.orchestra.run.smithi064.stdout:/ 2024-09-12T03:29:13.550 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/selinux 2024-09-12T03:29:13.550 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T03:29:13.550 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/mqueue 2024-09-12T03:29:13.550 INFO:teuthology.orchestra.run.smithi064.stdout:/dev/hugepages 2024-09-12T03:29:13.550 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/debug 2024-09-12T03:29:13.550 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/tracing 2024-09-12T03:29:13.551 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/fs/fuse/connections 2024-09-12T03:29:13.551 INFO:teuthology.orchestra.run.smithi064.stdout:/sys/kernel/config 2024-09-12T03:29:13.551 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-sysctl.service 2024-09-12T03:29:13.551 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-12T03:29:13.551 INFO:teuthology.orchestra.run.smithi064.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-12T03:29:13.551 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/nfs/rpc_pipefs 2024-09-12T03:29:13.551 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/1000 2024-09-12T03:29:13.551 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/ceph 2024-09-12T03:29:13.551 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay 2024-09-12T03:29:13.551 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/ee33a1d8e4eaa1e034e23579453d7974193472acfcc3f767e0dd3d4df36a3590/merged 2024-09-12T03:29:13.551 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/659cd9feec649f7e9ef720891b9b8dee9cd5f2da02847139e453f34cea210194/merged 2024-09-12T03:29:13.551 INFO:teuthology.orchestra.run.smithi064.stdout:/run/user/0 2024-09-12T03:29:13.551 INFO:teuthology.orchestra.run.smithi064.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T03:29:13.552 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/6657365ee9908bf70289e1e2b4eba2994c383e00f4e1271d1910290a4e352dfb/merged 2024-09-12T03:29:13.552 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/d63e2a8e350f2e9f6e0b5aaea9b095d7e0b824ac35353c9dfb67c8bbf5259935/merged 2024-09-12T03:29:13.552 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/d01078f963b9aca8da0d3791bbb79a913c48a122c7a98b85055d0e0b00bdae45/merged 2024-09-12T03:29:13.552 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/12ca51601f86349c78af34a3c17cb3ca77bb030d12b3c8f618a5a555c1466e7e/merged 2024-09-12T03:29:13.552 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/29a3719ab7c8a64396f80d3a1c6095fbc7965ffe2b1e0730f1b22eacf0ef7da5/merged 2024-09-12T03:29:13.552 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/7870e610b0a225b40a938f58f36210d45d2c00dfdaf3db8248049e82bc99279a/merged 2024-09-12T03:29:13.552 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/a27f9d3c9d12f78c7f07ff9d3dee29c8d26c31554950b6dc66444fc30bd57047/merged 2024-09-12T03:29:13.552 INFO:teuthology.orchestra.run.smithi064.stdout:/var/lib/containers/storage/overlay/e71f1074293252655b8551c7acdc167b8003259d1051bf1071b7e92e0506ef0a/merged 2024-09-12T03:29:13.552 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns 2024-09-12T03:29:13.552 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T03:29:13.552 INFO:teuthology.orchestra.run.smithi064.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T03:29:13.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:29:13.553 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2024-09-12T03:29:13.554 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:2024-09-12T03:29:13.552+0000 7ff7b8522480 -1 init, newargv = 0x562096231b40 newargc=15 2024-09-12T03:29:13.554 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[84997]: starting ceph client 2024-09-12T03:29:13.563 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:29:13 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:29:13.301+0000 7f64eb5bc040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T03:29:13.564 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:29:13 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:29:13.419+0000 7f64eb5bc040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T03:29:13.569 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[84997]: starting fuse 2024-09-12T03:29:13.584 INFO:teuthology.orchestra.run.smithi064.stdout:96 2024-09-12T03:29:13.585 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [96] 2024-09-12T03:29:13.585 DEBUG:teuthology.orchestra.run.smithi064:> sudo stdin-killer -- python3 -c ' 2024-09-12T03:29:13.585 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2024-09-12T03:29:13.585 DEBUG:teuthology.orchestra.run.smithi064:> import re 2024-09-12T03:29:13.585 DEBUG:teuthology.orchestra.run.smithi064:> import os 2024-09-12T03:29:13.585 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2024-09-12T03:29:13.585 DEBUG:teuthology.orchestra.run.smithi064:> 2024-09-12T03:29:13.585 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2024-09-12T03:29:13.586 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-12T03:29:13.586 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2024-09-12T03:29:13.586 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-12T03:29:13.586 DEBUG:teuthology.orchestra.run.smithi064:> 2024-09-12T03:29:13.586 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2024-09-12T03:29:13.586 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2024-09-12T03:29:13.586 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2024-09-12T03:29:13.586 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2024-09-12T03:29:13.586 DEBUG:teuthology.orchestra.run.smithi064:> 2024-09-12T03:29:13.586 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2024-09-12T03:29:13.586 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-12T03:29:13.586 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-12T03:29:13.586 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-12T03:29:13.586 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2024-09-12T03:29:13.587 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2024-09-12T03:29:13.587 DEBUG:teuthology.orchestra.run.smithi064:> return f 2024-09-12T03:29:13.587 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-12T03:29:13.587 DEBUG:teuthology.orchestra.run.smithi064:> 2024-09-12T03:29:13.587 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.0")) 2024-09-12T03:29:13.587 DEBUG:teuthology.orchestra.run.smithi064:> ' 2024-09-12T03:29:13.707 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.0.84997.asok 2024-09-12T03:29:13.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:29:13.718 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.84997.asok 2024-09-12T03:29:13.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:29:13.718 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.84997.asok status 2024-09-12T03:29:13.825 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:29:13 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:29:13.547+0000 7faa05c55040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:29:13.825 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:29:13 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:29:13.611+0000 7faa05c55040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T03:29:13.825 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:29:13 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:29:13.757+0000 7faa05c55040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T03:29:13.841 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:29:13 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:29:13.563+0000 7f64eb5bc040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:29:13.841 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:29:13 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:29:13.626+0000 7f64eb5bc040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T03:29:13.842 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:29:13 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:29:13.774+0000 7f64eb5bc040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T03:29:13.842 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:29:13 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:29:13.841+0000 7f64eb5bc040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T03:29:13.857 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T03:29:13.857 INFO:teuthology.orchestra.run.smithi064.stdout: "metadata": { 2024-09-12T03:29:13.857 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", 2024-09-12T03:29:13.857 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T03:29:13.857 INFO:teuthology.orchestra.run.smithi064.stdout: "entity_id": "0", 2024-09-12T03:29:13.857 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": "smithi064", 2024-09-12T03:29:13.857 INFO:teuthology.orchestra.run.smithi064.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-12T03:29:13.857 INFO:teuthology.orchestra.run.smithi064.stdout: "pid": "84997", 2024-09-12T03:29:13.857 INFO:teuthology.orchestra.run.smithi064.stdout: "root": "/volumes/qa/sv_0/2158fa2b-94db-46f4-9312-ea8d5f733d5c" 2024-09-12T03:29:13.857 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:29:13.857 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_count": 0, 2024-09-12T03:29:13.857 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_pinned_count": 0, 2024-09-12T03:29:13.858 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 15255, 2024-09-12T03:29:13.858 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2024-09-12T03:29:13.858 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2024-09-12T03:29:13.858 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2024-09-12T03:29:13.858 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 15255 2024-09-12T03:29:13.858 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:29:13.858 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2024-09-12T03:29:13.858 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2024-09-12T03:29:13.858 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2024-09-12T03:29:13.858 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 2730200417 2024-09-12T03:29:13.858 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:29:13.858 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:29:13.858 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2024-09-12T03:29:13.858 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2024-09-12T03:29:13.858 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2024-09-12T03:29:13.859 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 2730200417 2024-09-12T03:29:13.859 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:29:13.859 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.15255 192.168.0.1:0/2730200417", 2024-09-12T03:29:13.859 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "192.168.0.1:0/2730200417", 2024-09-12T03:29:13.859 INFO:teuthology.orchestra.run.smithi064.stdout: "inode_count": 5, 2024-09-12T03:29:13.859 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_epoch": 24, 2024-09-12T03:29:13.859 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch": 92, 2024-09-12T03:29:13.859 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch_barrier": 0, 2024-09-12T03:29:13.859 INFO:teuthology.orchestra.run.smithi064.stdout: "blocklisted": false, 2024-09-12T03:29:13.859 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs" 2024-09-12T03:29:13.859 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T03:29:13.860 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:13 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:29:13.579+0000 7fa5bfb90040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T03:29:13.860 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:13 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:29:13.703+0000 7fa5bfb90040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T03:29:13.860 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:13 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:29:13.856+0000 7fa5bfb90040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:29:13.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:29:13.866 DEBUG:teuthology.orchestra.run.smithi064:> stat --file-system '--printf=%T 2024-09-12T03:29:13.866 DEBUG:teuthology.orchestra.run.smithi064:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-12T03:29:13.883 INFO:teuthology.orchestra.run.smithi064.stdout:fuseblk 2024-09-12T03:29:13.883 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-12T03:29:13.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:29:13.883 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-12T03:29:13.952 INFO:teuthology.run_tasks:Running task exec... 2024-09-12T03:29:13.962 INFO:teuthology.task.exec:Executing custom commands... 2024-09-12T03:29:13.962 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi064.front.sepia.ceph.com 2024-09-12T03:29:13.962 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-12T03:29:14.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.100+0000 7fad43326640 1 -- 172.21.15.64:0/1276097118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad3c104ef0 msgr2=0x7fad3c1072e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:14.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.100+0000 7fad43326640 1 --2- 172.21.15.64:0/1276097118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad3c104ef0 0x7fad3c1072e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fad2c009900 tx=0x7fad2c02a410 comp rx=0 tx=0).stop 2024-09-12T03:29:14.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.100+0000 7fad43326640 1 -- 172.21.15.64:0/1276097118 shutdown_connections 2024-09-12T03:29:14.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.100+0000 7fad43326640 1 --2- 172.21.15.64:0/1276097118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad3c107820 0x7fad3c109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.100+0000 7fad43326640 1 --2- 172.21.15.64:0/1276097118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad3c104ef0 0x7fad3c1072e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.100+0000 7fad43326640 1 --2- 172.21.15.64:0/1276097118 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad3c1025c0 0x7fad3c1049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.100+0000 7fad43326640 1 -- 172.21.15.64:0/1276097118 >> 172.21.15.64:0/1276097118 conn(0x7fad3c0fc1e0 msgr2=0x7fad3c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:14.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.100+0000 7fad43326640 1 -- 172.21.15.64:0/1276097118 shutdown_connections 2024-09-12T03:29:14.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.100+0000 7fad43326640 1 -- 172.21.15.64:0/1276097118 wait complete. 2024-09-12T03:29:14.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.100+0000 7fad43326640 1 Processor -- start 2024-09-12T03:29:14.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.100+0000 7fad43326640 1 -- start start 2024-09-12T03:29:14.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.101+0000 7fad43326640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad3c1025c0 0x7fad3c197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:14.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.101+0000 7fad43326640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad3c104ef0 0x7fad3c1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:14.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.101+0000 7fad43326640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad3c107820 0x7fad3c19f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:14.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.101+0000 7fad43326640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fad3c10d6a0 con 0x7fad3c104ef0 2024-09-12T03:29:14.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.101+0000 7fad43326640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fad3c10d520 con 0x7fad3c107820 2024-09-12T03:29:14.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.101+0000 7fad43326640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fad3c10d820 con 0x7fad3c1025c0 2024-09-12T03:29:14.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.101+0000 7fad408c1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad3c104ef0 0x7fad3c1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:14.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.101+0000 7fad410c2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad3c1025c0 0x7fad3c197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:14.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.101+0000 7fad408c1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad3c104ef0 0x7fad3c1982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36284/0 (socket says 172.21.15.64:36284) 2024-09-12T03:29:14.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.101+0000 7fad410c2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad3c1025c0 0x7fad3c197da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.64:41038/0 (socket says 172.21.15.64:41038) 2024-09-12T03:29:14.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.101+0000 7fad408c1640 1 -- 172.21.15.64:0/1426924515 learned_addr learned my addr 172.21.15.64:0/1426924515 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:14.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.101+0000 7fad418c3640 1 --2- 172.21.15.64:0/1426924515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad3c107820 0x7fad3c19f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:14.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.101+0000 7fad408c1640 1 -- 172.21.15.64:0/1426924515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad3c1025c0 msgr2=0x7fad3c197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:14.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.101+0000 7fad408c1640 1 --2- 172.21.15.64:0/1426924515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad3c1025c0 0x7fad3c197da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.101+0000 7fad408c1640 1 -- 172.21.15.64:0/1426924515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad3c107820 msgr2=0x7fad3c19f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:14.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.101+0000 7fad408c1640 1 --2- 172.21.15.64:0/1426924515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad3c107820 0x7fad3c19f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.101+0000 7fad408c1640 1 -- 172.21.15.64:0/1426924515 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad3c19f9b0 con 0x7fad3c104ef0 2024-09-12T03:29:14.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.101+0000 7fad410c2640 1 --2- 172.21.15.64:0/1426924515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad3c1025c0 0x7fad3c197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:14.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.101+0000 7fad418c3640 1 --2- 172.21.15.64:0/1426924515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad3c107820 0x7fad3c19f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:14.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.101+0000 7fad408c1640 1 --2- 172.21.15.64:0/1426924515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad3c104ef0 0x7fad3c1982e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fad2c0314e0 tx=0x7fad2c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:14.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.101+0000 7fad327fc640 1 -- 172.21.15.64:0/1426924515 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2c03f030 con 0x7fad3c104ef0 2024-09-12T03:29:14.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.101+0000 7fad327fc640 1 -- 172.21.15.64:0/1426924515 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad2c003aa0 con 0x7fad3c104ef0 2024-09-12T03:29:14.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.102+0000 7fad43326640 1 -- 172.21.15.64:0/1426924515 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad3c19fc40 con 0x7fad3c104ef0 2024-09-12T03:29:14.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.102+0000 7fad327fc640 1 -- 172.21.15.64:0/1426924515 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2c035070 con 0x7fad3c104ef0 2024-09-12T03:29:14.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.102+0000 7fad43326640 1 -- 172.21.15.64:0/1426924515 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad3c1a0130 con 0x7fad3c104ef0 2024-09-12T03:29:14.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.103+0000 7fad43326640 1 -- 172.21.15.64:0/1426924515 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad3c0fdcd0 con 0x7fad3c104ef0 2024-09-12T03:29:14.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.104+0000 7fad327fc640 1 -- 172.21.15.64:0/1426924515 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7fad2c042030 con 0x7fad3c104ef0 2024-09-12T03:29:14.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.105+0000 7fad327fc640 1 --2- 172.21.15.64:0/1426924515 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fad280a2c80 0x7fad280a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:14.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.105+0000 7fad410c2640 1 -- 172.21.15.64:0/1426924515 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fad280a2c80 msgr2=0x7fad280a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/759931848 2024-09-12T03:29:14.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.105+0000 7fad327fc640 1 -- 172.21.15.64:0/1426924515 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9237+0+0 (secure 0 0 0) 0x7fad2c0efbe0 con 0x7fad3c104ef0 2024-09-12T03:29:14.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.105+0000 7fad410c2640 1 --2- 172.21.15.64:0/1426924515 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fad280a2c80 0x7fad280a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T03:29:14.111 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:29:13 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:29:13.825+0000 7faa05c55040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T03:29:14.111 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:29:13 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:29:13.884+0000 7faa05c55040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T03:29:14.111 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:29:14 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:29:14.042+0000 7faa05c55040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T03:29:14.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.109+0000 7fad327fc640 1 -- 172.21.15.64:0/1426924515 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fad2c0bc100 con 0x7fad3c104ef0 2024-09-12T03:29:14.130 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:29:13 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:29:13.902+0000 7f64eb5bc040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T03:29:14.130 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:29:14 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:29:14.063+0000 7f64eb5bc040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T03:29:14.163 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:13 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:29:13.922+0000 7fa5bfb90040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T03:29:14.163 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:14 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:29:14.087+0000 7fa5bfb90040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T03:29:14.163 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:14 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:29:14.161+0000 7fa5bfb90040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T03:29:14.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.306+0000 7fad410c2640 1 -- 172.21.15.64:0/1426924515 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fad280a2c80 msgr2=0x7fad280a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/759931848 2024-09-12T03:29:14.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.306+0000 7fad410c2640 1 --2- 172.21.15.64:0/1426924515 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fad280a2c80 0x7fad280a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T03:29:14.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.323+0000 7fad43326640 1 -- 172.21.15.64:0/1426924515 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7fad3c062df0 con 0x7fad3c104ef0 2024-09-12T03:29:14.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.375+0000 7fad327fc640 1 -- 172.21.15.64:0/1426924515 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v25) ==== 117+0+0 (secure 0 0 0) 0x7fad2c0c0400 con 0x7fad3c104ef0 2024-09-12T03:29:14.377 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:29:14 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:29:14.109+0000 7faa05c55040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T03:29:14.377 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:29:14 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:29:14.176+0000 7faa05c55040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T03:29:14.377 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:29:14 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:29:14.258+0000 7faa05c55040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T03:29:14.377 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:29:14 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:29:14.317+0000 7faa05c55040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T03:29:14.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.377+0000 7fad43326640 1 -- 172.21.15.64:0/1426924515 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fad280a2c80 msgr2=0x7fad280a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:29:14.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.377+0000 7fad43326640 1 --2- 172.21.15.64:0/1426924515 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fad280a2c80 0x7fad280a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.377+0000 7fad43326640 1 -- 172.21.15.64:0/1426924515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad3c104ef0 msgr2=0x7fad3c1982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:14.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.377+0000 7fad43326640 1 --2- 172.21.15.64:0/1426924515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad3c104ef0 0x7fad3c1982e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fad2c0314e0 tx=0x7fad2c004270 comp rx=0 tx=0).stop 2024-09-12T03:29:14.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.377+0000 7fad43326640 1 -- 172.21.15.64:0/1426924515 shutdown_connections 2024-09-12T03:29:14.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.377+0000 7fad43326640 1 --2- 172.21.15.64:0/1426924515 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7fad280a2c80 0x7fad280a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.377+0000 7fad43326640 1 --2- 172.21.15.64:0/1426924515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad3c107820 0x7fad3c19f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.377+0000 7fad43326640 1 --2- 172.21.15.64:0/1426924515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad3c104ef0 0x7fad3c1982e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.377+0000 7fad43326640 1 --2- 172.21.15.64:0/1426924515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad3c1025c0 0x7fad3c197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.377+0000 7fad43326640 1 -- 172.21.15.64:0/1426924515 >> 172.21.15.64:0/1426924515 conn(0x7fad3c0fc1e0 msgr2=0x7fad3c102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:14.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.377+0000 7fad43326640 1 -- 172.21.15.64:0/1426924515 shutdown_connections 2024-09-12T03:29:14.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:14.377+0000 7fad43326640 1 -- 172.21.15.64:0/1426924515 wait complete. 2024-09-12T03:29:14.380 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:29:14 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:29:14.129+0000 7f64eb5bc040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T03:29:14.380 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:29:14 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:29:14.197+0000 7f64eb5bc040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T03:29:14.380 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:29:14 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:29:14.280+0000 7f64eb5bc040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T03:29:14.381 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:29:14 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:29:14.339+0000 7f64eb5bc040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T03:29:14.389 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-12T03:29:14.744 INFO:teuthology.run_tasks:Running task exec... 2024-09-12T03:29:15.336 INFO:teuthology.task.exec:Executing custom commands... 2024-09-12T03:29:15.336 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi064.front.sepia.ceph.com 2024-09-12T03:29:15.336 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-09-12T03:29:15.338 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:14 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/1426924515' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-12T03:29:15.338 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:29:14 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:29:14.376+0000 7faa05c55040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T03:29:15.338 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:14 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/1426924515' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-12T03:29:15.338 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:29:14 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:29:14.400+0000 7f64eb5bc040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T03:29:15.339 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:14 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/1426924515' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-12T03:29:15.339 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:14 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:29:14.226+0000 7fa5bfb90040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T03:29:15.339 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:14 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:29:14.396+0000 7fa5bfb90040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T03:29:15.339 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:14 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:29:14.465+0000 7fa5bfb90040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T03:29:15.339 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:14 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:29:14.535+0000 7fa5bfb90040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T03:29:15.339 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:14 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:29:14.622+0000 7fa5bfb90040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T03:29:15.339 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:14 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:29:14.684+0000 7fa5bfb90040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T03:29:15.339 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:14 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:29:14.746+0000 7fa5bfb90040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T03:29:15.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.451+0000 7f49d7fff640 1 -- 172.21.15.64:0/3770261065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49d8104d90 msgr2=0x7f49d8107180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:15.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.451+0000 7f49d7fff640 1 --2- 172.21.15.64:0/3770261065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49d8104d90 0x7f49d8107180 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009900 tx=0x7f49cc02a410 comp rx=0 tx=0).stop 2024-09-12T03:29:15.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.451+0000 7f49d7fff640 1 -- 172.21.15.64:0/3770261065 shutdown_connections 2024-09-12T03:29:15.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.451+0000 7f49d7fff640 1 --2- 172.21.15.64:0/3770261065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49d81076c0 0x7f49d8109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:15.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.451+0000 7f49d7fff640 1 --2- 172.21.15.64:0/3770261065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49d8104d90 0x7f49d8107180 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:15.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.451+0000 7f49d7fff640 1 --2- 172.21.15.64:0/3770261065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49d8102460 0x7f49d8104850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:15.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.452+0000 7f49d7fff640 1 -- 172.21.15.64:0/3770261065 >> 172.21.15.64:0/3770261065 conn(0x7f49d80fbfa0 msgr2=0x7f49d80fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:15.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.452+0000 7f49d7fff640 1 -- 172.21.15.64:0/3770261065 shutdown_connections 2024-09-12T03:29:15.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.452+0000 7f49d7fff640 1 -- 172.21.15.64:0/3770261065 wait complete. 2024-09-12T03:29:15.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.452+0000 7f49d7fff640 1 Processor -- start 2024-09-12T03:29:15.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.453+0000 7f49d7fff640 1 -- start start 2024-09-12T03:29:15.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.453+0000 7f49d7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49d8102460 0x7f49d8197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:15.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.453+0000 7f49d7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49d8104d90 0x7f49d81981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:15.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.453+0000 7f49d7fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49d81076c0 0x7f49d819f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:15.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.453+0000 7f49d7fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f49d810d480 con 0x7f49d8104d90 2024-09-12T03:29:15.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.453+0000 7f49d7fff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f49d810d300 con 0x7f49d8102460 2024-09-12T03:29:15.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.453+0000 7f49d7fff640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f49d810d600 con 0x7f49d81076c0 2024-09-12T03:29:15.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.453+0000 7f49d67fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49d8104d90 0x7f49d81981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:15.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.453+0000 7f49d67fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49d8104d90 0x7f49d81981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36316/0 (socket says 172.21.15.64:36316) 2024-09-12T03:29:15.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.453+0000 7f49d67fc640 1 -- 172.21.15.64:0/3675311424 learned_addr learned my addr 172.21.15.64:0/3675311424 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:15.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.453+0000 7f49d6ffd640 1 --2- 172.21.15.64:0/3675311424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49d8102460 0x7f49d8197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:15.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.453+0000 7f49d77fe640 1 --2- 172.21.15.64:0/3675311424 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49d81076c0 0x7f49d819f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:15.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.453+0000 7f49d67fc640 1 -- 172.21.15.64:0/3675311424 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49d81076c0 msgr2=0x7f49d819f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:15.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.453+0000 7f49d67fc640 1 --2- 172.21.15.64:0/3675311424 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49d81076c0 0x7f49d819f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:15.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.453+0000 7f49d67fc640 1 -- 172.21.15.64:0/3675311424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49d8102460 msgr2=0x7f49d8197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:15.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.453+0000 7f49d67fc640 1 --2- 172.21.15.64:0/3675311424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49d8102460 0x7f49d8197c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:15.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.453+0000 7f49d67fc640 1 -- 172.21.15.64:0/3675311424 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d819f880 con 0x7f49d8104d90 2024-09-12T03:29:15.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.453+0000 7f49d6ffd640 1 --2- 172.21.15.64:0/3675311424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49d8102460 0x7f49d8197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:15.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.454+0000 7f49d67fc640 1 --2- 172.21.15.64:0/3675311424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49d8104d90 0x7f49d81981b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f49cc0314e0 tx=0x7f49cc031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:15.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.454+0000 7f49bffff640 1 -- 172.21.15.64:0/3675311424 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49cc03f030 con 0x7f49d8104d90 2024-09-12T03:29:15.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.454+0000 7f49bffff640 1 -- 172.21.15.64:0/3675311424 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49cc0044b0 con 0x7f49d8104d90 2024-09-12T03:29:15.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.454+0000 7f49d7fff640 1 -- 172.21.15.64:0/3675311424 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d819fb10 con 0x7f49d8104d90 2024-09-12T03:29:15.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.454+0000 7f49bffff640 1 -- 172.21.15.64:0/3675311424 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49cc035070 con 0x7f49d8104d90 2024-09-12T03:29:15.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.454+0000 7f49d7fff640 1 -- 172.21.15.64:0/3675311424 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d81a0000 con 0x7f49d8104d90 2024-09-12T03:29:15.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.455+0000 7f49d7fff640 1 -- 172.21.15.64:0/3675311424 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49d80fda90 con 0x7f49d8104d90 2024-09-12T03:29:15.458 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:29:15 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z[36591]: 2024-09-12T03:29:15.194+0000 7f64eb5bc040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T03:29:15.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:15 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/1426924515' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-12T03:29:15.459 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:15 smithi195 ceph-mon[34786]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:29:15.459 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:15 smithi195 ceph-mon[34786]: Standby manager daemon y restarted 2024-09-12T03:29:15.459 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:15 smithi195 ceph-mon[34786]: Standby manager daemon y started 2024-09-12T03:29:15.459 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:15 smithi195 ceph-mon[34786]: from='mgr.? 172.21.15.115:0/895675849' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T03:29:15.459 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:15 smithi195 ceph-mon[34786]: from='mgr.? 172.21.15.115:0/895675849' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:29:15.459 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:15 smithi195 ceph-mon[34786]: from='mgr.? 172.21.15.115:0/895675849' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T03:29:15.459 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:15 smithi195 ceph-mon[34786]: from='mgr.? 172.21.15.115:0/895675849' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:29:15.459 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:15 smithi195 ceph-mon[34786]: Standby manager daemon z restarted 2024-09-12T03:29:15.459 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:15 smithi195 ceph-mon[34786]: Standby manager daemon z started 2024-09-12T03:29:15.459 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:15 smithi195 ceph-mon[34786]: from='mgr.? 172.21.15.195:0/629377003' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T03:29:15.459 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:15 smithi195 ceph-mon[34786]: from='mgr.? 172.21.15.195:0/629377003' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:29:15.459 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:15 smithi195 ceph-mon[34786]: from='mgr.? 172.21.15.195:0/629377003' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T03:29:15.459 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:15 smithi195 ceph-mon[34786]: from='mgr.? 172.21.15.195:0/629377003' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:29:15.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.457+0000 7f49bffff640 1 -- 172.21.15.64:0/3675311424 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 20) ==== 137850+0+0 (secure 0 0 0) 0x7f49cc042030 con 0x7f49d8104d90 2024-09-12T03:29:15.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.458+0000 7f49bffff640 1 --2- 172.21.15.64:0/3675311424 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f49c00a2bd0 0x7f49c00a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:15.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.458+0000 7f49d6ffd640 1 -- 172.21.15.64:0/3675311424 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f49c00a2bd0 msgr2=0x7f49c00a5090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/759931848 2024-09-12T03:29:15.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.458+0000 7f49bffff640 1 -- 172.21.15.64:0/3675311424 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9237+0+0 (secure 0 0 0) 0x7f49cc0efdb0 con 0x7f49d8104d90 2024-09-12T03:29:15.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.458+0000 7f49d6ffd640 1 --2- 172.21.15.64:0/3675311424 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f49c00a2bd0 0x7f49c00a5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T03:29:15.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.460+0000 7f49bffff640 1 -- 172.21.15.64:0/3675311424 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f49cc0bc2d0 con 0x7f49d8104d90 2024-09-12T03:29:15.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.608+0000 7f49bffff640 1 -- 172.21.15.64:0/3675311424 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 21) ==== 137518+0+0 (secure 0 0 0) 0x7f49cc0b88b0 con 0x7f49d8104d90 2024-09-12T03:29:15.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.609+0000 7f49bffff640 1 -- 172.21.15.64:0/3675311424 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f49c00a2bd0 msgr2=0x7f49c00a5090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:29:15.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.609+0000 7f49bffff640 1 --2- 172.21.15.64:0/3675311424 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f49c00a2bd0 0x7f49c00a5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:15.642 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:29:15 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y[36168]: 2024-09-12T03:29:15.167+0000 7faa05c55040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T03:29:15.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:15 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/1426924515' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-12T03:29:15.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:15 smithi115 ceph-mon[34801]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:29:15.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:15 smithi115 ceph-mon[34801]: Standby manager daemon y restarted 2024-09-12T03:29:15.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:15 smithi115 ceph-mon[34801]: Standby manager daemon y started 2024-09-12T03:29:15.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:15 smithi115 ceph-mon[34801]: from='mgr.? 172.21.15.115:0/895675849' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T03:29:15.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:15 smithi115 ceph-mon[34801]: from='mgr.? 172.21.15.115:0/895675849' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:29:15.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:15 smithi115 ceph-mon[34801]: from='mgr.? 172.21.15.115:0/895675849' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T03:29:15.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:15 smithi115 ceph-mon[34801]: from='mgr.? 172.21.15.115:0/895675849' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:29:15.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:15 smithi115 ceph-mon[34801]: Standby manager daemon z restarted 2024-09-12T03:29:15.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:15 smithi115 ceph-mon[34801]: Standby manager daemon z started 2024-09-12T03:29:15.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:15 smithi115 ceph-mon[34801]: from='mgr.? 172.21.15.195:0/629377003' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T03:29:15.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:15 smithi115 ceph-mon[34801]: from='mgr.? 172.21.15.195:0/629377003' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:29:15.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:15 smithi115 ceph-mon[34801]: from='mgr.? 172.21.15.195:0/629377003' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T03:29:15.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:15 smithi115 ceph-mon[34801]: from='mgr.? 172.21.15.195:0/629377003' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:29:15.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.649+0000 7f49d7fff640 1 -- 172.21.15.64:0/3675311424 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f49d8062df0 con 0x7f49d8104d90 2024-09-12T03:29:15.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.653+0000 7f49bffff640 1 -- 172.21.15.64:0/3675311424 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v29)=0 v29) ==== 129+0+0 (secure 0 0 0) 0x7f49cc0b4b70 con 0x7f49d8104d90 2024-09-12T03:29:15.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.655+0000 7f49d7fff640 1 -- 172.21.15.64:0/3675311424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49d8104d90 msgr2=0x7f49d81981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:15.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.655+0000 7f49d7fff640 1 --2- 172.21.15.64:0/3675311424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49d8104d90 0x7f49d81981b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f49cc0314e0 tx=0x7f49cc031a30 comp rx=0 tx=0).stop 2024-09-12T03:29:15.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.656+0000 7f49d7fff640 1 -- 172.21.15.64:0/3675311424 shutdown_connections 2024-09-12T03:29:15.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.656+0000 7f49d7fff640 1 --2- 172.21.15.64:0/3675311424 >> [v2:172.21.15.64:6800/759931848,v1:172.21.15.64:6801/759931848] conn(0x7f49c00a2bd0 0x7f49c00a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:15.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.656+0000 7f49d7fff640 1 --2- 172.21.15.64:0/3675311424 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49d81076c0 0x7f49d819f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:15.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.656+0000 7f49d7fff640 1 --2- 172.21.15.64:0/3675311424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49d8104d90 0x7f49d81981b0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:15.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.656+0000 7f49d7fff640 1 --2- 172.21.15.64:0/3675311424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49d8102460 0x7f49d8197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:15.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.656+0000 7f49d7fff640 1 -- 172.21.15.64:0/3675311424 >> 172.21.15.64:0/3675311424 conn(0x7f49d80fbfa0 msgr2=0x7f49d8102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:15.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.656+0000 7f49d7fff640 1 -- 172.21.15.64:0/3675311424 shutdown_connections 2024-09-12T03:29:15.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.656+0000 7f49d7fff640 1 -- 172.21.15.64:0/3675311424 wait complete. 2024-09-12T03:29:15.667 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:15 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:29:15.594+0000 7fa5bfb90040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T03:29:15.667 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:15 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/1426924515' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-12T03:29:15.667 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:15 smithi064 ceph-mon[30952]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:29:15.667 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:15 smithi064 ceph-mon[30952]: Standby manager daemon y restarted 2024-09-12T03:29:15.667 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:15 smithi064 ceph-mon[30952]: Standby manager daemon y started 2024-09-12T03:29:15.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:15 smithi064 ceph-mon[30952]: from='mgr.? 172.21.15.115:0/895675849' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T03:29:15.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:15 smithi064 ceph-mon[30952]: from='mgr.? 172.21.15.115:0/895675849' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:29:15.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:15 smithi064 ceph-mon[30952]: from='mgr.? 172.21.15.115:0/895675849' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T03:29:15.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:15 smithi064 ceph-mon[30952]: from='mgr.? 172.21.15.115:0/895675849' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:29:15.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:15 smithi064 ceph-mon[30952]: Standby manager daemon z restarted 2024-09-12T03:29:15.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:15 smithi064 ceph-mon[30952]: Standby manager daemon z started 2024-09-12T03:29:15.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:15 smithi064 ceph-mon[30952]: from='mgr.? 172.21.15.195:0/629377003' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T03:29:15.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:15 smithi064 ceph-mon[30952]: from='mgr.? 172.21.15.195:0/629377003' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:29:15.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:15 smithi064 ceph-mon[30952]: from='mgr.? 172.21.15.195:0/629377003' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T03:29:15.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:15 smithi064 ceph-mon[30952]: from='mgr.? 172.21.15.195:0/629377003' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:29:15.669 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-09-12T03:29:15.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.813+0000 7f558354e640 1 -- 172.21.15.64:0/350010176 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f557409b350 msgr2=0x7f557409d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:15.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.813+0000 7f558354e640 1 --2- 172.21.15.64:0/350010176 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f557409b350 0x7f557409d7c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f557c050280 tx=0x7f557c096c10 comp rx=0 tx=0).stop 2024-09-12T03:29:15.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.813+0000 7f558354e640 1 -- 172.21.15.64:0/350010176 shutdown_connections 2024-09-12T03:29:15.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.813+0000 7f558354e640 1 --2- 172.21.15.64:0/350010176 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f557409b350 0x7f557409d7c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:15.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.813+0000 7f558354e640 1 --2- 172.21.15.64:0/350010176 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5574098a20 0x7f557409ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:15.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.813+0000 7f558354e640 1 --2- 172.21.15.64:0/350010176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55740960f0 0x7f55740984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:15.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.813+0000 7f558354e640 1 -- 172.21.15.64:0/350010176 >> 172.21.15.64:0/350010176 conn(0x7f557408fcb0 msgr2=0x7f5574092110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:15.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.814+0000 7f558354e640 1 -- 172.21.15.64:0/350010176 shutdown_connections 2024-09-12T03:29:15.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.814+0000 7f558354e640 1 -- 172.21.15.64:0/350010176 wait complete. 2024-09-12T03:29:15.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.815+0000 7f558354e640 1 Processor -- start 2024-09-12T03:29:15.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.815+0000 7f558354e640 1 -- start start 2024-09-12T03:29:15.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.815+0000 7f558354e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55740960f0 0x7f55740a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:15.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.815+0000 7f558354e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5574098a20 0x7f55740a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:15.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.815+0000 7f558354e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f557409b350 0x7f55740a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:15.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.815+0000 7f558354e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f55740a10b0 con 0x7f5574098a20 2024-09-12T03:29:15.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.815+0000 7f558354e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f55740a0f30 con 0x7f55740960f0 2024-09-12T03:29:15.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.815+0000 7f558354e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f55740a1230 con 0x7f557409b350 2024-09-12T03:29:15.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.816+0000 7f5582d4d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f557409b350 0x7f55740a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:15.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.816+0000 7f5582d4d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f557409b350 0x7f55740a1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.64:41122/0 (socket says 172.21.15.64:41122) 2024-09-12T03:29:15.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.816+0000 7f5582d4d640 1 -- 172.21.15.64:0/2871322457 learned_addr learned my addr 172.21.15.64:0/2871322457 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:15.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.816+0000 7f5581d4b640 1 --2- 172.21.15.64:0/2871322457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5574098a20 0x7f55740a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:15.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.816+0000 7f558254c640 1 --2- 172.21.15.64:0/2871322457 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55740960f0 0x7f55740a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:15.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.816+0000 7f558254c640 1 -- 172.21.15.64:0/2871322457 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f557409b350 msgr2=0x7f55740a1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:15.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.816+0000 7f558254c640 1 --2- 172.21.15.64:0/2871322457 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f557409b350 0x7f55740a1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:15.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.816+0000 7f558254c640 1 -- 172.21.15.64:0/2871322457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5574098a20 msgr2=0x7f55740a5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:15.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.816+0000 7f558254c640 1 --2- 172.21.15.64:0/2871322457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5574098a20 0x7f55740a5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:15.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.816+0000 7f558254c640 1 -- 172.21.15.64:0/2871322457 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55740a2800 con 0x7f55740960f0 2024-09-12T03:29:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.816+0000 7f558254c640 1 --2- 172.21.15.64:0/2871322457 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55740960f0 0x7f55740a53d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f557800e9d0 tx=0x7f557800ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.816+0000 7f55737fe640 1 -- 172.21.15.64:0/2871322457 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557800cde0 con 0x7f55740960f0 2024-09-12T03:29:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.816+0000 7f558354e640 1 -- 172.21.15.64:0/2871322457 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f557414acc0 con 0x7f55740960f0 2024-09-12T03:29:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.817+0000 7f55737fe640 1 -- 172.21.15.64:0/2871322457 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5578005030 con 0x7f55740960f0 2024-09-12T03:29:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.817+0000 7f55737fe640 1 -- 172.21.15.64:0/2871322457 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5578004a00 con 0x7f55740960f0 2024-09-12T03:29:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.817+0000 7f558354e640 1 -- 172.21.15.64:0/2871322457 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f557414b070 con 0x7f55740960f0 2024-09-12T03:29:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.819+0000 7f558354e640 1 -- 172.21.15.64:0/2871322457 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f557409ce60 con 0x7f55740960f0 2024-09-12T03:29:15.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.820+0000 7f55737fe640 1 -- 172.21.15.64:0/2871322457 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 21) ==== 137518+0+0 (secure 0 0 0) 0x7f5578004c60 con 0x7f55740960f0 2024-09-12T03:29:15.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.821+0000 7f55737fe640 1 -- 172.21.15.64:0/2871322457 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f55780cf230 con 0x7f55740960f0 2024-09-12T03:29:15.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:15.825+0000 7f55737fe640 1 -- 172.21.15.64:0/2871322457 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f557809c040 con 0x7f55740960f0 2024-09-12T03:29:16.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.109+0000 7f558354e640 1 -- 172.21.15.64:0/2871322457 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f55740028c0 con 0x7f55740960f0 2024-09-12T03:29:16.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.115+0000 7f55737fe640 1 -- 172.21.15.64:0/2871322457 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v30)=0 v30) ==== 138+0+0 (secure 0 0 0) 0x7f557809b940 con 0x7f55740960f0 2024-09-12T03:29:16.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.117+0000 7f558354e640 1 -- 172.21.15.64:0/2871322457 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55740960f0 msgr2=0x7f55740a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:16.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.117+0000 7f558354e640 1 --2- 172.21.15.64:0/2871322457 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55740960f0 0x7f55740a53d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f557800e9d0 tx=0x7f557800ee90 comp rx=0 tx=0).stop 2024-09-12T03:29:16.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.117+0000 7f558354e640 1 -- 172.21.15.64:0/2871322457 shutdown_connections 2024-09-12T03:29:16.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.117+0000 7f558354e640 1 --2- 172.21.15.64:0/2871322457 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f557409b350 0x7f55740a1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:16.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.117+0000 7f558354e640 1 --2- 172.21.15.64:0/2871322457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5574098a20 0x7f55740a5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:16.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.117+0000 7f558354e640 1 --2- 172.21.15.64:0/2871322457 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55740960f0 0x7f55740a53d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:16.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.117+0000 7f558354e640 1 -- 172.21.15.64:0/2871322457 >> 172.21.15.64:0/2871322457 conn(0x7f557408fcb0 msgr2=0x7f5574096b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:16.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.117+0000 7f558354e640 1 -- 172.21.15.64:0/2871322457 shutdown_connections 2024-09-12T03:29:16.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.117+0000 7f558354e640 1 -- 172.21.15.64:0/2871322457 wait complete. 2024-09-12T03:29:16.133 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-09-12T03:29:16.233 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:15 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:29:15.917+0000 7fa529967640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:29:16.233 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:15 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:29:15.917+0000 7fa529967640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:29:16.233 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:15 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:29:15.917+0000 7fa529967640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:29:16.233 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:15 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:29:15.917+0000 7fa529967640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:29:16.233 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:15 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:29:15.917+0000 7fa529967640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:29:16.233 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:15 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:29:15.919+0000 7fa526961640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:29:16.233 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:15 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:29:15.919+0000 7fa526961640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:29:16.234 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:15 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:29:15.919+0000 7fa526961640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:29:16.234 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:15 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:29:15.919+0000 7fa526961640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:29:16.234 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:15 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:29:15.919+0000 7fa526961640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:29:16.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.240+0000 7f061b4e6640 1 -- 172.21.15.64:0/1798522522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0614102600 msgr2=0x7f0614108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:16.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.240+0000 7f061b4e6640 1 --2- 172.21.15.64:0/1798522522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0614102600 0x7f0614108e90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f060c009ca0 tx=0x7f060c02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:29:16.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.240+0000 7f061b4e6640 1 -- 172.21.15.64:0/1798522522 shutdown_connections 2024-09-12T03:29:16.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.241+0000 7f061b4e6640 1 --2- 172.21.15.64:0/1798522522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0614102600 0x7f0614108e90 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:16.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.241+0000 7f061b4e6640 1 --2- 172.21.15.64:0/1798522522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0614101c40 0x7f06141020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:16.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.241+0000 7f061b4e6640 1 --2- 172.21.15.64:0/1798522522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0614100a50 0x7f0614100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:16.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.241+0000 7f061b4e6640 1 -- 172.21.15.64:0/1798522522 >> 172.21.15.64:0/1798522522 conn(0x7f06140fc1b0 msgr2=0x7f06140fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:16.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.241+0000 7f061b4e6640 1 -- 172.21.15.64:0/1798522522 shutdown_connections 2024-09-12T03:29:16.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.241+0000 7f061b4e6640 1 -- 172.21.15.64:0/1798522522 wait complete. 2024-09-12T03:29:16.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.241+0000 7f061b4e6640 1 Processor -- start 2024-09-12T03:29:16.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.241+0000 7f061b4e6640 1 -- start start 2024-09-12T03:29:16.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.242+0000 7f061b4e6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0614100a50 0x7f0614199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:16.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.242+0000 7f061b4e6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0614101c40 0x7f061419a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:16.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.242+0000 7f061b4e6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0614102600 0x7f06141a1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:16.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.242+0000 7f061b4e6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f061410c560 con 0x7f0614102600 2024-09-12T03:29:16.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.242+0000 7f061b4e6640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f061410c3e0 con 0x7f0614100a50 2024-09-12T03:29:16.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.242+0000 7f061b4e6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f061410c6e0 con 0x7f0614101c40 2024-09-12T03:29:16.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.242+0000 7f0619282640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0614100a50 0x7f0614199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:16.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.242+0000 7f0619a83640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0614102600 0x7f06141a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:16.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.242+0000 7f0619282640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0614100a50 0x7f0614199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.64:57724/0 (socket says 172.21.15.64:57724) 2024-09-12T03:29:16.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.242+0000 7f0618a81640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0614101c40 0x7f061419a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:16.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.242+0000 7f0619282640 1 -- 172.21.15.64:0/332388636 learned_addr learned my addr 172.21.15.64:0/332388636 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:16.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.242+0000 7f0619a83640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0614102600 0x7f06141a1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36452/0 (socket says 172.21.15.64:36452) 2024-09-12T03:29:16.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.242+0000 7f0619282640 1 -- 172.21.15.64:0/332388636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0614101c40 msgr2=0x7f061419a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:16.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.242+0000 7f0619282640 1 --2- 172.21.15.64:0/332388636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0614101c40 0x7f061419a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:16.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.242+0000 7f0619282640 1 -- 172.21.15.64:0/332388636 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0614102600 msgr2=0x7f06141a1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:16.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.242+0000 7f0619282640 1 --2- 172.21.15.64:0/332388636 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0614102600 0x7f06141a1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:16.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.242+0000 7f0619282640 1 -- 172.21.15.64:0/332388636 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06141a1ac0 con 0x7f0614100a50 2024-09-12T03:29:16.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.242+0000 7f0619a83640 1 --2- 172.21.15.64:0/332388636 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0614102600 0x7f06141a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:16.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.242+0000 7f0618a81640 1 --2- 172.21.15.64:0/332388636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0614101c40 0x7f061419a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:16.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.242+0000 7f0619282640 1 --2- 172.21.15.64:0/332388636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0614100a50 0x7f0614199ea0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f061000d950 tx=0x7f061000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:16.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.243+0000 7f060a7fc640 1 -- 172.21.15.64:0/332388636 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0610017020 con 0x7f0614100a50 2024-09-12T03:29:16.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.243+0000 7f061b4e6640 1 -- 172.21.15.64:0/332388636 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06141a1db0 con 0x7f0614100a50 2024-09-12T03:29:16.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.243+0000 7f060a7fc640 1 -- 172.21.15.64:0/332388636 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0610009c30 con 0x7f0614100a50 2024-09-12T03:29:16.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.243+0000 7f060a7fc640 1 -- 172.21.15.64:0/332388636 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061000f620 con 0x7f0614100a50 2024-09-12T03:29:16.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.243+0000 7f061b4e6640 1 -- 172.21.15.64:0/332388636 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06141a2220 con 0x7f0614100a50 2024-09-12T03:29:16.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.244+0000 7f061b4e6640 1 -- 172.21.15.64:0/332388636 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06141a24b0 con 0x7f0614100a50 2024-09-12T03:29:16.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.245+0000 7f060a7fc640 1 -- 172.21.15.64:0/332388636 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 21) ==== 137518+0+0 (secure 0 0 0) 0x7f061001a020 con 0x7f0614100a50 2024-09-12T03:29:16.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.246+0000 7f060a7fc640 1 -- 172.21.15.64:0/332388636 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f06100ceeb0 con 0x7f0614100a50 2024-09-12T03:29:16.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.251+0000 7f060a7fc640 1 -- 172.21.15.64:0/332388636 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0610021020 con 0x7f0614100a50 2024-09-12T03:29:16.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.439+0000 7f061b4e6640 1 -- 172.21.15.64:0/332388636 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f06141a2dc0 con 0x7f0614100a50 2024-09-12T03:29:16.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.444+0000 7f060a7fc640 1 -- 172.21.15.64:0/332388636 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v31)=0 v31) ==== 133+0+0 (secure 0 0 0) 0x7f061009bcc0 con 0x7f0614100a50 2024-09-12T03:29:16.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.447+0000 7f061b4e6640 1 -- 172.21.15.64:0/332388636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0614100a50 msgr2=0x7f0614199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:16.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.447+0000 7f061b4e6640 1 --2- 172.21.15.64:0/332388636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0614100a50 0x7f0614199ea0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f061000d950 tx=0x7f061000de10 comp rx=0 tx=0).stop 2024-09-12T03:29:16.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.447+0000 7f061b4e6640 1 -- 172.21.15.64:0/332388636 shutdown_connections 2024-09-12T03:29:16.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.447+0000 7f061b4e6640 1 --2- 172.21.15.64:0/332388636 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0614102600 0x7f06141a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:16.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.447+0000 7f061b4e6640 1 --2- 172.21.15.64:0/332388636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0614101c40 0x7f061419a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:16.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.447+0000 7f061b4e6640 1 --2- 172.21.15.64:0/332388636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0614100a50 0x7f0614199ea0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:16.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.447+0000 7f061b4e6640 1 -- 172.21.15.64:0/332388636 >> 172.21.15.64:0/332388636 conn(0x7f06140fc1b0 msgr2=0x7f06140fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:16.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.447+0000 7f061b4e6640 1 -- 172.21.15.64:0/332388636 shutdown_connections 2024-09-12T03:29:16.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.447+0000 7f061b4e6640 1 -- 172.21.15.64:0/332388636 wait complete. 2024-09-12T03:29:16.464 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-12T03:29:16.490 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-09-12T03:29:16.490 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: Active manager daemon x restarted 2024-09-12T03:29:16.490 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: Activating manager daemon x 2024-09-12T03:29:16.490 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: osdmap e93: 12 total, 12 up, 12 in 2024-09-12T03:29:16.490 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: mgrmap e21: x(active, starting, since 0.012404s), standbys: y, z 2024-09-12T03:29:16.490 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:29:16.490 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:29:16.490 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:29:16.490 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T03:29:16.490 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T03:29:16.491 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T03:29:16.491 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T03:29:16.491 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T03:29:16.491 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T03:29:16.491 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:29:16.491 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:29:16.491 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:29:16.491 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:29:16.491 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:29:16.491 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:29:16.491 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T03:29:16.491 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T03:29:16.491 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T03:29:16.491 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:29:16.491 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:29:16.492 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:29:16.492 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:29:16.492 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:29:16.492 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:29:16.492 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:29:16.492 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:29:16.492 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:29:16.492 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:29:16.492 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:29:16.492 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:29:16.492 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T03:29:16.492 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T03:29:16.492 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T03:29:16.492 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: Manager daemon x is now available 2024-09-12T03:29:16.493 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/3675311424' entity='client.admin' 2024-09-12T03:29:16.493 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:29:16.493 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T03:29:16.493 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T03:29:16.493 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T03:29:16.493 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T03:29:16.493 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:16 smithi064 ceph-mon[30952]: from='client.? ' entity='client.admin' 2024-09-12T03:29:16.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.576+0000 7f99b5d33640 1 -- 172.21.15.64:0/1956841801 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b0102de0 msgr2=0x7f99b0103260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:16.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.576+0000 7f99b5d33640 1 --2- 172.21.15.64:0/1956841801 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b0102de0 0x7f99b0103260 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f99a4009900 tx=0x7f99a402a410 comp rx=0 tx=0).stop 2024-09-12T03:29:16.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.576+0000 7f99b5d33640 1 -- 172.21.15.64:0/1956841801 shutdown_connections 2024-09-12T03:29:16.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.576+0000 7f99b5d33640 1 --2- 172.21.15.64:0/1956841801 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99b01037a0 0x7f99b010a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:16.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.576+0000 7f99b5d33640 1 --2- 172.21.15.64:0/1956841801 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b0102de0 0x7f99b0103260 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:16.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.576+0000 7f99b5d33640 1 --2- 172.21.15.64:0/1956841801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99b00ffe10 0x7f99b0100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:16.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.576+0000 7f99b5d33640 1 -- 172.21.15.64:0/1956841801 >> 172.21.15.64:0/1956841801 conn(0x7f99b00fb930 msgr2=0x7f99b00fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:16.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.576+0000 7f99b5d33640 1 -- 172.21.15.64:0/1956841801 shutdown_connections 2024-09-12T03:29:16.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.576+0000 7f99b5d33640 1 -- 172.21.15.64:0/1956841801 wait complete. 2024-09-12T03:29:16.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.577+0000 7f99b5d33640 1 Processor -- start 2024-09-12T03:29:16.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.577+0000 7f99b5d33640 1 -- start start 2024-09-12T03:29:16.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.578+0000 7f99b5d33640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99b00ffe10 0x7f99b0102750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:16.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.578+0000 7f99b5d33640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99b0102de0 0x7f99b0100d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:16.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.578+0000 7f99b5d33640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b01037a0 0x7f99b01012d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:16.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.578+0000 7f99b5d33640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f99b010d6e0 con 0x7f99b01037a0 2024-09-12T03:29:16.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.578+0000 7f99b5d33640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f99b010d560 con 0x7f99b00ffe10 2024-09-12T03:29:16.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.578+0000 7f99b5d33640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f99b010d860 con 0x7f99b0102de0 2024-09-12T03:29:16.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.578+0000 7f99af7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99b00ffe10 0x7f99b0102750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:16.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.578+0000 7f99affff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b01037a0 0x7f99b01012d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:16.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.578+0000 7f99af7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99b00ffe10 0x7f99b0102750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.64:57734/0 (socket says 172.21.15.64:57734) 2024-09-12T03:29:16.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.578+0000 7f99af7fe640 1 -- 172.21.15.64:0/4285694812 learned_addr learned my addr 172.21.15.64:0/4285694812 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:16.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.578+0000 7f99aeffd640 1 --2- 172.21.15.64:0/4285694812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99b0102de0 0x7f99b0100d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:16.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.578+0000 7f99af7fe640 1 -- 172.21.15.64:0/4285694812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99b0102de0 msgr2=0x7f99b0100d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:16.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.578+0000 7f99af7fe640 1 --2- 172.21.15.64:0/4285694812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99b0102de0 0x7f99b0100d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:16.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.578+0000 7f99af7fe640 1 -- 172.21.15.64:0/4285694812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b01037a0 msgr2=0x7f99b01012d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:16.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.578+0000 7f99af7fe640 1 --2- 172.21.15.64:0/4285694812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b01037a0 0x7f99b01012d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:16.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.578+0000 7f99af7fe640 1 -- 172.21.15.64:0/4285694812 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99b0101810 con 0x7f99b00ffe10 2024-09-12T03:29:16.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.578+0000 7f99affff640 1 --2- 172.21.15.64:0/4285694812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b01037a0 0x7f99b01012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:16.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.578+0000 7f99af7fe640 1 --2- 172.21.15.64:0/4285694812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99b00ffe10 0x7f99b0102750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f99a000e9c0 tx=0x7f99a000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:16.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.579+0000 7f99acff9640 1 -- 172.21.15.64:0/4285694812 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a000cd80 con 0x7f99b00ffe10 2024-09-12T03:29:16.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.579+0000 7f99acff9640 1 -- 172.21.15.64:0/4285694812 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99a00035f0 con 0x7f99b00ffe10 2024-09-12T03:29:16.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.579+0000 7f99b5d33640 1 -- 172.21.15.64:0/4285694812 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99b0101b00 con 0x7f99b00ffe10 2024-09-12T03:29:16.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.579+0000 7f99acff9640 1 -- 172.21.15.64:0/4285694812 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a0010690 con 0x7f99b00ffe10 2024-09-12T03:29:16.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.579+0000 7f99b5d33640 1 -- 172.21.15.64:0/4285694812 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99b01a67c0 con 0x7f99b00ffe10 2024-09-12T03:29:16.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.580+0000 7f99b5d33640 1 -- 172.21.15.64:0/4285694812 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99b0100210 con 0x7f99b00ffe10 2024-09-12T03:29:16.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.583+0000 7f99acff9640 1 -- 172.21.15.64:0/4285694812 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 21) ==== 137518+0+0 (secure 0 0 0) 0x7f99a00037a0 con 0x7f99b00ffe10 2024-09-12T03:29:16.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.583+0000 7f99acff9640 1 -- 172.21.15.64:0/4285694812 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f99a00cf770 con 0x7f99b00ffe10 2024-09-12T03:29:16.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.586+0000 7f99acff9640 1 -- 172.21.15.64:0/4285694812 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f99a009c5e0 con 0x7f99b00ffe10 2024-09-12T03:29:16.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.614+0000 7f99acff9640 1 -- 172.21.15.64:0/4285694812 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7f99a0097c30 con 0x7f99b00ffe10 2024-09-12T03:29:16.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.615+0000 7f99acff9640 1 --2- 172.21.15.64:0/4285694812 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f99980ab250 0x7f99980ad640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:16.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.621+0000 7f99aeffd640 1 --2- 172.21.15.64:0/4285694812 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f99980ab250 0x7f99980ad640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:16.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.627+0000 7f99aeffd640 1 --2- 172.21.15.64:0/4285694812 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f99980ab250 0x7f99980ad640 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f99a40314e0 tx=0x7f99a4005a20 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:16.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-09-12T03:29:16.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: Active manager daemon x restarted 2024-09-12T03:29:16.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: Activating manager daemon x 2024-09-12T03:29:16.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: osdmap e93: 12 total, 12 up, 12 in 2024-09-12T03:29:16.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: mgrmap e21: x(active, starting, since 0.012404s), standbys: y, z 2024-09-12T03:29:16.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:29:16.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:29:16.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:29:16.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T03:29:16.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T03:29:16.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T03:29:16.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T03:29:16.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T03:29:16.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T03:29:16.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:29:16.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:29:16.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:29:16.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:29:16.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:29:16.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:29:16.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T03:29:16.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T03:29:16.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T03:29:16.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:29:16.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:29:16.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:29:16.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:29:16.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:29:16.645 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:29:16.645 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:29:16.645 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:29:16.645 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:29:16.645 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:29:16.645 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:29:16.645 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:29:16.645 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T03:29:16.645 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T03:29:16.645 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T03:29:16.645 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: Manager daemon x is now available 2024-09-12T03:29:16.645 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/3675311424' entity='client.admin' 2024-09-12T03:29:16.645 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:29:16.646 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T03:29:16.646 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T03:29:16.646 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T03:29:16.646 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T03:29:16.646 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:16 smithi115 ceph-mon[34801]: from='client.? ' entity='client.admin' 2024-09-12T03:29:16.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-09-12T03:29:16.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: Active manager daemon x restarted 2024-09-12T03:29:16.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: Activating manager daemon x 2024-09-12T03:29:16.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: osdmap e93: 12 total, 12 up, 12 in 2024-09-12T03:29:16.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: mgrmap e21: x(active, starting, since 0.012404s), standbys: y, z 2024-09-12T03:29:16.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:29:16.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:29:16.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:29:16.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T03:29:16.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T03:29:16.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T03:29:16.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T03:29:16.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T03:29:16.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T03:29:16.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:29:16.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:29:16.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:29:16.708 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:29:16.708 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:29:16.708 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:29:16.708 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T03:29:16.708 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T03:29:16.708 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T03:29:16.708 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:29:16.708 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:29:16.708 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:29:16.708 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:29:16.708 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:29:16.708 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:29:16.708 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:29:16.709 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:29:16.709 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:29:16.709 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:29:16.709 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:29:16.709 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:29:16.709 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T03:29:16.709 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T03:29:16.709 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T03:29:16.709 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: Manager daemon x is now available 2024-09-12T03:29:16.709 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/3675311424' entity='client.admin' 2024-09-12T03:29:16.709 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:29:16.709 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T03:29:16.709 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T03:29:16.709 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T03:29:16.709 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T03:29:16.710 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:16 smithi195 ceph-mon[34786]: from='client.? ' entity='client.admin' 2024-09-12T03:29:16.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.808+0000 7f99b5d33640 1 -- 172.21.15.64:0/4285694812 --> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f99b0062630 con 0x7f99980ab250 2024-09-12T03:29:16.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.866+0000 7f99acff9640 1 -- 172.21.15.64:0/4285694812 <== mgr.15243 v2:172.21.15.64:6800/3960824462 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f99b0062630 con 0x7f99980ab250 2024-09-12T03:29:16.867 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule set for path / 2024-09-12T03:29:16.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.868+0000 7f99b5d33640 1 -- 172.21.15.64:0/4285694812 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f99980ab250 msgr2=0x7f99980ad640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:16.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.868+0000 7f99b5d33640 1 --2- 172.21.15.64:0/4285694812 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f99980ab250 0x7f99980ad640 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f99a40314e0 tx=0x7f99a4005a20 comp rx=0 tx=0).stop 2024-09-12T03:29:16.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.868+0000 7f99b5d33640 1 -- 172.21.15.64:0/4285694812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99b00ffe10 msgr2=0x7f99b0102750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:16.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.868+0000 7f99b5d33640 1 --2- 172.21.15.64:0/4285694812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99b00ffe10 0x7f99b0102750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f99a000e9c0 tx=0x7f99a000ee80 comp rx=0 tx=0).stop 2024-09-12T03:29:16.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.868+0000 7f99b5d33640 1 -- 172.21.15.64:0/4285694812 shutdown_connections 2024-09-12T03:29:16.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.868+0000 7f99b5d33640 1 --2- 172.21.15.64:0/4285694812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b01037a0 0x7f99b01012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:16.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.868+0000 7f99b5d33640 1 --2- 172.21.15.64:0/4285694812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99b0102de0 0x7f99b0100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:16.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.868+0000 7f99b5d33640 1 --2- 172.21.15.64:0/4285694812 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f99980ab250 0x7f99980ad640 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:16.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.868+0000 7f99b5d33640 1 --2- 172.21.15.64:0/4285694812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99b00ffe10 0x7f99b0102750 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:16.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.868+0000 7f99b5d33640 1 -- 172.21.15.64:0/4285694812 >> 172.21.15.64:0/4285694812 conn(0x7f99b00fb930 msgr2=0x7f99b0106200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:16.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.868+0000 7f99b5d33640 1 -- 172.21.15.64:0/4285694812 shutdown_connections 2024-09-12T03:29:16.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:16.868+0000 7f99b5d33640 1 -- 172.21.15.64:0/4285694812 wait complete. 2024-09-12T03:29:16.880 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-12T03:29:17.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.030+0000 7fd14ece6640 1 -- 172.21.15.64:0/3210480489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1481016f0 msgr2=0x7fd148101ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:17.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.030+0000 7fd14ece6640 1 --2- 172.21.15.64:0/3210480489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1481016f0 0x7fd148101ba0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd140009ca0 tx=0x7fd14002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.030+0000 7fd14ece6640 1 -- 172.21.15.64:0/3210480489 shutdown_connections 2024-09-12T03:29:17.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.030+0000 7fd14ece6640 1 --2- 172.21.15.64:0/3210480489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1481016f0 0x7fd148101ba0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.030+0000 7fd14ece6640 1 --2- 172.21.15.64:0/3210480489 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd148100d50 0x7fd1481011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.030+0000 7fd14ece6640 1 --2- 172.21.15.64:0/3210480489 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd148100410 0x7fd148100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.030+0000 7fd14ece6640 1 -- 172.21.15.64:0/3210480489 >> 172.21.15.64:0/3210480489 conn(0x7fd1480fc1e0 msgr2=0x7fd1480fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:17.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.030+0000 7fd14ece6640 1 -- 172.21.15.64:0/3210480489 shutdown_connections 2024-09-12T03:29:17.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.030+0000 7fd14ece6640 1 -- 172.21.15.64:0/3210480489 wait complete. 2024-09-12T03:29:17.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.030+0000 7fd14ece6640 1 Processor -- start 2024-09-12T03:29:17.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.030+0000 7fd14ece6640 1 -- start start 2024-09-12T03:29:17.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.030+0000 7fd14ece6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd148100410 0x7fd148199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:17.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.030+0000 7fd14ece6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd148100d50 0x7fd14819a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:17.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.031+0000 7fd14ece6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1481016f0 0x7fd1481a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:17.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.031+0000 7fd14ece6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd14810f800 con 0x7fd1481016f0 2024-09-12T03:29:17.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.031+0000 7fd14ece6640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd14810f680 con 0x7fd148100d50 2024-09-12T03:29:17.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.031+0000 7fd14ece6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd14810f980 con 0x7fd148100410 2024-09-12T03:29:17.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.031+0000 7fd14d283640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1481016f0 0x7fd1481a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:17.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.031+0000 7fd147fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd148100d50 0x7fd14819a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:17.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.031+0000 7fd147fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd148100d50 0x7fd14819a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.64:57758/0 (socket says 172.21.15.64:57758) 2024-09-12T03:29:17.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.031+0000 7fd14d283640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1481016f0 0x7fd1481a1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36502/0 (socket says 172.21.15.64:36502) 2024-09-12T03:29:17.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.031+0000 7fd147fff640 1 -- 172.21.15.64:0/5316263 learned_addr learned my addr 172.21.15.64:0/5316263 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:17.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.031+0000 7fd14ca82640 1 --2- 172.21.15.64:0/5316263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd148100410 0x7fd148199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:17.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.031+0000 7fd147fff640 1 -- 172.21.15.64:0/5316263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd148100410 msgr2=0x7fd148199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:17.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.031+0000 7fd147fff640 1 --2- 172.21.15.64:0/5316263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd148100410 0x7fd148199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.031+0000 7fd147fff640 1 -- 172.21.15.64:0/5316263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1481016f0 msgr2=0x7fd1481a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:17.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.031+0000 7fd147fff640 1 --2- 172.21.15.64:0/5316263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1481016f0 0x7fd1481a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.031+0000 7fd147fff640 1 -- 172.21.15.64:0/5316263 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1481a1a80 con 0x7fd148100d50 2024-09-12T03:29:17.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.031+0000 7fd14d283640 1 --2- 172.21.15.64:0/5316263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1481016f0 0x7fd1481a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:17.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.031+0000 7fd147fff640 1 --2- 172.21.15.64:0/5316263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd148100d50 0x7fd14819a3a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd13400e9f0 tx=0x7fd13400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:17.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.032+0000 7fd145ffb640 1 -- 172.21.15.64:0/5316263 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13400cd80 con 0x7fd148100d50 2024-09-12T03:29:17.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.032+0000 7fd145ffb640 1 -- 172.21.15.64:0/5316263 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1340054d0 con 0x7fd148100d50 2024-09-12T03:29:17.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.032+0000 7fd145ffb640 1 -- 172.21.15.64:0/5316263 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd134010490 con 0x7fd148100d50 2024-09-12T03:29:17.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.032+0000 7fd14ece6640 1 -- 172.21.15.64:0/5316263 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1481a1d70 con 0x7fd148100d50 2024-09-12T03:29:17.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.032+0000 7fd14ece6640 1 -- 172.21.15.64:0/5316263 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1481a2210 con 0x7fd148100d50 2024-09-12T03:29:17.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.033+0000 7fd14ece6640 1 -- 172.21.15.64:0/5316263 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd14810b160 con 0x7fd148100d50 2024-09-12T03:29:17.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.034+0000 7fd145ffb640 1 -- 172.21.15.64:0/5316263 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7fd134005020 con 0x7fd148100d50 2024-09-12T03:29:17.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.035+0000 7fd145ffb640 1 --2- 172.21.15.64:0/5316263 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fd1380a2a40 0x7fd1380a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:17.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.035+0000 7fd145ffb640 1 -- 172.21.15.64:0/5316263 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7fd1340cf490 con 0x7fd148100d50 2024-09-12T03:29:17.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.035+0000 7fd14ca82640 1 --2- 172.21.15.64:0/5316263 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fd1380a2a40 0x7fd1380a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:17.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.035+0000 7fd14ca82640 1 --2- 172.21.15.64:0/5316263 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fd1380a2a40 0x7fd1380a4f00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd13c009bd0 tx=0x7fd13c005f90 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:17.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.039+0000 7fd145ffb640 1 -- 172.21.15.64:0/5316263 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd13409c1f0 con 0x7fd148100d50 2024-09-12T03:29:17.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.241+0000 7fd14ece6640 1 -- 172.21.15.64:0/5316263 --> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fd1480626d0 con 0x7fd1380a2a40 2024-09-12T03:29:17.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.299+0000 7fd145ffb640 1 -- 172.21.15.64:0/5316263 <== mgr.15243 v2:172.21.15.64:6800/3960824462 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fd1480626d0 con 0x7fd1380a2a40 2024-09-12T03:29:17.300 INFO:teuthology.orchestra.run.smithi064.stdout:Retention added to path / 2024-09-12T03:29:17.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.302+0000 7fd14ece6640 1 -- 172.21.15.64:0/5316263 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fd1380a2a40 msgr2=0x7fd1380a4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:17.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.302+0000 7fd14ece6640 1 --2- 172.21.15.64:0/5316263 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fd1380a2a40 0x7fd1380a4f00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd13c009bd0 tx=0x7fd13c005f90 comp rx=0 tx=0).stop 2024-09-12T03:29:17.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.302+0000 7fd14ece6640 1 -- 172.21.15.64:0/5316263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd148100d50 msgr2=0x7fd14819a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:17.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.302+0000 7fd14ece6640 1 --2- 172.21.15.64:0/5316263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd148100d50 0x7fd14819a3a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd13400e9f0 tx=0x7fd13400edf0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.302+0000 7fd14ece6640 1 -- 172.21.15.64:0/5316263 shutdown_connections 2024-09-12T03:29:17.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.302+0000 7fd14ece6640 1 --2- 172.21.15.64:0/5316263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1481016f0 0x7fd1481a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.302+0000 7fd14ece6640 1 --2- 172.21.15.64:0/5316263 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fd1380a2a40 0x7fd1380a4f00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.302+0000 7fd14ece6640 1 --2- 172.21.15.64:0/5316263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd148100d50 0x7fd14819a3a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.302+0000 7fd14ece6640 1 --2- 172.21.15.64:0/5316263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd148100410 0x7fd148199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.302+0000 7fd14ece6640 1 -- 172.21.15.64:0/5316263 >> 172.21.15.64:0/5316263 conn(0x7fd1480fc1e0 msgr2=0x7fd1480fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:17.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.302+0000 7fd14ece6640 1 -- 172.21.15.64:0/5316263 shutdown_connections 2024-09-12T03:29:17.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.302+0000 7fd14ece6640 1 -- 172.21.15.64:0/5316263 wait complete. 2024-09-12T03:29:17.317 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-12T03:29:17.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.420+0000 7f2f71d0a640 1 -- 172.21.15.64:0/440600767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f6c100a40 msgr2=0x7f2f6c100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:17.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.420+0000 7f2f71d0a640 1 --2- 172.21.15.64:0/440600767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f6c100a40 0x7f2f6c100e40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c009900 tx=0x7f2f5c02a410 comp rx=0 tx=0).stop 2024-09-12T03:29:17.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.420+0000 7f2f71d0a640 1 -- 172.21.15.64:0/440600767 shutdown_connections 2024-09-12T03:29:17.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.420+0000 7f2f71d0a640 1 --2- 172.21.15.64:0/440600767 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f6c1025f0 0x7f2f6c108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.420+0000 7f2f71d0a640 1 --2- 172.21.15.64:0/440600767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f6c101c30 0x7f2f6c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.420+0000 7f2f71d0a640 1 --2- 172.21.15.64:0/440600767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f6c100a40 0x7f2f6c100e40 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.420+0000 7f2f71d0a640 1 -- 172.21.15.64:0/440600767 >> 172.21.15.64:0/440600767 conn(0x7f2f6c0fc1e0 msgr2=0x7f2f6c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:17.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.420+0000 7f2f71d0a640 1 -- 172.21.15.64:0/440600767 shutdown_connections 2024-09-12T03:29:17.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.420+0000 7f2f71d0a640 1 -- 172.21.15.64:0/440600767 wait complete. 2024-09-12T03:29:17.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.421+0000 7f2f71d0a640 1 Processor -- start 2024-09-12T03:29:17.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.421+0000 7f2f71d0a640 1 -- start start 2024-09-12T03:29:17.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.421+0000 7f2f71d0a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f6c100a40 0x7f2f6c197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:17.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.421+0000 7f2f71d0a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f6c101c30 0x7f2f6c198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:17.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.421+0000 7f2f71d0a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f6c1025f0 0x7f2f6c19f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:17.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.421+0000 7f2f71d0a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c10c5a0 con 0x7f2f6c1025f0 2024-09-12T03:29:17.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.421+0000 7f2f71d0a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c10c420 con 0x7f2f6c100a40 2024-09-12T03:29:17.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.421+0000 7f2f71d0a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c10c720 con 0x7f2f6c101c30 2024-09-12T03:29:17.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.422+0000 7f2f6bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f6c1025f0 0x7f2f6c19f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:17.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.422+0000 7f2f6b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f6c100a40 0x7f2f6c197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:17.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.422+0000 7f2f6bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f6c1025f0 0x7f2f6c19f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36524/0 (socket says 172.21.15.64:36524) 2024-09-12T03:29:17.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.422+0000 7f2f6bfff640 1 -- 172.21.15.64:0/2380701408 learned_addr learned my addr 172.21.15.64:0/2380701408 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:17.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.422+0000 7f2f6affd640 1 --2- 172.21.15.64:0/2380701408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f6c101c30 0x7f2f6c198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:17.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.422+0000 7f2f6bfff640 1 -- 172.21.15.64:0/2380701408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f6c101c30 msgr2=0x7f2f6c198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:17.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.422+0000 7f2f6bfff640 1 --2- 172.21.15.64:0/2380701408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f6c101c30 0x7f2f6c198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.422+0000 7f2f6bfff640 1 -- 172.21.15.64:0/2380701408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f6c100a40 msgr2=0x7f2f6c197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:17.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.422+0000 7f2f6bfff640 1 --2- 172.21.15.64:0/2380701408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f6c100a40 0x7f2f6c197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.422+0000 7f2f6bfff640 1 -- 172.21.15.64:0/2380701408 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f6c19f950 con 0x7f2f6c1025f0 2024-09-12T03:29:17.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.422+0000 7f2f6bfff640 1 --2- 172.21.15.64:0/2380701408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f6c1025f0 0x7f2f6c19f2b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2f6400eaa0 tx=0x7f2f6400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:17.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.422+0000 7f2f68ff9640 1 -- 172.21.15.64:0/2380701408 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6400cdb0 con 0x7f2f6c1025f0 2024-09-12T03:29:17.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.422+0000 7f2f68ff9640 1 -- 172.21.15.64:0/2380701408 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f64005020 con 0x7f2f6c1025f0 2024-09-12T03:29:17.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.422+0000 7f2f71d0a640 1 -- 172.21.15.64:0/2380701408 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f6c19fc40 con 0x7f2f6c1025f0 2024-09-12T03:29:17.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.422+0000 7f2f68ff9640 1 -- 172.21.15.64:0/2380701408 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f640116d0 con 0x7f2f6c1025f0 2024-09-12T03:29:17.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.422+0000 7f2f71d0a640 1 -- 172.21.15.64:0/2380701408 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f6c1a0160 con 0x7f2f6c1025f0 2024-09-12T03:29:17.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.423+0000 7f2f71d0a640 1 -- 172.21.15.64:0/2380701408 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f6c1a0390 con 0x7f2f6c1025f0 2024-09-12T03:29:17.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.425+0000 7f2f68ff9640 1 -- 172.21.15.64:0/2380701408 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7f2f640051d0 con 0x7f2f6c1025f0 2024-09-12T03:29:17.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.425+0000 7f2f68ff9640 1 --2- 172.21.15.64:0/2380701408 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f2f580a2ac0 0x7f2f580a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:17.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.426+0000 7f2f68ff9640 1 -- 172.21.15.64:0/2380701408 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f2f640cf250 con 0x7f2f6c1025f0 2024-09-12T03:29:17.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.426+0000 7f2f6b7fe640 1 --2- 172.21.15.64:0/2380701408 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f2f580a2ac0 0x7f2f580a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:17.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.426+0000 7f2f6b7fe640 1 --2- 172.21.15.64:0/2380701408 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f2f580a2ac0 0x7f2f580a4f80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c005ab0 tx=0x7f2f5c005a20 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:17.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.430+0000 7f2f68ff9640 1 -- 172.21.15.64:0/2380701408 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2f64016030 con 0x7f2f6c1025f0 2024-09-12T03:29:17.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:17 smithi064 ceph-mon[30952]: from='client.? ' entity='client.admin' 2024-09-12T03:29:17.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:17 smithi064 ceph-mon[30952]: mgrmap e22: x(active, since 1.01649s), standbys: y, z 2024-09-12T03:29:17.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:17 smithi064 ceph-mon[30952]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:29:17.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:17 smithi064 ceph-mon[30952]: from='client.25028 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:29:17.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:17 smithi064 ceph-mon[30952]: from='client.25034 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:29:17.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.651+0000 7f2f71d0a640 1 -- 172.21.15.64:0/2380701408 --> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f2f6c1a0560 con 0x7f2f580a2ac0 2024-09-12T03:29:17.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.658+0000 7f2f68ff9640 1 -- 172.21.15.64:0/2380701408 <== mgr.15243 v2:172.21.15.64:6800/3960824462 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f2f6c1a0560 con 0x7f2f580a2ac0 2024-09-12T03:29:17.659 INFO:teuthology.orchestra.run.smithi064.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-12T00:00:00", "created": "2024-09-12T03:29:16", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-12T03:29:17.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.660+0000 7f2f71d0a640 1 -- 172.21.15.64:0/2380701408 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f2f580a2ac0 msgr2=0x7f2f580a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:17.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.660+0000 7f2f71d0a640 1 --2- 172.21.15.64:0/2380701408 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f2f580a2ac0 0x7f2f580a4f80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c005ab0 tx=0x7f2f5c005a20 comp rx=0 tx=0).stop 2024-09-12T03:29:17.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.660+0000 7f2f71d0a640 1 -- 172.21.15.64:0/2380701408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f6c1025f0 msgr2=0x7f2f6c19f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:17.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.660+0000 7f2f71d0a640 1 --2- 172.21.15.64:0/2380701408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f6c1025f0 0x7f2f6c19f2b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2f6400eaa0 tx=0x7f2f6400ef60 comp rx=0 tx=0).stop 2024-09-12T03:29:17.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.660+0000 7f2f71d0a640 1 -- 172.21.15.64:0/2380701408 shutdown_connections 2024-09-12T03:29:17.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.660+0000 7f2f71d0a640 1 --2- 172.21.15.64:0/2380701408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f6c1025f0 0x7f2f6c19f2b0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.660+0000 7f2f71d0a640 1 --2- 172.21.15.64:0/2380701408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f6c101c30 0x7f2f6c198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.660+0000 7f2f71d0a640 1 --2- 172.21.15.64:0/2380701408 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f2f580a2ac0 0x7f2f580a4f80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.660+0000 7f2f71d0a640 1 --2- 172.21.15.64:0/2380701408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f6c100a40 0x7f2f6c197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.660+0000 7f2f71d0a640 1 -- 172.21.15.64:0/2380701408 >> 172.21.15.64:0/2380701408 conn(0x7f2f6c0fc1e0 msgr2=0x7f2f6c0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:17.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.660+0000 7f2f71d0a640 1 -- 172.21.15.64:0/2380701408 shutdown_connections 2024-09-12T03:29:17.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.660+0000 7f2f71d0a640 1 -- 172.21.15.64:0/2380701408 wait complete. 2024-09-12T03:29:17.673 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-12T03:29:17.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:17 smithi195 ceph-mon[34786]: from='client.? ' entity='client.admin' 2024-09-12T03:29:17.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:17 smithi195 ceph-mon[34786]: mgrmap e22: x(active, since 1.01649s), standbys: y, z 2024-09-12T03:29:17.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:17 smithi195 ceph-mon[34786]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:29:17.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:17 smithi195 ceph-mon[34786]: from='client.25028 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:29:17.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:17 smithi195 ceph-mon[34786]: from='client.25034 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:29:17.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.795+0000 7f176d623640 1 -- 172.21.15.64:0/816283916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1768102d50 msgr2=0x7f17681031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:17.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.795+0000 7f176d623640 1 --2- 172.21.15.64:0/816283916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1768102d50 0x7f17681031d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f175c009900 tx=0x7f175c02a410 comp rx=0 tx=0).stop 2024-09-12T03:29:17.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.795+0000 7f176d623640 1 -- 172.21.15.64:0/816283916 shutdown_connections 2024-09-12T03:29:17.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.795+0000 7f176d623640 1 --2- 172.21.15.64:0/816283916 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1768103710 0x7f1768109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.795+0000 7f176d623640 1 --2- 172.21.15.64:0/816283916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1768102d50 0x7f17681031d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.795+0000 7f176d623640 1 --2- 172.21.15.64:0/816283916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1768101b60 0x7f1768101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.795+0000 7f176d623640 1 -- 172.21.15.64:0/816283916 >> 172.21.15.64:0/816283916 conn(0x7f17680fd2c0 msgr2=0x7f17680ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:17.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.795+0000 7f176d623640 1 -- 172.21.15.64:0/816283916 shutdown_connections 2024-09-12T03:29:17.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.795+0000 7f176d623640 1 -- 172.21.15.64:0/816283916 wait complete. 2024-09-12T03:29:17.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.796+0000 7f176d623640 1 Processor -- start 2024-09-12T03:29:17.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.796+0000 7f176d623640 1 -- start start 2024-09-12T03:29:17.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.797+0000 7f176d623640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1768101b60 0x7f1768197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:17.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.797+0000 7f176d623640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1768102d50 0x7f17681982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:17.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.797+0000 7f176d623640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1768103710 0x7f176819f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:17.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.797+0000 7f176d623640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f176810d6c0 con 0x7f1768103710 2024-09-12T03:29:17.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.797+0000 7f176d623640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f176810d540 con 0x7f1768101b60 2024-09-12T03:29:17.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.797+0000 7f176d623640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f176810d840 con 0x7f1768102d50 2024-09-12T03:29:17.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.797+0000 7f1766ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1768101b60 0x7f1768197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:17.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.797+0000 7f1766ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1768101b60 0x7f1768197d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.64:58152/0 (socket says 172.21.15.64:58152) 2024-09-12T03:29:17.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.797+0000 7f1766ffd640 1 -- 172.21.15.64:0/3427809367 learned_addr learned my addr 172.21.15.64:0/3427809367 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:17.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.797+0000 7f17667fc640 1 --2- 172.21.15.64:0/3427809367 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1768102d50 0x7f17681982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:17.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.797+0000 7f17677fe640 1 --2- 172.21.15.64:0/3427809367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1768103710 0x7f176819f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:17.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.798+0000 7f1766ffd640 1 -- 172.21.15.64:0/3427809367 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1768102d50 msgr2=0x7f17681982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:17.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.798+0000 7f1766ffd640 1 --2- 172.21.15.64:0/3427809367 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1768102d50 0x7f17681982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.798+0000 7f1766ffd640 1 -- 172.21.15.64:0/3427809367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1768103710 msgr2=0x7f176819f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:17.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.798+0000 7f1766ffd640 1 --2- 172.21.15.64:0/3427809367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1768103710 0x7f176819f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:17.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.798+0000 7f1766ffd640 1 -- 172.21.15.64:0/3427809367 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f176819f990 con 0x7f1768101b60 2024-09-12T03:29:17.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.798+0000 7f1766ffd640 1 --2- 172.21.15.64:0/3427809367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1768101b60 0x7f1768197d70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1758009d20 tx=0x7f175800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:17.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.798+0000 7f174ffff640 1 -- 172.21.15.64:0/3427809367 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1758010630 con 0x7f1768101b60 2024-09-12T03:29:17.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.798+0000 7f174ffff640 1 -- 172.21.15.64:0/3427809367 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1758010c80 con 0x7f1768101b60 2024-09-12T03:29:17.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.798+0000 7f174ffff640 1 -- 172.21.15.64:0/3427809367 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1758018d70 con 0x7f1768101b60 2024-09-12T03:29:17.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.798+0000 7f176d623640 1 -- 172.21.15.64:0/3427809367 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f176819fc80 con 0x7f1768101b60 2024-09-12T03:29:17.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.798+0000 7f176d623640 1 -- 172.21.15.64:0/3427809367 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17681a0150 con 0x7f1768101b60 2024-09-12T03:29:17.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.800+0000 7f176d623640 1 -- 172.21.15.64:0/3427809367 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1768109010 con 0x7f1768101b60 2024-09-12T03:29:17.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.801+0000 7f174ffff640 1 -- 172.21.15.64:0/3427809367 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7f175801f020 con 0x7f1768101b60 2024-09-12T03:29:17.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.802+0000 7f174ffff640 1 --2- 172.21.15.64:0/3427809367 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f17500a2a40 0x7f17500a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:17.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.802+0000 7f174ffff640 1 -- 172.21.15.64:0/3427809367 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f17580cf5c0 con 0x7f1768101b60 2024-09-12T03:29:17.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.802+0000 7f17667fc640 1 --2- 172.21.15.64:0/3427809367 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f17500a2a40 0x7f17500a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:17.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.802+0000 7f17667fc640 1 --2- 172.21.15.64:0/3427809367 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f17500a2a40 0x7f17500a4f00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f175c005ab0 tx=0x7f175c005a20 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:17.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:17.806+0000 7f174ffff640 1 -- 172.21.15.64:0/3427809367 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f175809c320 con 0x7f1768101b60 2024-09-12T03:29:17.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:17 smithi115 ceph-mon[34801]: from='client.? ' entity='client.admin' 2024-09-12T03:29:17.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:17 smithi115 ceph-mon[34801]: mgrmap e22: x(active, since 1.01649s), standbys: y, z 2024-09-12T03:29:17.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:17 smithi115 ceph-mon[34801]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:29:17.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:17 smithi115 ceph-mon[34801]: from='client.25028 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:29:17.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:17 smithi115 ceph-mon[34801]: from='client.25034 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:29:18.022 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:17 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:29:17.857+0000 7fa509327640 -1 log_channel(cephadm) log [ERR] : [12/Sep/2024:03:29:17] ENGINE Error in HTTPServer.serve 2024-09-12T03:29:18.023 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:17 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: Traceback (most recent call last): 2024-09-12T03:29:18.023 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:17 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T03:29:18.023 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:17 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: self._connections.run(self.expiration_interval) 2024-09-12T03:29:18.023 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:17 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T03:29:18.023 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:17 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: self._run(expiration_interval) 2024-09-12T03:29:18.023 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:17 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T03:29:18.023 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:17 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T03:29:18.023 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:17 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T03:29:18.023 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:17 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T03:29:18.023 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:17 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T03:29:18.023 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:17 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: s = self.context.wrap_socket( 2024-09-12T03:29:18.023 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:17 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T03:29:18.024 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:17 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: return self.sslsocket_class._create( 2024-09-12T03:29:18.024 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:17 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T03:29:18.024 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:17 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: self.do_handshake() 2024-09-12T03:29:18.024 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:17 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T03:29:18.024 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:17 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: self._sslobj.do_handshake() 2024-09-12T03:29:18.024 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:17 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T03:29:18.024 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:29:17 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:29:18.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:18.038+0000 7f176d623640 1 -- 172.21.15.64:0/3427809367 --> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f1768062630 con 0x7f17500a2a40 2024-09-12T03:29:18.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:18.045+0000 7f174ffff640 1 -- 172.21.15.64:0/3427809367 <== mgr.15243 v2:172.21.15.64:6800/3960824462 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f1768062630 con 0x7f17500a2a40 2024-09-12T03:29:18.046 INFO:teuthology.orchestra.run.smithi064.stdout:/ 1m 6m3h 2024-09-12T03:29:18.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:18.047+0000 7f176d623640 1 -- 172.21.15.64:0/3427809367 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f17500a2a40 msgr2=0x7f17500a4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:18.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:18.047+0000 7f176d623640 1 --2- 172.21.15.64:0/3427809367 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f17500a2a40 0x7f17500a4f00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f175c005ab0 tx=0x7f175c005a20 comp rx=0 tx=0).stop 2024-09-12T03:29:18.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:18.047+0000 7f176d623640 1 -- 172.21.15.64:0/3427809367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1768101b60 msgr2=0x7f1768197d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:18.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:18.047+0000 7f176d623640 1 --2- 172.21.15.64:0/3427809367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1768101b60 0x7f1768197d70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1758009d20 tx=0x7f175800c6a0 comp rx=0 tx=0).stop 2024-09-12T03:29:18.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:18.047+0000 7f176d623640 1 -- 172.21.15.64:0/3427809367 shutdown_connections 2024-09-12T03:29:18.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:18.047+0000 7f176d623640 1 --2- 172.21.15.64:0/3427809367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1768103710 0x7f176819f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:18.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:18.047+0000 7f176d623640 1 --2- 172.21.15.64:0/3427809367 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1768102d50 0x7f17681982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:18.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:18.047+0000 7f176d623640 1 --2- 172.21.15.64:0/3427809367 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f17500a2a40 0x7f17500a4f00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:18.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:18.047+0000 7f176d623640 1 --2- 172.21.15.64:0/3427809367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1768101b60 0x7f1768197d70 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:18.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:18.047+0000 7f176d623640 1 -- 172.21.15.64:0/3427809367 >> 172.21.15.64:0/3427809367 conn(0x7f17680fd2c0 msgr2=0x7f17681061e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:18.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:18.047+0000 7f176d623640 1 -- 172.21.15.64:0/3427809367 shutdown_connections 2024-09-12T03:29:18.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:29:18.047+0000 7f176d623640 1 -- 172.21.15.64:0/3427809367 wait complete. 2024-09-12T03:29:18.059 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-12T03:29:18.087 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-12T03:29:18.097 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-12T03:29:18.105 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi064.front.sepia.ceph.com 2024-09-12T03:29:18.106 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-12T03:29:18.129 INFO:teuthology.run_tasks:Running task workunit... 2024-09-12T03:29:18.144 INFO:tasks.workunit:Pulling workunits from ref 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-12T03:29:18.144 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-12T03:29:18.144 INFO:tasks.workunit:timeout=3h 2024-09-12T03:29:18.144 INFO:tasks.workunit:cleanup=True 2024-09-12T03:29:18.145 DEBUG:teuthology.orchestra.run.smithi064:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-12T03:29:18.163 INFO:teuthology.orchestra.run.smithi064.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-12T03:29:18.163 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-09-12T03:29:18.163 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 60h/96d Inode: 1 Links: 2 2024-09-12T03:29:18.163 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:29:18.163 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:fusefs_t:s0 2024-09-12T03:29:18.163 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-09-12 03:27:40.748995757 +0000 2024-09-12T03:29:18.163 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-09-12 03:27:40.748995757 +0000 2024-09-12T03:29:18.163 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-09-12 03:29:13.950993813 +0000 2024-09-12T03:29:18.163 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: - 2024-09-12T03:29:18.164 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-12T03:29:18.164 DEBUG:teuthology.orchestra.run.smithi064:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-12T03:29:18.237 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-12T03:29:18.294 INFO:tasks.workunit.client.0.smithi064.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-12T03:29:19.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: [12/Sep/2024:03:29:17] ENGINE Bus STARTING 2024-09-12T03:29:19.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:29:19.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: [12/Sep/2024:03:29:17] ENGINE Serving on http://172.21.15.64:8765 2024-09-12T03:29:19.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:19.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:19.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:19.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:19.144 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: [12/Sep/2024:03:29:17] ENGINE Serving on https://172.21.15.64:7150 2024-09-12T03:29:19.144 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: [12/Sep/2024:03:29:17] ENGINE Bus STARTED 2024-09-12T03:29:19.144 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: [12/Sep/2024:03:29:17] ENGINE Error in HTTPServer.serve 2024-09-12T03:29:19.144 INFO:journalctl@ceph.mon.b.smithi115.stdout: Traceback (most recent call last): 2024-09-12T03:29:19.144 INFO:journalctl@ceph.mon.b.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T03:29:19.144 INFO:journalctl@ceph.mon.b.smithi115.stdout: self._connections.run(self.expiration_interval) 2024-09-12T03:29:19.144 INFO:journalctl@ceph.mon.b.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T03:29:19.144 INFO:journalctl@ceph.mon.b.smithi115.stdout: self._run(expiration_interval) 2024-09-12T03:29:19.144 INFO:journalctl@ceph.mon.b.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T03:29:19.144 INFO:journalctl@ceph.mon.b.smithi115.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T03:29:19.144 INFO:journalctl@ceph.mon.b.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T03:29:19.144 INFO:journalctl@ceph.mon.b.smithi115.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T03:29:19.144 INFO:journalctl@ceph.mon.b.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T03:29:19.145 INFO:journalctl@ceph.mon.b.smithi115.stdout: s = self.context.wrap_socket( 2024-09-12T03:29:19.145 INFO:journalctl@ceph.mon.b.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T03:29:19.145 INFO:journalctl@ceph.mon.b.smithi115.stdout: return self.sslsocket_class._create( 2024-09-12T03:29:19.145 INFO:journalctl@ceph.mon.b.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T03:29:19.145 INFO:journalctl@ceph.mon.b.smithi115.stdout: self.do_handshake() 2024-09-12T03:29:19.145 INFO:journalctl@ceph.mon.b.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T03:29:19.145 INFO:journalctl@ceph.mon.b.smithi115.stdout: self._sslobj.do_handshake() 2024-09-12T03:29:19.145 INFO:journalctl@ceph.mon.b.smithi115.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T03:29:19.145 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:19.145 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:19.145 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: from='client.25052 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:29:19.145 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-09-12T03:29:19.145 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:19.146 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:19.146 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:19.146 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.146 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:19.146 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.146 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.146 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.146 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.147 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.147 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.147 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.147 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.147 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.147 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.147 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.147 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.147 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.147 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.147 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.147 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:19.147 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:19.148 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.148 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.148 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.148 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.148 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.148 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.148 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.148 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.148 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:29:19.148 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:18 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:29:19.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: [12/Sep/2024:03:29:17] ENGINE Bus STARTING 2024-09-12T03:29:19.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:29:19.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: [12/Sep/2024:03:29:17] ENGINE Serving on http://172.21.15.64:8765 2024-09-12T03:29:19.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:19.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:19.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:19.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:19.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: [12/Sep/2024:03:29:17] ENGINE Serving on https://172.21.15.64:7150 2024-09-12T03:29:19.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: [12/Sep/2024:03:29:17] ENGINE Bus STARTED 2024-09-12T03:29:19.208 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: [12/Sep/2024:03:29:17] ENGINE Error in HTTPServer.serve 2024-09-12T03:29:19.208 INFO:journalctl@ceph.mon.c.smithi195.stdout: Traceback (most recent call last): 2024-09-12T03:29:19.208 INFO:journalctl@ceph.mon.c.smithi195.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T03:29:19.208 INFO:journalctl@ceph.mon.c.smithi195.stdout: self._connections.run(self.expiration_interval) 2024-09-12T03:29:19.208 INFO:journalctl@ceph.mon.c.smithi195.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T03:29:19.208 INFO:journalctl@ceph.mon.c.smithi195.stdout: self._run(expiration_interval) 2024-09-12T03:29:19.208 INFO:journalctl@ceph.mon.c.smithi195.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T03:29:19.208 INFO:journalctl@ceph.mon.c.smithi195.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T03:29:19.208 INFO:journalctl@ceph.mon.c.smithi195.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T03:29:19.208 INFO:journalctl@ceph.mon.c.smithi195.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T03:29:19.208 INFO:journalctl@ceph.mon.c.smithi195.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T03:29:19.208 INFO:journalctl@ceph.mon.c.smithi195.stdout: s = self.context.wrap_socket( 2024-09-12T03:29:19.209 INFO:journalctl@ceph.mon.c.smithi195.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T03:29:19.209 INFO:journalctl@ceph.mon.c.smithi195.stdout: return self.sslsocket_class._create( 2024-09-12T03:29:19.209 INFO:journalctl@ceph.mon.c.smithi195.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T03:29:19.209 INFO:journalctl@ceph.mon.c.smithi195.stdout: self.do_handshake() 2024-09-12T03:29:19.209 INFO:journalctl@ceph.mon.c.smithi195.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T03:29:19.209 INFO:journalctl@ceph.mon.c.smithi195.stdout: self._sslobj.do_handshake() 2024-09-12T03:29:19.209 INFO:journalctl@ceph.mon.c.smithi195.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T03:29:19.209 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:19.209 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:19.209 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: from='client.25052 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:29:19.209 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-09-12T03:29:19.209 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:19.209 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:19.210 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:19.210 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.210 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:19.210 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.210 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.210 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.210 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.210 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.210 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.210 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.210 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.210 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.210 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.211 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.211 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.211 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.211 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.211 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.211 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:19.212 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:19.212 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.212 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.212 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.212 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.212 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.212 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.212 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.212 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.212 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:29:19.212 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:18 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:29:19.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: [12/Sep/2024:03:29:17] ENGINE Bus STARTING 2024-09-12T03:29:19.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:29:19.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: [12/Sep/2024:03:29:17] ENGINE Serving on http://172.21.15.64:8765 2024-09-12T03:29:19.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:19.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:19.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:19.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:19.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: [12/Sep/2024:03:29:17] ENGINE Serving on https://172.21.15.64:7150 2024-09-12T03:29:19.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: [12/Sep/2024:03:29:17] ENGINE Bus STARTED 2024-09-12T03:29:19.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: [12/Sep/2024:03:29:17] ENGINE Error in HTTPServer.serve 2024-09-12T03:29:19.273 INFO:journalctl@ceph.mon.a.smithi064.stdout: Traceback (most recent call last): 2024-09-12T03:29:19.273 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T03:29:19.273 INFO:journalctl@ceph.mon.a.smithi064.stdout: self._connections.run(self.expiration_interval) 2024-09-12T03:29:19.273 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T03:29:19.273 INFO:journalctl@ceph.mon.a.smithi064.stdout: self._run(expiration_interval) 2024-09-12T03:29:19.273 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T03:29:19.273 INFO:journalctl@ceph.mon.a.smithi064.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T03:29:19.273 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T03:29:19.274 INFO:journalctl@ceph.mon.a.smithi064.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T03:29:19.274 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T03:29:19.274 INFO:journalctl@ceph.mon.a.smithi064.stdout: s = self.context.wrap_socket( 2024-09-12T03:29:19.274 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T03:29:19.274 INFO:journalctl@ceph.mon.a.smithi064.stdout: return self.sslsocket_class._create( 2024-09-12T03:29:19.274 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T03:29:19.274 INFO:journalctl@ceph.mon.a.smithi064.stdout: self.do_handshake() 2024-09-12T03:29:19.274 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T03:29:19.274 INFO:journalctl@ceph.mon.a.smithi064.stdout: self._sslobj.do_handshake() 2024-09-12T03:29:19.274 INFO:journalctl@ceph.mon.a.smithi064.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T03:29:19.274 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:19.274 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:19.274 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: from='client.25052 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:29:19.274 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-09-12T03:29:19.274 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:19.275 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:19.275 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:19.275 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.275 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:19.275 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.275 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.275 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.275 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.275 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.275 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.276 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.276 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.276 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.276 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.276 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.276 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.276 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.276 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.276 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.276 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:19.276 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:19.276 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.277 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.277 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.277 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.277 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.277 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.277 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.277 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:29:19.277 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:29:19.277 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:18 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:29:20.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:19 smithi115 ceph-mon[34801]: Adjusting osd_memory_target on smithi115 to 176.0M 2024-09-12T03:29:20.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:19 smithi115 ceph-mon[34801]: Unable to set osd_memory_target on smithi115 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-12T03:29:20.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:19 smithi115 ceph-mon[34801]: Adjusting osd_memory_target on smithi195 to 176.0M 2024-09-12T03:29:20.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:19 smithi115 ceph-mon[34801]: Unable to set osd_memory_target on smithi195 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-12T03:29:20.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:19 smithi115 ceph-mon[34801]: Adjusting osd_memory_target on smithi064 to 176.0M 2024-09-12T03:29:20.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:19 smithi115 ceph-mon[34801]: Unable to set osd_memory_target on smithi064 to 184586752: error parsing value: Value '184586752' is below minimum 939524096 2024-09-12T03:29:20.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:19 smithi115 ceph-mon[34801]: Updating smithi064:/etc/ceph/ceph.conf 2024-09-12T03:29:20.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:19 smithi115 ceph-mon[34801]: Updating smithi115:/etc/ceph/ceph.conf 2024-09-12T03:29:20.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:19 smithi115 ceph-mon[34801]: Updating smithi195:/etc/ceph/ceph.conf 2024-09-12T03:29:20.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:19 smithi115 ceph-mon[34801]: Updating smithi115:/var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:29:20.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:19 smithi115 ceph-mon[34801]: Updating smithi195:/var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:29:20.144 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:19 smithi115 ceph-mon[34801]: Updating smithi064:/var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:29:20.144 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:19 smithi115 ceph-mon[34801]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:29:20.144 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:19 smithi115 ceph-mon[34801]: Updating smithi195:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:29:20.144 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:19 smithi115 ceph-mon[34801]: Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:29:20.144 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:19 smithi115 ceph-mon[34801]: Updating smithi115:/var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:29:20.144 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:19 smithi115 ceph-mon[34801]: Updating smithi195:/var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:29:20.144 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:19 smithi115 ceph-mon[34801]: Updating smithi064:/var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:29:20.144 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:19 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:20.144 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:19 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:20.144 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:19 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:20.144 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:19 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:20.144 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:19 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:20.144 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:19 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:20.144 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:19 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:20.145 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:19 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:29:20.145 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:19 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:29:20.145 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:19 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:29:20.145 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:19 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:20.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:19 smithi195 ceph-mon[34786]: Adjusting osd_memory_target on smithi115 to 176.0M 2024-09-12T03:29:20.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:19 smithi195 ceph-mon[34786]: Unable to set osd_memory_target on smithi115 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-12T03:29:20.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:19 smithi195 ceph-mon[34786]: Adjusting osd_memory_target on smithi195 to 176.0M 2024-09-12T03:29:20.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:19 smithi195 ceph-mon[34786]: Unable to set osd_memory_target on smithi195 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-12T03:29:20.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:19 smithi195 ceph-mon[34786]: Adjusting osd_memory_target on smithi064 to 176.0M 2024-09-12T03:29:20.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:19 smithi195 ceph-mon[34786]: Unable to set osd_memory_target on smithi064 to 184586752: error parsing value: Value '184586752' is below minimum 939524096 2024-09-12T03:29:20.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:19 smithi195 ceph-mon[34786]: Updating smithi064:/etc/ceph/ceph.conf 2024-09-12T03:29:20.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:19 smithi195 ceph-mon[34786]: Updating smithi115:/etc/ceph/ceph.conf 2024-09-12T03:29:20.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:19 smithi195 ceph-mon[34786]: Updating smithi195:/etc/ceph/ceph.conf 2024-09-12T03:29:20.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:19 smithi195 ceph-mon[34786]: Updating smithi115:/var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:29:20.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:19 smithi195 ceph-mon[34786]: Updating smithi195:/var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:29:20.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:19 smithi195 ceph-mon[34786]: Updating smithi064:/var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:29:20.208 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:19 smithi195 ceph-mon[34786]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:29:20.208 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:19 smithi195 ceph-mon[34786]: Updating smithi195:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:29:20.208 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:19 smithi195 ceph-mon[34786]: Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:29:20.208 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:19 smithi195 ceph-mon[34786]: Updating smithi115:/var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:29:20.208 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:19 smithi195 ceph-mon[34786]: Updating smithi195:/var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:29:20.208 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:19 smithi195 ceph-mon[34786]: Updating smithi064:/var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:29:20.208 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:19 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:20.208 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:19 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:20.208 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:19 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:20.208 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:19 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:20.208 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:19 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:20.208 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:19 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:20.208 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:19 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:20.208 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:19 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:29:20.209 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:19 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:29:20.209 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:19 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:29:20.209 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:19 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:20.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:19 smithi064 ceph-mon[30952]: Adjusting osd_memory_target on smithi115 to 176.0M 2024-09-12T03:29:20.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:19 smithi064 ceph-mon[30952]: Unable to set osd_memory_target on smithi115 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-12T03:29:20.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:19 smithi064 ceph-mon[30952]: Adjusting osd_memory_target on smithi195 to 176.0M 2024-09-12T03:29:20.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:19 smithi064 ceph-mon[30952]: Unable to set osd_memory_target on smithi195 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-12T03:29:20.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:19 smithi064 ceph-mon[30952]: Adjusting osd_memory_target on smithi064 to 176.0M 2024-09-12T03:29:20.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:19 smithi064 ceph-mon[30952]: Unable to set osd_memory_target on smithi064 to 184586752: error parsing value: Value '184586752' is below minimum 939524096 2024-09-12T03:29:20.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:19 smithi064 ceph-mon[30952]: Updating smithi064:/etc/ceph/ceph.conf 2024-09-12T03:29:20.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:19 smithi064 ceph-mon[30952]: Updating smithi115:/etc/ceph/ceph.conf 2024-09-12T03:29:20.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:19 smithi064 ceph-mon[30952]: Updating smithi195:/etc/ceph/ceph.conf 2024-09-12T03:29:20.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:19 smithi064 ceph-mon[30952]: Updating smithi115:/var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:29:20.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:19 smithi064 ceph-mon[30952]: Updating smithi195:/var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:29:20.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:19 smithi064 ceph-mon[30952]: Updating smithi064:/var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:29:20.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:19 smithi064 ceph-mon[30952]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:29:20.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:19 smithi064 ceph-mon[30952]: Updating smithi195:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:29:20.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:19 smithi064 ceph-mon[30952]: Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:29:20.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:19 smithi064 ceph-mon[30952]: Updating smithi115:/var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:29:20.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:19 smithi064 ceph-mon[30952]: Updating smithi195:/var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:29:20.273 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:19 smithi064 ceph-mon[30952]: Updating smithi064:/var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:29:20.274 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:19 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:20.274 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:19 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:20.274 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:19 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:20.274 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:19 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:20.274 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:19 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:20.274 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:19 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:20.274 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:19 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:20.274 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:19 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:29:20.274 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:19 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:29:20.274 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:19 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:29:20.274 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:19 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:29:21.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:20 smithi115 ceph-mon[34801]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:29:21.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:20 smithi115 ceph-mon[34801]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-09-12T03:29:21.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:20 smithi195 ceph-mon[34786]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:29:21.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:20 smithi195 ceph-mon[34786]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-09-12T03:29:21.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:20 smithi064 ceph-mon[30952]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:29:21.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:20 smithi064 ceph-mon[30952]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-09-12T03:29:23.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:22 smithi115 ceph-mon[34801]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:29:23.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:22 smithi195 ceph-mon[34786]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:29:23.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:22 smithi064 ceph-mon[30952]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:29:25.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:24 smithi115 ceph-mon[34801]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 76 op/s 2024-09-12T03:29:25.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:24 smithi195 ceph-mon[34786]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 76 op/s 2024-09-12T03:29:25.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:24 smithi064 ceph-mon[30952]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 76 op/s 2024-09-12T03:29:27.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:26 smithi115 ceph-mon[34801]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 102 KiB/s wr, 59 op/s 2024-09-12T03:29:27.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:26 smithi195 ceph-mon[34786]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 102 KiB/s wr, 59 op/s 2024-09-12T03:29:27.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:26 smithi064 ceph-mon[30952]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 102 KiB/s wr, 59 op/s 2024-09-12T03:29:29.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:28 smithi115 ceph-mon[34801]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-09-12T03:29:29.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:28 smithi195 ceph-mon[34786]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-09-12T03:29:29.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:28 smithi064 ceph-mon[30952]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-09-12T03:29:31.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:30 smithi115 ceph-mon[34801]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-09-12T03:29:31.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:30 smithi195 ceph-mon[34786]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-09-12T03:29:31.271 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:30 smithi064 ceph-mon[30952]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-09-12T03:29:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.275+0000 7f9239e0e640 1 -- 172.21.15.64:0/3095667621 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f923410d490 msgr2=0x7f9234114fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.275+0000 7f9239e0e640 1 --2- 172.21.15.64:0/3095667621 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f923410d490 0x7f9234114fe0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f922c009900 tx=0x7f922c02a410 comp rx=0 tx=0).stop 2024-09-12T03:29:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.275+0000 7f9239e0e640 1 -- 172.21.15.64:0/3095667621 shutdown_connections 2024-09-12T03:29:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.275+0000 7f9239e0e640 1 --2- 172.21.15.64:0/3095667621 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f923410d490 0x7f9234114fe0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.275+0000 7f9239e0e640 1 --2- 172.21.15.64:0/3095667621 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9234108cf0 0x7f923410cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.275+0000 7f9239e0e640 1 --2- 172.21.15.64:0/3095667621 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9234108340 0x7f9234108720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.275+0000 7f9239e0e640 1 -- 172.21.15.64:0/3095667621 >> 172.21.15.64:0/3095667621 conn(0x7f9234077a40 msgr2=0x7f9234079e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.275+0000 7f9239e0e640 1 -- 172.21.15.64:0/3095667621 shutdown_connections 2024-09-12T03:29:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.275+0000 7f9239e0e640 1 -- 172.21.15.64:0/3095667621 wait complete. 2024-09-12T03:29:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.276+0000 7f9239e0e640 1 Processor -- start 2024-09-12T03:29:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.276+0000 7f9239e0e640 1 -- start start 2024-09-12T03:29:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.276+0000 7f9239e0e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9234108340 0x7f92341a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.276+0000 7f9239e0e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f923410d490 0x7f92341a4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.276+0000 7f9239e0e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f923419e640 0x7f923419eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.276+0000 7f9239e0e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9234118370 con 0x7f9234108340 2024-09-12T03:29:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.276+0000 7f9239e0e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f92341181f0 con 0x7f923419e640 2024-09-12T03:29:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.276+0000 7f9239e0e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f92341184f0 con 0x7f923410d490 2024-09-12T03:29:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.276+0000 7f92337fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9234108340 0x7f92341a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.276+0000 7f92337fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9234108340 0x7f92341a4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33206/0 (socket says 172.21.15.64:33206) 2024-09-12T03:29:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.276+0000 7f92337fe640 1 -- 172.21.15.64:0/2377636227 learned_addr learned my addr 172.21.15.64:0/2377636227 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.277+0000 7f9233fff640 1 --2- 172.21.15.64:0/2377636227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f923419e640 0x7f923419eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.277+0000 7f9232ffd640 1 --2- 172.21.15.64:0/2377636227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f923410d490 0x7f92341a4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.277+0000 7f92337fe640 1 -- 172.21.15.64:0/2377636227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f923410d490 msgr2=0x7f92341a4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.277+0000 7f92337fe640 1 --2- 172.21.15.64:0/2377636227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f923410d490 0x7f92341a4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.277+0000 7f92337fe640 1 -- 172.21.15.64:0/2377636227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f923419e640 msgr2=0x7f923419eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.277+0000 7f92337fe640 1 --2- 172.21.15.64:0/2377636227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f923419e640 0x7f923419eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.277+0000 7f92337fe640 1 -- 172.21.15.64:0/2377636227 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f923419f240 con 0x7f9234108340 2024-09-12T03:29:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.277+0000 7f9232ffd640 1 --2- 172.21.15.64:0/2377636227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f923410d490 0x7f92341a4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:29:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.277+0000 7f9233fff640 1 --2- 172.21.15.64:0/2377636227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f923419e640 0x7f923419eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:29:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.277+0000 7f92337fe640 1 --2- 172.21.15.64:0/2377636227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9234108340 0x7f92341a4580 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f922400db80 tx=0x7f922400b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.277+0000 7f9230ff9640 1 -- 172.21.15.64:0/2377636227 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9224017070 con 0x7f9234108340 2024-09-12T03:29:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.277+0000 7f9230ff9640 1 -- 172.21.15.64:0/2377636227 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9224003b70 con 0x7f9234108340 2024-09-12T03:29:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.277+0000 7f9239e0e640 1 -- 172.21.15.64:0/2377636227 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92341ab320 con 0x7f9234108340 2024-09-12T03:29:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.277+0000 7f9230ff9640 1 -- 172.21.15.64:0/2377636227 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92240104c0 con 0x7f9234108340 2024-09-12T03:29:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.277+0000 7f9239e0e640 1 -- 172.21.15.64:0/2377636227 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92341ab780 con 0x7f9234108340 2024-09-12T03:29:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.280+0000 7f9230ff9640 1 -- 172.21.15.64:0/2377636227 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f9224004020 con 0x7f9234108340 2024-09-12T03:29:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.280+0000 7f923cc87740 1 Processor -- start 2024-09-12T03:29:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.280+0000 7f923cc87740 1 -- start start 2024-09-12T03:29:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.280+0000 7f923cc87740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a489b78af0 0x55a489b78ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.280+0000 7f923cc87740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a489b79410 0x55a489b83770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.280+0000 7f923cc87740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a489b83cb0 0x55a489b860a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.280+0000 7f923cc87740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a489add2f0 con 0x55a489b79410 2024-09-12T03:29:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.280+0000 7f923cc87740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a489a85270 con 0x55a489b78af0 2024-09-12T03:29:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.280+0000 7f923cc87740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a489ad92b0 con 0x55a489b83cb0 2024-09-12T03:29:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.280+0000 7f9230ff9640 1 --2- 172.21.15.64:0/2377636227 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f921c0a2ec0 0x7f921c0a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.280+0000 7f92337fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a489b79410 0x55a489b83770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.280+0000 7f92337fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a489b79410 0x55a489b83770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33208/0 (socket says 172.21.15.64:33208) 2024-09-12T03:29:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.280+0000 7f92337fe640 1 -- 172.21.15.64:0/505378770 learned_addr learned my addr 172.21.15.64:0/505378770 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:31.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.280+0000 7f9233fff640 1 --2- 172.21.15.64:0/505378770 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a489b78af0 0x55a489b78ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:31.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.281+0000 7f9233fff640 1 --2- 172.21.15.64:0/2377636227 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f921c0a2ec0 0x7f921c0a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:31.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.281+0000 7f9232ffd640 1 --2- 172.21.15.64:0/505378770 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a489b83cb0 0x55a489b860a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:31.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.281+0000 7f9230ff9640 1 -- 172.21.15.64:0/2377636227 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f92240cfaa0 con 0x7f9234108340 2024-09-12T03:29:31.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.281+0000 7f92337fe640 1 -- 172.21.15.64:0/505378770 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a489b83cb0 msgr2=0x55a489b860a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:31.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.281+0000 7f92337fe640 1 --2- 172.21.15.64:0/505378770 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a489b83cb0 0x55a489b860a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:31.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.281+0000 7f92337fe640 1 -- 172.21.15.64:0/505378770 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a489b78af0 msgr2=0x55a489b78ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:31.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.281+0000 7f92337fe640 1 --2- 172.21.15.64:0/505378770 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a489b78af0 0x55a489b78ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:31.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.281+0000 7f92337fe640 1 -- 172.21.15.64:0/505378770 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a489a84340 con 0x55a489b79410 2024-09-12T03:29:31.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.281+0000 7f9232ffd640 1 --2- 172.21.15.64:0/505378770 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a489b83cb0 0x55a489b860a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:29:31.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.281+0000 7f9233fff640 1 --2- 172.21.15.64:0/505378770 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a489b78af0 0x55a489b78ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:31.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.281+0000 7f92337fe640 1 --2- 172.21.15.64:0/505378770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a489b79410 0x55a489b83770 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f922400db50 tx=0x7f922401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9b06944d2f9e20 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:31.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.281+0000 7f9233fff640 1 --2- 172.21.15.64:0/2377636227 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f921c0a2ec0 0x7f921c0a5380 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f922c0057c0 tx=0x7f922c009260 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:31.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.281+0000 7f9219ffb640 1 -- 172.21.15.64:0/505378770 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9224004310 con 0x55a489b79410 2024-09-12T03:29:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.282+0000 7f9219ffb640 1 -- 172.21.15.64:0/505378770 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f922409ba80 con 0x55a489b79410 2024-09-12T03:29:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.282+0000 7f9219ffb640 1 -- 172.21.15.64:0/505378770 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92240a46f0 con 0x55a489b79410 2024-09-12T03:29:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.282+0000 7f923cc87740 1 -- 172.21.15.64:0/505378770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a489b79410 msgr2=0x55a489b83770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.282+0000 7f923cc87740 1 --2- 172.21.15.64:0/505378770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a489b79410 0x55a489b83770 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f922400db50 tx=0x7f922401c000 comp rx=0 tx=0).stop 2024-09-12T03:29:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.282+0000 7f923cc87740 1 -- 172.21.15.64:0/505378770 shutdown_connections 2024-09-12T03:29:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.282+0000 7f923cc87740 1 --2- 172.21.15.64:0/505378770 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a489b83cb0 0x55a489b860a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.282+0000 7f923cc87740 1 --2- 172.21.15.64:0/505378770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a489b79410 0x55a489b83770 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.282+0000 7f923cc87740 1 --2- 172.21.15.64:0/505378770 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a489b78af0 0x55a489b78ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.282+0000 7f923cc87740 1 -- 172.21.15.64:0/505378770 >> 172.21.15.64:0/505378770 conn(0x55a489b75060 msgr2=0x55a489b75440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.282+0000 7f923cc87740 1 -- 172.21.15.64:0/505378770 shutdown_connections 2024-09-12T03:29:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.282+0000 7f923cc87740 1 -- 172.21.15.64:0/505378770 wait complete. 2024-09-12T03:29:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.283+0000 7f923cc87740 1 -- start start 2024-09-12T03:29:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.283+0000 7f923cc87740 1 Processor -- start 2024-09-12T03:29:31.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.283+0000 7f923cc87740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a489b78af0 0x55a489b80c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:31.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.283+0000 7f923cc87740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a489b79410 0x55a489b811b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:31.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.283+0000 7f923cc87740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a489b83cb0 0x55a489b816f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:31.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.283+0000 7f923cc87740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a489b86820 con 0x55a489b83cb0 2024-09-12T03:29:31.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.283+0000 7f923cc87740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a489a66290 con 0x55a489b78af0 2024-09-12T03:29:31.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.283+0000 7f923cc87740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a489b869a0 con 0x55a489b79410 2024-09-12T03:29:31.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.283+0000 7f923cc87740 10 client.0 _ll_register_callbacks cb 0x55a489a97140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:31.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.283+0000 7f9233fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a489b83cb0 0x55a489b816f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:31.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.283+0000 7f9233fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a489b83cb0 0x55a489b816f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33212/0 (socket says 172.21.15.64:33212) 2024-09-12T03:29:31.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.283+0000 7f9233fff640 1 -- 172.21.15.64:0/3394686906 learned_addr learned my addr 172.21.15.64:0/3394686906 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:31.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.283+0000 7f9232ffd640 1 --2- 172.21.15.64:0/3394686906 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a489b79410 0x55a489b811b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:31.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.283+0000 7f92337fe640 1 --2- 172.21.15.64:0/3394686906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a489b78af0 0x55a489b80c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.284+0000 7f9233fff640 1 -- 172.21.15.64:0/3394686906 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a489b79410 msgr2=0x55a489b811b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.284+0000 7f9233fff640 1 --2- 172.21.15.64:0/3394686906 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a489b79410 0x55a489b811b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.284+0000 7f9233fff640 1 -- 172.21.15.64:0/3394686906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a489b78af0 msgr2=0x55a489b80c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.284+0000 7f9233fff640 1 --2- 172.21.15.64:0/3394686906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a489b78af0 0x55a489b80c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.284+0000 7f9233fff640 1 -- 172.21.15.64:0/3394686906 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4897f0210 con 0x55a489b83cb0 2024-09-12T03:29:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.284+0000 7f92337fe640 1 --2- 172.21.15.64:0/3394686906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a489b78af0 0x55a489b80c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.284+0000 7f9233fff640 1 --2- 172.21.15.64:0/3394686906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a489b83cb0 0x55a489b816f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f922c033000 tx=0x7f922c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b593392ed2f7bf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.284+0000 7f9212ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:29:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.284+0000 7f9212ffd640 1 -- 172.21.15.64:0/3394686906 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922c04f020 con 0x55a489b83cb0 2024-09-12T03:29:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.284+0000 7f9212ffd640 1 -- 172.21.15.64:0/3394686906 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f922c003a00 con 0x55a489b83cb0 2024-09-12T03:29:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.284+0000 7f923cc87740 1 -- 172.21.15.64:0/3394686906 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a489aeaa20 con 0x55a489b83cb0 2024-09-12T03:29:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.284+0000 7f9212ffd640 1 -- 172.21.15.64:0/3394686906 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f922c05a050 con 0x55a489b83cb0 2024-09-12T03:29:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.284+0000 7f923cc87740 10 client.15354 fetch_fsmap learned FSMap version 25 2024-09-12T03:29:31.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.284+0000 7f923cc87740 1 -- 172.21.15.64:0/3394686906 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a4898bc620 con 0x55a489b83cb0 2024-09-12T03:29:31.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.285+0000 7f9212ffd640 1 -- 172.21.15.64:0/3394686906 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55a489aeaa20 con 0x55a489b83cb0 2024-09-12T03:29:31.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.285+0000 7f9212ffd640 1 -- 172.21.15.64:0/3394686906 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f922c03a040 con 0x55a489b83cb0 2024-09-12T03:29:31.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.285+0000 7f923cc87740 10 client.15354 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:29:31.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.285+0000 7f923cc87740 10 client.15354 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:29:31.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.285+0000 7f923cc87740 20 client.15354 populate_metadata read hostname 'smithi064' 2024-09-12T03:29:31.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.285+0000 7f923cc87740 1 --2- 172.21.15.64:0/3394686906 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55a489b93c60 0x55a489b960b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:31.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.285+0000 7f923cc87740 4 client.15354 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:29:31.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.285+0000 7f923cc87740 1 -- 172.21.15.64:0/3394686906 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a489ae4d70 con 0x55a489b93c60 2024-09-12T03:29:31.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.285+0000 7f92337fe640 1 --2- 172.21.15.64:0/3394686906 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55a489b93c60 0x55a489b960b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:31.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.286+0000 7f9212ffd640 10 client.15354 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:29:31.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.286+0000 7f92337fe640 1 --2- 172.21.15.64:0/3394686906 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55a489b93c60 0x55a489b960b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=707b91b6bf4a7f86 server_cookie=48d8cee2008ac4a6 in_seq=0 out_seq=0 2024-09-12T03:29:31.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.286+0000 7f9212ffd640 1 -- 172.21.15.64:0/3394686906 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a489ae4d70 con 0x55a489b93c60 2024-09-12T03:29:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.286+0000 7f9212ffd640 10 client.15354 handle_command_reply: tid=0 2024-09-12T03:29:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.286+0000 7f923cc87740 1 client.15354 shutdown 2024-09-12T03:29:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.287+0000 7f923cc87740 1 -- 172.21.15.64:0/3394686906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a489b83cb0 msgr2=0x55a489b816f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.287+0000 7f923cc87740 1 --2- 172.21.15.64:0/3394686906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a489b83cb0 0x55a489b816f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f922c033000 tx=0x7f922c02a410 comp rx=0 tx=0).stop 2024-09-12T03:29:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.287+0000 7f923cc87740 1 -- 172.21.15.64:0/3394686906 shutdown_connections 2024-09-12T03:29:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.287+0000 7f923cc87740 1 --2- 172.21.15.64:0/3394686906 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55a489b93c60 0x55a489b960b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.287+0000 7f923cc87740 1 --2- 172.21.15.64:0/3394686906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a489b83cb0 0x55a489b816f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.287+0000 7f923cc87740 1 --2- 172.21.15.64:0/3394686906 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a489b79410 0x55a489b811b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.287+0000 7f9212ffd640 0 client.15354 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:29:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.287+0000 7f923cc87740 1 --2- 172.21.15.64:0/3394686906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a489b78af0 0x55a489b80c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.287+0000 7f923cc87740 1 -- 172.21.15.64:0/3394686906 >> 172.21.15.64:0/3394686906 conn(0x55a489b75060 msgr2=0x55a489b7adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.287+0000 7f923cc87740 1 -- 172.21.15.64:0/3394686906 shutdown_connections 2024-09-12T03:29:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.287+0000 7f923cc87740 1 -- 172.21.15.64:0/3394686906 wait complete. 2024-09-12T03:29:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.287+0000 7f923cc87740 20 client.15354 trim_cache size 0 max 16384 2024-09-12T03:29:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.301+0000 7f923cc87740 1 Processor -- start 2024-09-12T03:29:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.302+0000 7f923cc87740 1 -- start start 2024-09-12T03:29:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.302+0000 7f923cc87740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a489b78af0 0x55a489b81520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:31.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.302+0000 7f923cc87740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a489b79410 0x55a489b7acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:31.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.302+0000 7f923cc87740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a489b83cb0 0x55a489b9b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:31.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.302+0000 7f923cc87740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f92341b7ee0 con 0x55a489b78af0 2024-09-12T03:29:31.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.302+0000 7f923cc87740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f92341b57b0 con 0x55a489b83cb0 2024-09-12T03:29:31.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.302+0000 7f923cc87740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f92341b3600 con 0x55a489b79410 2024-09-12T03:29:31.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.302+0000 7f92337fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a489b78af0 0x55a489b81520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:31.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.302+0000 7f92337fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a489b78af0 0x55a489b81520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33228/0 (socket says 172.21.15.64:33228) 2024-09-12T03:29:31.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.302+0000 7f92337fe640 1 -- 172.21.15.64:0/2249987198 learned_addr learned my addr 172.21.15.64:0/2249987198 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:31.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.302+0000 7f92337fe640 1 -- 172.21.15.64:0/2249987198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a489b79410 msgr2=0x55a489b7acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:31.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.302+0000 7f9232ffd640 1 --2- 172.21.15.64:0/2249987198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a489b79410 0x55a489b7acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:31.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.302+0000 7f92337fe640 1 --2- 172.21.15.64:0/2249987198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a489b79410 0x55a489b7acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:31.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.302+0000 7f92337fe640 1 -- 172.21.15.64:0/2249987198 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a489b83cb0 msgr2=0x55a489b9b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:31.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.302+0000 7f92337fe640 1 --2- 172.21.15.64:0/2249987198 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a489b83cb0 0x55a489b9b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.302+0000 7f92337fe640 1 -- 172.21.15.64:0/2249987198 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a489b81a60 con 0x55a489b78af0 2024-09-12T03:29:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.302+0000 7f9232ffd640 1 --2- 172.21.15.64:0/2249987198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a489b79410 0x55a489b7acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:29:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.302+0000 7f92337fe640 1 --2- 172.21.15.64:0/2249987198 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a489b78af0 0x55a489b81520 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f92240cce40 tx=0x7f922400df10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a79bddd4b594960a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.302+0000 7f92197fa640 1 -- 172.21.15.64:0/2249987198 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92240b9070 con 0x55a489b78af0 2024-09-12T03:29:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.303+0000 7f92197fa640 1 -- 172.21.15.64:0/2249987198 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92240212a0 con 0x55a489b78af0 2024-09-12T03:29:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.303+0000 7f92197fa640 1 -- 172.21.15.64:0/2249987198 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92240cd010 con 0x55a489b78af0 2024-09-12T03:29:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.303+0000 7f923cc87740 1 -- 172.21.15.64:0/2249987198 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a489b78af0 msgr2=0x55a489b81520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.303+0000 7f923cc87740 1 --2- 172.21.15.64:0/2249987198 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a489b78af0 0x55a489b81520 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f92240cce40 tx=0x7f922400df10 comp rx=0 tx=0).stop 2024-09-12T03:29:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.303+0000 7f923cc87740 1 -- 172.21.15.64:0/2249987198 shutdown_connections 2024-09-12T03:29:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.303+0000 7f923cc87740 1 --2- 172.21.15.64:0/2249987198 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a489b83cb0 0x55a489b9b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.303+0000 7f923cc87740 1 --2- 172.21.15.64:0/2249987198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a489b79410 0x55a489b7acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.303+0000 7f923cc87740 1 --2- 172.21.15.64:0/2249987198 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a489b78af0 0x55a489b81520 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.303+0000 7f923cc87740 1 -- 172.21.15.64:0/2249987198 >> 172.21.15.64:0/2249987198 conn(0x55a489b75060 msgr2=0x55a489b7a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.303+0000 7f923cc87740 1 -- 172.21.15.64:0/2249987198 shutdown_connections 2024-09-12T03:29:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.303+0000 7f923cc87740 1 -- 172.21.15.64:0/2249987198 wait complete. 2024-09-12T03:29:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.304+0000 7f923cc87740 1 -- start start 2024-09-12T03:29:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.304+0000 7f923cc87740 1 Processor -- start 2024-09-12T03:29:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.304+0000 7f923cc87740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a489b78af0 0x55a489b871b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.304+0000 7f923cc87740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a489b79410 0x55a489b876f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.304+0000 7f923cc87740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a489b83cb0 0x55a489bfc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.304+0000 7f923cc87740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f92341b3c70 con 0x55a489b79410 2024-09-12T03:29:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.304+0000 7f923cc87740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f92341b50e0 con 0x55a489b83cb0 2024-09-12T03:29:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.304+0000 7f923cc87740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f92341b4340 con 0x55a489b78af0 2024-09-12T03:29:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.304+0000 7f923cc87740 10 client.0 _ll_register_callbacks cb 0x7f922c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.304+0000 7f9232ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a489b79410 0x55a489b876f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.304+0000 7f9232ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a489b79410 0x55a489b876f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33242/0 (socket says 172.21.15.64:33242) 2024-09-12T03:29:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.304+0000 7f9232ffd640 1 -- 172.21.15.64:0/789912081 learned_addr learned my addr 172.21.15.64:0/789912081 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.304+0000 7f92337fe640 1 --2- 172.21.15.64:0/789912081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a489b78af0 0x55a489b871b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.304+0000 7f9233fff640 1 --2- 172.21.15.64:0/789912081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a489b83cb0 0x55a489bfc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.304+0000 7f9232ffd640 1 -- 172.21.15.64:0/789912081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a489b78af0 msgr2=0x55a489b871b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.305+0000 7f9232ffd640 1 --2- 172.21.15.64:0/789912081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a489b78af0 0x55a489b871b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.305+0000 7f9232ffd640 1 -- 172.21.15.64:0/789912081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a489b83cb0 msgr2=0x55a489bfc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.305+0000 7f9232ffd640 1 --2- 172.21.15.64:0/789912081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a489b83cb0 0x55a489bfc980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.305+0000 7f9232ffd640 1 -- 172.21.15.64:0/789912081 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a489bfcfc0 con 0x55a489b79410 2024-09-12T03:29:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.305+0000 7f9232ffd640 1 --2- 172.21.15.64:0/789912081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a489b79410 0x55a489b876f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f923419fe90 tx=0x7f9228013900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7096b894ac071428 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.305+0000 7f9212ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:29:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.305+0000 7f9212ffd640 1 -- 172.21.15.64:0/789912081 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9228004260 con 0x55a489b79410 2024-09-12T03:29:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.305+0000 7f9212ffd640 1 -- 172.21.15.64:0/789912081 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f9228004400 con 0x55a489b79410 2024-09-12T03:29:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.305+0000 7f923cc87740 1 -- 172.21.15.64:0/789912081 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92240212a0 con 0x55a489b79410 2024-09-12T03:29:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.305+0000 7f9212ffd640 1 -- 172.21.15.64:0/789912081 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f92280035f0 con 0x55a489b79410 2024-09-12T03:29:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.305+0000 7f923cc87740 10 client.15360 fetch_fsmap learned FSMap version 25 2024-09-12T03:29:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.305+0000 7f923cc87740 1 -- 172.21.15.64:0/789912081 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a489bfe2b0 con 0x55a489b79410 2024-09-12T03:29:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.305+0000 7f9212ffd640 1 -- 172.21.15.64:0/789912081 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92240212a0 con 0x55a489b79410 2024-09-12T03:29:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.305+0000 7f9212ffd640 1 -- 172.21.15.64:0/789912081 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f922800f040 con 0x55a489b79410 2024-09-12T03:29:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.305+0000 7f923cc87740 10 client.15360 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:29:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.305+0000 7f923cc87740 10 client.15360 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:29:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.305+0000 7f923cc87740 20 client.15360 populate_metadata read hostname 'smithi064' 2024-09-12T03:29:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.306+0000 7f923cc87740 1 --2- 172.21.15.64:0/789912081 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55a489bfeae0 0x55a489c00f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.306+0000 7f923cc87740 4 client.15360 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:29:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.306+0000 7f923cc87740 1 -- 172.21.15.64:0/789912081 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a489a61900 con 0x55a489bfeae0 2024-09-12T03:29:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.306+0000 7f92337fe640 1 --2- 172.21.15.64:0/789912081 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55a489bfeae0 0x55a489c00f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.306+0000 7f92337fe640 1 --2- 172.21.15.64:0/789912081 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55a489bfeae0 0x55a489c00f00 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0956c0dc17fcc7e server_cookie=c6cfb2bfc0850646 in_seq=0 out_seq=0 2024-09-12T03:29:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.306+0000 7f9212ffd640 10 client.15360 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:29:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.320+0000 7f9212ffd640 1 -- 172.21.15.64:0/789912081 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a489a61900 con 0x55a489bfeae0 2024-09-12T03:29:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.320+0000 7f9212ffd640 10 client.15360 handle_command_reply: tid=0 2024-09-12T03:29:31.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.320+0000 7f923cc87740 1 client.15360 shutdown 2024-09-12T03:29:31.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.320+0000 7f923cc87740 1 -- 172.21.15.64:0/789912081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a489b79410 msgr2=0x55a489b876f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:31.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.320+0000 7f923cc87740 1 --2- 172.21.15.64:0/789912081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a489b79410 0x55a489b876f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f923419fe90 tx=0x7f9228013900 comp rx=0 tx=0).stop 2024-09-12T03:29:31.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.320+0000 7f923cc87740 1 -- 172.21.15.64:0/789912081 shutdown_connections 2024-09-12T03:29:31.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.320+0000 7f923cc87740 1 --2- 172.21.15.64:0/789912081 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55a489bfeae0 0x55a489c00f00 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:31.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.320+0000 7f923cc87740 1 --2- 172.21.15.64:0/789912081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a489b83cb0 0x55a489bfc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:31.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.320+0000 7f923cc87740 1 --2- 172.21.15.64:0/789912081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a489b79410 0x55a489b876f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:31.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.320+0000 7f9212ffd640 0 client.15360 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:29:31.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.320+0000 7f923cc87740 1 --2- 172.21.15.64:0/789912081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a489b78af0 0x55a489b871b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:31.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.320+0000 7f923cc87740 1 -- 172.21.15.64:0/789912081 >> 172.21.15.64:0/789912081 conn(0x55a489b75060 msgr2=0x55a489b741c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:31.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.320+0000 7f923cc87740 1 -- 172.21.15.64:0/789912081 shutdown_connections 2024-09-12T03:29:31.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.320+0000 7f923cc87740 1 -- 172.21.15.64:0/789912081 wait complete. 2024-09-12T03:29:31.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.320+0000 7f923cc87740 20 client.15360 trim_cache size 0 max 16384 2024-09-12T03:29:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:29:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:29:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:29:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:29:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.321+0000 7f9239e0e640 1 -- 172.21.15.64:0/2377636227 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f921c0a2ec0 msgr2=0x7f921c0a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.321+0000 7f9239e0e640 1 --2- 172.21.15.64:0/2377636227 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f921c0a2ec0 0x7f921c0a5380 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f922c0057c0 tx=0x7f922c009260 comp rx=0 tx=0).stop 2024-09-12T03:29:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.321+0000 7f9239e0e640 1 -- 172.21.15.64:0/2377636227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9234108340 msgr2=0x7f92341a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.321+0000 7f9239e0e640 1 --2- 172.21.15.64:0/2377636227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9234108340 0x7f92341a4580 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f922400db80 tx=0x7f922400b630 comp rx=0 tx=0).stop 2024-09-12T03:29:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.321+0000 7f9239e0e640 1 -- 172.21.15.64:0/2377636227 shutdown_connections 2024-09-12T03:29:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.321+0000 7f9239e0e640 1 --2- 172.21.15.64:0/2377636227 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f921c0a2ec0 0x7f921c0a5380 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.321+0000 7f9239e0e640 1 --2- 172.21.15.64:0/2377636227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f923419e640 0x7f923419eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.321+0000 7f9239e0e640 1 --2- 172.21.15.64:0/2377636227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f923410d490 0x7f92341a4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.321+0000 7f9239e0e640 1 --2- 172.21.15.64:0/2377636227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9234108340 0x7f92341a4580 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.321+0000 7f9239e0e640 1 -- 172.21.15.64:0/2377636227 >> 172.21.15.64:0/2377636227 conn(0x7f9234077a40 msgr2=0x7f9234079e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.321+0000 7f9239e0e640 1 -- 172.21.15.64:0/2377636227 shutdown_connections 2024-09-12T03:29:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:31.321+0000 7f9239e0e640 1 -- 172.21.15.64:0/2377636227 wait complete. 2024-09-12T03:29:33.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:32 smithi115 ceph-mon[34801]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 49 op/s 2024-09-12T03:29:33.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:32 smithi195 ceph-mon[34786]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 49 op/s 2024-09-12T03:29:33.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:32 smithi064 ceph-mon[30952]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 49 op/s 2024-09-12T03:29:35.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:34 smithi115 ceph-mon[34801]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 79 KiB/s wr, 54 op/s 2024-09-12T03:29:35.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:34 smithi195 ceph-mon[34786]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 79 KiB/s wr, 54 op/s 2024-09-12T03:29:35.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:34 smithi064 ceph-mon[30952]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 79 KiB/s wr, 54 op/s 2024-09-12T03:29:37.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:36 smithi115 ceph-mon[34801]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-12T03:29:37.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:36 smithi195 ceph-mon[34786]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-12T03:29:37.271 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:36 smithi064 ceph-mon[30952]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-12T03:29:39.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:38 smithi115 ceph-mon[34801]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-09-12T03:29:39.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:38 smithi195 ceph-mon[34786]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-09-12T03:29:39.271 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:38 smithi064 ceph-mon[30952]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-09-12T03:29:41.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:40 smithi115 ceph-mon[34801]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-09-12T03:29:41.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:40 smithi195 ceph-mon[34786]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-09-12T03:29:41.271 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:40 smithi064 ceph-mon[30952]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-09-12T03:29:43.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:42 smithi115 ceph-mon[34801]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-09-12T03:29:43.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:42 smithi195 ceph-mon[34786]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-09-12T03:29:43.271 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:42 smithi064 ceph-mon[30952]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-09-12T03:29:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.412+0000 7fce1a8ac640 1 -- 172.21.15.64:0/2860587240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce14102d00 msgr2=0x7fce141030e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.412+0000 7fce1a8ac640 1 --2- 172.21.15.64:0/2860587240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce14102d00 0x7fce141030e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fce04009900 tx=0x7fce0402a480 comp rx=0 tx=0).stop 2024-09-12T03:29:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.412+0000 7fce1a8ac640 1 -- 172.21.15.64:0/2860587240 shutdown_connections 2024-09-12T03:29:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.412+0000 7fce1a8ac640 1 --2- 172.21.15.64:0/2860587240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce1410c0d0 0x7fce1410e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.412+0000 7fce1a8ac640 1 --2- 172.21.15.64:0/2860587240 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce14103620 0x7fce1410bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.412+0000 7fce1a8ac640 1 --2- 172.21.15.64:0/2860587240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce14102d00 0x7fce141030e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.412+0000 7fce1a8ac640 1 -- 172.21.15.64:0/2860587240 >> 172.21.15.64:0/2860587240 conn(0x7fce14077990 msgr2=0x7fce140fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.412+0000 7fce1a8ac640 1 -- 172.21.15.64:0/2860587240 shutdown_connections 2024-09-12T03:29:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.412+0000 7fce1a8ac640 1 -- 172.21.15.64:0/2860587240 wait complete. 2024-09-12T03:29:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.413+0000 7fce1a8ac640 1 Processor -- start 2024-09-12T03:29:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.413+0000 7fce1a8ac640 1 -- start start 2024-09-12T03:29:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.413+0000 7fce1a8ac640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce14102d00 0x7fce14198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.413+0000 7fce1a8ac640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce14103620 0x7fce14198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.413+0000 7fce1a8ac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce1410c0d0 0x7fce1419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.413+0000 7fce1a8ac640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fce14111da0 con 0x7fce1410c0d0 2024-09-12T03:29:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.413+0000 7fce1a8ac640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fce14111c20 con 0x7fce14103620 2024-09-12T03:29:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.413+0000 7fce1a8ac640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fce14111f20 con 0x7fce14102d00 2024-09-12T03:29:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.413+0000 7fce18e49640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce1410c0d0 0x7fce1419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.413+0000 7fce13fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce14102d00 0x7fce14198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.413+0000 7fce13fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce14102d00 0x7fce14198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.64:54346/0 (socket says 172.21.15.64:54346) 2024-09-12T03:29:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.413+0000 7fce18e49640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce1410c0d0 0x7fce1419cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43942/0 (socket says 172.21.15.64:43942) 2024-09-12T03:29:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.413+0000 7fce13fff640 1 -- 172.21.15.64:0/267962051 learned_addr learned my addr 172.21.15.64:0/267962051 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.413+0000 7fce137fe640 1 --2- 172.21.15.64:0/267962051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce14103620 0x7fce14198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.414+0000 7fce18e49640 1 -- 172.21.15.64:0/267962051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce14102d00 msgr2=0x7fce14198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.414+0000 7fce18e49640 1 --2- 172.21.15.64:0/267962051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce14102d00 0x7fce14198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.414+0000 7fce18e49640 1 -- 172.21.15.64:0/267962051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce14103620 msgr2=0x7fce14198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.414+0000 7fce18e49640 1 --2- 172.21.15.64:0/267962051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce14103620 0x7fce14198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.414+0000 7fce18e49640 1 -- 172.21.15.64:0/267962051 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce1419d340 con 0x7fce1410c0d0 2024-09-12T03:29:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.414+0000 7fce18e49640 1 --2- 172.21.15.64:0/267962051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce1410c0d0 0x7fce1419cc10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fce0c00ca30 tx=0x7fce0c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.414+0000 7fce117fa640 1 -- 172.21.15.64:0/267962051 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0c004d20 con 0x7fce1410c0d0 2024-09-12T03:29:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.414+0000 7fce117fa640 1 -- 172.21.15.64:0/267962051 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce0c005020 con 0x7fce1410c0d0 2024-09-12T03:29:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.414+0000 7fce117fa640 1 -- 172.21.15.64:0/267962051 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0c00f650 con 0x7fce1410c0d0 2024-09-12T03:29:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.414+0000 7fce1a8ac640 1 -- 172.21.15.64:0/267962051 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce1419d630 con 0x7fce1410c0d0 2024-09-12T03:29:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.414+0000 7fce1a8ac640 1 -- 172.21.15.64:0/267962051 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce141a4ec0 con 0x7fce1410c0d0 2024-09-12T03:29:44.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.416+0000 7fce117fa640 1 -- 172.21.15.64:0/267962051 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7fce0c0051d0 con 0x7fce1410c0d0 2024-09-12T03:29:44.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.417+0000 7fce1d64a740 1 Processor -- start 2024-09-12T03:29:44.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.417+0000 7fce1d64a740 1 -- start start 2024-09-12T03:29:44.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.417+0000 7fce1d64a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd2367aaf0 0x55fd2367aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:44.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.417+0000 7fce1d64a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd2367b410 0x55fd23685770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:44.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.417+0000 7fce1d64a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd23685cb0 0x55fd236880a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:44.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.417+0000 7fce1d64a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55fd235df2f0 con 0x55fd2367aaf0 2024-09-12T03:29:44.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.417+0000 7fce117fa640 1 --2- 172.21.15.64:0/267962051 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fcdfc0a2e40 0x7fcdfc0a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:44.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.417+0000 7fce1d64a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55fd23587270 con 0x55fd23685cb0 2024-09-12T03:29:44.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.417+0000 7fce1d64a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fd235db2b0 con 0x55fd2367b410 2024-09-12T03:29:44.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.418+0000 7fce137fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd2367aaf0 0x55fd2367aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:44.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.418+0000 7fce137fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd2367aaf0 0x55fd2367aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43956/0 (socket says 172.21.15.64:43956) 2024-09-12T03:29:44.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.418+0000 7fce137fe640 1 -- 172.21.15.64:0/1742914354 learned_addr learned my addr 172.21.15.64:0/1742914354 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.418+0000 7fce18e49640 1 --2- 172.21.15.64:0/267962051 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fcdfc0a2e40 0x7fcdfc0a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.418+0000 7fce13fff640 1 --2- 172.21.15.64:0/1742914354 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd2367b410 0x55fd23685770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.418+0000 7fce117fa640 1 -- 172.21.15.64:0/267962051 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7fce0c0cfc40 con 0x7fce1410c0d0 2024-09-12T03:29:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.418+0000 7fce137fe640 1 --2- 172.21.15.64:0/1742914354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd23685cb0 0x55fd236880a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.418+0000 7fce137fe640 1 -- 172.21.15.64:0/1742914354 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd2367b410 msgr2=0x55fd23685770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.418+0000 7fce137fe640 1 --2- 172.21.15.64:0/1742914354 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd2367b410 0x55fd23685770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.418+0000 7fce137fe640 1 -- 172.21.15.64:0/1742914354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd23685cb0 msgr2=0x55fd236880a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.418+0000 7fce137fe640 1 --2- 172.21.15.64:0/1742914354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd23685cb0 0x55fd236880a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.418+0000 7fce137fe640 1 -- 172.21.15.64:0/1742914354 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd23586340 con 0x55fd2367aaf0 2024-09-12T03:29:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.418+0000 7fce137fe640 1 --2- 172.21.15.64:0/1742914354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd2367aaf0 0x55fd2367aed0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fce08009cb0 tx=0x7fce080096f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78057e918185d5b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.418+0000 7fce18e49640 1 --2- 172.21.15.64:0/267962051 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fcdfc0a2e40 0x7fcdfc0a5300 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fce0c0ccf00 tx=0x7fce0c091050 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.418+0000 7fcdfa7fc640 1 -- 172.21.15.64:0/1742914354 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce08009e60 con 0x55fd2367aaf0 2024-09-12T03:29:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.418+0000 7fcdfa7fc640 1 -- 172.21.15.64:0/1742914354 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce0801a070 con 0x55fd2367aaf0 2024-09-12T03:29:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.418+0000 7fcdfa7fc640 1 -- 172.21.15.64:0/1742914354 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce080125d0 con 0x55fd2367aaf0 2024-09-12T03:29:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.419+0000 7fce1d64a740 1 -- 172.21.15.64:0/1742914354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd2367aaf0 msgr2=0x55fd2367aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.419+0000 7fce1d64a740 1 --2- 172.21.15.64:0/1742914354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd2367aaf0 0x55fd2367aed0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fce08009cb0 tx=0x7fce080096f0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.419+0000 7fce1d64a740 1 -- 172.21.15.64:0/1742914354 shutdown_connections 2024-09-12T03:29:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.419+0000 7fce1d64a740 1 --2- 172.21.15.64:0/1742914354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd23685cb0 0x55fd236880a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.419+0000 7fce1d64a740 1 --2- 172.21.15.64:0/1742914354 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd2367b410 0x55fd23685770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.419+0000 7fce1d64a740 1 --2- 172.21.15.64:0/1742914354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd2367aaf0 0x55fd2367aed0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.419+0000 7fce1d64a740 1 -- 172.21.15.64:0/1742914354 >> 172.21.15.64:0/1742914354 conn(0x55fd23677060 msgr2=0x55fd23677440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.419+0000 7fce1d64a740 1 -- 172.21.15.64:0/1742914354 shutdown_connections 2024-09-12T03:29:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.419+0000 7fce1d64a740 1 -- 172.21.15.64:0/1742914354 wait complete. 2024-09-12T03:29:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.420+0000 7fce1d64a740 1 -- start start 2024-09-12T03:29:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.420+0000 7fce1d64a740 1 Processor -- start 2024-09-12T03:29:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.420+0000 7fce1d64a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd2367aaf0 0x55fd23682b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.420+0000 7fce1d64a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd2367b410 0x55fd23683070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.420+0000 7fce1d64a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd23685cb0 0x55fd236835b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.420+0000 7fce1d64a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55fd23688880 con 0x55fd2367b410 2024-09-12T03:29:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.420+0000 7fce1d64a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55fd23568290 con 0x55fd23685cb0 2024-09-12T03:29:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.420+0000 7fce1d64a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fd23688a00 con 0x55fd2367aaf0 2024-09-12T03:29:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.420+0000 7fce1d64a740 10 client.0 _ll_register_callbacks cb 0x55fd23599140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:44.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.420+0000 7fce13fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd2367b410 0x55fd23683070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:44.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.420+0000 7fce13fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd2367b410 0x55fd23683070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43966/0 (socket says 172.21.15.64:43966) 2024-09-12T03:29:44.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.420+0000 7fce13fff640 1 -- 172.21.15.64:0/3224033430 learned_addr learned my addr 172.21.15.64:0/3224033430 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:44.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.420+0000 7fce137fe640 1 --2- 172.21.15.64:0/3224033430 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd2367aaf0 0x55fd23682b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:44.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.420+0000 7fce137fe640 1 --2- 172.21.15.64:0/3224033430 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd23685cb0 0x55fd236835b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:44.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.421+0000 7fce13fff640 1 -- 172.21.15.64:0/3224033430 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd2367aaf0 msgr2=0x55fd23682b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:44.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.421+0000 7fce13fff640 1 --2- 172.21.15.64:0/3224033430 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd2367aaf0 0x55fd23682b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.421+0000 7fce13fff640 1 -- 172.21.15.64:0/3224033430 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd23685cb0 msgr2=0x55fd236835b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:44.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.421+0000 7fce13fff640 1 --2- 172.21.15.64:0/3224033430 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd23685cb0 0x55fd236835b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.421+0000 7fce13fff640 1 -- 172.21.15.64:0/3224033430 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd232f2210 con 0x55fd2367b410 2024-09-12T03:29:44.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.421+0000 7fce13fff640 1 --2- 172.21.15.64:0/3224033430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd2367b410 0x55fd23683070 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fce04000c00 tx=0x7fce040316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c44d0e19ac8079b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:44.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.421+0000 7fcdf37fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:29:44.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.421+0000 7fcdf37fe640 1 -- 172.21.15.64:0/3224033430 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce04035040 con 0x55fd2367b410 2024-09-12T03:29:44.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.421+0000 7fcdf37fe640 1 -- 172.21.15.64:0/3224033430 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7fce04004450 con 0x55fd2367b410 2024-09-12T03:29:44.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.421+0000 7fce1d64a740 1 -- 172.21.15.64:0/3224033430 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce0801a070 con 0x55fd2367b410 2024-09-12T03:29:44.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.421+0000 7fcdf37fe640 1 -- 172.21.15.64:0/3224033430 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fce04051050 con 0x55fd2367b410 2024-09-12T03:29:44.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.421+0000 7fce1d64a740 10 client.15372 fetch_fsmap learned FSMap version 25 2024-09-12T03:29:44.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.421+0000 7fce1d64a740 1 -- 172.21.15.64:0/3224033430 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fd233be620 con 0x55fd2367b410 2024-09-12T03:29:44.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.421+0000 7fcdf37fe640 1 -- 172.21.15.64:0/3224033430 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce0801a070 con 0x55fd2367b410 2024-09-12T03:29:44.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.422+0000 7fcdf37fe640 1 -- 172.21.15.64:0/3224033430 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fce04048510 con 0x55fd2367b410 2024-09-12T03:29:44.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.422+0000 7fce1d64a740 10 client.15372 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:29:44.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.422+0000 7fce1d64a740 10 client.15372 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:29:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.422+0000 7fce1d64a740 20 client.15372 populate_metadata read hostname 'smithi064' 2024-09-12T03:29:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.422+0000 7fce1d64a740 1 --2- 172.21.15.64:0/3224033430 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55fd23695bd0 0x55fd23697fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.422+0000 7fce1d64a740 4 client.15372 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:29:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.422+0000 7fce1d64a740 1 -- 172.21.15.64:0/3224033430 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd235eca20 con 0x55fd23695bd0 2024-09-12T03:29:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.422+0000 7fce137fe640 1 --2- 172.21.15.64:0/3224033430 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55fd23695bd0 0x55fd23697fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.423+0000 7fce137fe640 1 --2- 172.21.15.64:0/3224033430 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55fd23695bd0 0x55fd23697fc0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec7afc93cad041d7 server_cookie=22db1e0db3b83f74 in_seq=0 out_seq=0 2024-09-12T03:29:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.423+0000 7fcdf37fe640 10 client.15372 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:29:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.423+0000 7fcdf37fe640 1 -- 172.21.15.64:0/3224033430 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fd235eca20 con 0x55fd23695bd0 2024-09-12T03:29:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.423+0000 7fcdf37fe640 10 client.15372 handle_command_reply: tid=0 2024-09-12T03:29:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.423+0000 7fce1d64a740 1 client.15372 shutdown 2024-09-12T03:29:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.423+0000 7fce1d64a740 1 -- 172.21.15.64:0/3224033430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd2367b410 msgr2=0x55fd23683070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.423+0000 7fce1d64a740 1 --2- 172.21.15.64:0/3224033430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd2367b410 0x55fd23683070 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fce04000c00 tx=0x7fce040316d0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.423+0000 7fce1d64a740 1 -- 172.21.15.64:0/3224033430 shutdown_connections 2024-09-12T03:29:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.423+0000 7fce1d64a740 1 --2- 172.21.15.64:0/3224033430 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55fd23695bd0 0x55fd23697fc0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.423+0000 7fce1d64a740 1 --2- 172.21.15.64:0/3224033430 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd23685cb0 0x55fd236835b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.423+0000 7fce1d64a740 1 --2- 172.21.15.64:0/3224033430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd2367b410 0x55fd23683070 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.423+0000 7fce1d64a740 1 --2- 172.21.15.64:0/3224033430 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd2367aaf0 0x55fd23682b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.424+0000 7fcdf37fe640 0 client.15372 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:29:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.424+0000 7fce1d64a740 1 -- 172.21.15.64:0/3224033430 >> 172.21.15.64:0/3224033430 conn(0x55fd23677060 msgr2=0x55fd2367cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.424+0000 7fce1d64a740 1 -- 172.21.15.64:0/3224033430 shutdown_connections 2024-09-12T03:29:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.424+0000 7fce1d64a740 1 -- 172.21.15.64:0/3224033430 wait complete. 2024-09-12T03:29:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.424+0000 7fce1d64a740 20 client.15372 trim_cache size 0 max 16384 2024-09-12T03:29:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.438+0000 7fce1d64a740 1 Processor -- start 2024-09-12T03:29:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.438+0000 7fce1d64a740 1 -- start start 2024-09-12T03:29:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.438+0000 7fce1d64a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd2367aaf0 0x55fd23684aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.438+0000 7fce1d64a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd2367b410 0x55fd236833e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.438+0000 7fce1d64a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd23685cb0 0x55fd2369d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.438+0000 7fce1d64a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fce141b1810 con 0x55fd2367aaf0 2024-09-12T03:29:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.438+0000 7fce1d64a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fce141af0e0 con 0x55fd2367b410 2024-09-12T03:29:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.438+0000 7fce1d64a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fce141acf30 con 0x55fd23685cb0 2024-09-12T03:29:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.439+0000 7fce137fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd2367aaf0 0x55fd23684aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.439+0000 7fce137fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd2367aaf0 0x55fd23684aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43968/0 (socket says 172.21.15.64:43968) 2024-09-12T03:29:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.439+0000 7fce137fe640 1 -- 172.21.15.64:0/2906556526 learned_addr learned my addr 172.21.15.64:0/2906556526 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.439+0000 7fce13fff640 1 --2- 172.21.15.64:0/2906556526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd2367b410 0x55fd236833e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.439+0000 7fce137fe640 1 -- 172.21.15.64:0/2906556526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd23685cb0 msgr2=0x55fd2369d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.439+0000 7fce137fe640 1 --2- 172.21.15.64:0/2906556526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd23685cb0 0x55fd2369d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.439+0000 7fce137fe640 1 -- 172.21.15.64:0/2906556526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd2367b410 msgr2=0x55fd236833e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.439+0000 7fce137fe640 1 --2- 172.21.15.64:0/2906556526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd2367b410 0x55fd236833e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.439+0000 7fce137fe640 1 -- 172.21.15.64:0/2906556526 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd2367d940 con 0x55fd2367aaf0 2024-09-12T03:29:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.439+0000 7fce13fff640 1 --2- 172.21.15.64:0/2906556526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd2367b410 0x55fd236833e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:29:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.439+0000 7fce137fe640 1 --2- 172.21.15.64:0/2906556526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd2367aaf0 0x55fd23684aa0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fce08009cb0 tx=0x7fce080096f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=704ec90047a31717 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.439+0000 7fcdf9ffb640 1 -- 172.21.15.64:0/2906556526 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce080156a0 con 0x55fd2367aaf0 2024-09-12T03:29:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.439+0000 7fcdf9ffb640 1 -- 172.21.15.64:0/2906556526 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce0801a040 con 0x55fd2367aaf0 2024-09-12T03:29:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.439+0000 7fcdf9ffb640 1 -- 172.21.15.64:0/2906556526 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0802ec40 con 0x55fd2367aaf0 2024-09-12T03:29:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.440+0000 7fce1d64a740 1 -- 172.21.15.64:0/2906556526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd2367aaf0 msgr2=0x55fd23684aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.440+0000 7fce1d64a740 1 --2- 172.21.15.64:0/2906556526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd2367aaf0 0x55fd23684aa0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fce08009cb0 tx=0x7fce080096f0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.440+0000 7fce1d64a740 1 -- 172.21.15.64:0/2906556526 shutdown_connections 2024-09-12T03:29:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.440+0000 7fce1d64a740 1 --2- 172.21.15.64:0/2906556526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd23685cb0 0x55fd2369d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.440+0000 7fce1d64a740 1 --2- 172.21.15.64:0/2906556526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd2367b410 0x55fd236833e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.440+0000 7fce1d64a740 1 --2- 172.21.15.64:0/2906556526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd2367aaf0 0x55fd23684aa0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.440+0000 7fce1d64a740 1 -- 172.21.15.64:0/2906556526 >> 172.21.15.64:0/2906556526 conn(0x55fd23677060 msgr2=0x55fd23684390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.440+0000 7fce1d64a740 1 -- 172.21.15.64:0/2906556526 shutdown_connections 2024-09-12T03:29:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.440+0000 7fce1d64a740 1 -- 172.21.15.64:0/2906556526 wait complete. 2024-09-12T03:29:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.440+0000 7fce1d64a740 1 -- start start 2024-09-12T03:29:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.440+0000 7fce1d64a740 1 Processor -- start 2024-09-12T03:29:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.441+0000 7fce1d64a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd2367b410 0x55fd2368e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.441+0000 7fce1d64a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd23685cb0 0x55fd23689560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.441+0000 7fce1d64a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd23689aa0 0x55fd23689f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.441+0000 7fce1d64a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fce141ad5a0 con 0x55fd23689aa0 2024-09-12T03:29:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.441+0000 7fce1d64a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fce141aea10 con 0x55fd2367b410 2024-09-12T03:29:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.441+0000 7fce1d64a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fce141adc70 con 0x55fd23685cb0 2024-09-12T03:29:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.441+0000 7fce1d64a740 10 client.0 _ll_register_callbacks cb 0x7fce04032b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.441+0000 7fce137fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd23689aa0 0x55fd23689f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.441+0000 7fce137fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd23689aa0 0x55fd23689f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43984/0 (socket says 172.21.15.64:43984) 2024-09-12T03:29:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.441+0000 7fce137fe640 1 -- 172.21.15.64:0/2135990160 learned_addr learned my addr 172.21.15.64:0/2135990160 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.441+0000 7fce137fe640 1 --2- 172.21.15.64:0/2135990160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd2367b410 0x55fd2368e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.441+0000 7fce13fff640 1 --2- 172.21.15.64:0/2135990160 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd23685cb0 0x55fd23689560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.441+0000 7fce137fe640 1 -- 172.21.15.64:0/2135990160 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd23685cb0 msgr2=0x55fd23689560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.441+0000 7fce137fe640 1 --2- 172.21.15.64:0/2135990160 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd23685cb0 0x55fd23689560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.441+0000 7fce137fe640 1 -- 172.21.15.64:0/2135990160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd2367b410 msgr2=0x55fd2368e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.441+0000 7fce137fe640 1 --2- 172.21.15.64:0/2135990160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd2367b410 0x55fd2368e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.441+0000 7fce137fe640 1 -- 172.21.15.64:0/2135990160 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd236ff880 con 0x55fd23689aa0 2024-09-12T03:29:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.441+0000 7fce137fe640 1 --2- 172.21.15.64:0/2135990160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd23689aa0 0x55fd23689f20 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fce0802e920 tx=0x7fce0802e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b762d7f297164ce5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.441+0000 7fcdf37fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:29:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.441+0000 7fcdf37fe640 1 -- 172.21.15.64:0/2135990160 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0803f060 con 0x55fd23689aa0 2024-09-12T03:29:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.441+0000 7fcdf37fe640 1 -- 172.21.15.64:0/2135990160 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7fce0802e9c0 con 0x55fd23689aa0 2024-09-12T03:29:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.441+0000 7fce1d64a740 1 -- 172.21.15.64:0/2135990160 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce0801a040 con 0x55fd23689aa0 2024-09-12T03:29:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.442+0000 7fcdf37fe640 1 -- 172.21.15.64:0/2135990160 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fce0802ebc0 con 0x55fd23689aa0 2024-09-12T03:29:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.442+0000 7fce1d64a740 10 client.15378 fetch_fsmap learned FSMap version 25 2024-09-12T03:29:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.442+0000 7fce1d64a740 1 -- 172.21.15.64:0/2135990160 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fd23700aa0 con 0x55fd23689aa0 2024-09-12T03:29:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.442+0000 7fcdf37fe640 1 -- 172.21.15.64:0/2135990160 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce0801a040 con 0x55fd23689aa0 2024-09-12T03:29:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.442+0000 7fcdf37fe640 1 -- 172.21.15.64:0/2135990160 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fce08039070 con 0x55fd23689aa0 2024-09-12T03:29:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.442+0000 7fce1d64a740 10 client.15378 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:29:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.442+0000 7fce1d64a740 10 client.15378 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:29:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.442+0000 7fce1d64a740 20 client.15378 populate_metadata read hostname 'smithi064' 2024-09-12T03:29:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.442+0000 7fce1d64a740 1 --2- 172.21.15.64:0/2135990160 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55fd23700cc0 0x55fd237030b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.442+0000 7fce1d64a740 4 client.15378 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:29:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.442+0000 7fce1d64a740 1 -- 172.21.15.64:0/2135990160 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fd235e6d70 con 0x55fd23700cc0 2024-09-12T03:29:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.442+0000 7fce13fff640 1 --2- 172.21.15.64:0/2135990160 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55fd23700cc0 0x55fd237030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.443+0000 7fcdf37fe640 10 client.15378 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:29:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.443+0000 7fce13fff640 1 --2- 172.21.15.64:0/2135990160 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55fd23700cc0 0x55fd237030b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46ee6fae8413e34c server_cookie=aa2132b82a919a2 in_seq=0 out_seq=0 2024-09-12T03:29:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.450+0000 7fcdf37fe640 1 -- 172.21.15.64:0/2135990160 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fd235e6d70 con 0x55fd23700cc0 2024-09-12T03:29:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.450+0000 7fcdf37fe640 10 client.15378 handle_command_reply: tid=0 2024-09-12T03:29:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.450+0000 7fce1d64a740 1 client.15378 shutdown 2024-09-12T03:29:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.450+0000 7fce1d64a740 1 -- 172.21.15.64:0/2135990160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd23689aa0 msgr2=0x55fd23689f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.450+0000 7fce1d64a740 1 --2- 172.21.15.64:0/2135990160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd23689aa0 0x55fd23689f20 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fce0802e920 tx=0x7fce0802e820 comp rx=0 tx=0).stop 2024-09-12T03:29:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.450+0000 7fce1d64a740 1 -- 172.21.15.64:0/2135990160 shutdown_connections 2024-09-12T03:29:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.450+0000 7fce1d64a740 1 --2- 172.21.15.64:0/2135990160 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55fd23700cc0 0x55fd237030b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.450+0000 7fce1d64a740 1 --2- 172.21.15.64:0/2135990160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd23689aa0 0x55fd23689f20 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.450+0000 7fce1d64a740 1 --2- 172.21.15.64:0/2135990160 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd23685cb0 0x55fd23689560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.450+0000 7fcdf37fe640 0 client.15378 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:29:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.450+0000 7fce1d64a740 1 --2- 172.21.15.64:0/2135990160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd2367b410 0x55fd2368e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.450+0000 7fce1d64a740 1 -- 172.21.15.64:0/2135990160 >> 172.21.15.64:0/2135990160 conn(0x55fd23677060 msgr2=0x55fd236761c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.450+0000 7fce1d64a740 1 -- 172.21.15.64:0/2135990160 shutdown_connections 2024-09-12T03:29:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.450+0000 7fce1d64a740 1 -- 172.21.15.64:0/2135990160 wait complete. 2024-09-12T03:29:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.451+0000 7fce1d64a740 20 client.15378 trim_cache size 0 max 16384 2024-09-12T03:29:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:29:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:29:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:29:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:29:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.451+0000 7fce1a8ac640 1 -- 172.21.15.64:0/267962051 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fcdfc0a2e40 msgr2=0x7fcdfc0a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.451+0000 7fce1a8ac640 1 --2- 172.21.15.64:0/267962051 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fcdfc0a2e40 0x7fcdfc0a5300 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fce0c0ccf00 tx=0x7fce0c091050 comp rx=0 tx=0).stop 2024-09-12T03:29:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.451+0000 7fce1a8ac640 1 -- 172.21.15.64:0/267962051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce1410c0d0 msgr2=0x7fce1419cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.451+0000 7fce1a8ac640 1 --2- 172.21.15.64:0/267962051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce1410c0d0 0x7fce1419cc10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fce0c00ca30 tx=0x7fce0c00ce30 comp rx=0 tx=0).stop 2024-09-12T03:29:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.451+0000 7fce1a8ac640 1 -- 172.21.15.64:0/267962051 shutdown_connections 2024-09-12T03:29:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.451+0000 7fce1a8ac640 1 --2- 172.21.15.64:0/267962051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce1410c0d0 0x7fce1419cc10 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.451+0000 7fce1a8ac640 1 --2- 172.21.15.64:0/267962051 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fcdfc0a2e40 0x7fcdfc0a5300 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.451+0000 7fce1a8ac640 1 --2- 172.21.15.64:0/267962051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce14103620 0x7fce14198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.451+0000 7fce1a8ac640 1 --2- 172.21.15.64:0/267962051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce14102d00 0x7fce14198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.451+0000 7fce1a8ac640 1 -- 172.21.15.64:0/267962051 >> 172.21.15.64:0/267962051 conn(0x7fce14077990 msgr2=0x7fce140fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.451+0000 7fce1a8ac640 1 -- 172.21.15.64:0/267962051 shutdown_connections 2024-09-12T03:29:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:44.451+0000 7fce1a8ac640 1 -- 172.21.15.64:0/267962051 wait complete. 2024-09-12T03:29:45.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:44 smithi115 ceph-mon[34801]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1023 B/s wr, 9 op/s 2024-09-12T03:29:45.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:44 smithi195 ceph-mon[34786]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1023 B/s wr, 9 op/s 2024-09-12T03:29:45.271 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:44 smithi064 ceph-mon[30952]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1023 B/s wr, 9 op/s 2024-09-12T03:29:47.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:46 smithi115 ceph-mon[34801]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:29:47.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:46 smithi195 ceph-mon[34786]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:29:47.271 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:46 smithi064 ceph-mon[30952]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:29:49.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:48 smithi115 ceph-mon[34801]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-12T03:29:49.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:48 smithi195 ceph-mon[34786]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-12T03:29:49.271 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:48 smithi064 ceph-mon[30952]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-12T03:29:51.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:50 smithi115 ceph-mon[34801]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-12T03:29:51.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:50 smithi195 ceph-mon[34786]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-12T03:29:51.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:50 smithi064 ceph-mon[30952]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-12T03:29:53.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:52 smithi115 ceph-mon[34801]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-12T03:29:53.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:52 smithi195 ceph-mon[34786]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-12T03:29:53.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:52 smithi064 ceph-mon[30952]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-12T03:29:55.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:54 smithi115 ceph-mon[34801]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-12T03:29:55.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:54 smithi195 ceph-mon[34786]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-12T03:29:55.271 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:54 smithi064 ceph-mon[30952]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-12T03:29:57.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:56 smithi115 ceph-mon[34801]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-12T03:29:57.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:56 smithi195 ceph-mon[34786]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-12T03:29:57.271 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:56 smithi064 ceph-mon[30952]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-12T03:29:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.578+0000 7f80e8847640 1 -- 172.21.15.64:0/1259635256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80e410c0d0 msgr2=0x7f80e410e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.578+0000 7f80e8847640 1 --2- 172.21.15.64:0/1259635256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80e410c0d0 0x7f80e410e550 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f80dc009ca0 tx=0x7f80dc02a950 comp rx=0 tx=0).stop 2024-09-12T03:29:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.578+0000 7f80e8847640 1 -- 172.21.15.64:0/1259635256 shutdown_connections 2024-09-12T03:29:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.578+0000 7f80e8847640 1 --2- 172.21.15.64:0/1259635256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80e410c0d0 0x7f80e410e550 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.578+0000 7f80e8847640 1 --2- 172.21.15.64:0/1259635256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80e41035a0 0x7f80e410ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.578+0000 7f80e8847640 1 --2- 172.21.15.64:0/1259635256 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80e4102c80 0x7f80e4103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.578+0000 7f80e8847640 1 -- 172.21.15.64:0/1259635256 >> 172.21.15.64:0/1259635256 conn(0x7f80e4077990 msgr2=0x7f80e40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.578+0000 7f80e8847640 1 -- 172.21.15.64:0/1259635256 shutdown_connections 2024-09-12T03:29:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.578+0000 7f80e8847640 1 -- 172.21.15.64:0/1259635256 wait complete. 2024-09-12T03:29:57.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.579+0000 7f80e8847640 1 Processor -- start 2024-09-12T03:29:57.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.579+0000 7f80e8847640 1 -- start start 2024-09-12T03:29:57.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.579+0000 7f80e8847640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80e4102c80 0x7f80e4198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.579+0000 7f80e8847640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80e41035a0 0x7f80e4198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.579+0000 7f80e8847640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80e410c0d0 0x7f80e419cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.579+0000 7f80e8847640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f80e4111d20 con 0x7f80e4102c80 2024-09-12T03:29:57.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.579+0000 7f80e8847640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f80e4111ba0 con 0x7f80e41035a0 2024-09-12T03:29:57.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.579+0000 7f80e8847640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f80e4111ea0 con 0x7f80e410c0d0 2024-09-12T03:29:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.579+0000 7f80e259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80e4102c80 0x7f80e4198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.579+0000 7f80e259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80e4102c80 0x7f80e4198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45952/0 (socket says 172.21.15.64:45952) 2024-09-12T03:29:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.579+0000 7f80e259c640 1 -- 172.21.15.64:0/1412303926 learned_addr learned my addr 172.21.15.64:0/1412303926 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.579+0000 7f80dbfff640 1 --2- 172.21.15.64:0/1412303926 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80e41035a0 0x7f80e4198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.580+0000 7f80e259c640 1 -- 172.21.15.64:0/1412303926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80e410c0d0 msgr2=0x7f80e419cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.580+0000 7f80e259c640 1 --2- 172.21.15.64:0/1412303926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80e410c0d0 0x7f80e419cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.580+0000 7f80e259c640 1 -- 172.21.15.64:0/1412303926 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80e41035a0 msgr2=0x7f80e4198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.580+0000 7f80e259c640 1 --2- 172.21.15.64:0/1412303926 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80e41035a0 0x7f80e4198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.580+0000 7f80e259c640 1 -- 172.21.15.64:0/1412303926 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e419d240 con 0x7f80e4102c80 2024-09-12T03:29:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.580+0000 7f80e259c640 1 --2- 172.21.15.64:0/1412303926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80e4102c80 0x7f80e4198250 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f80d400d950 tx=0x7f80d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.580+0000 7f80db7fe640 1 -- 172.21.15.64:0/1412303926 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d4017070 con 0x7f80e4102c80 2024-09-12T03:29:57.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.580+0000 7f80db7fe640 1 -- 172.21.15.64:0/1412303926 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80d4009c30 con 0x7f80e4102c80 2024-09-12T03:29:57.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.580+0000 7f80e8847640 1 -- 172.21.15.64:0/1412303926 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80e419d530 con 0x7f80e4102c80 2024-09-12T03:29:57.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.580+0000 7f80e8847640 1 -- 172.21.15.64:0/1412303926 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80e41a4dc0 con 0x7f80e4102c80 2024-09-12T03:29:57.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.580+0000 7f80db7fe640 1 -- 172.21.15.64:0/1412303926 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d400f620 con 0x7f80e4102c80 2024-09-12T03:29:57.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.583+0000 7f80eb5e0740 1 Processor -- start 2024-09-12T03:29:57.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.583+0000 7f80eb5e0740 1 -- start start 2024-09-12T03:29:57.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.583+0000 7f80db7fe640 1 -- 172.21.15.64:0/1412303926 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f80d40109a0 con 0x7f80e4102c80 2024-09-12T03:29:57.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.583+0000 7f80eb5e0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556328b519d0 0x556328b51db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.583+0000 7f80eb5e0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556328b522f0 0x556328b60b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.583+0000 7f80eb5e0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556328b610c0 0x556328b634b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.583+0000 7f80eb5e0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556328ab62f0 con 0x556328b522f0 2024-09-12T03:29:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.583+0000 7f80eb5e0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556328a5e270 con 0x556328b610c0 2024-09-12T03:29:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.583+0000 7f80eb5e0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556328ab22b0 con 0x556328b519d0 2024-09-12T03:29:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.584+0000 7f80e259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556328b522f0 0x556328b60b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.584+0000 7f80dbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556328b610c0 0x556328b634b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.584+0000 7f80dbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556328b610c0 0x556328b634b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.64:36778/0 (socket says 172.21.15.64:36778) 2024-09-12T03:29:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.584+0000 7f80e259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556328b522f0 0x556328b60b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45966/0 (socket says 172.21.15.64:45966) 2024-09-12T03:29:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.584+0000 7f80dbfff640 1 -- 172.21.15.64:0/118715206 learned_addr learned my addr 172.21.15.64:0/118715206 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.584+0000 7f80e2d9d640 1 --2- 172.21.15.64:0/118715206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556328b519d0 0x556328b51db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.584+0000 7f80dbfff640 1 -- 172.21.15.64:0/118715206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556328b519d0 msgr2=0x556328b51db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.584+0000 7f80dbfff640 1 --2- 172.21.15.64:0/118715206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556328b519d0 0x556328b51db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.584+0000 7f80dbfff640 1 -- 172.21.15.64:0/118715206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556328b522f0 msgr2=0x556328b60b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.584+0000 7f80dbfff640 1 --2- 172.21.15.64:0/118715206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556328b522f0 0x556328b60b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.584+0000 7f80dbfff640 1 -- 172.21.15.64:0/118715206 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556328a5d340 con 0x556328b610c0 2024-09-12T03:29:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.584+0000 7f80dbfff640 1 --2- 172.21.15.64:0/118715206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556328b610c0 0x556328b634b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f80cc00bc50 tx=0x7f80cc0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=599330f7369c3086 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.584+0000 7f80e2d9d640 1 --2- 172.21.15.64:0/118715206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556328b519d0 0x556328b51db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:29:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.585+0000 7f80db7fe640 1 --2- 172.21.15.64:0/1412303926 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f80d00a2ec0 0x7f80d00a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.585+0000 7f80db7fe640 1 -- 172.21.15.64:0/1412303926 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f80d40cf4e0 con 0x7f80e4102c80 2024-09-12T03:29:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.585+0000 7f80e2d9d640 1 --2- 172.21.15.64:0/1412303926 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f80d00a2ec0 0x7f80d00a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.586+0000 7f80d8ff9640 1 -- 172.21.15.64:0/118715206 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80cc01c480 con 0x556328b610c0 2024-09-12T03:29:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.586+0000 7f80d8ff9640 1 -- 172.21.15.64:0/118715206 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80cc01cad0 con 0x556328b610c0 2024-09-12T03:29:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.586+0000 7f80d8ff9640 1 -- 172.21.15.64:0/118715206 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80cc011b50 con 0x556328b610c0 2024-09-12T03:29:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.587+0000 7f80eb5e0740 1 -- 172.21.15.64:0/118715206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556328b610c0 msgr2=0x556328b634b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.587+0000 7f80e2d9d640 1 --2- 172.21.15.64:0/1412303926 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f80d00a2ec0 0x7f80d00a5380 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f80dc004af0 tx=0x7f80dc004a40 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.587+0000 7f80eb5e0740 1 --2- 172.21.15.64:0/118715206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556328b610c0 0x556328b634b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f80cc00bc50 tx=0x7f80cc0107a0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.587+0000 7f80eb5e0740 1 -- 172.21.15.64:0/118715206 shutdown_connections 2024-09-12T03:29:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.587+0000 7f80eb5e0740 1 --2- 172.21.15.64:0/118715206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556328b610c0 0x556328b634b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.587+0000 7f80eb5e0740 1 --2- 172.21.15.64:0/118715206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556328b522f0 0x556328b60b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.587+0000 7f80eb5e0740 1 --2- 172.21.15.64:0/118715206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556328b519d0 0x556328b51db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.587+0000 7f80eb5e0740 1 -- 172.21.15.64:0/118715206 >> 172.21.15.64:0/118715206 conn(0x556328b4e060 msgr2=0x556328b4e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.587+0000 7f80eb5e0740 1 -- 172.21.15.64:0/118715206 shutdown_connections 2024-09-12T03:29:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.587+0000 7f80eb5e0740 1 -- 172.21.15.64:0/118715206 wait complete. 2024-09-12T03:29:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.587+0000 7f80eb5e0740 1 -- start start 2024-09-12T03:29:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.588+0000 7f80eb5e0740 1 Processor -- start 2024-09-12T03:29:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.588+0000 7f80eb5e0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556328b519d0 0x556328b5e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.588+0000 7f80eb5e0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556328b522f0 0x556328b5e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.588+0000 7f80eb5e0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556328b610c0 0x556328b5eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.588+0000 7f80e2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556328b610c0 0x556328b5eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.588+0000 7f80e2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556328b610c0 0x556328b5eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45974/0 (socket says 172.21.15.64:45974) 2024-09-12T03:29:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.588+0000 7f80e259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556328b519d0 0x556328b5e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.588+0000 7f80e259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556328b519d0 0x556328b5e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.64:36794/0 (socket says 172.21.15.64:36794) 2024-09-12T03:29:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.588+0000 7f80e2d9d640 1 -- 172.21.15.64:0/4081518665 learned_addr learned my addr 172.21.15.64:0/4081518665 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.588+0000 7f80dbfff640 1 --2- 172.21.15.64:0/4081518665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556328b522f0 0x556328b5e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.588+0000 7f80eb5e0740 1 -- 172.21.15.64:0/4081518665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556328b63cf0 con 0x556328b610c0 2024-09-12T03:29:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.588+0000 7f80eb5e0740 1 -- 172.21.15.64:0/4081518665 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556328a3f290 con 0x556328b519d0 2024-09-12T03:29:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.589+0000 7f80eb5e0740 1 -- 172.21.15.64:0/4081518665 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556328b63e70 con 0x556328b522f0 2024-09-12T03:29:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.589+0000 7f80e2d9d640 1 -- 172.21.15.64:0/4081518665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556328b522f0 msgr2=0x556328b5e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.589+0000 7f80e2d9d640 1 --2- 172.21.15.64:0/4081518665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556328b522f0 0x556328b5e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.589+0000 7f80e2d9d640 1 -- 172.21.15.64:0/4081518665 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556328b519d0 msgr2=0x556328b5e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.589+0000 7f80e2d9d640 1 --2- 172.21.15.64:0/4081518665 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556328b519d0 0x556328b5e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.589+0000 7f80e2d9d640 1 -- 172.21.15.64:0/4081518665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563287c9210 con 0x556328b610c0 2024-09-12T03:29:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.589+0000 7f80e2d9d640 1 --2- 172.21.15.64:0/4081518665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556328b610c0 0x556328b5eb20 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f80dc02f650 tx=0x7f80dc0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55d1a735972c1497 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.589+0000 7f80c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:29:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.589+0000 7f80eb5e0740 10 client.0 _ll_register_callbacks cb 0x556328a70140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.589+0000 7f80eb5e0740 1 -- 172.21.15.64:0/4081518665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80cc01cad0 con 0x556328b610c0 2024-09-12T03:29:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.589+0000 7f80c9ffb640 1 -- 172.21.15.64:0/4081518665 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80dc034070 con 0x556328b610c0 2024-09-12T03:29:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.590+0000 7f80c9ffb640 1 -- 172.21.15.64:0/4081518665 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f80dc004450 con 0x556328b610c0 2024-09-12T03:29:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.590+0000 7f80c9ffb640 1 -- 172.21.15.64:0/4081518665 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f80dc02f680 con 0x556328b610c0 2024-09-12T03:29:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.590+0000 7f80eb5e0740 10 client.15390 fetch_fsmap learned FSMap version 25 2024-09-12T03:29:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.590+0000 7f80eb5e0740 1 -- 172.21.15.64:0/4081518665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({fsmap=25}) -- 0x556328895620 con 0x556328b610c0 2024-09-12T03:29:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.590+0000 7f80c9ffb640 1 -- 172.21.15.64:0/4081518665 <== mon.0 v2:172.21.15.64:3300/0 4 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f80dc04c070 con 0x556328b610c0 2024-09-12T03:29:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.590+0000 7f80eb5e0740 10 client.15390 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:29:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.590+0000 7f80eb5e0740 10 client.15390 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:29:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.590+0000 7f80eb5e0740 20 client.15390 populate_metadata read hostname 'smithi064' 2024-09-12T03:29:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.590+0000 7f80eb5e0740 1 --2- 172.21.15.64:0/4081518665 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x556328b70ec0 0x556328b73350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.590+0000 7f80eb5e0740 4 client.15390 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:29:57.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.590+0000 7f80eb5e0740 1 -- 172.21.15.64:0/4081518665 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f80cc01cad0 con 0x556328b70ec0 2024-09-12T03:29:57.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.591+0000 7f80e259c640 1 --2- 172.21.15.64:0/4081518665 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x556328b70ec0 0x556328b73350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.591+0000 7f80c9ffb640 10 client.15390 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:29:57.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.591+0000 7f80e259c640 1 --2- 172.21.15.64:0/4081518665 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x556328b70ec0 0x556328b73350 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3516c89e013f916d server_cookie=24c0a0bf31f0dd35 in_seq=0 out_seq=0 2024-09-12T03:29:57.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.592+0000 7f80c9ffb640 1 -- 172.21.15.64:0/4081518665 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f80cc01cad0 con 0x556328b70ec0 2024-09-12T03:29:57.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.592+0000 7f80c9ffb640 10 client.15390 handle_command_reply: tid=0 2024-09-12T03:29:57.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.592+0000 7f80eb5e0740 1 client.15390 shutdown 2024-09-12T03:29:57.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.592+0000 7f80eb5e0740 1 -- 172.21.15.64:0/4081518665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556328b610c0 msgr2=0x556328b5eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:57.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.592+0000 7f80eb5e0740 1 --2- 172.21.15.64:0/4081518665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556328b610c0 0x556328b5eb20 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f80dc02f650 tx=0x7f80dc0095c0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.592+0000 7f80eb5e0740 1 -- 172.21.15.64:0/4081518665 shutdown_connections 2024-09-12T03:29:57.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.592+0000 7f80eb5e0740 1 --2- 172.21.15.64:0/4081518665 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x556328b70ec0 0x556328b73350 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.592+0000 7f80eb5e0740 1 --2- 172.21.15.64:0/4081518665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556328b610c0 0x556328b5eb20 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.592+0000 7f80eb5e0740 1 --2- 172.21.15.64:0/4081518665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556328b522f0 0x556328b5e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.592+0000 7f80c9ffb640 0 client.15390 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:29:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.592+0000 7f80eb5e0740 1 --2- 172.21.15.64:0/4081518665 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556328b519d0 0x556328b5e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.592+0000 7f80eb5e0740 1 -- 172.21.15.64:0/4081518665 >> 172.21.15.64:0/4081518665 conn(0x556328b4e060 msgr2=0x556328b53d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.592+0000 7f80eb5e0740 1 -- 172.21.15.64:0/4081518665 shutdown_connections 2024-09-12T03:29:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.592+0000 7f80eb5e0740 1 -- 172.21.15.64:0/4081518665 wait complete. 2024-09-12T03:29:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.592+0000 7f80eb5e0740 20 client.15390 trim_cache size 0 max 16384 2024-09-12T03:29:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.607+0000 7f80eb5e0740 1 Processor -- start 2024-09-12T03:29:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.607+0000 7f80eb5e0740 1 -- start start 2024-09-12T03:29:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.607+0000 7f80eb5e0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556328b519d0 0x556328b6d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.607+0000 7f80eb5e0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556328b522f0 0x556328b78c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.607+0000 7f80eb5e0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556328b610c0 0x556328b791a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.607+0000 7f80eb5e0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f80e41af630 con 0x556328b610c0 2024-09-12T03:29:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.607+0000 7f80eb5e0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f80e41acf00 con 0x556328b519d0 2024-09-12T03:29:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.607+0000 7f80eb5e0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f80e41aada0 con 0x556328b522f0 2024-09-12T03:29:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.607+0000 7f80e2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556328b610c0 0x556328b791a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.607+0000 7f80e2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556328b610c0 0x556328b791a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45976/0 (socket says 172.21.15.64:45976) 2024-09-12T03:29:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.607+0000 7f80e2d9d640 1 -- 172.21.15.64:0/1814844928 learned_addr learned my addr 172.21.15.64:0/1814844928 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.607+0000 7f80e259c640 1 --2- 172.21.15.64:0/1814844928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556328b519d0 0x556328b6d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.607+0000 7f80dbfff640 1 --2- 172.21.15.64:0/1814844928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556328b522f0 0x556328b78c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.607+0000 7f80e2d9d640 1 -- 172.21.15.64:0/1814844928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556328b522f0 msgr2=0x556328b78c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.607+0000 7f80e2d9d640 1 --2- 172.21.15.64:0/1814844928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556328b522f0 0x556328b78c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.607+0000 7f80e2d9d640 1 -- 172.21.15.64:0/1814844928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556328b519d0 msgr2=0x556328b6d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.607+0000 7f80e2d9d640 1 --2- 172.21.15.64:0/1814844928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556328b519d0 0x556328b6d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.607+0000 7f80e2d9d640 1 -- 172.21.15.64:0/1814844928 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556328b6cf90 con 0x556328b610c0 2024-09-12T03:29:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.607+0000 7f80e259c640 1 --2- 172.21.15.64:0/1814844928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556328b519d0 0x556328b6d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:29:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.608+0000 7f80e2d9d640 1 --2- 172.21.15.64:0/1814844928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556328b610c0 0x556328b791a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f80dc02fe20 tx=0x7f80dc048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4475b6e006c6d7f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.608+0000 7f80cbfff640 1 -- 172.21.15.64:0/1814844928 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80dc05f070 con 0x556328b610c0 2024-09-12T03:29:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.608+0000 7f80cbfff640 1 -- 172.21.15.64:0/1814844928 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80dc043b60 con 0x556328b610c0 2024-09-12T03:29:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.608+0000 7f80cbfff640 1 -- 172.21.15.64:0/1814844928 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80dc04e900 con 0x556328b610c0 2024-09-12T03:29:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.609+0000 7f80eb5e0740 1 -- 172.21.15.64:0/1814844928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556328b610c0 msgr2=0x556328b791a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.609+0000 7f80eb5e0740 1 --2- 172.21.15.64:0/1814844928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556328b610c0 0x556328b791a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f80dc02fe20 tx=0x7f80dc048000 comp rx=0 tx=0).stop 2024-09-12T03:29:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.609+0000 7f80eb5e0740 1 -- 172.21.15.64:0/1814844928 shutdown_connections 2024-09-12T03:29:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.609+0000 7f80eb5e0740 1 --2- 172.21.15.64:0/1814844928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556328b610c0 0x556328b791a0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.609+0000 7f80eb5e0740 1 --2- 172.21.15.64:0/1814844928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556328b522f0 0x556328b78c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.609+0000 7f80eb5e0740 1 --2- 172.21.15.64:0/1814844928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556328b519d0 0x556328b6d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.609+0000 7f80eb5e0740 1 -- 172.21.15.64:0/1814844928 >> 172.21.15.64:0/1814844928 conn(0x556328b4e060 msgr2=0x556328b60120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.609+0000 7f80eb5e0740 1 -- 172.21.15.64:0/1814844928 shutdown_connections 2024-09-12T03:29:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.609+0000 7f80eb5e0740 1 -- 172.21.15.64:0/1814844928 wait complete. 2024-09-12T03:29:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.610+0000 7f80eb5e0740 1 -- start start 2024-09-12T03:29:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.610+0000 7f80eb5e0740 1 Processor -- start 2024-09-12T03:29:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.610+0000 7f80eb5e0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556328b519d0 0x556328b65230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.610+0000 7f80eb5e0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556328b522f0 0x556328b65770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.610+0000 7f80eb5e0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556328b610c0 0x556328bdac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.610+0000 7f80eb5e0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f80e41ab3c0 con 0x556328b522f0 2024-09-12T03:29:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.610+0000 7f80eb5e0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f80e41ac830 con 0x556328b519d0 2024-09-12T03:29:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.610+0000 7f80eb5e0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f80e41aba90 con 0x556328b610c0 2024-09-12T03:29:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.610+0000 7f80eb5e0740 10 client.0 _ll_register_callbacks cb 0x7f80dc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.610+0000 7f80dbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556328b522f0 0x556328b65770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.611+0000 7f80e2d9d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556328b610c0 0x556328bdac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.611+0000 7f80dbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556328b522f0 0x556328b65770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45992/0 (socket says 172.21.15.64:45992) 2024-09-12T03:29:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.611+0000 7f80dbfff640 1 -- 172.21.15.64:0/523563689 learned_addr learned my addr 172.21.15.64:0/523563689 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:29:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.611+0000 7f80e259c640 1 --2- 172.21.15.64:0/523563689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556328b519d0 0x556328b65230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.611+0000 7f80e2d9d640 1 -- 172.21.15.64:0/523563689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556328b519d0 msgr2=0x556328b65230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.611+0000 7f80e2d9d640 1 --2- 172.21.15.64:0/523563689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556328b519d0 0x556328b65230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.611+0000 7f80e2d9d640 1 -- 172.21.15.64:0/523563689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556328b522f0 msgr2=0x556328b65770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.611+0000 7f80e2d9d640 1 --2- 172.21.15.64:0/523563689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556328b522f0 0x556328b65770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.611+0000 7f80e2d9d640 1 -- 172.21.15.64:0/523563689 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556328b66070 con 0x556328b610c0 2024-09-12T03:29:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.611+0000 7f80dbfff640 1 --2- 172.21.15.64:0/523563689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556328b522f0 0x556328b65770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:29:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.612+0000 7f80e2d9d640 1 --2- 172.21.15.64:0/523563689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556328b610c0 0x556328bdac30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f80dc03f040 tx=0x7f80dc043a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c983a3039c7b48b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.612+0000 7f80c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-09-12T03:29:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.612+0000 7f80c9ffb640 1 -- 172.21.15.64:0/523563689 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80dc05f070 con 0x556328b610c0 2024-09-12T03:29:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.612+0000 7f80eb5e0740 1 -- 172.21.15.64:0/523563689 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80dc043b60 con 0x556328b610c0 2024-09-12T03:29:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.612+0000 7f80c9ffb640 1 -- 172.21.15.64:0/523563689 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f80dc04e780 con 0x556328b610c0 2024-09-12T03:29:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.612+0000 7f80c9ffb640 1 -- 172.21.15.64:0/523563689 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f80dc06b050 con 0x556328b610c0 2024-09-12T03:29:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.612+0000 7f80eb5e0740 10 client.25066 fetch_fsmap learned FSMap version 25 2024-09-12T03:29:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.612+0000 7f80eb5e0740 1 -- 172.21.15.64:0/523563689 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556328bdc2b0 con 0x556328b610c0 2024-09-12T03:29:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.612+0000 7f80c9ffb640 1 -- 172.21.15.64:0/523563689 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80dc043b60 con 0x556328b610c0 2024-09-12T03:29:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.612+0000 7f80c9ffb640 1 -- 172.21.15.64:0/523563689 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f80dc04c070 con 0x556328b610c0 2024-09-12T03:29:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.612+0000 7f80eb5e0740 10 client.25066 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:29:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.612+0000 7f80eb5e0740 10 client.25066 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:29:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.613+0000 7f80eb5e0740 20 client.25066 populate_metadata read hostname 'smithi064' 2024-09-12T03:29:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.613+0000 7f80eb5e0740 1 --2- 172.21.15.64:0/523563689 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x556328bdc910 0x556328bded00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.613+0000 7f80eb5e0740 4 client.25066 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:29:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.613+0000 7f80eb5e0740 1 -- 172.21.15.64:0/523563689 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556328ac3a20 con 0x556328bdc910 2024-09-12T03:29:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.613+0000 7f80e259c640 1 --2- 172.21.15.64:0/523563689 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x556328bdc910 0x556328bded00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.613+0000 7f80e259c640 1 --2- 172.21.15.64:0/523563689 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x556328bdc910 0x556328bded00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dedf27129d9f2ba server_cookie=984e78522e43671f in_seq=0 out_seq=0 2024-09-12T03:29:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.613+0000 7f80c9ffb640 10 client.25066 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:29:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.619+0000 7f80c9ffb640 1 -- 172.21.15.64:0/523563689 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556328ac3a20 con 0x556328bdc910 2024-09-12T03:29:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.619+0000 7f80c9ffb640 10 client.25066 handle_command_reply: tid=0 2024-09-12T03:29:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.619+0000 7f80eb5e0740 1 client.25066 shutdown 2024-09-12T03:29:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.619+0000 7f80eb5e0740 1 -- 172.21.15.64:0/523563689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556328b610c0 msgr2=0x556328bdac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.619+0000 7f80eb5e0740 1 --2- 172.21.15.64:0/523563689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556328b610c0 0x556328bdac30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f80dc03f040 tx=0x7f80dc043a50 comp rx=0 tx=0).stop 2024-09-12T03:29:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.619+0000 7f80eb5e0740 1 -- 172.21.15.64:0/523563689 shutdown_connections 2024-09-12T03:29:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.619+0000 7f80eb5e0740 1 --2- 172.21.15.64:0/523563689 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x556328bdc910 0x556328bded00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.619+0000 7f80eb5e0740 1 --2- 172.21.15.64:0/523563689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556328b610c0 0x556328bdac30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.619+0000 7f80eb5e0740 1 --2- 172.21.15.64:0/523563689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556328b522f0 0x556328b65770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.619+0000 7f80eb5e0740 1 --2- 172.21.15.64:0/523563689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556328b519d0 0x556328b65230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.619+0000 7f80eb5e0740 1 -- 172.21.15.64:0/523563689 >> 172.21.15.64:0/523563689 conn(0x556328b4e060 msgr2=0x556328b4c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:57.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.619+0000 7f80c9ffb640 0 client.25066 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:29:57.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.619+0000 7f80eb5e0740 1 -- 172.21.15.64:0/523563689 shutdown_connections 2024-09-12T03:29:57.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.619+0000 7f80eb5e0740 1 -- 172.21.15.64:0/523563689 wait complete. 2024-09-12T03:29:57.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.619+0000 7f80eb5e0740 20 client.25066 trim_cache size 0 max 16384 2024-09-12T03:29:57.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:29:57.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:29:57.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:29:57.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:29:57.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.620+0000 7f80e8847640 1 -- 172.21.15.64:0/1412303926 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f80d00a2ec0 msgr2=0x7f80d00a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:57.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.620+0000 7f80e8847640 1 --2- 172.21.15.64:0/1412303926 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f80d00a2ec0 0x7f80d00a5380 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f80dc004af0 tx=0x7f80dc004a40 comp rx=0 tx=0).stop 2024-09-12T03:29:57.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.620+0000 7f80e8847640 1 -- 172.21.15.64:0/1412303926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80e4102c80 msgr2=0x7f80e4198250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:57.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.620+0000 7f80e8847640 1 --2- 172.21.15.64:0/1412303926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80e4102c80 0x7f80e4198250 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f80d400d950 tx=0x7f80d400de10 comp rx=0 tx=0).stop 2024-09-12T03:29:57.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.620+0000 7f80e8847640 1 -- 172.21.15.64:0/1412303926 shutdown_connections 2024-09-12T03:29:57.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.620+0000 7f80e8847640 1 --2- 172.21.15.64:0/1412303926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80e410c0d0 0x7f80e419cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.620+0000 7f80e8847640 1 --2- 172.21.15.64:0/1412303926 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f80d00a2ec0 0x7f80d00a5380 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.620+0000 7f80e8847640 1 --2- 172.21.15.64:0/1412303926 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80e41035a0 0x7f80e4198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.620+0000 7f80e8847640 1 --2- 172.21.15.64:0/1412303926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80e4102c80 0x7f80e4198250 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.620+0000 7f80e8847640 1 -- 172.21.15.64:0/1412303926 >> 172.21.15.64:0/1412303926 conn(0x7f80e4077990 msgr2=0x7f80e40fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.620+0000 7f80e8847640 1 -- 172.21.15.64:0/1412303926 shutdown_connections 2024-09-12T03:29:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:29:57.620+0000 7f80e8847640 1 -- 172.21.15.64:0/1412303926 wait complete. 2024-09-12T03:29:59.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:29:58 smithi115 ceph-mon[34801]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-12T03:29:59.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:29:58 smithi195 ceph-mon[34786]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-12T03:29:59.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:29:58 smithi064 ceph-mon[30952]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-12T03:30:00.772 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:30:00 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:30:00.310+0000 7fa512bfa640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:30:00.772 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:30:00 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:30:00.310+0000 7fa512bfa640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:30:00.772 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:30:00 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:30:00.310+0000 7fa512bfa640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:30:00.772 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:30:00 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:30:00.310+0000 7fa512bfa640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:30:00.772 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:30:00 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-12T03:30:00.310+0000 7fa512bfa640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:30:01.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:30:00 smithi115 ceph-mon[34801]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:30:01.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:30:00 smithi115 ceph-mon[34801]: overall HEALTH_OK 2024-09-12T03:30:01.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:30:00 smithi195 ceph-mon[34786]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:30:01.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:30:00 smithi195 ceph-mon[34786]: overall HEALTH_OK 2024-09-12T03:30:01.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:30:00 smithi064 ceph-mon[30952]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:30:01.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:30:00 smithi064 ceph-mon[30952]: overall HEALTH_OK 2024-09-12T03:30:03.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:30:02 smithi115 ceph-mon[34801]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-12T03:30:03.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:30:02 smithi115 ceph-mon[34801]: mgrmap e25: x(active, since 46s), standbys: y, z 2024-09-12T03:30:03.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:30:02 smithi195 ceph-mon[34786]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-12T03:30:03.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:30:02 smithi195 ceph-mon[34786]: mgrmap e25: x(active, since 46s), standbys: y, z 2024-09-12T03:30:03.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:30:02 smithi064 ceph-mon[30952]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-12T03:30:03.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:30:02 smithi064 ceph-mon[30952]: mgrmap e25: x(active, since 46s), standbys: y, z 2024-09-12T03:30:05.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:30:04 smithi115 ceph-mon[34801]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-09-12T03:30:05.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:30:04 smithi195 ceph-mon[34786]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-09-12T03:30:05.271 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:30:04 smithi064 ceph-mon[30952]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-09-12T03:30:07.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:30:06 smithi115 ceph-mon[34801]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-12T03:30:07.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:30:06 smithi195 ceph-mon[34786]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-12T03:30:07.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:30:06 smithi064 ceph-mon[30952]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-12T03:30:09.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:30:08 smithi115 ceph-mon[34801]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-12T03:30:09.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:30:08 smithi195 ceph-mon[34786]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-12T03:30:09.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:30:08 smithi064 ceph-mon[30952]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-12T03:30:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.710+0000 7f84f0711640 1 -- 172.21.15.64:0/2435839687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84e8100860 msgr2=0x7f84e8100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.710+0000 7f84f0711640 1 --2- 172.21.15.64:0/2435839687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84e8100860 0x7f84e8100c40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f84e4009900 tx=0x7f84e402a410 comp rx=0 tx=0).stop 2024-09-12T03:30:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.711+0000 7f84f0711640 1 -- 172.21.15.64:0/2435839687 shutdown_connections 2024-09-12T03:30:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.711+0000 7f84f0711640 1 --2- 172.21.15.64:0/2435839687 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84e810e410 0x7f84e8110800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.711+0000 7f84f0711640 1 --2- 172.21.15.64:0/2435839687 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84e8101210 0x7f84e810ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.711+0000 7f84f0711640 1 --2- 172.21.15.64:0/2435839687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84e8100860 0x7f84e8100c40 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.711+0000 7f84f0711640 1 -- 172.21.15.64:0/2435839687 >> 172.21.15.64:0/2435839687 conn(0x7f84e8077990 msgr2=0x7f84e80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.711+0000 7f84f0711640 1 -- 172.21.15.64:0/2435839687 shutdown_connections 2024-09-12T03:30:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.711+0000 7f84f0711640 1 -- 172.21.15.64:0/2435839687 wait complete. 2024-09-12T03:30:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.711+0000 7f84f0711640 1 Processor -- start 2024-09-12T03:30:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.711+0000 7f84f0711640 1 -- start start 2024-09-12T03:30:10.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.712+0000 7f84f0711640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84e8100860 0x7f84e819a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:10.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.712+0000 7f84f0711640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84e8101210 0x7f84e819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:10.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.712+0000 7f84f0711640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84e810e410 0x7f84e819ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:10.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.712+0000 7f84f0711640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f84e8113f90 con 0x7f84e8101210 2024-09-12T03:30:10.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.712+0000 7f84f0711640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f84e8113e10 con 0x7f84e810e410 2024-09-12T03:30:10.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.712+0000 7f84f0711640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f84e8114110 con 0x7f84e8100860 2024-09-12T03:30:10.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.712+0000 7f84edcac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84e8101210 0x7f84e819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.712+0000 7f84edcac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84e8101210 0x7f84e819aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56556/0 (socket says 172.21.15.64:56556) 2024-09-12T03:30:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.712+0000 7f84edcac640 1 -- 172.21.15.64:0/524561378 learned_addr learned my addr 172.21.15.64:0/524561378 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:30:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.712+0000 7f84ee4ad640 1 --2- 172.21.15.64:0/524561378 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84e8100860 0x7f84e819a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.712+0000 7f84eecae640 1 --2- 172.21.15.64:0/524561378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84e810e410 0x7f84e819ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.712+0000 7f84edcac640 1 -- 172.21.15.64:0/524561378 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84e8100860 msgr2=0x7f84e819a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.712+0000 7f84edcac640 1 --2- 172.21.15.64:0/524561378 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84e8100860 0x7f84e819a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.712+0000 7f84edcac640 1 -- 172.21.15.64:0/524561378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84e810e410 msgr2=0x7f84e819ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.712+0000 7f84edcac640 1 --2- 172.21.15.64:0/524561378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84e810e410 0x7f84e819ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.712+0000 7f84edcac640 1 -- 172.21.15.64:0/524561378 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e819f550 con 0x7f84e8101210 2024-09-12T03:30:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.712+0000 7f84eecae640 1 --2- 172.21.15.64:0/524561378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84e810e410 0x7f84e819ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:30:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.713+0000 7f84edcac640 1 --2- 172.21.15.64:0/524561378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84e8101210 0x7f84e819aaa0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f84d800e9f0 tx=0x7f84d800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.713+0000 7f84df7fe640 1 -- 172.21.15.64:0/524561378 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d800cd80 con 0x7f84e8101210 2024-09-12T03:30:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.713+0000 7f84df7fe640 1 -- 172.21.15.64:0/524561378 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84d80108e0 con 0x7f84e8101210 2024-09-12T03:30:10.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.713+0000 7f84f0711640 1 -- 172.21.15.64:0/524561378 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84e819f840 con 0x7f84e8101210 2024-09-12T03:30:10.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.713+0000 7f84df7fe640 1 -- 172.21.15.64:0/524561378 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d8018920 con 0x7f84e8101210 2024-09-12T03:30:10.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.713+0000 7f84f0711640 1 -- 172.21.15.64:0/524561378 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84e81a70d0 con 0x7f84e8101210 2024-09-12T03:30:10.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.715+0000 7f84df7fe640 1 -- 172.21.15.64:0/524561378 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7f84d8010a90 con 0x7f84e8101210 2024-09-12T03:30:10.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.716+0000 7f84df7fe640 1 --2- 172.21.15.64:0/524561378 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f84d40a2fc0 0x7f84d40a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:10.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.716+0000 7f84df7fe640 1 -- 172.21.15.64:0/524561378 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f84d80cfed0 con 0x7f84e8101210 2024-09-12T03:30:10.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.716+0000 7f84ee4ad640 1 --2- 172.21.15.64:0/524561378 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f84d40a2fc0 0x7f84d40a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.716+0000 7f84f34a5740 1 Processor -- start 2024-09-12T03:30:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.717+0000 7f84f34a5740 1 -- start start 2024-09-12T03:30:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.717+0000 7f84ee4ad640 1 --2- 172.21.15.64:0/524561378 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f84d40a2fc0 0x7f84d40a5480 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f84e4000c00 tx=0x7f84e4031400 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.717+0000 7f84f34a5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba90b21af0 0x55ba90b21ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.717+0000 7f84f34a5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ba90b22410 0x55ba90b2c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.717+0000 7f84f34a5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ba90b2ccb0 0x55ba90b2f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.717+0000 7f84f34a5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ba90a862f0 con 0x55ba90b21af0 2024-09-12T03:30:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.717+0000 7f84f34a5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ba90a2e270 con 0x55ba90b2ccb0 2024-09-12T03:30:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.717+0000 7f84f34a5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ba90a822b0 con 0x55ba90b22410 2024-09-12T03:30:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.717+0000 7f84ee4ad640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba90b21af0 0x55ba90b21ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.717+0000 7f84ee4ad640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba90b21af0 0x55ba90b21ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56564/0 (socket says 172.21.15.64:56564) 2024-09-12T03:30:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.717+0000 7f84ee4ad640 1 -- 172.21.15.64:0/1821304176 learned_addr learned my addr 172.21.15.64:0/1821304176 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:30:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.717+0000 7f84edcac640 1 --2- 172.21.15.64:0/1821304176 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ba90b22410 0x55ba90b2c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.717+0000 7f84eecae640 1 --2- 172.21.15.64:0/1821304176 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ba90b2ccb0 0x55ba90b2f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.718+0000 7f84ee4ad640 1 -- 172.21.15.64:0/1821304176 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ba90b22410 msgr2=0x55ba90b2c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.718+0000 7f84ee4ad640 1 --2- 172.21.15.64:0/1821304176 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ba90b22410 0x55ba90b2c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.718+0000 7f84ee4ad640 1 -- 172.21.15.64:0/1821304176 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ba90b2ccb0 msgr2=0x55ba90b2f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.718+0000 7f84ee4ad640 1 --2- 172.21.15.64:0/1821304176 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ba90b2ccb0 0x55ba90b2f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.718+0000 7f84ee4ad640 1 -- 172.21.15.64:0/1821304176 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba90a2d340 con 0x55ba90b21af0 2024-09-12T03:30:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.718+0000 7f84eecae640 1 --2- 172.21.15.64:0/1821304176 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ba90b2ccb0 0x55ba90b2f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:30:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.718+0000 7f84ee4ad640 1 --2- 172.21.15.64:0/1821304176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba90b21af0 0x55ba90b21ed0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f84e4033000 tx=0x7f84e403c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3262cccf1f82c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.718+0000 7f84dcff9640 1 -- 172.21.15.64:0/1821304176 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e4035050 con 0x55ba90b21af0 2024-09-12T03:30:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.718+0000 7f84dcff9640 1 -- 172.21.15.64:0/1821304176 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84e404d070 con 0x55ba90b21af0 2024-09-12T03:30:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.718+0000 7f84dcff9640 1 -- 172.21.15.64:0/1821304176 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e40039f0 con 0x55ba90b21af0 2024-09-12T03:30:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.719+0000 7f84f34a5740 1 -- 172.21.15.64:0/1821304176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba90b21af0 msgr2=0x55ba90b21ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.719+0000 7f84f34a5740 1 --2- 172.21.15.64:0/1821304176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba90b21af0 0x55ba90b21ed0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f84e4033000 tx=0x7f84e403c000 comp rx=0 tx=0).stop 2024-09-12T03:30:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.719+0000 7f84f34a5740 1 -- 172.21.15.64:0/1821304176 shutdown_connections 2024-09-12T03:30:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.719+0000 7f84f34a5740 1 --2- 172.21.15.64:0/1821304176 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ba90b2ccb0 0x55ba90b2f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.720+0000 7f84f34a5740 1 --2- 172.21.15.64:0/1821304176 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ba90b22410 0x55ba90b2c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.720+0000 7f84f34a5740 1 --2- 172.21.15.64:0/1821304176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba90b21af0 0x55ba90b21ed0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.720+0000 7f84f34a5740 1 -- 172.21.15.64:0/1821304176 >> 172.21.15.64:0/1821304176 conn(0x55ba90b1e060 msgr2=0x55ba90b1e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.720+0000 7f84f34a5740 1 -- 172.21.15.64:0/1821304176 shutdown_connections 2024-09-12T03:30:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.720+0000 7f84f34a5740 1 -- 172.21.15.64:0/1821304176 wait complete. 2024-09-12T03:30:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.720+0000 7f84f34a5740 1 -- start start 2024-09-12T03:30:10.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.720+0000 7f84f34a5740 1 Processor -- start 2024-09-12T03:30:10.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.721+0000 7f84f34a5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba90b21af0 0x55ba90b29b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:10.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.721+0000 7f84f34a5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ba90b22410 0x55ba90b2a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:10.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.721+0000 7f84f34a5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ba90b2ccb0 0x55ba90b2a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:10.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.721+0000 7f84ee4ad640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba90b21af0 0x55ba90b29b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:10.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.721+0000 7f84f34a5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ba90b2f880 con 0x55ba90b21af0 2024-09-12T03:30:10.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.721+0000 7f84f34a5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ba90a0f290 con 0x55ba90b2ccb0 2024-09-12T03:30:10.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.721+0000 7f84f34a5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ba90b2fa00 con 0x55ba90b22410 2024-09-12T03:30:10.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.721+0000 7f84ee4ad640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba90b21af0 0x55ba90b29b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56566/0 (socket says 172.21.15.64:56566) 2024-09-12T03:30:10.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.721+0000 7f84ee4ad640 1 -- 172.21.15.64:0/976726121 learned_addr learned my addr 172.21.15.64:0/976726121 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:30:10.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.721+0000 7f84f34a5740 10 client.0 _ll_register_callbacks cb 0x55ba90a40140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:10.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.721+0000 7f84edcac640 1 --2- 172.21.15.64:0/976726121 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ba90b22410 0x55ba90b2a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:10.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.722+0000 7f84eecae640 1 --2- 172.21.15.64:0/976726121 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ba90b2ccb0 0x55ba90b2a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.722+0000 7f84edcac640 1 -- 172.21.15.64:0/976726121 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ba90b2ccb0 msgr2=0x55ba90b2a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.722+0000 7f84edcac640 1 --2- 172.21.15.64:0/976726121 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ba90b2ccb0 0x55ba90b2a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.722+0000 7f84edcac640 1 -- 172.21.15.64:0/976726121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba90b21af0 msgr2=0x55ba90b29b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.722+0000 7f84edcac640 1 --2- 172.21.15.64:0/976726121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba90b21af0 0x55ba90b29b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.722+0000 7f84edcac640 1 -- 172.21.15.64:0/976726121 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba90799210 con 0x55ba90b22410 2024-09-12T03:30:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.722+0000 7f84ee4ad640 1 --2- 172.21.15.64:0/976726121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba90b21af0 0x55ba90b29b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:30:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.722+0000 7f84edcac640 1 --2- 172.21.15.64:0/976726121 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ba90b22410 0x55ba90b2a0c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f84d80cd070 tx=0x7f84d8010ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d3a86df25f123ce server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.722+0000 7f84d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-09-12T03:30:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.722+0000 7f84d1ffb640 1 -- 172.21.15.64:0/976726121 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d80a0120 con 0x55ba90b22410 2024-09-12T03:30:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.722+0000 7f84d1ffb640 1 -- 172.21.15.64:0/976726121 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f84d80cd350 con 0x55ba90b22410 2024-09-12T03:30:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.722+0000 7f84f34a5740 1 -- 172.21.15.64:0/976726121 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84e404d070 con 0x55ba90b22410 2024-09-12T03:30:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.722+0000 7f84d1ffb640 1 -- 172.21.15.64:0/976726121 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f84d80a49e0 con 0x55ba90b22410 2024-09-12T03:30:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.723+0000 7f84f34a5740 10 client.25084 fetch_fsmap learned FSMap version 25 2024-09-12T03:30:10.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.723+0000 7f84f34a5740 1 -- 172.21.15.64:0/976726121 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ba90865620 con 0x55ba90b22410 2024-09-12T03:30:10.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.723+0000 7f84d1ffb640 1 -- 172.21.15.64:0/976726121 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84e404d070 con 0x55ba90b22410 2024-09-12T03:30:10.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.723+0000 7f84d1ffb640 1 -- 172.21.15.64:0/976726121 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f84d80b2070 con 0x55ba90b22410 2024-09-12T03:30:10.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.723+0000 7f84f34a5740 10 client.25084 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:30:10.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.723+0000 7f84f34a5740 10 client.25084 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:30:10.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.723+0000 7f84f34a5740 20 client.25084 populate_metadata read hostname 'smithi064' 2024-09-12T03:30:10.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.723+0000 7f84f34a5740 1 --2- 172.21.15.64:0/976726121 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55ba90b3cc50 0x55ba90b3f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:10.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.723+0000 7f84f34a5740 4 client.25084 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:30:10.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.723+0000 7f84f34a5740 1 -- 172.21.15.64:0/976726121 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba90a93a20 con 0x55ba90b3cc50 2024-09-12T03:30:10.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.723+0000 7f84ee4ad640 1 --2- 172.21.15.64:0/976726121 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55ba90b3cc50 0x55ba90b3f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:10.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.724+0000 7f84d1ffb640 10 client.25084 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:30:10.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.724+0000 7f84ee4ad640 1 --2- 172.21.15.64:0/976726121 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55ba90b3cc50 0x55ba90b3f060 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aef96ef537e4327d server_cookie=5199fe116387bfdc in_seq=0 out_seq=0 2024-09-12T03:30:10.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.725+0000 7f84d1ffb640 1 -- 172.21.15.64:0/976726121 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ba90a93a20 con 0x55ba90b3cc50 2024-09-12T03:30:10.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.725+0000 7f84d1ffb640 10 client.25084 handle_command_reply: tid=0 2024-09-12T03:30:10.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.725+0000 7f84f34a5740 1 client.25084 shutdown 2024-09-12T03:30:10.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.725+0000 7f84f34a5740 1 -- 172.21.15.64:0/976726121 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ba90b22410 msgr2=0x55ba90b2a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:10.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.725+0000 7f84f34a5740 1 --2- 172.21.15.64:0/976726121 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ba90b22410 0x55ba90b2a0c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f84d80cd070 tx=0x7f84d8010ea0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.725+0000 7f84f34a5740 1 -- 172.21.15.64:0/976726121 shutdown_connections 2024-09-12T03:30:10.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.725+0000 7f84f34a5740 1 --2- 172.21.15.64:0/976726121 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55ba90b3cc50 0x55ba90b3f060 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.725+0000 7f84f34a5740 1 --2- 172.21.15.64:0/976726121 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ba90b2ccb0 0x55ba90b2a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.725+0000 7f84d1ffb640 0 client.25084 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:30:10.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.725+0000 7f84f34a5740 1 --2- 172.21.15.64:0/976726121 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ba90b22410 0x55ba90b2a0c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.725+0000 7f84f34a5740 1 --2- 172.21.15.64:0/976726121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba90b21af0 0x55ba90b29b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.725+0000 7f84f34a5740 1 -- 172.21.15.64:0/976726121 >> 172.21.15.64:0/976726121 conn(0x55ba90b1e060 msgr2=0x55ba90b23d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:10.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.725+0000 7f84f34a5740 1 -- 172.21.15.64:0/976726121 shutdown_connections 2024-09-12T03:30:10.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.725+0000 7f84f34a5740 1 -- 172.21.15.64:0/976726121 wait complete. 2024-09-12T03:30:10.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.725+0000 7f84f34a5740 20 client.25084 trim_cache size 0 max 16384 2024-09-12T03:30:10.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.753+0000 7f84f34a5740 1 Processor -- start 2024-09-12T03:30:10.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.753+0000 7f84f34a5740 1 -- start start 2024-09-12T03:30:10.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.754+0000 7f84f34a5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ba90b21af0 0x55ba90b2a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:10.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.754+0000 7f84f34a5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba90b22410 0x55ba90b2baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:10.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.754+0000 7f84f34a5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ba90b2ccb0 0x55ba90b44970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:10.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.754+0000 7f84f34a5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f84e81b3a80 con 0x55ba90b22410 2024-09-12T03:30:10.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.754+0000 7f84f34a5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f84e81b1350 con 0x55ba90b21af0 2024-09-12T03:30:10.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.754+0000 7f84f34a5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f84e81af1a0 con 0x55ba90b2ccb0 2024-09-12T03:30:10.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.754+0000 7f84edcac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba90b22410 0x55ba90b2baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:10.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.754+0000 7f84edcac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba90b22410 0x55ba90b2baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56578/0 (socket says 172.21.15.64:56578) 2024-09-12T03:30:10.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.754+0000 7f84edcac640 1 -- 172.21.15.64:0/3814007194 learned_addr learned my addr 172.21.15.64:0/3814007194 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:30:10.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.754+0000 7f84eecae640 1 --2- 172.21.15.64:0/3814007194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ba90b2ccb0 0x55ba90b44970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:10.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.754+0000 7f84ee4ad640 1 --2- 172.21.15.64:0/3814007194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ba90b21af0 0x55ba90b2a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.754+0000 7f84edcac640 1 -- 172.21.15.64:0/3814007194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ba90b2ccb0 msgr2=0x55ba90b44970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.754+0000 7f84edcac640 1 --2- 172.21.15.64:0/3814007194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ba90b2ccb0 0x55ba90b44970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.754+0000 7f84edcac640 1 -- 172.21.15.64:0/3814007194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ba90b21af0 msgr2=0x55ba90b2a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.754+0000 7f84edcac640 1 --2- 172.21.15.64:0/3814007194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ba90b21af0 0x55ba90b2a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.754+0000 7f84edcac640 1 -- 172.21.15.64:0/3814007194 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba90b22240 con 0x55ba90b22410 2024-09-12T03:30:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.754+0000 7f84ee4ad640 1 --2- 172.21.15.64:0/3814007194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ba90b21af0 0x55ba90b2a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:30:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.754+0000 7f84eecae640 1 --2- 172.21.15.64:0/3814007194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ba90b2ccb0 0x55ba90b44970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:30:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.755+0000 7f84edcac640 1 --2- 172.21.15.64:0/3814007194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba90b22410 0x55ba90b2baf0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f84d8098740 tx=0x7f84d8018ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=294d56da03cd56ae server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.755+0000 7f84d3fff640 1 -- 172.21.15.64:0/3814007194 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d80a0120 con 0x55ba90b22410 2024-09-12T03:30:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.755+0000 7f84d3fff640 1 -- 172.21.15.64:0/3814007194 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84d8097ac0 con 0x55ba90b22410 2024-09-12T03:30:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.755+0000 7f84d3fff640 1 -- 172.21.15.64:0/3814007194 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d809b6d0 con 0x55ba90b22410 2024-09-12T03:30:10.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.756+0000 7f84f34a5740 1 -- 172.21.15.64:0/3814007194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba90b22410 msgr2=0x55ba90b2baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:10.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.756+0000 7f84f34a5740 1 --2- 172.21.15.64:0/3814007194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba90b22410 0x55ba90b2baf0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f84d8098740 tx=0x7f84d8018ac0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.756+0000 7f84f34a5740 1 -- 172.21.15.64:0/3814007194 shutdown_connections 2024-09-12T03:30:10.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.756+0000 7f84f34a5740 1 --2- 172.21.15.64:0/3814007194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ba90b2ccb0 0x55ba90b44970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.756+0000 7f84f34a5740 1 --2- 172.21.15.64:0/3814007194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba90b22410 0x55ba90b2baf0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.756+0000 7f84f34a5740 1 --2- 172.21.15.64:0/3814007194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ba90b21af0 0x55ba90b2a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.756+0000 7f84f34a5740 1 -- 172.21.15.64:0/3814007194 >> 172.21.15.64:0/3814007194 conn(0x55ba90b1e060 msgr2=0x55ba90b2b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:10.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.756+0000 7f84f34a5740 1 -- 172.21.15.64:0/3814007194 shutdown_connections 2024-09-12T03:30:10.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.756+0000 7f84f34a5740 1 -- 172.21.15.64:0/3814007194 wait complete. 2024-09-12T03:30:10.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.756+0000 7f84f34a5740 1 -- start start 2024-09-12T03:30:10.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.757+0000 7f84f34a5740 1 Processor -- start 2024-09-12T03:30:10.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.757+0000 7f84f34a5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba90b21af0 0x55ba90b34f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:10.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.757+0000 7f84f34a5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ba90b2ccb0 0x55ba90b35490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:10.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.757+0000 7f84f34a5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ba90b26440 0x55ba90b268a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:10.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.757+0000 7f84f34a5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f84e81af810 con 0x55ba90b21af0 2024-09-12T03:30:10.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.757+0000 7f84f34a5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f84e81b0c80 con 0x55ba90b2ccb0 2024-09-12T03:30:10.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.757+0000 7f84f34a5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f84e81afee0 con 0x55ba90b26440 2024-09-12T03:30:10.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.757+0000 7f84f34a5740 10 client.0 _ll_register_callbacks cb 0x7f84d809b8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:10.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.757+0000 7f84ee4ad640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba90b21af0 0x55ba90b34f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.757+0000 7f84ee4ad640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba90b21af0 0x55ba90b34f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56588/0 (socket says 172.21.15.64:56588) 2024-09-12T03:30:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.757+0000 7f84ee4ad640 1 -- 172.21.15.64:0/3552202772 learned_addr learned my addr 172.21.15.64:0/3552202772 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:30:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.757+0000 7f84eecae640 1 --2- 172.21.15.64:0/3552202772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ba90b26440 0x55ba90b268a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.757+0000 7f84edcac640 1 --2- 172.21.15.64:0/3552202772 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ba90b2ccb0 0x55ba90b35490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.758+0000 7f84ee4ad640 1 -- 172.21.15.64:0/3552202772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ba90b26440 msgr2=0x55ba90b268a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.758+0000 7f84ee4ad640 1 --2- 172.21.15.64:0/3552202772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ba90b26440 0x55ba90b268a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.758+0000 7f84ee4ad640 1 -- 172.21.15.64:0/3552202772 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ba90b2ccb0 msgr2=0x55ba90b35490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.758+0000 7f84ee4ad640 1 --2- 172.21.15.64:0/3552202772 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ba90b2ccb0 0x55ba90b35490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.758+0000 7f84ee4ad640 1 -- 172.21.15.64:0/3552202772 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba90b270d0 con 0x55ba90b21af0 2024-09-12T03:30:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.758+0000 7f84edcac640 1 --2- 172.21.15.64:0/3552202772 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ba90b2ccb0 0x55ba90b35490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:30:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.758+0000 7f84ee4ad640 1 --2- 172.21.15.64:0/3552202772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba90b21af0 0x55ba90b34f50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f84e4003660 tx=0x7f84e40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dec6de5a29a398 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.758+0000 7f84d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:30:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.758+0000 7f84d1ffb640 1 -- 172.21.15.64:0/3552202772 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e403f040 con 0x55ba90b21af0 2024-09-12T03:30:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.758+0000 7f84d1ffb640 1 -- 172.21.15.64:0/3552202772 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f84e4068b40 con 0x55ba90b21af0 2024-09-12T03:30:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.758+0000 7f84f34a5740 1 -- 172.21.15.64:0/3552202772 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84d8097ac0 con 0x55ba90b21af0 2024-09-12T03:30:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.758+0000 7f84d1ffb640 1 -- 172.21.15.64:0/3552202772 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f84e4075050 con 0x55ba90b21af0 2024-09-12T03:30:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.758+0000 7f84f34a5740 10 client.15420 fetch_fsmap learned FSMap version 25 2024-09-12T03:30:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.759+0000 7f84f34a5740 1 -- 172.21.15.64:0/3552202772 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ba90ba7920 con 0x55ba90b21af0 2024-09-12T03:30:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.759+0000 7f84d1ffb640 1 -- 172.21.15.64:0/3552202772 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84d8097ac0 con 0x55ba90b21af0 2024-09-12T03:30:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.759+0000 7f84d1ffb640 1 -- 172.21.15.64:0/3552202772 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f84e4070030 con 0x55ba90b21af0 2024-09-12T03:30:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.759+0000 7f84f34a5740 10 client.15420 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:30:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.759+0000 7f84f34a5740 10 client.15420 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:30:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.759+0000 7f84f34a5740 20 client.15420 populate_metadata read hostname 'smithi064' 2024-09-12T03:30:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.759+0000 7f84f34a5740 1 --2- 172.21.15.64:0/3552202772 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55ba90ba7cb0 0x55ba90baa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:10.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.759+0000 7f84f34a5740 4 client.15420 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:30:10.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.759+0000 7f84f34a5740 1 -- 172.21.15.64:0/3552202772 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ba90a8dd70 con 0x55ba90ba7cb0 2024-09-12T03:30:10.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.760+0000 7f84edcac640 1 --2- 172.21.15.64:0/3552202772 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55ba90ba7cb0 0x55ba90baa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:10.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.760+0000 7f84d1ffb640 10 client.15420 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:30:10.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.760+0000 7f84edcac640 1 --2- 172.21.15.64:0/3552202772 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55ba90ba7cb0 0x55ba90baa0a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2448c92b3bfbcff2 server_cookie=8fd68f6c5f6c59f3 in_seq=0 out_seq=0 2024-09-12T03:30:10.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.766+0000 7f84d1ffb640 1 -- 172.21.15.64:0/3552202772 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ba90a8dd70 con 0x55ba90ba7cb0 2024-09-12T03:30:10.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.766+0000 7f84d1ffb640 10 client.15420 handle_command_reply: tid=0 2024-09-12T03:30:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.766+0000 7f84f34a5740 1 client.15420 shutdown 2024-09-12T03:30:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.766+0000 7f84f34a5740 1 -- 172.21.15.64:0/3552202772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba90b21af0 msgr2=0x55ba90b34f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.766+0000 7f84f34a5740 1 --2- 172.21.15.64:0/3552202772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba90b21af0 0x55ba90b34f50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f84e4003660 tx=0x7f84e40045e0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.766+0000 7f84f34a5740 1 -- 172.21.15.64:0/3552202772 shutdown_connections 2024-09-12T03:30:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.766+0000 7f84f34a5740 1 --2- 172.21.15.64:0/3552202772 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55ba90ba7cb0 0x55ba90baa0a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.766+0000 7f84f34a5740 1 --2- 172.21.15.64:0/3552202772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ba90b26440 0x55ba90b268a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.766+0000 7f84d1ffb640 0 client.15420 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:30:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.766+0000 7f84f34a5740 1 --2- 172.21.15.64:0/3552202772 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ba90b2ccb0 0x55ba90b35490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.766+0000 7f84f34a5740 1 --2- 172.21.15.64:0/3552202772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ba90b21af0 0x55ba90b34f50 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.766+0000 7f84f34a5740 1 -- 172.21.15.64:0/3552202772 >> 172.21.15.64:0/3552202772 conn(0x55ba90b1e060 msgr2=0x55ba90b23d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.766+0000 7f84f34a5740 1 -- 172.21.15.64:0/3552202772 shutdown_connections 2024-09-12T03:30:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.766+0000 7f84f34a5740 1 -- 172.21.15.64:0/3552202772 wait complete. 2024-09-12T03:30:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.766+0000 7f84f34a5740 20 client.15420 trim_cache size 0 max 16384 2024-09-12T03:30:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:30:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:30:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:30:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:30:10.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.767+0000 7f84f0711640 1 -- 172.21.15.64:0/524561378 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f84d40a2fc0 msgr2=0x7f84d40a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:10.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.767+0000 7f84f0711640 1 --2- 172.21.15.64:0/524561378 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f84d40a2fc0 0x7f84d40a5480 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f84e4000c00 tx=0x7f84e4031400 comp rx=0 tx=0).stop 2024-09-12T03:30:10.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.767+0000 7f84f0711640 1 -- 172.21.15.64:0/524561378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84e8101210 msgr2=0x7f84e819aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:10.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.767+0000 7f84f0711640 1 --2- 172.21.15.64:0/524561378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84e8101210 0x7f84e819aaa0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f84d800e9f0 tx=0x7f84d800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.767+0000 7f84f0711640 1 -- 172.21.15.64:0/524561378 shutdown_connections 2024-09-12T03:30:10.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.767+0000 7f84f0711640 1 --2- 172.21.15.64:0/524561378 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f84d40a2fc0 0x7f84d40a5480 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.767+0000 7f84f0711640 1 --2- 172.21.15.64:0/524561378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84e810e410 0x7f84e819ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.767+0000 7f84f0711640 1 --2- 172.21.15.64:0/524561378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84e8101210 0x7f84e819aaa0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.767+0000 7f84f0711640 1 --2- 172.21.15.64:0/524561378 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84e8100860 0x7f84e819a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.767+0000 7f84f0711640 1 -- 172.21.15.64:0/524561378 >> 172.21.15.64:0/524561378 conn(0x7f84e8077990 msgr2=0x7f84e80ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.767+0000 7f84f0711640 1 -- 172.21.15.64:0/524561378 shutdown_connections 2024-09-12T03:30:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:10.767+0000 7f84f0711640 1 -- 172.21.15.64:0/524561378 wait complete. 2024-09-12T03:30:11.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:30:10 smithi115 ceph-mon[34801]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-12T03:30:11.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:30:10 smithi195 ceph-mon[34786]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-12T03:30:11.271 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:30:10 smithi064 ceph-mon[30952]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-12T03:30:13.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:30:12 smithi115 ceph-mon[34801]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-12T03:30:13.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:30:12 smithi195 ceph-mon[34786]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-12T03:30:13.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:30:12 smithi064 ceph-mon[30952]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-12T03:30:15.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:30:14 smithi115 ceph-mon[34801]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-09-12T03:30:15.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:30:14 smithi195 ceph-mon[34786]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-09-12T03:30:15.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:30:14 smithi064 ceph-mon[30952]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-09-12T03:30:17.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:30:16 smithi195 ceph-mon[34786]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-12T03:30:17.271 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:30:16 smithi064 ceph-mon[30952]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-12T03:30:17.279 INFO:tasks.workunit.client.0.smithi064.stderr:Updating files: 92% (11936/12903) Updating files: 93% (12000/12903) Updating files: 94% (12129/12903) Updating files: 95% (12258/12903) Updating files: 96% (12387/12903) Updating files: 97% (12516/12903) Updating files: 98% (12645/12903) Updating files: 99% (12774/12903) Updating files: 100% (12903/12903) Updating files: 100% (12903/12903), done. 2024-09-12T03:30:17.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:30:16 smithi115 ceph-mon[34801]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-12T03:30:17.879 INFO:tasks.workunit.client.0.smithi064.stderr:Note: switching to '086c737fa75c22d521a3932c59b31c5e1d1c9ef4'. 2024-09-12T03:30:17.879 INFO:tasks.workunit.client.0.smithi064.stderr: 2024-09-12T03:30:17.880 INFO:tasks.workunit.client.0.smithi064.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-12T03:30:17.880 INFO:tasks.workunit.client.0.smithi064.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-12T03:30:17.880 INFO:tasks.workunit.client.0.smithi064.stderr:state without impacting any branches by switching back to a branch. 2024-09-12T03:30:17.880 INFO:tasks.workunit.client.0.smithi064.stderr: 2024-09-12T03:30:17.880 INFO:tasks.workunit.client.0.smithi064.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-12T03:30:17.880 INFO:tasks.workunit.client.0.smithi064.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-12T03:30:17.880 INFO:tasks.workunit.client.0.smithi064.stderr: 2024-09-12T03:30:17.880 INFO:tasks.workunit.client.0.smithi064.stderr: git switch -c 2024-09-12T03:30:17.880 INFO:tasks.workunit.client.0.smithi064.stderr: 2024-09-12T03:30:17.880 INFO:tasks.workunit.client.0.smithi064.stderr:Or undo this operation with: 2024-09-12T03:30:17.880 INFO:tasks.workunit.client.0.smithi064.stderr: 2024-09-12T03:30:17.880 INFO:tasks.workunit.client.0.smithi064.stderr: git switch - 2024-09-12T03:30:17.880 INFO:tasks.workunit.client.0.smithi064.stderr: 2024-09-12T03:30:17.881 INFO:tasks.workunit.client.0.smithi064.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-12T03:30:17.881 INFO:tasks.workunit.client.0.smithi064.stderr: 2024-09-12T03:30:17.881 INFO:tasks.workunit.client.0.smithi064.stderr:HEAD is now at 086c737fa75 Merge pull request #59733 from zdover23/wip-doc-2024-09-11-backport-59635-to-squid 2024-09-12T03:30:17.885 DEBUG:teuthology.orchestra.run.smithi064:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-12T03:30:17.960 INFO:tasks.workunit.client.0.smithi064.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-12T03:30:17.962 INFO:tasks.workunit.client.0.smithi064.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-12T03:30:17.962 INFO:tasks.workunit.client.0.smithi064.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-12T03:30:18.683 INFO:tasks.workunit.client.0.smithi064.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-12T03:30:18.759 INFO:tasks.workunit.client.0.smithi064.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-12T03:30:18.802 INFO:tasks.workunit.client.0.smithi064.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-12T03:30:18.804 INFO:tasks.workunit.client.0.smithi064.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-12T03:30:18.804 INFO:tasks.workunit.client.0.smithi064.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-12T03:30:18.844 INFO:tasks.workunit.client.0.smithi064.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-12T03:30:18.848 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-12T03:30:18.848 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-12T03:30:18.905 INFO:tasks.workunit:Running workunits matching fs/test_o_trunc.sh on client.0... 2024-09-12T03:30:18.906 INFO:tasks.workunit:Running workunit fs/test_o_trunc.sh... 2024-09-12T03:30:18.906 DEBUG:teuthology.orchestra.run.smithi064:workunit test fs/test_o_trunc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-09-12T03:30:18.971 INFO:tasks.workunit.client.0.smithi064.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-09-12T03:30:18.972 INFO:tasks.workunit.client.0.smithi064.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs 2024-09-12T03:30:18.972 INFO:tasks.workunit.client.0.smithi064.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc trunc.foo 600 2024-09-12T03:30:19.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:30:18 smithi195 ceph-mon[34786]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-12T03:30:19.271 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:30:18 smithi064 ceph-mon[30952]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-12T03:30:19.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:30:18 smithi115 ceph-mon[34801]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-12T03:30:20.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:30:19 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:30:20.271 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:30:19 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:30:20.294 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:30:19 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:30:21.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:30:21 smithi064 ceph-mon[30952]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-12T03:30:21.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:30:21 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:30:21.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:30:21 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:30:21.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:30:21 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:30:21.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:30:21 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:30:21.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:30:21 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:30:21.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:30:21 smithi115 ceph-mon[34801]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-12T03:30:21.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:30:21 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:30:21.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:30:21 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:30:21.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:30:21 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:30:21.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:30:21 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:30:21.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:30:21 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:30:21.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:30:21 smithi195 ceph-mon[34786]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-12T03:30:21.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:30:21 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:30:21.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:30:21 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:30:21.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:30:21 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:30:21.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:30:21 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:30:21.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:30:21 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:30:23.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:30:23 smithi064 ceph-mon[30952]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.0 KiB/s wr, 8 op/s 2024-09-12T03:30:23.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:30:23 smithi115 ceph-mon[34801]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.0 KiB/s wr, 8 op/s 2024-09-12T03:30:23.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:30:23 smithi195 ceph-mon[34786]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.0 KiB/s wr, 8 op/s 2024-09-12T03:30:23.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.869+0000 7f70c536d640 1 -- 172.21.15.64:0/3212934159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70c01009f0 msgr2=0x7f70c0100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:23.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.869+0000 7f70c536d640 1 --2- 172.21.15.64:0/3212934159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70c01009f0 0x7f70c0100e50 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f70b4009930 tx=0x7f70b402a420 comp rx=0 tx=0).stop 2024-09-12T03:30:23.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.869+0000 7f70c536d640 1 -- 172.21.15.64:0/3212934159 shutdown_connections 2024-09-12T03:30:23.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.869+0000 7f70c536d640 1 --2- 172.21.15.64:0/3212934159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70c0101390 0x7f70c010d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.869+0000 7f70c536d640 1 --2- 172.21.15.64:0/3212934159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70c01009f0 0x7f70c0100e50 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.869+0000 7f70c536d640 1 --2- 172.21.15.64:0/3212934159 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70c0106a00 0x7f70c0106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.869+0000 7f70c536d640 1 -- 172.21.15.64:0/3212934159 >> 172.21.15.64:0/3212934159 conn(0x7f70c00fc6b0 msgr2=0x7f70c00fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:23.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.869+0000 7f70c536d640 1 -- 172.21.15.64:0/3212934159 shutdown_connections 2024-09-12T03:30:23.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.869+0000 7f70c536d640 1 -- 172.21.15.64:0/3212934159 wait complete. 2024-09-12T03:30:23.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.870+0000 7f70c536d640 1 Processor -- start 2024-09-12T03:30:23.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.870+0000 7f70c536d640 1 -- start start 2024-09-12T03:30:23.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.870+0000 7f70c536d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70c01009f0 0x7f70c01982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.870+0000 7f70c536d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70c0101390 0x7f70c0198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.870+0000 7f70c536d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70c0106a00 0x7f70c019cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.870+0000 7f70c536d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f70c0110cf0 con 0x7f70c0101390 2024-09-12T03:30:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.870+0000 7f70c536d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f70c0110b70 con 0x7f70c0106a00 2024-09-12T03:30:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.870+0000 7f70c536d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f70c0110e70 con 0x7f70c01009f0 2024-09-12T03:30:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.870+0000 7f70be7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70c0101390 0x7f70c0198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.870+0000 7f70be7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70c0101390 0x7f70c0198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45968/0 (socket says 172.21.15.64:45968) 2024-09-12T03:30:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.870+0000 7f70be7fc640 1 -- 172.21.15.64:0/3965478654 learned_addr learned my addr 172.21.15.64:0/3965478654 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:30:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.871+0000 7f70bf7fe640 1 --2- 172.21.15.64:0/3965478654 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70c0106a00 0x7f70c019cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.871+0000 7f70beffd640 1 --2- 172.21.15.64:0/3965478654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70c01009f0 0x7f70c01982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.871+0000 7f70be7fc640 1 -- 172.21.15.64:0/3965478654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70c01009f0 msgr2=0x7f70c01982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.871+0000 7f70be7fc640 1 --2- 172.21.15.64:0/3965478654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70c01009f0 0x7f70c01982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.871+0000 7f70be7fc640 1 -- 172.21.15.64:0/3965478654 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70c0106a00 msgr2=0x7f70c019cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.871+0000 7f70be7fc640 1 --2- 172.21.15.64:0/3965478654 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70c0106a00 0x7f70c019cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.871+0000 7f70be7fc640 1 -- 172.21.15.64:0/3965478654 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c019d2c0 con 0x7f70c0101390 2024-09-12T03:30:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.871+0000 7f70be7fc640 1 --2- 172.21.15.64:0/3965478654 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70c0101390 0x7f70c0198810 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f70b4005940 tx=0x7f70b40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.871+0000 7f70a7fff640 1 -- 172.21.15.64:0/3965478654 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b403f030 con 0x7f70c0101390 2024-09-12T03:30:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.871+0000 7f70a7fff640 1 -- 172.21.15.64:0/3965478654 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70b40318a0 con 0x7f70c0101390 2024-09-12T03:30:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.871+0000 7f70c536d640 1 -- 172.21.15.64:0/3965478654 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70c019d550 con 0x7f70c0101390 2024-09-12T03:30:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.871+0000 7f70a7fff640 1 -- 172.21.15.64:0/3965478654 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b4035070 con 0x7f70c0101390 2024-09-12T03:30:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.871+0000 7f70c536d640 1 -- 172.21.15.64:0/3965478654 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70c01a4e40 con 0x7f70c0101390 2024-09-12T03:30:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.874+0000 7f70a7fff640 1 -- 172.21.15.64:0/3965478654 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7f70b4042030 con 0x7f70c0101390 2024-09-12T03:30:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.874+0000 7f70c81e7740 1 Processor -- start 2024-09-12T03:30:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.874+0000 7f70c81e7740 1 -- start start 2024-09-12T03:30:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.875+0000 7f70c81e7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638bb13faf0 0x5638bb13fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.875+0000 7f70c81e7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5638bb140410 0x5638bb14a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.875+0000 7f70c81e7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bb14acb0 0x5638bb14d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.875+0000 7f70c81e7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5638bb0a42f0 con 0x5638bb14acb0 2024-09-12T03:30:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.875+0000 7f70c81e7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5638bb04c270 con 0x5638bb13faf0 2024-09-12T03:30:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.875+0000 7f70c81e7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5638bb0a02b0 con 0x5638bb140410 2024-09-12T03:30:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.875+0000 7f70a7fff640 1 --2- 172.21.15.64:0/3965478654 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f70a80a2ec0 0x7f70a80a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.875+0000 7f70be7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bb14acb0 0x5638bb14d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.875+0000 7f70be7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bb14acb0 0x5638bb14d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45978/0 (socket says 172.21.15.64:45978) 2024-09-12T03:30:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.875+0000 7f70be7fc640 1 -- 172.21.15.64:0/1383676945 learned_addr learned my addr 172.21.15.64:0/1383676945 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:30:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.875+0000 7f70bf7fe640 1 --2- 172.21.15.64:0/1383676945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638bb13faf0 0x5638bb13fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.875+0000 7f70beffd640 1 --2- 172.21.15.64:0/1383676945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5638bb140410 0x5638bb14a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.875+0000 7f70bf7fe640 1 --2- 172.21.15.64:0/3965478654 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f70a80a2ec0 0x7f70a80a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.875+0000 7f70a7fff640 1 -- 172.21.15.64:0/3965478654 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f70b40efda0 con 0x7f70c0101390 2024-09-12T03:30:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.875+0000 7f70be7fc640 1 -- 172.21.15.64:0/1383676945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5638bb140410 msgr2=0x5638bb14a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.875+0000 7f70be7fc640 1 --2- 172.21.15.64:0/1383676945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5638bb140410 0x5638bb14a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.875+0000 7f70be7fc640 1 -- 172.21.15.64:0/1383676945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638bb13faf0 msgr2=0x5638bb13fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.875+0000 7f70be7fc640 1 --2- 172.21.15.64:0/1383676945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638bb13faf0 0x5638bb13fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.875+0000 7f70be7fc640 1 -- 172.21.15.64:0/1383676945 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638bb04b340 con 0x5638bb14acb0 2024-09-12T03:30:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.875+0000 7f70be7fc640 1 --2- 172.21.15.64:0/1383676945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bb14acb0 0x5638bb14d0a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f70b4009900 tx=0x7f70b4047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2354cdcc9c012df server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.875+0000 7f70bf7fe640 1 --2- 172.21.15.64:0/3965478654 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f70a80a2ec0 0x7f70a80a5380 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f70b8005c40 tx=0x7f70b8009bb0 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.876+0000 7f70a57fa640 1 -- 172.21.15.64:0/1383676945 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b4049070 con 0x5638bb14acb0 2024-09-12T03:30:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.876+0000 7f70a57fa640 1 -- 172.21.15.64:0/1383676945 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70b4003e20 con 0x5638bb14acb0 2024-09-12T03:30:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.876+0000 7f70a57fa640 1 -- 172.21.15.64:0/1383676945 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b40b8380 con 0x5638bb14acb0 2024-09-12T03:30:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.876+0000 7f70c81e7740 1 -- 172.21.15.64:0/1383676945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bb14acb0 msgr2=0x5638bb14d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:23.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.876+0000 7f70c81e7740 1 --2- 172.21.15.64:0/1383676945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bb14acb0 0x5638bb14d0a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f70b4009900 tx=0x7f70b4047000 comp rx=0 tx=0).stop 2024-09-12T03:30:23.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.876+0000 7f70c81e7740 1 -- 172.21.15.64:0/1383676945 shutdown_connections 2024-09-12T03:30:23.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.876+0000 7f70c81e7740 1 --2- 172.21.15.64:0/1383676945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bb14acb0 0x5638bb14d0a0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.876+0000 7f70c81e7740 1 --2- 172.21.15.64:0/1383676945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5638bb140410 0x5638bb14a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.876+0000 7f70c81e7740 1 --2- 172.21.15.64:0/1383676945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638bb13faf0 0x5638bb13fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.876+0000 7f70c81e7740 1 -- 172.21.15.64:0/1383676945 >> 172.21.15.64:0/1383676945 conn(0x5638bb13c060 msgr2=0x5638bb13c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:23.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.876+0000 7f70c81e7740 1 -- 172.21.15.64:0/1383676945 shutdown_connections 2024-09-12T03:30:23.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.876+0000 7f70c81e7740 1 -- 172.21.15.64:0/1383676945 wait complete. 2024-09-12T03:30:23.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.877+0000 7f70c81e7740 1 -- start start 2024-09-12T03:30:23.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.877+0000 7f70c81e7740 1 Processor -- start 2024-09-12T03:30:23.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.877+0000 7f70c81e7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638bb13faf0 0x5638bb147bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.877+0000 7f70c81e7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5638bb140410 0x5638bb148100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.877+0000 7f70c81e7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bb14acb0 0x5638bb148640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.877+0000 7f70c81e7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5638bb14d640 con 0x5638bb14acb0 2024-09-12T03:30:23.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.877+0000 7f70c81e7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5638bb02d290 con 0x5638bb13faf0 2024-09-12T03:30:23.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.877+0000 7f70c81e7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5638bb14d7c0 con 0x5638bb140410 2024-09-12T03:30:23.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.877+0000 7f70c81e7740 10 client.0 _ll_register_callbacks cb 0x5638bb05e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:23.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.877+0000 7f70bf7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bb14acb0 0x5638bb148640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.877+0000 7f70bf7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bb14acb0 0x5638bb148640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45980/0 (socket says 172.21.15.64:45980) 2024-09-12T03:30:23.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.877+0000 7f70bf7fe640 1 -- 172.21.15.64:0/3235150700 learned_addr learned my addr 172.21.15.64:0/3235150700 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:30:23.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.878+0000 7f70be7fc640 1 --2- 172.21.15.64:0/3235150700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5638bb140410 0x5638bb148100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.878+0000 7f70beffd640 1 --2- 172.21.15.64:0/3235150700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638bb13faf0 0x5638bb147bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.878+0000 7f70bf7fe640 1 -- 172.21.15.64:0/3235150700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5638bb140410 msgr2=0x5638bb148100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:23.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.878+0000 7f70bf7fe640 1 --2- 172.21.15.64:0/3235150700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5638bb140410 0x5638bb148100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.878+0000 7f70bf7fe640 1 -- 172.21.15.64:0/3235150700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638bb13faf0 msgr2=0x5638bb147bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:23.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.878+0000 7f70bf7fe640 1 --2- 172.21.15.64:0/3235150700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638bb13faf0 0x5638bb147bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.878+0000 7f70bf7fe640 1 -- 172.21.15.64:0/3235150700 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638badb7210 con 0x5638bb14acb0 2024-09-12T03:30:23.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.878+0000 7f70beffd640 1 --2- 172.21.15.64:0/3235150700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638bb13faf0 0x5638bb147bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:30:23.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.878+0000 7f70bf7fe640 1 --2- 172.21.15.64:0/3235150700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bb14acb0 0x5638bb148640 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f70b8010010 tx=0x7f70b8005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=414b84108469e7d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:23.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.878+0000 7f70a2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:30:23.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.878+0000 7f70a2ff5640 1 -- 172.21.15.64:0/3235150700 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b8020870 con 0x5638bb14acb0 2024-09-12T03:30:23.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.878+0000 7f70a2ff5640 1 -- 172.21.15.64:0/3235150700 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f70b8020a10 con 0x5638bb14acb0 2024-09-12T03:30:23.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.878+0000 7f70c81e7740 1 -- 172.21.15.64:0/3235150700 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5638bb0b1a20 con 0x5638bb14acb0 2024-09-12T03:30:23.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.878+0000 7f70a2ff5640 1 -- 172.21.15.64:0/3235150700 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f70b8020c10 con 0x5638bb14acb0 2024-09-12T03:30:23.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.878+0000 7f70c81e7740 10 client.15432 fetch_fsmap learned FSMap version 25 2024-09-12T03:30:23.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.878+0000 7f70c81e7740 1 -- 172.21.15.64:0/3235150700 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5638bae83620 con 0x5638bb14acb0 2024-09-12T03:30:23.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.879+0000 7f70a2ff5640 1 -- 172.21.15.64:0/3235150700 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5638bb0b1a20 con 0x5638bb14acb0 2024-09-12T03:30:23.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.879+0000 7f70a2ff5640 1 -- 172.21.15.64:0/3235150700 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f70b8027070 con 0x5638bb14acb0 2024-09-12T03:30:23.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.879+0000 7f70c81e7740 10 client.15432 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:30:23.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.879+0000 7f70c81e7740 10 client.15432 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:30:23.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.879+0000 7f70c81e7740 20 client.15432 populate_metadata read hostname 'smithi064' 2024-09-12T03:30:23.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.879+0000 7f70c81e7740 1 --2- 172.21.15.64:0/3235150700 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5638bb15a920 0x5638bb15cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.879+0000 7f70c81e7740 4 client.15432 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:30:23.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.879+0000 7f70c81e7740 1 -- 172.21.15.64:0/3235150700 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638bb0abd70 con 0x5638bb15a920 2024-09-12T03:30:23.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.879+0000 7f70beffd640 1 --2- 172.21.15.64:0/3235150700 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5638bb15a920 0x5638bb15cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.880+0000 7f70beffd640 1 --2- 172.21.15.64:0/3235150700 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5638bb15a920 0x5638bb15cd90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d07f6cb479793008 server_cookie=afe2881bc3246dc in_seq=0 out_seq=0 2024-09-12T03:30:23.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.880+0000 7f70a2ff5640 10 client.15432 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:30:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.880+0000 7f70a2ff5640 1 -- 172.21.15.64:0/3235150700 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5638bb0abd70 con 0x5638bb15a920 2024-09-12T03:30:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.880+0000 7f70a2ff5640 10 client.15432 handle_command_reply: tid=0 2024-09-12T03:30:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.880+0000 7f70c81e7740 1 client.15432 shutdown 2024-09-12T03:30:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.881+0000 7f70c81e7740 1 -- 172.21.15.64:0/3235150700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bb14acb0 msgr2=0x5638bb148640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.881+0000 7f70c81e7740 1 --2- 172.21.15.64:0/3235150700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bb14acb0 0x5638bb148640 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f70b8010010 tx=0x7f70b8005bd0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.881+0000 7f70c81e7740 1 -- 172.21.15.64:0/3235150700 shutdown_connections 2024-09-12T03:30:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.881+0000 7f70c81e7740 1 --2- 172.21.15.64:0/3235150700 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5638bb15a920 0x5638bb15cd90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.881+0000 7f70c81e7740 1 --2- 172.21.15.64:0/3235150700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bb14acb0 0x5638bb148640 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.881+0000 7f70c81e7740 1 --2- 172.21.15.64:0/3235150700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5638bb140410 0x5638bb148100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.881+0000 7f70c81e7740 1 --2- 172.21.15.64:0/3235150700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638bb13faf0 0x5638bb147bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.881+0000 7f70c81e7740 1 -- 172.21.15.64:0/3235150700 >> 172.21.15.64:0/3235150700 conn(0x5638bb13c060 msgr2=0x5638bb141d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.881+0000 7f70a2ff5640 0 client.15432 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:30:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.881+0000 7f70c81e7740 1 -- 172.21.15.64:0/3235150700 shutdown_connections 2024-09-12T03:30:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.881+0000 7f70c81e7740 1 -- 172.21.15.64:0/3235150700 wait complete. 2024-09-12T03:30:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.881+0000 7f70c81e7740 20 client.15432 trim_cache size 0 max 16384 2024-09-12T03:30:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.895+0000 7f70c81e7740 1 Processor -- start 2024-09-12T03:30:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.895+0000 7f70c81e7740 1 -- start start 2024-09-12T03:30:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.896+0000 7f70c81e7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638bb13faf0 0x5638bb146ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.896+0000 7f70c81e7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5638bb140410 0x5638bb148470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.896+0000 7f70c81e7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bb14acb0 0x5638bb1626f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.896+0000 7f70c81e7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f70c01b1880 con 0x5638bb14acb0 2024-09-12T03:30:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.896+0000 7f70c81e7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f70c01af150 con 0x5638bb13faf0 2024-09-12T03:30:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.896+0000 7f70c81e7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f70c01acfa0 con 0x5638bb140410 2024-09-12T03:30:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.896+0000 7f70bf7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bb14acb0 0x5638bb1626f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.896+0000 7f70bf7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bb14acb0 0x5638bb1626f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45994/0 (socket says 172.21.15.64:45994) 2024-09-12T03:30:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.896+0000 7f70bf7fe640 1 -- 172.21.15.64:0/1575716192 learned_addr learned my addr 172.21.15.64:0/1575716192 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:30:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.896+0000 7f70be7fc640 1 --2- 172.21.15.64:0/1575716192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5638bb140410 0x5638bb148470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.896+0000 7f70beffd640 1 --2- 172.21.15.64:0/1575716192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638bb13faf0 0x5638bb146ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.896+0000 7f70bf7fe640 1 -- 172.21.15.64:0/1575716192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5638bb140410 msgr2=0x5638bb148470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.896+0000 7f70bf7fe640 1 --2- 172.21.15.64:0/1575716192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5638bb140410 0x5638bb148470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.896+0000 7f70bf7fe640 1 -- 172.21.15.64:0/1575716192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638bb13faf0 msgr2=0x5638bb146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.896+0000 7f70bf7fe640 1 --2- 172.21.15.64:0/1575716192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638bb13faf0 0x5638bb146ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.896+0000 7f70bf7fe640 1 -- 172.21.15.64:0/1575716192 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638bb15d410 con 0x5638bb14acb0 2024-09-12T03:30:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.896+0000 7f70beffd640 1 --2- 172.21.15.64:0/1575716192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638bb13faf0 0x5638bb146ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:30:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.896+0000 7f70bf7fe640 1 --2- 172.21.15.64:0/1575716192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bb14acb0 0x5638bb1626f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f70b800ffe0 tx=0x7f70b8011860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3871c3c9881371f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.897+0000 7f70a4ff9640 1 -- 172.21.15.64:0/1575716192 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b803c070 con 0x5638bb14acb0 2024-09-12T03:30:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.897+0000 7f70a4ff9640 1 -- 172.21.15.64:0/1575716192 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70b8009480 con 0x5638bb14acb0 2024-09-12T03:30:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.897+0000 7f70a4ff9640 1 -- 172.21.15.64:0/1575716192 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b80297b0 con 0x5638bb14acb0 2024-09-12T03:30:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.897+0000 7f70c81e7740 1 -- 172.21.15.64:0/1575716192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bb14acb0 msgr2=0x5638bb1626f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.897+0000 7f70c81e7740 1 --2- 172.21.15.64:0/1575716192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bb14acb0 0x5638bb1626f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f70b800ffe0 tx=0x7f70b8011860 comp rx=0 tx=0).stop 2024-09-12T03:30:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.897+0000 7f70c81e7740 1 -- 172.21.15.64:0/1575716192 shutdown_connections 2024-09-12T03:30:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.897+0000 7f70c81e7740 1 --2- 172.21.15.64:0/1575716192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bb14acb0 0x5638bb1626f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.897+0000 7f70c81e7740 1 --2- 172.21.15.64:0/1575716192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5638bb140410 0x5638bb148470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.897+0000 7f70c81e7740 1 --2- 172.21.15.64:0/1575716192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638bb13faf0 0x5638bb146ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.897+0000 7f70c81e7740 1 -- 172.21.15.64:0/1575716192 >> 172.21.15.64:0/1575716192 conn(0x5638bb13c060 msgr2=0x5638bb149c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.897+0000 7f70c81e7740 1 -- 172.21.15.64:0/1575716192 shutdown_connections 2024-09-12T03:30:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.897+0000 7f70c81e7740 1 -- 172.21.15.64:0/1575716192 wait complete. 2024-09-12T03:30:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.897+0000 7f70c81e7740 1 -- start start 2024-09-12T03:30:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.898+0000 7f70c81e7740 1 Processor -- start 2024-09-12T03:30:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.898+0000 7f70c81e7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5638bb13faf0 0x5638bb152f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.898+0000 7f70c81e7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bb140410 0x5638bb14e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.898+0000 7f70c81e7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638bb14acb0 0x5638bb14e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.898+0000 7f70c81e7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f70c01ad610 con 0x5638bb140410 2024-09-12T03:30:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.898+0000 7f70c81e7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f70c01aea80 con 0x5638bb14acb0 2024-09-12T03:30:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.898+0000 7f70c81e7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f70c01adce0 con 0x5638bb13faf0 2024-09-12T03:30:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.898+0000 7f70c81e7740 10 client.0 _ll_register_callbacks cb 0x7f70b8011550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.898+0000 7f70be7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bb140410 0x5638bb14e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.898+0000 7f70be7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bb140410 0x5638bb14e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46000/0 (socket says 172.21.15.64:46000) 2024-09-12T03:30:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.898+0000 7f70be7fc640 1 -- 172.21.15.64:0/2980788088 learned_addr learned my addr 172.21.15.64:0/2980788088 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:30:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.898+0000 7f70bf7fe640 1 --2- 172.21.15.64:0/2980788088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638bb14acb0 0x5638bb14e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.898+0000 7f70beffd640 1 --2- 172.21.15.64:0/2980788088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5638bb13faf0 0x5638bb152f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.898+0000 7f70be7fc640 1 -- 172.21.15.64:0/2980788088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5638bb13faf0 msgr2=0x5638bb152f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.898+0000 7f70be7fc640 1 --2- 172.21.15.64:0/2980788088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5638bb13faf0 0x5638bb152f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.898+0000 7f70be7fc640 1 -- 172.21.15.64:0/2980788088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638bb14acb0 msgr2=0x5638bb14e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.898+0000 7f70be7fc640 1 --2- 172.21.15.64:0/2980788088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638bb14acb0 0x5638bb14e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.898+0000 7f70be7fc640 1 -- 172.21.15.64:0/2980788088 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638bb1372d0 con 0x5638bb140410 2024-09-12T03:30:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.898+0000 7f70bf7fe640 1 --2- 172.21.15.64:0/2980788088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638bb14acb0 0x5638bb14e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:30:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.898+0000 7f70be7fc640 1 --2- 172.21.15.64:0/2980788088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bb140410 0x5638bb14e270 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f70b4005c20 tx=0x7f70b402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed66ddab432e5a14 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.898+0000 7f70a2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:30:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.899+0000 7f70a2ff5640 1 -- 172.21.15.64:0/2980788088 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b40d9070 con 0x5638bb140410 2024-09-12T03:30:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.899+0000 7f70a2ff5640 1 -- 172.21.15.64:0/2980788088 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f70b40ed420 con 0x5638bb140410 2024-09-12T03:30:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.899+0000 7f70c81e7740 1 -- 172.21.15.64:0/2980788088 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70b8009480 con 0x5638bb140410 2024-09-12T03:30:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.899+0000 7f70a2ff5640 1 -- 172.21.15.64:0/2980788088 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f70b4049040 con 0x5638bb140410 2024-09-12T03:30:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.899+0000 7f70c81e7740 10 client.15438 fetch_fsmap learned FSMap version 25 2024-09-12T03:30:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.899+0000 7f70c81e7740 1 -- 172.21.15.64:0/2980788088 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5638bb13ccf0 con 0x5638bb140410 2024-09-12T03:30:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.899+0000 7f70a2ff5640 1 -- 172.21.15.64:0/2980788088 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70b8009480 con 0x5638bb140410 2024-09-12T03:30:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.899+0000 7f70a2ff5640 1 -- 172.21.15.64:0/2980788088 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f70b40bb160 con 0x5638bb140410 2024-09-12T03:30:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.899+0000 7f70c81e7740 10 client.15438 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:30:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.899+0000 7f70c81e7740 10 client.15438 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:30:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.899+0000 7f70c81e7740 20 client.15438 populate_metadata read hostname 'smithi064' 2024-09-12T03:30:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.899+0000 7f70c81e7740 1 --2- 172.21.15.64:0/2980788088 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5638bb1c5690 0x5638bb1c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.899+0000 7f70c81e7740 4 client.15438 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:30:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.899+0000 7f70c81e7740 1 -- 172.21.15.64:0/2980788088 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5638bb028900 con 0x5638bb1c5690 2024-09-12T03:30:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.899+0000 7f70beffd640 1 --2- 172.21.15.64:0/2980788088 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5638bb1c5690 0x5638bb1c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.900+0000 7f70beffd640 1 --2- 172.21.15.64:0/2980788088 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5638bb1c5690 0x5638bb1c7ab0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc41283d82708cb1 server_cookie=d1e412c2e18a9d27 in_seq=0 out_seq=0 2024-09-12T03:30:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.900+0000 7f70a2ff5640 10 client.15438 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:30:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.909+0000 7f70a2ff5640 1 -- 172.21.15.64:0/2980788088 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5638bb028900 con 0x5638bb1c5690 2024-09-12T03:30:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.909+0000 7f70a2ff5640 10 client.15438 handle_command_reply: tid=0 2024-09-12T03:30:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.909+0000 7f70c81e7740 1 client.15438 shutdown 2024-09-12T03:30:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.910+0000 7f70c81e7740 1 -- 172.21.15.64:0/2980788088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bb140410 msgr2=0x5638bb14e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.910+0000 7f70c81e7740 1 --2- 172.21.15.64:0/2980788088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bb140410 0x5638bb14e270 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f70b4005c20 tx=0x7f70b402a420 comp rx=0 tx=0).stop 2024-09-12T03:30:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.910+0000 7f70c81e7740 1 -- 172.21.15.64:0/2980788088 shutdown_connections 2024-09-12T03:30:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.910+0000 7f70c81e7740 1 --2- 172.21.15.64:0/2980788088 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5638bb1c5690 0x5638bb1c7ab0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.910+0000 7f70c81e7740 1 --2- 172.21.15.64:0/2980788088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638bb14acb0 0x5638bb14e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.910+0000 7f70c81e7740 1 --2- 172.21.15.64:0/2980788088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638bb140410 0x5638bb14e270 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.910+0000 7f70c81e7740 1 --2- 172.21.15.64:0/2980788088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5638bb13faf0 0x5638bb152f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.910+0000 7f70a2ff5640 0 client.15438 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:30:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.910+0000 7f70c81e7740 1 -- 172.21.15.64:0/2980788088 >> 172.21.15.64:0/2980788088 conn(0x5638bb13c060 msgr2=0x5638bb13a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.910+0000 7f70c81e7740 1 -- 172.21.15.64:0/2980788088 shutdown_connections 2024-09-12T03:30:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.910+0000 7f70c81e7740 1 -- 172.21.15.64:0/2980788088 wait complete. 2024-09-12T03:30:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.910+0000 7f70c81e7740 20 client.15438 trim_cache size 0 max 16384 2024-09-12T03:30:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:30:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:30:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:30:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:30:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.910+0000 7f70c536d640 1 -- 172.21.15.64:0/3965478654 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f70a80a2ec0 msgr2=0x7f70a80a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.910+0000 7f70c536d640 1 --2- 172.21.15.64:0/3965478654 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f70a80a2ec0 0x7f70a80a5380 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f70b8005c40 tx=0x7f70b8009bb0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.911+0000 7f70c536d640 1 -- 172.21.15.64:0/3965478654 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70c0101390 msgr2=0x7f70c0198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.911+0000 7f70c536d640 1 --2- 172.21.15.64:0/3965478654 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70c0101390 0x7f70c0198810 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f70b4005940 tx=0x7f70b40042f0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.911+0000 7f70c536d640 1 -- 172.21.15.64:0/3965478654 shutdown_connections 2024-09-12T03:30:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.911+0000 7f70c536d640 1 --2- 172.21.15.64:0/3965478654 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f70a80a2ec0 0x7f70a80a5380 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.911+0000 7f70c536d640 1 --2- 172.21.15.64:0/3965478654 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70c0106a00 0x7f70c019cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.911+0000 7f70c536d640 1 --2- 172.21.15.64:0/3965478654 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70c0101390 0x7f70c0198810 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.911+0000 7f70c536d640 1 --2- 172.21.15.64:0/3965478654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70c01009f0 0x7f70c01982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.911+0000 7f70c536d640 1 -- 172.21.15.64:0/3965478654 >> 172.21.15.64:0/3965478654 conn(0x7f70c00fc6b0 msgr2=0x7f70c006a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.911+0000 7f70c536d640 1 -- 172.21.15.64:0/3965478654 shutdown_connections 2024-09-12T03:30:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:23.911+0000 7f70c536d640 1 -- 172.21.15.64:0/3965478654 wait complete. 2024-09-12T03:30:25.521 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:30:25 smithi064 ceph-mon[30952]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.7 KiB/s wr, 7 op/s 2024-09-12T03:30:25.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:30:25 smithi115 ceph-mon[34801]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.7 KiB/s wr, 7 op/s 2024-09-12T03:30:25.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:30:25 smithi195 ceph-mon[34786]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.7 KiB/s wr, 7 op/s 2024-09-12T03:30:27.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:30:27 smithi064 ceph-mon[30952]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.6 KiB/s wr, 6 op/s 2024-09-12T03:30:27.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:30:27 smithi115 ceph-mon[34801]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.6 KiB/s wr, 6 op/s 2024-09-12T03:30:27.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:30:27 smithi195 ceph-mon[34786]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.6 KiB/s wr, 6 op/s 2024-09-12T03:30:29.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:30:29 smithi064 ceph-mon[30952]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-12T03:30:29.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:30:29 smithi115 ceph-mon[34801]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-12T03:30:29.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:30:29 smithi195 ceph-mon[34786]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-12T03:30:31.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:30:31 smithi064 ceph-mon[30952]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T03:30:31.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:30:31 smithi115 ceph-mon[34801]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T03:30:31.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:30:31 smithi195 ceph-mon[34786]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T03:30:33.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:30:33 smithi064 ceph-mon[30952]: pgmap v41: 129 pgs: 129 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-09-12T03:30:33.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:30:33 smithi115 ceph-mon[34801]: pgmap v41: 129 pgs: 129 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-09-12T03:30:33.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:30:33 smithi195 ceph-mon[34786]: pgmap v41: 129 pgs: 129 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-09-12T03:30:35.521 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:30:35 smithi064 ceph-mon[30952]: pgmap v42: 129 pgs: 129 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T03:30:35.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:30:35 smithi115 ceph-mon[34801]: pgmap v42: 129 pgs: 129 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T03:30:35.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:30:35 smithi195 ceph-mon[34786]: pgmap v42: 129 pgs: 129 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T03:30:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.010+0000 7ff384f31640 1 -- 172.21.15.64:0/418735899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff380102ca0 msgr2=0x7ff380103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.010+0000 7ff384f31640 1 --2- 172.21.15.64:0/418735899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff380102ca0 0x7ff380103080 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff370009900 tx=0x7ff37002a480 comp rx=0 tx=0).stop 2024-09-12T03:30:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.011+0000 7ff384f31640 1 -- 172.21.15.64:0/418735899 shutdown_connections 2024-09-12T03:30:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.011+0000 7ff384f31640 1 --2- 172.21.15.64:0/418735899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff38010c0c0 0x7ff38010e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.011+0000 7ff384f31640 1 --2- 172.21.15.64:0/418735899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3801035c0 0x7ff38010baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.011+0000 7ff384f31640 1 --2- 172.21.15.64:0/418735899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff380102ca0 0x7ff380103080 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.011+0000 7ff384f31640 1 -- 172.21.15.64:0/418735899 >> 172.21.15.64:0/418735899 conn(0x7ff3800fc6f0 msgr2=0x7ff3800feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.011+0000 7ff384f31640 1 -- 172.21.15.64:0/418735899 shutdown_connections 2024-09-12T03:30:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.011+0000 7ff384f31640 1 -- 172.21.15.64:0/418735899 wait complete. 2024-09-12T03:30:37.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.011+0000 7ff384f31640 1 Processor -- start 2024-09-12T03:30:37.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.011+0000 7ff384f31640 1 -- start start 2024-09-12T03:30:37.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.012+0000 7ff384f31640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff380102ca0 0x7ff380198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:37.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.012+0000 7ff384f31640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3801035c0 0x7ff3801987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:37.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.012+0000 7ff384f31640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff38010c0c0 0x7ff38019cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:37.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.012+0000 7ff384f31640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff380111d90 con 0x7ff38010c0c0 2024-09-12T03:30:37.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.012+0000 7ff384f31640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff380111c10 con 0x7ff3801035c0 2024-09-12T03:30:37.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.012+0000 7ff384f31640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff380111f10 con 0x7ff380102ca0 2024-09-12T03:30:37.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.012+0000 7ff37ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff38010c0c0 0x7ff38019cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:37.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.012+0000 7ff37e59c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff380102ca0 0x7ff380198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:37.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.012+0000 7ff37ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff38010c0c0 0x7ff38019cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59192/0 (socket says 172.21.15.64:59192) 2024-09-12T03:30:37.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.012+0000 7ff37ed9d640 1 -- 172.21.15.64:0/692496205 learned_addr learned my addr 172.21.15.64:0/692496205 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:30:37.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.012+0000 7ff37dd9b640 1 --2- 172.21.15.64:0/692496205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3801035c0 0x7ff3801987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:37.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.012+0000 7ff37ed9d640 1 -- 172.21.15.64:0/692496205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff380102ca0 msgr2=0x7ff380198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:37.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.012+0000 7ff37ed9d640 1 --2- 172.21.15.64:0/692496205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff380102ca0 0x7ff380198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.012+0000 7ff37ed9d640 1 -- 172.21.15.64:0/692496205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3801035c0 msgr2=0x7ff3801987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:37.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.012+0000 7ff37ed9d640 1 --2- 172.21.15.64:0/692496205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3801035c0 0x7ff3801987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.012+0000 7ff37ed9d640 1 -- 172.21.15.64:0/692496205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff38019d270 con 0x7ff38010c0c0 2024-09-12T03:30:37.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.012+0000 7ff37ed9d640 1 --2- 172.21.15.64:0/692496205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff38010c0c0 0x7ff38019cb40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff37800eaa0 tx=0x7ff37800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:37.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.013+0000 7ff36f7fe640 1 -- 172.21.15.64:0/692496205 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37800cdb0 con 0x7ff38010c0c0 2024-09-12T03:30:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.013+0000 7ff384f31640 1 -- 172.21.15.64:0/692496205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff38019d560 con 0x7ff38010c0c0 2024-09-12T03:30:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.013+0000 7ff36f7fe640 1 -- 172.21.15.64:0/692496205 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff378005020 con 0x7ff38010c0c0 2024-09-12T03:30:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.013+0000 7ff36f7fe640 1 -- 172.21.15.64:0/692496205 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3780116d0 con 0x7ff38010c0c0 2024-09-12T03:30:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.013+0000 7ff384f31640 1 -- 172.21.15.64:0/692496205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3801a4df0 con 0x7ff38010c0c0 2024-09-12T03:30:37.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.016+0000 7ff36f7fe640 1 -- 172.21.15.64:0/692496205 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7ff3780051d0 con 0x7ff38010c0c0 2024-09-12T03:30:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.016+0000 7ff387cd3740 1 Processor -- start 2024-09-12T03:30:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.016+0000 7ff387cd3740 1 -- start start 2024-09-12T03:30:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.017+0000 7ff387cd3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcab6d1af0 0x55bcab6d1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.017+0000 7ff387cd3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcab6d2410 0x55bcab6dc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.017+0000 7ff387cd3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcab6dccb0 0x55bcab6df0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.017+0000 7ff387cd3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bcab6362f0 con 0x55bcab6dccb0 2024-09-12T03:30:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.017+0000 7ff387cd3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55bcab5de270 con 0x55bcab6d1af0 2024-09-12T03:30:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.017+0000 7ff387cd3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55bcab6322b0 con 0x55bcab6d2410 2024-09-12T03:30:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.017+0000 7ff37dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcab6dccb0 0x55bcab6df0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.017+0000 7ff36f7fe640 1 --2- 172.21.15.64:0/692496205 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7ff3680a2ec0 0x7ff3680a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.017+0000 7ff37dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcab6dccb0 0x55bcab6df0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59196/0 (socket says 172.21.15.64:59196) 2024-09-12T03:30:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.017+0000 7ff37dd9b640 1 -- 172.21.15.64:0/2041289720 learned_addr learned my addr 172.21.15.64:0/2041289720 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:30:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.017+0000 7ff37dd9b640 1 --2- 172.21.15.64:0/2041289720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcab6d1af0 0x55bcab6d1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.017+0000 7ff37e59c640 1 --2- 172.21.15.64:0/2041289720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcab6d2410 0x55bcab6dc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.017+0000 7ff36f7fe640 1 -- 172.21.15.64:0/692496205 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7ff3780cfb30 con 0x7ff38010c0c0 2024-09-12T03:30:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.017+0000 7ff37ed9d640 1 --2- 172.21.15.64:0/692496205 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7ff3680a2ec0 0x7ff3680a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.017+0000 7ff37dd9b640 1 -- 172.21.15.64:0/2041289720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcab6d2410 msgr2=0x55bcab6dc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.017+0000 7ff37dd9b640 1 --2- 172.21.15.64:0/2041289720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcab6d2410 0x55bcab6dc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.017+0000 7ff37dd9b640 1 -- 172.21.15.64:0/2041289720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcab6d1af0 msgr2=0x55bcab6d1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.017+0000 7ff37dd9b640 1 --2- 172.21.15.64:0/2041289720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcab6d1af0 0x55bcab6d1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.017+0000 7ff37dd9b640 1 -- 172.21.15.64:0/2041289720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcab5dd340 con 0x55bcab6dccb0 2024-09-12T03:30:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.017+0000 7ff37dd9b640 1 --2- 172.21.15.64:0/2041289720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcab6dccb0 0x55bcab6df0a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff3740127c0 tx=0x7ff37400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0200e2570845b41 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.018+0000 7ff36cff9640 1 -- 172.21.15.64:0/2041289720 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff374012e00 con 0x55bcab6dccb0 2024-09-12T03:30:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.018+0000 7ff37ed9d640 1 --2- 172.21.15.64:0/692496205 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7ff3680a2ec0 0x7ff3680a5380 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff3780ccd90 tx=0x7ff3780cfe30 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.018+0000 7ff36cff9640 1 -- 172.21.15.64:0/2041289720 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff374010c20 con 0x55bcab6dccb0 2024-09-12T03:30:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.018+0000 7ff36cff9640 1 -- 172.21.15.64:0/2041289720 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37401ea70 con 0x55bcab6dccb0 2024-09-12T03:30:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.018+0000 7ff387cd3740 1 -- 172.21.15.64:0/2041289720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcab6dccb0 msgr2=0x55bcab6df0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.018+0000 7ff387cd3740 1 --2- 172.21.15.64:0/2041289720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcab6dccb0 0x55bcab6df0a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff3740127c0 tx=0x7ff37400b7c0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.018+0000 7ff387cd3740 1 -- 172.21.15.64:0/2041289720 shutdown_connections 2024-09-12T03:30:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.018+0000 7ff387cd3740 1 --2- 172.21.15.64:0/2041289720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcab6dccb0 0x55bcab6df0a0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.018+0000 7ff387cd3740 1 --2- 172.21.15.64:0/2041289720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcab6d2410 0x55bcab6dc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.018+0000 7ff387cd3740 1 --2- 172.21.15.64:0/2041289720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcab6d1af0 0x55bcab6d1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.018+0000 7ff387cd3740 1 -- 172.21.15.64:0/2041289720 >> 172.21.15.64:0/2041289720 conn(0x55bcab6ce060 msgr2=0x55bcab6ce440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.019+0000 7ff387cd3740 1 -- 172.21.15.64:0/2041289720 shutdown_connections 2024-09-12T03:30:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.019+0000 7ff387cd3740 1 -- 172.21.15.64:0/2041289720 wait complete. 2024-09-12T03:30:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.019+0000 7ff387cd3740 1 -- start start 2024-09-12T03:30:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.019+0000 7ff387cd3740 1 Processor -- start 2024-09-12T03:30:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.019+0000 7ff387cd3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcab6d1af0 0x55bcab6d9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.019+0000 7ff387cd3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcab6d2410 0x55bcab6da0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.019+0000 7ff387cd3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcab6dccb0 0x55bcab6da620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.019+0000 7ff387cd3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bcab6df880 con 0x55bcab6dccb0 2024-09-12T03:30:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.019+0000 7ff387cd3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55bcab5bf290 con 0x55bcab6d1af0 2024-09-12T03:30:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.019+0000 7ff387cd3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55bcab6dfa00 con 0x55bcab6d2410 2024-09-12T03:30:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.019+0000 7ff387cd3740 10 client.0 _ll_register_callbacks cb 0x55bcab5f0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:37.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.019+0000 7ff37dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcab6dccb0 0x55bcab6da620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:37.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.019+0000 7ff37dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcab6dccb0 0x55bcab6da620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59212/0 (socket says 172.21.15.64:59212) 2024-09-12T03:30:37.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.019+0000 7ff37dd9b640 1 -- 172.21.15.64:0/2710877316 learned_addr learned my addr 172.21.15.64:0/2710877316 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:30:37.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.020+0000 7ff37e59c640 1 --2- 172.21.15.64:0/2710877316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcab6d2410 0x55bcab6da0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:37.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.020+0000 7ff37dd9b640 1 --2- 172.21.15.64:0/2710877316 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcab6d1af0 0x55bcab6d9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:37.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.020+0000 7ff37dd9b640 1 -- 172.21.15.64:0/2710877316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcab6d2410 msgr2=0x55bcab6da0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:37.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.020+0000 7ff37dd9b640 1 --2- 172.21.15.64:0/2710877316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcab6d2410 0x55bcab6da0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.020+0000 7ff37dd9b640 1 -- 172.21.15.64:0/2710877316 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcab6d1af0 msgr2=0x55bcab6d9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:37.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.020+0000 7ff37dd9b640 1 --2- 172.21.15.64:0/2710877316 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcab6d1af0 0x55bcab6d9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.020+0000 7ff37dd9b640 1 -- 172.21.15.64:0/2710877316 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcab349210 con 0x55bcab6dccb0 2024-09-12T03:30:37.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.020+0000 7ff37dd9b640 1 --2- 172.21.15.64:0/2710877316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcab6dccb0 0x55bcab6da620 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff37401e6b0 tx=0x7ff374005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca4cd983cd888a82 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:37.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.020+0000 7ff365ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:30:37.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.020+0000 7ff365ffb640 1 -- 172.21.15.64:0/2710877316 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff374012e00 con 0x55bcab6dccb0 2024-09-12T03:30:37.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.020+0000 7ff365ffb640 1 -- 172.21.15.64:0/2710877316 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7ff374010770 con 0x55bcab6dccb0 2024-09-12T03:30:37.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.020+0000 7ff387cd3740 1 -- 172.21.15.64:0/2710877316 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff374010c20 con 0x55bcab6dccb0 2024-09-12T03:30:37.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.020+0000 7ff365ffb640 1 -- 172.21.15.64:0/2710877316 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff374010970 con 0x55bcab6dccb0 2024-09-12T03:30:37.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.020+0000 7ff387cd3740 10 client.15450 fetch_fsmap learned FSMap version 25 2024-09-12T03:30:37.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.020+0000 7ff387cd3740 1 -- 172.21.15.64:0/2710877316 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bcab415620 con 0x55bcab6dccb0 2024-09-12T03:30:37.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.021+0000 7ff365ffb640 1 -- 172.21.15.64:0/2710877316 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff374010c20 con 0x55bcab6dccb0 2024-09-12T03:30:37.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.021+0000 7ff365ffb640 1 -- 172.21.15.64:0/2710877316 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff374027030 con 0x55bcab6dccb0 2024-09-12T03:30:37.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.021+0000 7ff387cd3740 10 client.15450 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:30:37.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.021+0000 7ff387cd3740 10 client.15450 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:30:37.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.021+0000 7ff387cd3740 20 client.15450 populate_metadata read hostname 'smithi064' 2024-09-12T03:30:37.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.021+0000 7ff387cd3740 1 --2- 172.21.15.64:0/2710877316 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55bcab6ecb70 0x55bcab6eef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:37.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.021+0000 7ff387cd3740 4 client.15450 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:30:37.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.021+0000 7ff387cd3740 1 -- 172.21.15.64:0/2710877316 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcab643a20 con 0x55bcab6ecb70 2024-09-12T03:30:37.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.021+0000 7ff37e59c640 1 --2- 172.21.15.64:0/2710877316 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55bcab6ecb70 0x55bcab6eef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:37.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.022+0000 7ff37e59c640 1 --2- 172.21.15.64:0/2710877316 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55bcab6ecb70 0x55bcab6eef60 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d7eee1b7a8b538b server_cookie=7282891ce5e73716 in_seq=0 out_seq=0 2024-09-12T03:30:37.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.022+0000 7ff365ffb640 10 client.15450 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:30:37.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.022+0000 7ff365ffb640 1 -- 172.21.15.64:0/2710877316 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bcab643a20 con 0x55bcab6ecb70 2024-09-12T03:30:37.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.022+0000 7ff365ffb640 10 client.15450 handle_command_reply: tid=0 2024-09-12T03:30:37.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.022+0000 7ff387cd3740 1 client.15450 shutdown 2024-09-12T03:30:37.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.022+0000 7ff387cd3740 1 -- 172.21.15.64:0/2710877316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcab6dccb0 msgr2=0x55bcab6da620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:37.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.022+0000 7ff387cd3740 1 --2- 172.21.15.64:0/2710877316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcab6dccb0 0x55bcab6da620 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff37401e6b0 tx=0x7ff374005850 comp rx=0 tx=0).stop 2024-09-12T03:30:37.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.022+0000 7ff387cd3740 1 -- 172.21.15.64:0/2710877316 shutdown_connections 2024-09-12T03:30:37.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.022+0000 7ff387cd3740 1 --2- 172.21.15.64:0/2710877316 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55bcab6ecb70 0x55bcab6eef60 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.022+0000 7ff387cd3740 1 --2- 172.21.15.64:0/2710877316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcab6dccb0 0x55bcab6da620 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.022+0000 7ff387cd3740 1 --2- 172.21.15.64:0/2710877316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcab6d2410 0x55bcab6da0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.022+0000 7ff387cd3740 1 --2- 172.21.15.64:0/2710877316 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcab6d1af0 0x55bcab6d9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.022+0000 7ff387cd3740 1 -- 172.21.15.64:0/2710877316 >> 172.21.15.64:0/2710877316 conn(0x55bcab6ce060 msgr2=0x55bcab6d3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:37.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.022+0000 7ff365ffb640 0 client.15450 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:30:37.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.023+0000 7ff387cd3740 1 -- 172.21.15.64:0/2710877316 shutdown_connections 2024-09-12T03:30:37.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.023+0000 7ff387cd3740 1 -- 172.21.15.64:0/2710877316 wait complete. 2024-09-12T03:30:37.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.023+0000 7ff387cd3740 20 client.15450 trim_cache size 0 max 16384 2024-09-12T03:30:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.037+0000 7ff387cd3740 1 Processor -- start 2024-09-12T03:30:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.037+0000 7ff387cd3740 1 -- start start 2024-09-12T03:30:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.037+0000 7ff387cd3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcab6d1af0 0x55bcab6d8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.037+0000 7ff387cd3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcab6d2410 0x55bcab6da450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.037+0000 7ff387cd3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcab6dccb0 0x55bcab6f4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.037+0000 7ff387cd3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff3801b18c0 con 0x55bcab6d2410 2024-09-12T03:30:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.037+0000 7ff387cd3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff3801af190 con 0x55bcab6d1af0 2024-09-12T03:30:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.037+0000 7ff387cd3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff3801acfe0 con 0x55bcab6dccb0 2024-09-12T03:30:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.038+0000 7ff37e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcab6d2410 0x55bcab6da450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.038+0000 7ff37e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcab6d2410 0x55bcab6da450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59216/0 (socket says 172.21.15.64:59216) 2024-09-12T03:30:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.038+0000 7ff37e59c640 1 -- 172.21.15.64:0/1594601811 learned_addr learned my addr 172.21.15.64:0/1594601811 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:30:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.038+0000 7ff37dd9b640 1 --2- 172.21.15.64:0/1594601811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcab6d1af0 0x55bcab6d8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.038+0000 7ff37dd9b640 1 --2- 172.21.15.64:0/1594601811 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcab6dccb0 0x55bcab6f4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.038+0000 7ff37e59c640 1 -- 172.21.15.64:0/1594601811 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcab6dccb0 msgr2=0x55bcab6f4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.038+0000 7ff37e59c640 1 --2- 172.21.15.64:0/1594601811 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcab6dccb0 0x55bcab6f4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.038+0000 7ff37e59c640 1 -- 172.21.15.64:0/1594601811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcab6d1af0 msgr2=0x55bcab6d8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.038+0000 7ff37e59c640 1 --2- 172.21.15.64:0/1594601811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcab6d1af0 0x55bcab6d8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.038+0000 7ff37e59c640 1 -- 172.21.15.64:0/1594601811 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcab6d2240 con 0x55bcab6d2410 2024-09-12T03:30:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.038+0000 7ff37dd9b640 1 --2- 172.21.15.64:0/1594601811 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcab6dccb0 0x55bcab6f4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:30:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.038+0000 7ff37e59c640 1 --2- 172.21.15.64:0/1594601811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcab6d2410 0x55bcab6da450 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff37002a9a0 tx=0x7ff370005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd26a16ef1c9464c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.038+0000 7ff367fff640 1 -- 172.21.15.64:0/1594601811 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37004b070 con 0x55bcab6d2410 2024-09-12T03:30:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.038+0000 7ff367fff640 1 -- 172.21.15.64:0/1594601811 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3700054f0 con 0x55bcab6d2410 2024-09-12T03:30:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.038+0000 7ff367fff640 1 -- 172.21.15.64:0/1594601811 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff370035040 con 0x55bcab6d2410 2024-09-12T03:30:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.039+0000 7ff387cd3740 1 -- 172.21.15.64:0/1594601811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcab6d2410 msgr2=0x55bcab6da450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.039+0000 7ff387cd3740 1 --2- 172.21.15.64:0/1594601811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcab6d2410 0x55bcab6da450 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff37002a9a0 tx=0x7ff370005c00 comp rx=0 tx=0).stop 2024-09-12T03:30:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.039+0000 7ff387cd3740 1 -- 172.21.15.64:0/1594601811 shutdown_connections 2024-09-12T03:30:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.039+0000 7ff387cd3740 1 --2- 172.21.15.64:0/1594601811 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcab6dccb0 0x55bcab6f4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.039+0000 7ff387cd3740 1 --2- 172.21.15.64:0/1594601811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcab6d2410 0x55bcab6da450 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.039+0000 7ff387cd3740 1 --2- 172.21.15.64:0/1594601811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcab6d1af0 0x55bcab6d8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.039+0000 7ff387cd3740 1 -- 172.21.15.64:0/1594601811 >> 172.21.15.64:0/1594601811 conn(0x55bcab6ce060 msgr2=0x55bcab6dbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.039+0000 7ff387cd3740 1 -- 172.21.15.64:0/1594601811 shutdown_connections 2024-09-12T03:30:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.039+0000 7ff387cd3740 1 -- 172.21.15.64:0/1594601811 wait complete. 2024-09-12T03:30:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.039+0000 7ff387cd3740 1 -- start start 2024-09-12T03:30:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.039+0000 7ff387cd3740 1 Processor -- start 2024-09-12T03:30:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.040+0000 7ff387cd3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcab6d1af0 0x55bcab6e0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.040+0000 7ff387cd3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcab6d2410 0x55bcab6e0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.040+0000 7ff387cd3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcab6dccb0 0x55bcab74f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.040+0000 7ff387cd3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff3801ad650 con 0x55bcab6d1af0 2024-09-12T03:30:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.040+0000 7ff387cd3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff3801aeac0 con 0x55bcab6d2410 2024-09-12T03:30:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.040+0000 7ff387cd3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff3801add20 con 0x55bcab6dccb0 2024-09-12T03:30:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.040+0000 7ff387cd3740 10 client.0 _ll_register_callbacks cb 0x7ff37400be90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.040+0000 7ff37dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcab6d1af0 0x55bcab6e0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.040+0000 7ff37dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcab6d1af0 0x55bcab6e0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59222/0 (socket says 172.21.15.64:59222) 2024-09-12T03:30:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.040+0000 7ff37dd9b640 1 -- 172.21.15.64:0/3425159055 learned_addr learned my addr 172.21.15.64:0/3425159055 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:30:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.040+0000 7ff37e59c640 1 --2- 172.21.15.64:0/3425159055 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcab6d2410 0x55bcab6e0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.040+0000 7ff37dd9b640 1 -- 172.21.15.64:0/3425159055 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcab6dccb0 msgr2=0x55bcab74f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.040+0000 7ff37dd9b640 1 --2- 172.21.15.64:0/3425159055 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcab6dccb0 0x55bcab74f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.040+0000 7ff37dd9b640 1 -- 172.21.15.64:0/3425159055 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcab6d2410 msgr2=0x55bcab6e0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.040+0000 7ff37dd9b640 1 --2- 172.21.15.64:0/3425159055 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcab6d2410 0x55bcab6e0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.040+0000 7ff37dd9b640 1 -- 172.21.15.64:0/3425159055 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcab74fcb0 con 0x55bcab6d1af0 2024-09-12T03:30:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.040+0000 7ff37dd9b640 1 --2- 172.21.15.64:0/3425159055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcab6d1af0 0x55bcab6e0430 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff37401e470 tx=0x7ff37400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6755c15cc0bdec7a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.040+0000 7ff365ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:30:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.040+0000 7ff365ffb640 1 -- 172.21.15.64:0/3425159055 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37400b7c0 con 0x55bcab6d1af0 2024-09-12T03:30:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.040+0000 7ff365ffb640 1 -- 172.21.15.64:0/3425159055 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7ff374024040 con 0x55bcab6d1af0 2024-09-12T03:30:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.040+0000 7ff387cd3740 1 -- 172.21.15.64:0/3425159055 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3700054f0 con 0x55bcab6d1af0 2024-09-12T03:30:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.040+0000 7ff365ffb640 1 -- 172.21.15.64:0/3425159055 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff37400bab0 con 0x55bcab6d1af0 2024-09-12T03:30:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.040+0000 7ff387cd3740 10 client.15456 fetch_fsmap learned FSMap version 25 2024-09-12T03:30:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.041+0000 7ff387cd3740 1 -- 172.21.15.64:0/3425159055 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bcab757020 con 0x55bcab6d1af0 2024-09-12T03:30:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.041+0000 7ff365ffb640 1 -- 172.21.15.64:0/3425159055 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3700054f0 con 0x55bcab6d1af0 2024-09-12T03:30:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.041+0000 7ff365ffb640 1 -- 172.21.15.64:0/3425159055 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff374027040 con 0x55bcab6d1af0 2024-09-12T03:30:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.041+0000 7ff387cd3740 10 client.15456 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:30:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.041+0000 7ff387cd3740 10 client.15456 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:30:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.041+0000 7ff387cd3740 20 client.15456 populate_metadata read hostname 'smithi064' 2024-09-12T03:30:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.041+0000 7ff387cd3740 1 --2- 172.21.15.64:0/3425159055 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55bcab757710 0x55bcab759b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.041+0000 7ff387cd3740 4 client.15456 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:30:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.041+0000 7ff387cd3740 1 -- 172.21.15.64:0/3425159055 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bcab63dd70 con 0x55bcab757710 2024-09-12T03:30:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.041+0000 7ff37e59c640 1 --2- 172.21.15.64:0/3425159055 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55bcab757710 0x55bcab759b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.042+0000 7ff37e59c640 1 --2- 172.21.15.64:0/3425159055 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55bcab757710 0x55bcab759b30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31c6a445097b814d server_cookie=30d21dcc21fcb658 in_seq=0 out_seq=0 2024-09-12T03:30:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.042+0000 7ff365ffb640 10 client.15456 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:30:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.053+0000 7ff365ffb640 1 -- 172.21.15.64:0/3425159055 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bcab63dd70 con 0x55bcab757710 2024-09-12T03:30:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.053+0000 7ff365ffb640 10 client.15456 handle_command_reply: tid=0 2024-09-12T03:30:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.053+0000 7ff387cd3740 1 client.15456 shutdown 2024-09-12T03:30:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.054+0000 7ff387cd3740 1 -- 172.21.15.64:0/3425159055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcab6d1af0 msgr2=0x55bcab6e0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.054+0000 7ff387cd3740 1 --2- 172.21.15.64:0/3425159055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcab6d1af0 0x55bcab6e0430 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff37401e470 tx=0x7ff37400bd70 comp rx=0 tx=0).stop 2024-09-12T03:30:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.054+0000 7ff387cd3740 1 -- 172.21.15.64:0/3425159055 shutdown_connections 2024-09-12T03:30:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.054+0000 7ff387cd3740 1 --2- 172.21.15.64:0/3425159055 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55bcab757710 0x55bcab759b30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.054+0000 7ff387cd3740 1 --2- 172.21.15.64:0/3425159055 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcab6dccb0 0x55bcab74f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.054+0000 7ff387cd3740 1 --2- 172.21.15.64:0/3425159055 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcab6d2410 0x55bcab6e0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.054+0000 7ff365ffb640 0 client.15456 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:30:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.054+0000 7ff387cd3740 1 --2- 172.21.15.64:0/3425159055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcab6d1af0 0x55bcab6e0430 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.054+0000 7ff387cd3740 1 -- 172.21.15.64:0/3425159055 >> 172.21.15.64:0/3425159055 conn(0x55bcab6ce060 msgr2=0x55bcab6d3550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.054+0000 7ff387cd3740 1 -- 172.21.15.64:0/3425159055 shutdown_connections 2024-09-12T03:30:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.054+0000 7ff387cd3740 1 -- 172.21.15.64:0/3425159055 wait complete. 2024-09-12T03:30:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.054+0000 7ff387cd3740 20 client.15456 trim_cache size 0 max 16384 2024-09-12T03:30:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:30:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:30:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:30:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:30:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.054+0000 7ff384f31640 1 -- 172.21.15.64:0/692496205 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7ff3680a2ec0 msgr2=0x7ff3680a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.054+0000 7ff384f31640 1 --2- 172.21.15.64:0/692496205 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7ff3680a2ec0 0x7ff3680a5380 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff3780ccd90 tx=0x7ff3780cfe30 comp rx=0 tx=0).stop 2024-09-12T03:30:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.054+0000 7ff384f31640 1 -- 172.21.15.64:0/692496205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff38010c0c0 msgr2=0x7ff38019cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.054+0000 7ff384f31640 1 --2- 172.21.15.64:0/692496205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff38010c0c0 0x7ff38019cb40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff37800eaa0 tx=0x7ff37800ef60 comp rx=0 tx=0).stop 2024-09-12T03:30:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.054+0000 7ff384f31640 1 -- 172.21.15.64:0/692496205 shutdown_connections 2024-09-12T03:30:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.054+0000 7ff384f31640 1 --2- 172.21.15.64:0/692496205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff38010c0c0 0x7ff38019cb40 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.054+0000 7ff384f31640 1 --2- 172.21.15.64:0/692496205 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7ff3680a2ec0 0x7ff3680a5380 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.054+0000 7ff384f31640 1 --2- 172.21.15.64:0/692496205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3801035c0 0x7ff3801987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.054+0000 7ff384f31640 1 --2- 172.21.15.64:0/692496205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff380102ca0 0x7ff380198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.054+0000 7ff384f31640 1 -- 172.21.15.64:0/692496205 >> 172.21.15.64:0/692496205 conn(0x7ff3800fc6f0 msgr2=0x7ff3800fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.055+0000 7ff384f31640 1 -- 172.21.15.64:0/692496205 shutdown_connections 2024-09-12T03:30:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:37.055+0000 7ff384f31640 1 -- 172.21.15.64:0/692496205 wait complete. 2024-09-12T03:30:37.521 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:30:37 smithi064 ceph-mon[30952]: pgmap v43: 129 pgs: 129 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T03:30:37.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:30:37 smithi115 ceph-mon[34801]: pgmap v43: 129 pgs: 129 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T03:30:37.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:30:37 smithi195 ceph-mon[34786]: pgmap v43: 129 pgs: 129 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T03:30:39.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:30:39 smithi064 ceph-mon[30952]: pgmap v44: 129 pgs: 129 active+clean; 1.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 18 op/s 2024-09-12T03:30:39.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:30:39 smithi115 ceph-mon[34801]: pgmap v44: 129 pgs: 129 active+clean; 1.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 18 op/s 2024-09-12T03:30:39.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:30:39 smithi195 ceph-mon[34786]: pgmap v44: 129 pgs: 129 active+clean; 1.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 18 op/s 2024-09-12T03:30:41.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:30:41 smithi064 ceph-mon[30952]: pgmap v45: 129 pgs: 129 active+clean; 1.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-09-12T03:30:41.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:30:41 smithi115 ceph-mon[34801]: pgmap v45: 129 pgs: 129 active+clean; 1.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-09-12T03:30:41.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:30:41 smithi195 ceph-mon[34786]: pgmap v45: 129 pgs: 129 active+clean; 1.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-09-12T03:30:43.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:30:43 smithi064 ceph-mon[30952]: pgmap v46: 129 pgs: 129 active+clean; 1.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-09-12T03:30:43.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:30:43 smithi115 ceph-mon[34801]: pgmap v46: 129 pgs: 129 active+clean; 1.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-09-12T03:30:43.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:30:43 smithi195 ceph-mon[34786]: pgmap v46: 129 pgs: 129 active+clean; 1.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-09-12T03:30:45.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:30:45 smithi064 ceph-mon[30952]: pgmap v47: 129 pgs: 129 active+clean; 1.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-09-12T03:30:45.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:30:45 smithi115 ceph-mon[34801]: pgmap v47: 129 pgs: 129 active+clean; 1.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-09-12T03:30:45.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:30:45 smithi195 ceph-mon[34786]: pgmap v47: 129 pgs: 129 active+clean; 1.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-09-12T03:30:47.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:30:47 smithi064 ceph-mon[30952]: pgmap v48: 129 pgs: 129 active+clean; 1.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-09-12T03:30:47.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:30:47 smithi115 ceph-mon[34801]: pgmap v48: 129 pgs: 129 active+clean; 1.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-09-12T03:30:47.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:30:47 smithi195 ceph-mon[34786]: pgmap v48: 129 pgs: 129 active+clean; 1.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-09-12T03:30:49.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:30:49 smithi064 ceph-mon[30952]: pgmap v49: 129 pgs: 129 active+clean; 1.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 19 op/s 2024-09-12T03:30:49.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:30:49 smithi064 ceph-mon[30952]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-09-12T03:30:49.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:30:49 smithi115 ceph-mon[34801]: pgmap v49: 129 pgs: 129 active+clean; 1.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 19 op/s 2024-09-12T03:30:49.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:30:49 smithi115 ceph-mon[34801]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-09-12T03:30:49.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:30:49 smithi195 ceph-mon[34786]: pgmap v49: 129 pgs: 129 active+clean; 1.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 19 op/s 2024-09-12T03:30:49.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:30:49 smithi195 ceph-mon[34786]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-09-12T03:30:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.153+0000 7f202f445640 1 -- 172.21.15.64:0/321688512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f202810c110 msgr2=0x7f202810e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.153+0000 7f202f445640 1 --2- 172.21.15.64:0/321688512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f202810c110 0x7f202810e590 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2020009900 tx=0x7f202002a480 comp rx=0 tx=0).stop 2024-09-12T03:30:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.153+0000 7f202f445640 1 -- 172.21.15.64:0/321688512 shutdown_connections 2024-09-12T03:30:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.153+0000 7f202f445640 1 --2- 172.21.15.64:0/321688512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f202810c110 0x7f202810e590 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.153+0000 7f202f445640 1 --2- 172.21.15.64:0/321688512 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20281035e0 0x7f202810bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.153+0000 7f202f445640 1 --2- 172.21.15.64:0/321688512 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2028102cc0 0x7f20281030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.153+0000 7f202f445640 1 -- 172.21.15.64:0/321688512 >> 172.21.15.64:0/321688512 conn(0x7f20280fc6b0 msgr2=0x7f20280fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.154+0000 7f202f445640 1 -- 172.21.15.64:0/321688512 shutdown_connections 2024-09-12T03:30:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.154+0000 7f202f445640 1 -- 172.21.15.64:0/321688512 wait complete. 2024-09-12T03:30:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.154+0000 7f202f445640 1 Processor -- start 2024-09-12T03:30:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.154+0000 7f202f445640 1 -- start start 2024-09-12T03:30:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.154+0000 7f202f445640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2028102cc0 0x7f20281981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.154+0000 7f202f445640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20281035e0 0x7f2028198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.154+0000 7f202f445640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f202810c110 0x7f202819ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.154+0000 7f202f445640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2028111da0 con 0x7f202810c110 2024-09-12T03:30:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.154+0000 7f202f445640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2028111c20 con 0x7f20281035e0 2024-09-12T03:30:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.154+0000 7f202f445640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2028111f20 con 0x7f2028102cc0 2024-09-12T03:30:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.155+0000 7f202d9e2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f202810c110 0x7f202819ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.155+0000 7f202d9e2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f202810c110 0x7f202819ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48804/0 (socket says 172.21.15.64:48804) 2024-09-12T03:30:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.155+0000 7f202d9e2640 1 -- 172.21.15.64:0/4199947401 learned_addr learned my addr 172.21.15.64:0/4199947401 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:30:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.155+0000 7f202c9e0640 1 --2- 172.21.15.64:0/4199947401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20281035e0 0x7f2028198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.155+0000 7f202d1e1640 1 --2- 172.21.15.64:0/4199947401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2028102cc0 0x7f20281981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.155+0000 7f202d9e2640 1 -- 172.21.15.64:0/4199947401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2028102cc0 msgr2=0x7f20281981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.155+0000 7f202d9e2640 1 --2- 172.21.15.64:0/4199947401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2028102cc0 0x7f20281981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.155+0000 7f202d9e2640 1 -- 172.21.15.64:0/4199947401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20281035e0 msgr2=0x7f2028198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.155+0000 7f202d9e2640 1 --2- 172.21.15.64:0/4199947401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20281035e0 0x7f2028198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.155+0000 7f202d9e2640 1 -- 172.21.15.64:0/4199947401 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f202819d1c0 con 0x7f202810c110 2024-09-12T03:30:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.155+0000 7f202d1e1640 1 --2- 172.21.15.64:0/4199947401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2028102cc0 0x7f20281981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:30:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.155+0000 7f202d9e2640 1 --2- 172.21.15.64:0/4199947401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f202810c110 0x7f202819ca90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2020009a20 tx=0x7f20200042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.155+0000 7f20267fc640 1 -- 172.21.15.64:0/4199947401 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202003f030 con 0x7f202810c110 2024-09-12T03:30:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.155+0000 7f20267fc640 1 -- 172.21.15.64:0/4199947401 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20200044d0 con 0x7f202810c110 2024-09-12T03:30:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.155+0000 7f202f445640 1 -- 172.21.15.64:0/4199947401 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f202819d450 con 0x7f202810c110 2024-09-12T03:30:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.155+0000 7f20267fc640 1 -- 172.21.15.64:0/4199947401 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2020035070 con 0x7f202810c110 2024-09-12T03:30:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.155+0000 7f202f445640 1 -- 172.21.15.64:0/4199947401 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20281a4d40 con 0x7f202810c110 2024-09-12T03:30:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.158+0000 7f20267fc640 1 -- 172.21.15.64:0/4199947401 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f2020042030 con 0x7f202810c110 2024-09-12T03:30:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.158+0000 7f20321d9740 1 Processor -- start 2024-09-12T03:30:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.158+0000 7f20321d9740 1 -- start start 2024-09-12T03:30:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.159+0000 7f20321d9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a21f51af0 0x564a21f51ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.159+0000 7f20321d9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a21f52410 0x564a21f5c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.159+0000 7f20321d9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a21f5ccb0 0x564a21f5f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.159+0000 7f20321d9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564a21eb62f0 con 0x564a21f5ccb0 2024-09-12T03:30:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.159+0000 7f20321d9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564a21e5e270 con 0x564a21f52410 2024-09-12T03:30:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.159+0000 7f20321d9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564a21eb22b0 con 0x564a21f51af0 2024-09-12T03:30:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.159+0000 7f20267fc640 1 --2- 172.21.15.64:0/4199947401 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f20100a2da0 0x7f20100a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.159+0000 7f202c9e0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a21f5ccb0 0x564a21f5f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.159+0000 7f202c9e0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a21f5ccb0 0x564a21f5f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48820/0 (socket says 172.21.15.64:48820) 2024-09-12T03:30:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.159+0000 7f202c9e0640 1 -- 172.21.15.64:0/1798916297 learned_addr learned my addr 172.21.15.64:0/1798916297 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:30:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.159+0000 7f202d9e2640 1 --2- 172.21.15.64:0/4199947401 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f20100a2da0 0x7f20100a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.159+0000 7f202d1e1640 1 --2- 172.21.15.64:0/1798916297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a21f52410 0x564a21f5c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.159+0000 7f202c9e0640 1 --2- 172.21.15.64:0/1798916297 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a21f51af0 0x564a21f51ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.159+0000 7f20267fc640 1 -- 172.21.15.64:0/4199947401 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f20200f0210 con 0x7f202810c110 2024-09-12T03:30:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.159+0000 7f202c9e0640 1 -- 172.21.15.64:0/1798916297 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a21f51af0 msgr2=0x564a21f51ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.159+0000 7f202c9e0640 1 --2- 172.21.15.64:0/1798916297 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a21f51af0 0x564a21f51ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.159+0000 7f202c9e0640 1 -- 172.21.15.64:0/1798916297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a21f52410 msgr2=0x564a21f5c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.159+0000 7f202c9e0640 1 --2- 172.21.15.64:0/1798916297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a21f52410 0x564a21f5c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.159+0000 7f202c9e0640 1 -- 172.21.15.64:0/1798916297 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a21e5d340 con 0x564a21f5ccb0 2024-09-12T03:30:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.159+0000 7f202c9e0640 1 --2- 172.21.15.64:0/1798916297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a21f5ccb0 0x564a21f5f0a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f201c005e10 tx=0x7f201c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c7a2650dcec3dee server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.159+0000 7f202d9e2640 1 --2- 172.21.15.64:0/4199947401 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f20100a2da0 0x7f20100a5260 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f20200bc060 tx=0x7f202002a480 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.160+0000 7f200f7fe640 1 -- 172.21.15.64:0/1798916297 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201c015850 con 0x564a21f5ccb0 2024-09-12T03:30:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.160+0000 7f200f7fe640 1 -- 172.21.15.64:0/1798916297 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f201c017070 con 0x564a21f5ccb0 2024-09-12T03:30:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.160+0000 7f200f7fe640 1 -- 172.21.15.64:0/1798916297 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201c010670 con 0x564a21f5ccb0 2024-09-12T03:30:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.160+0000 7f20321d9740 1 -- 172.21.15.64:0/1798916297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a21f5ccb0 msgr2=0x564a21f5f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.160+0000 7f20321d9740 1 --2- 172.21.15.64:0/1798916297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a21f5ccb0 0x564a21f5f0a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f201c005e10 tx=0x7f201c00b4a0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.160+0000 7f20321d9740 1 -- 172.21.15.64:0/1798916297 shutdown_connections 2024-09-12T03:30:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.160+0000 7f20321d9740 1 --2- 172.21.15.64:0/1798916297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a21f5ccb0 0x564a21f5f0a0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.160+0000 7f20321d9740 1 --2- 172.21.15.64:0/1798916297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a21f52410 0x564a21f5c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.160+0000 7f20321d9740 1 --2- 172.21.15.64:0/1798916297 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a21f51af0 0x564a21f51ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.160+0000 7f20321d9740 1 -- 172.21.15.64:0/1798916297 >> 172.21.15.64:0/1798916297 conn(0x564a21f4e060 msgr2=0x564a21f4e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.160+0000 7f20321d9740 1 -- 172.21.15.64:0/1798916297 shutdown_connections 2024-09-12T03:30:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.161+0000 7f20321d9740 1 -- 172.21.15.64:0/1798916297 wait complete. 2024-09-12T03:30:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.161+0000 7f20321d9740 1 -- start start 2024-09-12T03:30:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.161+0000 7f20321d9740 1 Processor -- start 2024-09-12T03:30:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.161+0000 7f20321d9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a21f51af0 0x564a21f59bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.161+0000 7f20321d9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a21f52410 0x564a21f5a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.161+0000 7f20321d9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a21f5ccb0 0x564a21f5a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.161+0000 7f20321d9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564a21f5f880 con 0x564a21f52410 2024-09-12T03:30:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.161+0000 7f20321d9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564a21e3f290 con 0x564a21f5ccb0 2024-09-12T03:30:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.161+0000 7f20321d9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564a21f5fa00 con 0x564a21f51af0 2024-09-12T03:30:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.161+0000 7f20321d9740 10 client.0 _ll_register_callbacks cb 0x564a21e70140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.162+0000 7f202d1e1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a21f52410 0x564a21f5a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.162+0000 7f202d1e1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a21f52410 0x564a21f5a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48836/0 (socket says 172.21.15.64:48836) 2024-09-12T03:30:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.162+0000 7f202d1e1640 1 -- 172.21.15.64:0/3571156112 learned_addr learned my addr 172.21.15.64:0/3571156112 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:30:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.162+0000 7f202c9e0640 1 --2- 172.21.15.64:0/3571156112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a21f5ccb0 0x564a21f5a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.162+0000 7f202c9e0640 1 --2- 172.21.15.64:0/3571156112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a21f51af0 0x564a21f59bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.162+0000 7f202d1e1640 1 -- 172.21.15.64:0/3571156112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a21f51af0 msgr2=0x564a21f59bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.162+0000 7f202d1e1640 1 --2- 172.21.15.64:0/3571156112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a21f51af0 0x564a21f59bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.162+0000 7f202d1e1640 1 -- 172.21.15.64:0/3571156112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a21f5ccb0 msgr2=0x564a21f5a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.162+0000 7f202d1e1640 1 --2- 172.21.15.64:0/3571156112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a21f5ccb0 0x564a21f5a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.162+0000 7f202d1e1640 1 -- 172.21.15.64:0/3571156112 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a21bc9210 con 0x564a21f52410 2024-09-12T03:30:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.162+0000 7f202c9e0640 1 --2- 172.21.15.64:0/3571156112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a21f51af0 0x564a21f59bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:30:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.162+0000 7f202d1e1640 1 --2- 172.21.15.64:0/3571156112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a21f52410 0x564a21f5a0f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2018010b70 tx=0x7f201800e780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=919c9f25bcb8d3e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.162+0000 7f200cff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:30:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.162+0000 7f200cff9640 1 -- 172.21.15.64:0/3571156112 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201800b7b0 con 0x564a21f52410 2024-09-12T03:30:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.162+0000 7f200cff9640 1 -- 172.21.15.64:0/3571156112 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f201800b950 con 0x564a21f52410 2024-09-12T03:30:50.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.162+0000 7f20321d9740 1 -- 172.21.15.64:0/3571156112 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f201c017070 con 0x564a21f52410 2024-09-12T03:30:50.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.162+0000 7f200cff9640 1 -- 172.21.15.64:0/3571156112 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f201800bb50 con 0x564a21f52410 2024-09-12T03:30:50.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.162+0000 7f20321d9740 10 client.15468 fetch_fsmap learned FSMap version 25 2024-09-12T03:30:50.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.162+0000 7f20321d9740 1 -- 172.21.15.64:0/3571156112 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564a21c95620 con 0x564a21f52410 2024-09-12T03:30:50.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.163+0000 7f200cff9640 1 -- 172.21.15.64:0/3571156112 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f201c017070 con 0x564a21f52410 2024-09-12T03:30:50.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.163+0000 7f200cff9640 1 -- 172.21.15.64:0/3571156112 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2018017070 con 0x564a21f52410 2024-09-12T03:30:50.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.163+0000 7f20321d9740 10 client.15468 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:30:50.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.163+0000 7f20321d9740 10 client.15468 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:30:50.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.163+0000 7f20321d9740 20 client.15468 populate_metadata read hostname 'smithi064' 2024-09-12T03:30:50.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.163+0000 7f20321d9740 1 --2- 172.21.15.64:0/3571156112 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x564a21f6cd70 0x564a21f6f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:50.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.163+0000 7f20321d9740 4 client.15468 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:30:50.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.163+0000 7f20321d9740 1 -- 172.21.15.64:0/3571156112 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a21ec3a20 con 0x564a21f6cd70 2024-09-12T03:30:50.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.163+0000 7f202c9e0640 1 --2- 172.21.15.64:0/3571156112 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x564a21f6cd70 0x564a21f6f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:50.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.164+0000 7f200cff9640 10 client.15468 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:30:50.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.164+0000 7f202c9e0640 1 --2- 172.21.15.64:0/3571156112 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x564a21f6cd70 0x564a21f6f160 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3c693bc7502df2c server_cookie=608fa36fea28f805 in_seq=0 out_seq=0 2024-09-12T03:30:50.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.164+0000 7f200cff9640 1 -- 172.21.15.64:0/3571156112 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564a21ec3a20 con 0x564a21f6cd70 2024-09-12T03:30:50.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.164+0000 7f200cff9640 10 client.15468 handle_command_reply: tid=0 2024-09-12T03:30:50.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.164+0000 7f20321d9740 1 client.15468 shutdown 2024-09-12T03:30:50.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.164+0000 7f20321d9740 1 -- 172.21.15.64:0/3571156112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a21f52410 msgr2=0x564a21f5a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:50.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.164+0000 7f20321d9740 1 --2- 172.21.15.64:0/3571156112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a21f52410 0x564a21f5a0f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2018010b70 tx=0x7f201800e780 comp rx=0 tx=0).stop 2024-09-12T03:30:50.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.164+0000 7f20321d9740 1 -- 172.21.15.64:0/3571156112 shutdown_connections 2024-09-12T03:30:50.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.164+0000 7f20321d9740 1 --2- 172.21.15.64:0/3571156112 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x564a21f6cd70 0x564a21f6f160 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.164+0000 7f20321d9740 1 --2- 172.21.15.64:0/3571156112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a21f5ccb0 0x564a21f5a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.164+0000 7f20321d9740 1 --2- 172.21.15.64:0/3571156112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a21f52410 0x564a21f5a0f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.164+0000 7f20321d9740 1 --2- 172.21.15.64:0/3571156112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a21f51af0 0x564a21f59bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.165+0000 7f20321d9740 1 -- 172.21.15.64:0/3571156112 >> 172.21.15.64:0/3571156112 conn(0x564a21f4e060 msgr2=0x564a21f53d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:50.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.165+0000 7f200cff9640 0 client.15468 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:30:50.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.165+0000 7f20321d9740 1 -- 172.21.15.64:0/3571156112 shutdown_connections 2024-09-12T03:30:50.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.165+0000 7f20321d9740 1 -- 172.21.15.64:0/3571156112 wait complete. 2024-09-12T03:30:50.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.165+0000 7f20321d9740 20 client.15468 trim_cache size 0 max 16384 2024-09-12T03:30:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.179+0000 7f20321d9740 1 Processor -- start 2024-09-12T03:30:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.179+0000 7f20321d9740 1 -- start start 2024-09-12T03:30:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.179+0000 7f20321d9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a21f51af0 0x564a21f53c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.179+0000 7f20321d9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a21f52410 0x564a21f5a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.179+0000 7f20321d9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a21f5ccb0 0x564a21f74a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.179+0000 7f20321d9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f20281af5d0 con 0x564a21f52410 2024-09-12T03:30:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.179+0000 7f20321d9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f20281acea0 con 0x564a21f51af0 2024-09-12T03:30:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.179+0000 7f20321d9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f20281aacf0 con 0x564a21f5ccb0 2024-09-12T03:30:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.179+0000 7f202d1e1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a21f52410 0x564a21f5a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.180+0000 7f202d1e1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a21f52410 0x564a21f5a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48844/0 (socket says 172.21.15.64:48844) 2024-09-12T03:30:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.180+0000 7f202d1e1640 1 -- 172.21.15.64:0/750988358 learned_addr learned my addr 172.21.15.64:0/750988358 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:30:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.180+0000 7f202c9e0640 1 --2- 172.21.15.64:0/750988358 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a21f51af0 0x564a21f53c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.180+0000 7f202d1e1640 1 -- 172.21.15.64:0/750988358 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a21f5ccb0 msgr2=0x564a21f74a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.180+0000 7f202d1e1640 1 --2- 172.21.15.64:0/750988358 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a21f5ccb0 0x564a21f74a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.180+0000 7f202d1e1640 1 -- 172.21.15.64:0/750988358 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a21f51af0 msgr2=0x564a21f53c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.180+0000 7f202d1e1640 1 --2- 172.21.15.64:0/750988358 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a21f51af0 0x564a21f53c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.180+0000 7f202d1e1640 1 -- 172.21.15.64:0/750988358 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a21f58b60 con 0x564a21f52410 2024-09-12T03:30:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.180+0000 7f202d1e1640 1 --2- 172.21.15.64:0/750988358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a21f52410 0x564a21f5a460 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2018019f70 tx=0x7f201800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c0be70952dfa61 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.180+0000 7f200effd640 1 -- 172.21.15.64:0/750988358 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201802c020 con 0x564a21f52410 2024-09-12T03:30:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.180+0000 7f200effd640 1 -- 172.21.15.64:0/750988358 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2018018550 con 0x564a21f52410 2024-09-12T03:30:50.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.180+0000 7f200effd640 1 -- 172.21.15.64:0/750988358 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2018019440 con 0x564a21f52410 2024-09-12T03:30:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.181+0000 7f20321d9740 1 -- 172.21.15.64:0/750988358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a21f52410 msgr2=0x564a21f5a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.181+0000 7f20321d9740 1 --2- 172.21.15.64:0/750988358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a21f52410 0x564a21f5a460 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2018019f70 tx=0x7f201800ed00 comp rx=0 tx=0).stop 2024-09-12T03:30:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.181+0000 7f20321d9740 1 -- 172.21.15.64:0/750988358 shutdown_connections 2024-09-12T03:30:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.181+0000 7f20321d9740 1 --2- 172.21.15.64:0/750988358 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a21f5ccb0 0x564a21f74a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.181+0000 7f20321d9740 1 --2- 172.21.15.64:0/750988358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a21f52410 0x564a21f5a460 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.181+0000 7f20321d9740 1 --2- 172.21.15.64:0/750988358 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a21f51af0 0x564a21f53c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.181+0000 7f20321d9740 1 -- 172.21.15.64:0/750988358 >> 172.21.15.64:0/750988358 conn(0x564a21f4e060 msgr2=0x564a21f53550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.181+0000 7f20321d9740 1 -- 172.21.15.64:0/750988358 shutdown_connections 2024-09-12T03:30:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.181+0000 7f20321d9740 1 -- 172.21.15.64:0/750988358 wait complete. 2024-09-12T03:30:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.181+0000 7f20321d9740 1 -- start start 2024-09-12T03:30:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.181+0000 7f20321d9740 1 Processor -- start 2024-09-12T03:30:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.182+0000 7f20321d9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a21f51af0 0x564a21f652f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.182+0000 7f20321d9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a21f52410 0x564a21f567c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.182+0000 7f20321d9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a21f5ccb0 0x564a21f56d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.182+0000 7f20321d9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f20281ab360 con 0x564a21f52410 2024-09-12T03:30:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.182+0000 7f20321d9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f20281ac7d0 con 0x564a21f5ccb0 2024-09-12T03:30:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.182+0000 7f20321d9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f20281aba30 con 0x564a21f51af0 2024-09-12T03:30:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.182+0000 7f20321d9740 10 client.0 _ll_register_callbacks cb 0x7f20180089d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.182+0000 7f202d1e1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a21f52410 0x564a21f567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.182+0000 7f202d1e1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a21f52410 0x564a21f567c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48848/0 (socket says 172.21.15.64:48848) 2024-09-12T03:30:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.182+0000 7f202d1e1640 1 -- 172.21.15.64:0/4031318854 learned_addr learned my addr 172.21.15.64:0/4031318854 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:30:50.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.182+0000 7f202c9e0640 1 --2- 172.21.15.64:0/4031318854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a21f5ccb0 0x564a21f56d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:50.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.182+0000 7f202c9e0640 1 --2- 172.21.15.64:0/4031318854 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a21f51af0 0x564a21f652f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:50.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.182+0000 7f202d1e1640 1 -- 172.21.15.64:0/4031318854 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a21f51af0 msgr2=0x564a21f652f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:50.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.182+0000 7f202d1e1640 1 --2- 172.21.15.64:0/4031318854 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a21f51af0 0x564a21f652f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.182+0000 7f202d1e1640 1 -- 172.21.15.64:0/4031318854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a21f5ccb0 msgr2=0x564a21f56d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:50.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.182+0000 7f202d1e1640 1 --2- 172.21.15.64:0/4031318854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a21f5ccb0 0x564a21f56d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.182+0000 7f202d1e1640 1 -- 172.21.15.64:0/4031318854 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a21f54a80 con 0x564a21f52410 2024-09-12T03:30:50.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.182+0000 7f202c9e0640 1 --2- 172.21.15.64:0/4031318854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a21f5ccb0 0x564a21f56d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:30:50.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.182+0000 7f202d1e1640 1 --2- 172.21.15.64:0/4031318854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a21f52410 0x564a21f567c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2018005c40 tx=0x7f2018018910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=250aa934a9ab6ff8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:50.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.182+0000 7f200cff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:30:50.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.182+0000 7f200cff9640 1 -- 172.21.15.64:0/4031318854 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2018010250 con 0x564a21f52410 2024-09-12T03:30:50.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.182+0000 7f200cff9640 1 -- 172.21.15.64:0/4031318854 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f2018004b40 con 0x564a21f52410 2024-09-12T03:30:50.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.182+0000 7f20321d9740 1 -- 172.21.15.64:0/4031318854 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2018018550 con 0x564a21f52410 2024-09-12T03:30:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.183+0000 7f200cff9640 1 -- 172.21.15.64:0/4031318854 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2018004d40 con 0x564a21f52410 2024-09-12T03:30:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.183+0000 7f20321d9740 10 client.15474 fetch_fsmap learned FSMap version 25 2024-09-12T03:30:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.183+0000 7f20321d9740 1 -- 172.21.15.64:0/4031318854 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564a21fd76f0 con 0x564a21f52410 2024-09-12T03:30:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.183+0000 7f200cff9640 1 -- 172.21.15.64:0/4031318854 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2018018550 con 0x564a21f52410 2024-09-12T03:30:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.183+0000 7f200cff9640 1 -- 172.21.15.64:0/4031318854 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2018017050 con 0x564a21f52410 2024-09-12T03:30:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.183+0000 7f20321d9740 10 client.15474 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:30:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.183+0000 7f20321d9740 10 client.15474 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:30:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.183+0000 7f20321d9740 20 client.15474 populate_metadata read hostname 'smithi064' 2024-09-12T03:30:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.183+0000 7f20321d9740 1 --2- 172.21.15.64:0/4031318854 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x564a21fd7bd0 0x564a21fd9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.183+0000 7f20321d9740 4 client.15474 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:30:50.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.183+0000 7f20321d9740 1 -- 172.21.15.64:0/4031318854 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564a21ebdd70 con 0x564a21fd7bd0 2024-09-12T03:30:50.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.183+0000 7f202c9e0640 1 --2- 172.21.15.64:0/4031318854 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x564a21fd7bd0 0x564a21fd9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:50.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.184+0000 7f200cff9640 10 client.15474 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:30:50.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.184+0000 7f202c9e0640 1 --2- 172.21.15.64:0/4031318854 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x564a21fd7bd0 0x564a21fd9ff0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f4060cf5a2b198 server_cookie=3b066df6da21ee3e in_seq=0 out_seq=0 2024-09-12T03:30:50.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.197+0000 7f200cff9640 1 -- 172.21.15.64:0/4031318854 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564a21ebdd70 con 0x564a21fd7bd0 2024-09-12T03:30:50.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.197+0000 7f200cff9640 10 client.15474 handle_command_reply: tid=0 2024-09-12T03:30:50.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.197+0000 7f20321d9740 1 client.15474 shutdown 2024-09-12T03:30:50.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.197+0000 7f20321d9740 1 -- 172.21.15.64:0/4031318854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a21f52410 msgr2=0x564a21f567c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.197+0000 7f20321d9740 1 --2- 172.21.15.64:0/4031318854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a21f52410 0x564a21f567c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2018005c40 tx=0x7f2018018910 comp rx=0 tx=0).stop 2024-09-12T03:30:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.197+0000 7f20321d9740 1 -- 172.21.15.64:0/4031318854 shutdown_connections 2024-09-12T03:30:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.197+0000 7f20321d9740 1 --2- 172.21.15.64:0/4031318854 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x564a21fd7bd0 0x564a21fd9ff0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.197+0000 7f20321d9740 1 --2- 172.21.15.64:0/4031318854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a21f5ccb0 0x564a21f56d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.197+0000 7f20321d9740 1 --2- 172.21.15.64:0/4031318854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a21f52410 0x564a21f567c0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.197+0000 7f20321d9740 1 --2- 172.21.15.64:0/4031318854 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a21f51af0 0x564a21f652f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.197+0000 7f200cff9640 0 client.15474 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:30:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.197+0000 7f20321d9740 1 -- 172.21.15.64:0/4031318854 >> 172.21.15.64:0/4031318854 conn(0x564a21f4e060 msgr2=0x564a21f4d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.197+0000 7f20321d9740 1 -- 172.21.15.64:0/4031318854 shutdown_connections 2024-09-12T03:30:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.197+0000 7f20321d9740 1 -- 172.21.15.64:0/4031318854 wait complete. 2024-09-12T03:30:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.197+0000 7f20321d9740 20 client.15474 trim_cache size 0 max 16384 2024-09-12T03:30:50.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:30:50.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:30:50.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:30:50.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:30:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.198+0000 7f202f445640 1 -- 172.21.15.64:0/4199947401 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f20100a2da0 msgr2=0x7f20100a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.198+0000 7f202f445640 1 --2- 172.21.15.64:0/4199947401 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f20100a2da0 0x7f20100a5260 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f20200bc060 tx=0x7f202002a480 comp rx=0 tx=0).stop 2024-09-12T03:30:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.198+0000 7f202f445640 1 -- 172.21.15.64:0/4199947401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f202810c110 msgr2=0x7f202819ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.198+0000 7f202f445640 1 --2- 172.21.15.64:0/4199947401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f202810c110 0x7f202819ca90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2020009a20 tx=0x7f20200042e0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.198+0000 7f202f445640 1 -- 172.21.15.64:0/4199947401 shutdown_connections 2024-09-12T03:30:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.198+0000 7f202f445640 1 --2- 172.21.15.64:0/4199947401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f202810c110 0x7f202819ca90 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.198+0000 7f202f445640 1 --2- 172.21.15.64:0/4199947401 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f20100a2da0 0x7f20100a5260 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.198+0000 7f202f445640 1 --2- 172.21.15.64:0/4199947401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20281035e0 0x7f2028198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.198+0000 7f202f445640 1 --2- 172.21.15.64:0/4199947401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2028102cc0 0x7f20281981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.198+0000 7f202f445640 1 -- 172.21.15.64:0/4199947401 >> 172.21.15.64:0/4199947401 conn(0x7f20280fc6b0 msgr2=0x7f20280fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.198+0000 7f202f445640 1 -- 172.21.15.64:0/4199947401 shutdown_connections 2024-09-12T03:30:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:30:50.198+0000 7f202f445640 1 -- 172.21.15.64:0/4199947401 wait complete. 2024-09-12T03:30:51.521 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:30:51 smithi064 ceph-mon[30952]: pgmap v50: 129 pgs: 129 active+clean; 1.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-09-12T03:30:51.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:30:51 smithi115 ceph-mon[34801]: pgmap v50: 129 pgs: 129 active+clean; 1.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-09-12T03:30:51.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:30:51 smithi195 ceph-mon[34786]: pgmap v50: 129 pgs: 129 active+clean; 1.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-09-12T03:30:53.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:30:53 smithi115 ceph-mon[34801]: pgmap v51: 129 pgs: 129 active+clean; 1.8 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 21 op/s 2024-09-12T03:30:53.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:30:53 smithi195 ceph-mon[34786]: pgmap v51: 129 pgs: 129 active+clean; 1.8 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 21 op/s 2024-09-12T03:30:53.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:30:53 smithi064 ceph-mon[30952]: pgmap v51: 129 pgs: 129 active+clean; 1.8 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 21 op/s 2024-09-12T03:30:55.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:30:55 smithi115 ceph-mon[34801]: pgmap v52: 129 pgs: 129 active+clean; 1.8 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-12T03:30:55.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:30:55 smithi195 ceph-mon[34786]: pgmap v52: 129 pgs: 129 active+clean; 1.8 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-12T03:30:55.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:30:55 smithi064 ceph-mon[30952]: pgmap v52: 129 pgs: 129 active+clean; 1.8 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-12T03:30:57.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:30:57 smithi115 ceph-mon[34801]: pgmap v53: 129 pgs: 129 active+clean; 1.8 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-09-12T03:30:57.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:30:57 smithi195 ceph-mon[34786]: pgmap v53: 129 pgs: 129 active+clean; 1.8 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-09-12T03:30:57.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:30:57 smithi064 ceph-mon[30952]: pgmap v53: 129 pgs: 129 active+clean; 1.8 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-09-12T03:30:59.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:30:59 smithi115 ceph-mon[34801]: pgmap v54: 129 pgs: 129 active+clean; 1.9 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 18 op/s 2024-09-12T03:30:59.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:30:59 smithi195 ceph-mon[34786]: pgmap v54: 129 pgs: 129 active+clean; 1.9 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 18 op/s 2024-09-12T03:30:59.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:30:59 smithi064 ceph-mon[30952]: pgmap v54: 129 pgs: 129 active+clean; 1.9 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 18 op/s 2024-09-12T03:31:01.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:31:01 smithi115 ceph-mon[34801]: pgmap v55: 129 pgs: 129 active+clean; 1.9 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-09-12T03:31:01.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:31:01 smithi195 ceph-mon[34786]: pgmap v55: 129 pgs: 129 active+clean; 1.9 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-09-12T03:31:01.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:31:01 smithi064 ceph-mon[30952]: pgmap v55: 129 pgs: 129 active+clean; 1.9 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-09-12T03:31:03.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.299+0000 7f57a9b4b640 1 -- 172.21.15.64:0/1626644993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57a4102cc0 msgr2=0x7f57a41030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:03.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.299+0000 7f57a9b4b640 1 --2- 172.21.15.64:0/1626644993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57a4102cc0 0x7f57a41030a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5794009900 tx=0x7f579402a480 comp rx=0 tx=0).stop 2024-09-12T03:31:03.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.299+0000 7f57a9b4b640 1 -- 172.21.15.64:0/1626644993 shutdown_connections 2024-09-12T03:31:03.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.299+0000 7f57a9b4b640 1 --2- 172.21.15.64:0/1626644993 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57a410c110 0x7f57a410e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.299+0000 7f57a9b4b640 1 --2- 172.21.15.64:0/1626644993 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57a41035e0 0x7f57a410bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.299+0000 7f57a9b4b640 1 --2- 172.21.15.64:0/1626644993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57a4102cc0 0x7f57a41030a0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.299+0000 7f57a9b4b640 1 -- 172.21.15.64:0/1626644993 >> 172.21.15.64:0/1626644993 conn(0x7f57a40fc6b0 msgr2=0x7f57a40fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.299+0000 7f57a9b4b640 1 -- 172.21.15.64:0/1626644993 shutdown_connections 2024-09-12T03:31:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.299+0000 7f57a9b4b640 1 -- 172.21.15.64:0/1626644993 wait complete. 2024-09-12T03:31:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.299+0000 7f57a9b4b640 1 Processor -- start 2024-09-12T03:31:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.300+0000 7f57a9b4b640 1 -- start start 2024-09-12T03:31:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.300+0000 7f57a9b4b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57a4102cc0 0x7f57a41982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.300+0000 7f57a9b4b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57a41035e0 0x7f57a4198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.300+0000 7f57a9b4b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57a410c110 0x7f57a419cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.300+0000 7f57a9b4b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f57a4111da0 con 0x7f57a410c110 2024-09-12T03:31:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.300+0000 7f57a9b4b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f57a4111c20 con 0x7f57a4102cc0 2024-09-12T03:31:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.300+0000 7f57a9b4b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f57a4111f20 con 0x7f57a41035e0 2024-09-12T03:31:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.300+0000 7f57a3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57a410c110 0x7f57a419cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.300+0000 7f57a3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57a410c110 0x7f57a419cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59968/0 (socket says 172.21.15.64:59968) 2024-09-12T03:31:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.300+0000 7f57a3fff640 1 -- 172.21.15.64:0/3219229686 learned_addr learned my addr 172.21.15.64:0/3219229686 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:31:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.300+0000 7f57a2ffd640 1 --2- 172.21.15.64:0/3219229686 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57a41035e0 0x7f57a4198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.300+0000 7f57a37fe640 1 --2- 172.21.15.64:0/3219229686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57a4102cc0 0x7f57a41982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.300+0000 7f57a3fff640 1 -- 172.21.15.64:0/3219229686 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57a41035e0 msgr2=0x7f57a4198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.300+0000 7f57a3fff640 1 --2- 172.21.15.64:0/3219229686 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57a41035e0 0x7f57a4198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.300+0000 7f57a3fff640 1 -- 172.21.15.64:0/3219229686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57a4102cc0 msgr2=0x7f57a41982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.300+0000 7f57a3fff640 1 --2- 172.21.15.64:0/3219229686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57a4102cc0 0x7f57a41982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.300+0000 7f57a3fff640 1 -- 172.21.15.64:0/3219229686 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57a419d2d0 con 0x7f57a410c110 2024-09-12T03:31:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.300+0000 7f57a3fff640 1 --2- 172.21.15.64:0/3219229686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57a410c110 0x7f57a419cba0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f579c00eaa0 tx=0x7f579c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.301+0000 7f57a0ff9640 1 -- 172.21.15.64:0/3219229686 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579c00cdb0 con 0x7f57a410c110 2024-09-12T03:31:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.301+0000 7f57a0ff9640 1 -- 172.21.15.64:0/3219229686 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f579c005020 con 0x7f57a410c110 2024-09-12T03:31:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.301+0000 7f57a9b4b640 1 -- 172.21.15.64:0/3219229686 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57a419d5c0 con 0x7f57a410c110 2024-09-12T03:31:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.301+0000 7f57a0ff9640 1 -- 172.21.15.64:0/3219229686 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579c0116d0 con 0x7f57a410c110 2024-09-12T03:31:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.301+0000 7f57a9b4b640 1 -- 172.21.15.64:0/3219229686 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57a41a4e50 con 0x7f57a410c110 2024-09-12T03:31:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.303+0000 7f57a0ff9640 1 -- 172.21.15.64:0/3219229686 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f579c0051d0 con 0x7f57a410c110 2024-09-12T03:31:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.303+0000 7f57ac9c4740 1 Processor -- start 2024-09-12T03:31:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.304+0000 7f57ac9c4740 1 -- start start 2024-09-12T03:31:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.304+0000 7f57ac9c4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55693ddfb9d0 0x55693ddfbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.304+0000 7f57ac9c4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55693ddfc2f0 0x55693de0ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.304+0000 7f57ac9c4740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55693de0b0c0 0x55693de0d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.304+0000 7f57ac9c4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55693dd602f0 con 0x55693ddfb9d0 2024-09-12T03:31:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.304+0000 7f57ac9c4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55693dd08270 con 0x55693ddfc2f0 2024-09-12T03:31:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.304+0000 7f57ac9c4740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55693dd5c2b0 con 0x55693de0b0c0 2024-09-12T03:31:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.304+0000 7f57a2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55693ddfb9d0 0x55693ddfbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.304+0000 7f57a2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55693ddfb9d0 0x55693ddfbdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59984/0 (socket says 172.21.15.64:59984) 2024-09-12T03:31:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.304+0000 7f57a2ffd640 1 -- 172.21.15.64:0/2981866486 learned_addr learned my addr 172.21.15.64:0/2981866486 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:31:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.304+0000 7f57a0ff9640 1 --2- 172.21.15.64:0/3219229686 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f578c0a2e20 0x7f578c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:03.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.304+0000 7f57a2ffd640 1 --2- 172.21.15.64:0/2981866486 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55693de0b0c0 0x55693de0d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:03.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.304+0000 7f57a37fe640 1 --2- 172.21.15.64:0/2981866486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55693ddfc2f0 0x55693de0ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:03.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.304+0000 7f57a2ffd640 1 -- 172.21.15.64:0/2981866486 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55693de0b0c0 msgr2=0x55693de0d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:03.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.304+0000 7f57a2ffd640 1 --2- 172.21.15.64:0/2981866486 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55693de0b0c0 0x55693de0d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.304+0000 7f57a2ffd640 1 -- 172.21.15.64:0/2981866486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55693ddfc2f0 msgr2=0x55693de0ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:03.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.304+0000 7f57a2ffd640 1 --2- 172.21.15.64:0/2981866486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55693ddfc2f0 0x55693de0ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.304+0000 7f57a2ffd640 1 -- 172.21.15.64:0/2981866486 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55693dd07340 con 0x55693ddfb9d0 2024-09-12T03:31:03.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.304+0000 7f57a0ff9640 1 -- 172.21.15.64:0/3219229686 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f579c0cf830 con 0x7f57a410c110 2024-09-12T03:31:03.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.304+0000 7f57a3fff640 1 --2- 172.21.15.64:0/3219229686 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f578c0a2e20 0x7f578c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:03.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.304+0000 7f57a2ffd640 1 --2- 172.21.15.64:0/2981866486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55693ddfb9d0 0x55693ddfbdb0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5798012530 tx=0x7f579800b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c9b33cef060a84d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:03.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.305+0000 7f5789ffb640 1 -- 172.21.15.64:0/2981866486 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5798004030 con 0x55693ddfb9d0 2024-09-12T03:31:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.305+0000 7f5789ffb640 1 -- 172.21.15.64:0/2981866486 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57980041d0 con 0x55693ddfb9d0 2024-09-12T03:31:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.305+0000 7f5789ffb640 1 -- 172.21.15.64:0/2981866486 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5798010660 con 0x55693ddfb9d0 2024-09-12T03:31:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.305+0000 7f57ac9c4740 1 -- 172.21.15.64:0/2981866486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55693ddfb9d0 msgr2=0x55693ddfbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.305+0000 7f57a3fff640 1 --2- 172.21.15.64:0/3219229686 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f578c0a2e20 0x7f578c0a52e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f579c00efd0 tx=0x7f579c086040 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.305+0000 7f57ac9c4740 1 --2- 172.21.15.64:0/2981866486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55693ddfb9d0 0x55693ddfbdb0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5798012530 tx=0x7f579800b830 comp rx=0 tx=0).stop 2024-09-12T03:31:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.305+0000 7f57ac9c4740 1 -- 172.21.15.64:0/2981866486 shutdown_connections 2024-09-12T03:31:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.305+0000 7f57ac9c4740 1 --2- 172.21.15.64:0/2981866486 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55693de0b0c0 0x55693de0d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.305+0000 7f57ac9c4740 1 --2- 172.21.15.64:0/2981866486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55693ddfc2f0 0x55693de0ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.305+0000 7f57ac9c4740 1 --2- 172.21.15.64:0/2981866486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55693ddfb9d0 0x55693ddfbdb0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.305+0000 7f57ac9c4740 1 -- 172.21.15.64:0/2981866486 >> 172.21.15.64:0/2981866486 conn(0x55693ddf8060 msgr2=0x55693ddf8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.306+0000 7f57ac9c4740 1 -- 172.21.15.64:0/2981866486 shutdown_connections 2024-09-12T03:31:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.306+0000 7f57ac9c4740 1 -- 172.21.15.64:0/2981866486 wait complete. 2024-09-12T03:31:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.306+0000 7f57ac9c4740 1 -- start start 2024-09-12T03:31:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.306+0000 7f57ac9c4740 1 Processor -- start 2024-09-12T03:31:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.306+0000 7f57ac9c4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55693ddfb9d0 0x55693de08040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.306+0000 7f57ac9c4740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55693ddfc2f0 0x55693de08580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.306+0000 7f57ac9c4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55693de0b0c0 0x55693de08ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.307+0000 7f57ac9c4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55693de0dcf0 con 0x55693de0b0c0 2024-09-12T03:31:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.307+0000 7f57ac9c4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55693dce9290 con 0x55693ddfb9d0 2024-09-12T03:31:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.307+0000 7f57ac9c4740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55693de0de70 con 0x55693ddfc2f0 2024-09-12T03:31:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.307+0000 7f57ac9c4740 10 client.0 _ll_register_callbacks cb 0x55693dd1a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:31:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.307+0000 7f57a2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55693de0b0c0 0x55693de08ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.307+0000 7f57a2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55693de0b0c0 0x55693de08ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59990/0 (socket says 172.21.15.64:59990) 2024-09-12T03:31:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.307+0000 7f57a2ffd640 1 -- 172.21.15.64:0/186067025 learned_addr learned my addr 172.21.15.64:0/186067025 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:31:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.307+0000 7f57a2ffd640 1 --2- 172.21.15.64:0/186067025 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55693ddfb9d0 0x55693de08040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.307+0000 7f57a37fe640 1 --2- 172.21.15.64:0/186067025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55693ddfc2f0 0x55693de08580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.307+0000 7f57a2ffd640 1 -- 172.21.15.64:0/186067025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55693ddfc2f0 msgr2=0x55693de08580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.307+0000 7f57a2ffd640 1 --2- 172.21.15.64:0/186067025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55693ddfc2f0 0x55693de08580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.307+0000 7f57a2ffd640 1 -- 172.21.15.64:0/186067025 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55693ddfb9d0 msgr2=0x55693de08040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.307+0000 7f57a2ffd640 1 --2- 172.21.15.64:0/186067025 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55693ddfb9d0 0x55693de08040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.307+0000 7f57a2ffd640 1 -- 172.21.15.64:0/186067025 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55693da73210 con 0x55693de0b0c0 2024-09-12T03:31:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.307+0000 7f57a2ffd640 1 --2- 172.21.15.64:0/186067025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55693de0b0c0 0x55693de08ac0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5798003dc0 tx=0x7f5798003cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e08604137ae1a5d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.307+0000 7f5782ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:31:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.307+0000 7f5782ffd640 1 -- 172.21.15.64:0/186067025 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5798027040 con 0x55693de0b0c0 2024-09-12T03:31:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.307+0000 7f5782ffd640 1 -- 172.21.15.64:0/186067025 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f57980035f0 con 0x55693de0b0c0 2024-09-12T03:31:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.307+0000 7f57ac9c4740 1 -- 172.21.15.64:0/186067025 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57980041d0 con 0x55693de0b0c0 2024-09-12T03:31:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.308+0000 7f5782ffd640 1 -- 172.21.15.64:0/186067025 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f57980037f0 con 0x55693de0b0c0 2024-09-12T03:31:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.308+0000 7f57ac9c4740 10 client.15486 fetch_fsmap learned FSMap version 25 2024-09-12T03:31:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.308+0000 7f57ac9c4740 1 -- 172.21.15.64:0/186067025 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55693db3f620 con 0x55693de0b0c0 2024-09-12T03:31:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.308+0000 7f5782ffd640 1 -- 172.21.15.64:0/186067025 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57980041d0 con 0x55693de0b0c0 2024-09-12T03:31:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.308+0000 7f5782ffd640 1 -- 172.21.15.64:0/186067025 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f579801c020 con 0x55693de0b0c0 2024-09-12T03:31:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.308+0000 7f57ac9c4740 10 client.15486 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:31:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.308+0000 7f57ac9c4740 10 client.15486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:31:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.308+0000 7f57ac9c4740 20 client.15486 populate_metadata read hostname 'smithi064' 2024-09-12T03:31:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.308+0000 7f57ac9c4740 1 --2- 172.21.15.64:0/186067025 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55693de1af90 0x55693de1d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.308+0000 7f57ac9c4740 4 client.15486 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:31:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.308+0000 7f57ac9c4740 1 -- 172.21.15.64:0/186067025 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55693dd6da20 con 0x55693de1af90 2024-09-12T03:31:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.308+0000 7f57a37fe640 1 --2- 172.21.15.64:0/186067025 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55693de1af90 0x55693de1d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.309+0000 7f5782ffd640 10 client.15486 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:31:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.309+0000 7f57a37fe640 1 --2- 172.21.15.64:0/186067025 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55693de1af90 0x55693de1d380 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79c62d901c9be26d server_cookie=1dca777037a5783b in_seq=0 out_seq=0 2024-09-12T03:31:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.310+0000 7f5782ffd640 1 -- 172.21.15.64:0/186067025 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55693dd6da20 con 0x55693de1af90 2024-09-12T03:31:03.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.310+0000 7f5782ffd640 10 client.15486 handle_command_reply: tid=0 2024-09-12T03:31:03.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.310+0000 7f57ac9c4740 1 client.15486 shutdown 2024-09-12T03:31:03.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.310+0000 7f57ac9c4740 1 -- 172.21.15.64:0/186067025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55693de0b0c0 msgr2=0x55693de08ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:03.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.310+0000 7f57ac9c4740 1 --2- 172.21.15.64:0/186067025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55693de0b0c0 0x55693de08ac0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5798003dc0 tx=0x7f5798003cc0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.310+0000 7f57ac9c4740 1 -- 172.21.15.64:0/186067025 shutdown_connections 2024-09-12T03:31:03.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.310+0000 7f57ac9c4740 1 --2- 172.21.15.64:0/186067025 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55693de1af90 0x55693de1d380 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.310+0000 7f57ac9c4740 1 --2- 172.21.15.64:0/186067025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55693de0b0c0 0x55693de08ac0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.310+0000 7f57ac9c4740 1 --2- 172.21.15.64:0/186067025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55693ddfc2f0 0x55693de08580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.310+0000 7f57ac9c4740 1 --2- 172.21.15.64:0/186067025 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55693ddfb9d0 0x55693de08040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.310+0000 7f5782ffd640 0 client.15486 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:31:03.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.310+0000 7f57ac9c4740 1 -- 172.21.15.64:0/186067025 >> 172.21.15.64:0/186067025 conn(0x55693ddf8060 msgr2=0x55693ddfddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:03.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.310+0000 7f57ac9c4740 1 -- 172.21.15.64:0/186067025 shutdown_connections 2024-09-12T03:31:03.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.310+0000 7f57ac9c4740 1 -- 172.21.15.64:0/186067025 wait complete. 2024-09-12T03:31:03.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.310+0000 7f57ac9c4740 20 client.15486 trim_cache size 0 max 16384 2024-09-12T03:31:03.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.326+0000 7f57ac9c4740 1 Processor -- start 2024-09-12T03:31:03.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.326+0000 7f57ac9c4740 1 -- start start 2024-09-12T03:31:03.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.326+0000 7f57ac9c4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55693ddfb9d0 0x55693ddfdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:03.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.326+0000 7f57ac9c4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55693ddfc2f0 0x55693de22a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:03.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.326+0000 7f57ac9c4740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55693de0b0c0 0x55693de22f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:03.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.326+0000 7f57ac9c4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f57a41af720 con 0x55693ddfc2f0 2024-09-12T03:31:03.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.326+0000 7f57ac9c4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f57a41acff0 con 0x55693ddfb9d0 2024-09-12T03:31:03.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.326+0000 7f57ac9c4740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f57a41aae40 con 0x55693de0b0c0 2024-09-12T03:31:03.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.326+0000 7f57a37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55693ddfc2f0 0x55693de22a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:03.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.326+0000 7f57a37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55693ddfc2f0 0x55693de22a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59996/0 (socket says 172.21.15.64:59996) 2024-09-12T03:31:03.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.326+0000 7f57a37fe640 1 -- 172.21.15.64:0/3381316961 learned_addr learned my addr 172.21.15.64:0/3381316961 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:31:03.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.326+0000 7f57a2ffd640 1 --2- 172.21.15.64:0/3381316961 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55693ddfb9d0 0x55693ddfdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:03.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.326+0000 7f57a2ffd640 1 --2- 172.21.15.64:0/3381316961 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55693de0b0c0 0x55693de22f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:03.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.326+0000 7f57a37fe640 1 -- 172.21.15.64:0/3381316961 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55693de0b0c0 msgr2=0x55693de22f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:03.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.326+0000 7f57a37fe640 1 --2- 172.21.15.64:0/3381316961 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55693de0b0c0 0x55693de22f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.326+0000 7f57a37fe640 1 -- 172.21.15.64:0/3381316961 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55693ddfb9d0 msgr2=0x55693ddfdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:03.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.326+0000 7f57a37fe640 1 --2- 172.21.15.64:0/3381316961 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55693ddfb9d0 0x55693ddfdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.326+0000 7f57a37fe640 1 -- 172.21.15.64:0/3381316961 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55693ddfc120 con 0x55693ddfc2f0 2024-09-12T03:31:03.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.326+0000 7f57a37fe640 1 --2- 172.21.15.64:0/3381316961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55693ddfc2f0 0x55693de22a40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f579402a9a0 tx=0x7f5794005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25bea83cf93482a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:03.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.327+0000 7f57897fa640 1 -- 172.21.15.64:0/3381316961 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579404a070 con 0x55693ddfc2f0 2024-09-12T03:31:03.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.327+0000 7f57897fa640 1 -- 172.21.15.64:0/3381316961 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57940054f0 con 0x55693ddfc2f0 2024-09-12T03:31:03.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.327+0000 7f57897fa640 1 -- 172.21.15.64:0/3381316961 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5794035040 con 0x55693ddfc2f0 2024-09-12T03:31:03.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.327+0000 7f57ac9c4740 1 -- 172.21.15.64:0/3381316961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55693ddfc2f0 msgr2=0x55693de22a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:03.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.327+0000 7f57ac9c4740 1 --2- 172.21.15.64:0/3381316961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55693ddfc2f0 0x55693de22a40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f579402a9a0 tx=0x7f5794005c00 comp rx=0 tx=0).stop 2024-09-12T03:31:03.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.327+0000 7f57ac9c4740 1 -- 172.21.15.64:0/3381316961 shutdown_connections 2024-09-12T03:31:03.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.327+0000 7f57ac9c4740 1 --2- 172.21.15.64:0/3381316961 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55693de0b0c0 0x55693de22f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.327+0000 7f57ac9c4740 1 --2- 172.21.15.64:0/3381316961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55693ddfc2f0 0x55693de22a40 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.327+0000 7f57ac9c4740 1 --2- 172.21.15.64:0/3381316961 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55693ddfb9d0 0x55693ddfdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.327+0000 7f57ac9c4740 1 -- 172.21.15.64:0/3381316961 >> 172.21.15.64:0/3381316961 conn(0x55693ddf8060 msgr2=0x55693ddfd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.327+0000 7f57ac9c4740 1 -- 172.21.15.64:0/3381316961 shutdown_connections 2024-09-12T03:31:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.327+0000 7f57ac9c4740 1 -- 172.21.15.64:0/3381316961 wait complete. 2024-09-12T03:31:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.328+0000 7f57ac9c4740 1 -- start start 2024-09-12T03:31:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.328+0000 7f57ac9c4740 1 Processor -- start 2024-09-12T03:31:03.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.328+0000 7f57ac9c4740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55693ddfb9d0 0x55693de135a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.328+0000 7f57ac9c4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55693ddfc2f0 0x55693de0e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.328+0000 7f57ac9c4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55693de0b0c0 0x55693de0ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.328+0000 7f57ac9c4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f57a41ab4b0 con 0x55693de0b0c0 2024-09-12T03:31:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.328+0000 7f57ac9c4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f57a41ac920 con 0x55693ddfc2f0 2024-09-12T03:31:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.328+0000 7f57ac9c4740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f57a41abb80 con 0x55693ddfb9d0 2024-09-12T03:31:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.328+0000 7f57ac9c4740 10 client.0 _ll_register_callbacks cb 0x7f5798012410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:31:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.328+0000 7f57a2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55693de0b0c0 0x55693de0ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.328+0000 7f57a2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55693de0b0c0 0x55693de0ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60010/0 (socket says 172.21.15.64:60010) 2024-09-12T03:31:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.328+0000 7f57a2ffd640 1 -- 172.21.15.64:0/3062001960 learned_addr learned my addr 172.21.15.64:0/3062001960 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:31:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.328+0000 7f57a37fe640 1 --2- 172.21.15.64:0/3062001960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55693ddfc2f0 0x55693de0e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.328+0000 7f57a2ffd640 1 --2- 172.21.15.64:0/3062001960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55693ddfb9d0 0x55693de135a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.329+0000 7f57a2ffd640 1 -- 172.21.15.64:0/3062001960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55693ddfb9d0 msgr2=0x55693de135a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.329+0000 7f57a2ffd640 1 --2- 172.21.15.64:0/3062001960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55693ddfb9d0 0x55693de135a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.329+0000 7f57a2ffd640 1 -- 172.21.15.64:0/3062001960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55693ddfc2f0 msgr2=0x55693de0e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.329+0000 7f57a2ffd640 1 --2- 172.21.15.64:0/3062001960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55693ddfc2f0 0x55693de0e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.329+0000 7f57a2ffd640 1 -- 172.21.15.64:0/3062001960 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55693de0f3c0 con 0x55693de0b0c0 2024-09-12T03:31:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.329+0000 7f57a2ffd640 1 --2- 172.21.15.64:0/3062001960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55693de0b0c0 0x55693de0ee00 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5798000c00 tx=0x7f5798003cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6621cbb6d104f742 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.329+0000 7f5782ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:31:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.329+0000 7f5782ffd640 1 -- 172.21.15.64:0/3062001960 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579800b2e0 con 0x55693de0b0c0 2024-09-12T03:31:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.329+0000 7f5782ffd640 1 -- 172.21.15.64:0/3062001960 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f5798005200 con 0x55693de0b0c0 2024-09-12T03:31:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.329+0000 7f57ac9c4740 1 -- 172.21.15.64:0/3062001960 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57940054f0 con 0x55693de0b0c0 2024-09-12T03:31:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.329+0000 7f5782ffd640 1 -- 172.21.15.64:0/3062001960 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5798005610 con 0x55693de0b0c0 2024-09-12T03:31:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.329+0000 7f57ac9c4740 10 client.15492 fetch_fsmap learned FSMap version 25 2024-09-12T03:31:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.329+0000 7f57ac9c4740 1 -- 172.21.15.64:0/3062001960 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55693de857b0 con 0x55693de0b0c0 2024-09-12T03:31:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.329+0000 7f5782ffd640 1 -- 172.21.15.64:0/3062001960 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57940054f0 con 0x55693de0b0c0 2024-09-12T03:31:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.330+0000 7f5782ffd640 1 -- 172.21.15.64:0/3062001960 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f579801c040 con 0x55693de0b0c0 2024-09-12T03:31:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.330+0000 7f57ac9c4740 10 client.15492 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:31:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.330+0000 7f57ac9c4740 10 client.15492 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:31:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.330+0000 7f57ac9c4740 20 client.15492 populate_metadata read hostname 'smithi064' 2024-09-12T03:31:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.330+0000 7f57ac9c4740 1 --2- 172.21.15.64:0/3062001960 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55693de85e70 0x55693de88290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.330+0000 7f57ac9c4740 4 client.15492 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:31:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.330+0000 7f57ac9c4740 1 -- 172.21.15.64:0/3062001960 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55693dd67d70 con 0x55693de85e70 2024-09-12T03:31:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.330+0000 7f57a37fe640 1 --2- 172.21.15.64:0/3062001960 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55693de85e70 0x55693de88290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.330+0000 7f5782ffd640 10 client.15492 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:31:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.330+0000 7f57a37fe640 1 --2- 172.21.15.64:0/3062001960 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55693de85e70 0x55693de88290 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=542abe9633e377f0 server_cookie=6128c40c721c7b5f in_seq=0 out_seq=0 2024-09-12T03:31:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.343+0000 7f5782ffd640 1 -- 172.21.15.64:0/3062001960 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55693dd67d70 con 0x55693de85e70 2024-09-12T03:31:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.343+0000 7f5782ffd640 10 client.15492 handle_command_reply: tid=0 2024-09-12T03:31:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.343+0000 7f57ac9c4740 1 client.15492 shutdown 2024-09-12T03:31:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.344+0000 7f57ac9c4740 1 -- 172.21.15.64:0/3062001960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55693de0b0c0 msgr2=0x55693de0ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.344+0000 7f57ac9c4740 1 --2- 172.21.15.64:0/3062001960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55693de0b0c0 0x55693de0ee00 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5798000c00 tx=0x7f5798003cc0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.344+0000 7f57ac9c4740 1 -- 172.21.15.64:0/3062001960 shutdown_connections 2024-09-12T03:31:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.344+0000 7f57ac9c4740 1 --2- 172.21.15.64:0/3062001960 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55693de85e70 0x55693de88290 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.344+0000 7f57ac9c4740 1 --2- 172.21.15.64:0/3062001960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55693de0b0c0 0x55693de0ee00 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.344+0000 7f57ac9c4740 1 --2- 172.21.15.64:0/3062001960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55693ddfc2f0 0x55693de0e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.344+0000 7f5782ffd640 0 client.15492 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:31:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.344+0000 7f57ac9c4740 1 --2- 172.21.15.64:0/3062001960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55693ddfb9d0 0x55693de135a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.344+0000 7f57ac9c4740 1 -- 172.21.15.64:0/3062001960 >> 172.21.15.64:0/3062001960 conn(0x55693ddf8060 msgr2=0x55693ddf71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.344+0000 7f57ac9c4740 1 -- 172.21.15.64:0/3062001960 shutdown_connections 2024-09-12T03:31:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.344+0000 7f57ac9c4740 1 -- 172.21.15.64:0/3062001960 wait complete. 2024-09-12T03:31:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.344+0000 7f57ac9c4740 20 client.15492 trim_cache size 0 max 16384 2024-09-12T03:31:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:31:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:31:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:31:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:31:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.344+0000 7f57a9b4b640 1 -- 172.21.15.64:0/3219229686 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f578c0a2e20 msgr2=0x7f578c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.344+0000 7f57a9b4b640 1 --2- 172.21.15.64:0/3219229686 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f578c0a2e20 0x7f578c0a52e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f579c00efd0 tx=0x7f579c086040 comp rx=0 tx=0).stop 2024-09-12T03:31:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.344+0000 7f57a9b4b640 1 -- 172.21.15.64:0/3219229686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57a410c110 msgr2=0x7f57a419cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.344+0000 7f57a9b4b640 1 --2- 172.21.15.64:0/3219229686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57a410c110 0x7f57a419cba0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f579c00eaa0 tx=0x7f579c00ef60 comp rx=0 tx=0).stop 2024-09-12T03:31:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.344+0000 7f57a9b4b640 1 -- 172.21.15.64:0/3219229686 shutdown_connections 2024-09-12T03:31:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.344+0000 7f57a9b4b640 1 --2- 172.21.15.64:0/3219229686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57a410c110 0x7f57a419cba0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.344+0000 7f57a9b4b640 1 --2- 172.21.15.64:0/3219229686 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57a41035e0 0x7f57a4198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.344+0000 7f57a9b4b640 1 --2- 172.21.15.64:0/3219229686 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f578c0a2e20 0x7f578c0a52e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.344+0000 7f57a9b4b640 1 --2- 172.21.15.64:0/3219229686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57a4102cc0 0x7f57a41982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.344+0000 7f57a9b4b640 1 -- 172.21.15.64:0/3219229686 >> 172.21.15.64:0/3219229686 conn(0x7f57a40fc6b0 msgr2=0x7f57a40fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.345+0000 7f57a9b4b640 1 -- 172.21.15.64:0/3219229686 shutdown_connections 2024-09-12T03:31:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:03.345+0000 7f57a9b4b640 1 -- 172.21.15.64:0/3219229686 wait complete. 2024-09-12T03:31:03.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:31:03 smithi115 ceph-mon[34801]: pgmap v56: 129 pgs: 129 active+clean; 2.0 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 45 KiB/s wr, 33 op/s 2024-09-12T03:31:03.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:31:03 smithi195 ceph-mon[34786]: pgmap v56: 129 pgs: 129 active+clean; 2.0 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 45 KiB/s wr, 33 op/s 2024-09-12T03:31:03.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:31:03 smithi064 ceph-mon[30952]: pgmap v56: 129 pgs: 129 active+clean; 2.0 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 45 KiB/s wr, 33 op/s 2024-09-12T03:31:05.574 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:31:05 smithi064 ceph-mon[30952]: pgmap v57: 129 pgs: 129 active+clean; 2.0 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 36 KiB/s wr, 25 op/s 2024-09-12T03:31:05.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:31:05 smithi115 ceph-mon[34801]: pgmap v57: 129 pgs: 129 active+clean; 2.0 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 36 KiB/s wr, 25 op/s 2024-09-12T03:31:05.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:31:05 smithi195 ceph-mon[34786]: pgmap v57: 129 pgs: 129 active+clean; 2.0 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 36 KiB/s wr, 25 op/s 2024-09-12T03:31:07.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:31:07 smithi115 ceph-mon[34801]: pgmap v58: 129 pgs: 129 active+clean; 2.0 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 36 KiB/s wr, 25 op/s 2024-09-12T03:31:07.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:31:07 smithi195 ceph-mon[34786]: pgmap v58: 129 pgs: 129 active+clean; 2.0 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 36 KiB/s wr, 25 op/s 2024-09-12T03:31:07.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:31:07 smithi064 ceph-mon[30952]: pgmap v58: 129 pgs: 129 active+clean; 2.0 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 36 KiB/s wr, 25 op/s 2024-09-12T03:31:09.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:31:09 smithi115 ceph-mon[34801]: pgmap v59: 129 pgs: 129 active+clean; 2.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 48 KiB/s wr, 33 op/s 2024-09-12T03:31:09.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:31:09 smithi195 ceph-mon[34786]: pgmap v59: 129 pgs: 129 active+clean; 2.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 48 KiB/s wr, 33 op/s 2024-09-12T03:31:09.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:31:09 smithi064 ceph-mon[30952]: pgmap v59: 129 pgs: 129 active+clean; 2.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 48 KiB/s wr, 33 op/s 2024-09-12T03:31:11.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:31:11 smithi115 ceph-mon[34801]: pgmap v60: 129 pgs: 129 active+clean; 2.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 42 KiB/s wr, 28 op/s 2024-09-12T03:31:11.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:31:11 smithi195 ceph-mon[34786]: pgmap v60: 129 pgs: 129 active+clean; 2.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 42 KiB/s wr, 28 op/s 2024-09-12T03:31:11.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:31:11 smithi064 ceph-mon[30952]: pgmap v60: 129 pgs: 129 active+clean; 2.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 42 KiB/s wr, 28 op/s 2024-09-12T03:31:13.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:31:13 smithi115 ceph-mon[34801]: pgmap v61: 129 pgs: 129 active+clean; 2.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 51 KiB/s wr, 33 op/s 2024-09-12T03:31:13.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:31:13 smithi195 ceph-mon[34786]: pgmap v61: 129 pgs: 129 active+clean; 2.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 51 KiB/s wr, 33 op/s 2024-09-12T03:31:13.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:31:13 smithi064 ceph-mon[30952]: pgmap v61: 129 pgs: 129 active+clean; 2.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 51 KiB/s wr, 33 op/s 2024-09-12T03:31:15.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:31:15 smithi115 ceph-mon[34801]: pgmap v62: 129 pgs: 129 active+clean; 2.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-09-12T03:31:15.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:31:15 smithi195 ceph-mon[34786]: pgmap v62: 129 pgs: 129 active+clean; 2.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-09-12T03:31:15.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:31:15 smithi064 ceph-mon[30952]: pgmap v62: 129 pgs: 129 active+clean; 2.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-09-12T03:31:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.447+0000 7f864feba640 1 -- 172.21.15.64:0/705654253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f864810c0f0 msgr2=0x7f864810e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.447+0000 7f864feba640 1 --2- 172.21.15.64:0/705654253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f864810c0f0 0x7f864810e560 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8640009ca0 tx=0x7f864002a950 comp rx=0 tx=0).stop 2024-09-12T03:31:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.447+0000 7f864feba640 1 -- 172.21.15.64:0/705654253 shutdown_connections 2024-09-12T03:31:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.447+0000 7f864feba640 1 --2- 172.21.15.64:0/705654253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f864810c0f0 0x7f864810e560 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.447+0000 7f864feba640 1 --2- 172.21.15.64:0/705654253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8648103640 0x7f864810bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.447+0000 7f864feba640 1 --2- 172.21.15.64:0/705654253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8648102d20 0x7f8648103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.447+0000 7f864feba640 1 -- 172.21.15.64:0/705654253 >> 172.21.15.64:0/705654253 conn(0x7f8648077990 msgr2=0x7f86480fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.447+0000 7f864feba640 1 -- 172.21.15.64:0/705654253 shutdown_connections 2024-09-12T03:31:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.447+0000 7f864feba640 1 -- 172.21.15.64:0/705654253 wait complete. 2024-09-12T03:31:16.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.447+0000 7f864feba640 1 Processor -- start 2024-09-12T03:31:16.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.447+0000 7f864feba640 1 -- start start 2024-09-12T03:31:16.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.448+0000 7f864feba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8648102d20 0x7f8648100970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:16.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.448+0000 7f864feba640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8648103640 0x7f8648100eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:16.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.448+0000 7f864feba640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f864810c0f0 0x7f86481013f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:16.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.448+0000 7f864feba640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8648111d70 con 0x7f8648102d20 2024-09-12T03:31:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.448+0000 7f864feba640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8648111bf0 con 0x7f864810c0f0 2024-09-12T03:31:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.448+0000 7f864feba640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8648111ef0 con 0x7f8648103640 2024-09-12T03:31:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.448+0000 7f864dc56640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8648102d20 0x7f8648100970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.448+0000 7f864dc56640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8648102d20 0x7f8648100970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45798/0 (socket says 172.21.15.64:45798) 2024-09-12T03:31:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.448+0000 7f864dc56640 1 -- 172.21.15.64:0/4143665331 learned_addr learned my addr 172.21.15.64:0/4143665331 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:31:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.448+0000 7f864d455640 1 --2- 172.21.15.64:0/4143665331 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8648103640 0x7f8648100eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.448+0000 7f864dc56640 1 -- 172.21.15.64:0/4143665331 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8648103640 msgr2=0x7f8648100eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.448+0000 7f864dc56640 1 --2- 172.21.15.64:0/4143665331 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8648103640 0x7f8648100eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.448+0000 7f864dc56640 1 -- 172.21.15.64:0/4143665331 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f864810c0f0 msgr2=0x7f86481013f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.448+0000 7f864e457640 1 --2- 172.21.15.64:0/4143665331 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f864810c0f0 0x7f86481013f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:16.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.448+0000 7f864dc56640 1 --2- 172.21.15.64:0/4143665331 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f864810c0f0 0x7f86481013f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.448+0000 7f864dc56640 1 -- 172.21.15.64:0/4143665331 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86481a6700 con 0x7f8648102d20 2024-09-12T03:31:16.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.448+0000 7f864dc56640 1 --2- 172.21.15.64:0/4143665331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8648102d20 0x7f8648100970 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f864400e9f0 tx=0x7f864400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:16.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.449+0000 7f863effd640 1 -- 172.21.15.64:0/4143665331 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864400cd80 con 0x7f8648102d20 2024-09-12T03:31:16.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.449+0000 7f863effd640 1 -- 172.21.15.64:0/4143665331 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86440054d0 con 0x7f8648102d20 2024-09-12T03:31:16.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.449+0000 7f864feba640 1 -- 172.21.15.64:0/4143665331 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86481a69f0 con 0x7f8648102d20 2024-09-12T03:31:16.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.449+0000 7f863effd640 1 -- 172.21.15.64:0/4143665331 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8644010490 con 0x7f8648102d20 2024-09-12T03:31:16.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.449+0000 7f864feba640 1 -- 172.21.15.64:0/4143665331 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86481a6f40 con 0x7f8648102d20 2024-09-12T03:31:16.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.451+0000 7f863effd640 1 -- 172.21.15.64:0/4143665331 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f8644005020 con 0x7f8648102d20 2024-09-12T03:31:16.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.451+0000 7f8652c5c740 1 Processor -- start 2024-09-12T03:31:16.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.452+0000 7f8652c5c740 1 -- start start 2024-09-12T03:31:16.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.452+0000 7f8652c5c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56297b37daf0 0x56297b37ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:16.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.452+0000 7f8652c5c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56297b37e410 0x56297b388770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:16.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.452+0000 7f8652c5c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56297b388cb0 0x56297b38b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:16.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.452+0000 7f8652c5c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56297b2e22f0 con 0x56297b388cb0 2024-09-12T03:31:16.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.452+0000 7f8652c5c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56297b28a270 con 0x56297b37e410 2024-09-12T03:31:16.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.452+0000 7f8652c5c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56297b2de2b0 con 0x56297b37daf0 2024-09-12T03:31:16.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.452+0000 7f863effd640 1 --2- 172.21.15.64:0/4143665331 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f86340a2e20 0x7f86340a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:16.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.452+0000 7f864d455640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56297b388cb0 0x56297b38b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:16.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.452+0000 7f864d455640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56297b388cb0 0x56297b38b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45810/0 (socket says 172.21.15.64:45810) 2024-09-12T03:31:16.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.452+0000 7f864d455640 1 -- 172.21.15.64:0/4274836846 learned_addr learned my addr 172.21.15.64:0/4274836846 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:31:16.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.452+0000 7f864e457640 1 --2- 172.21.15.64:0/4274836846 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56297b37daf0 0x56297b37ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:16.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.452+0000 7f864dc56640 1 --2- 172.21.15.64:0/4274836846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56297b37e410 0x56297b388770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:16.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.452+0000 7f864e457640 1 --2- 172.21.15.64:0/4143665331 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f86340a2e20 0x7f86340a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:16.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.452+0000 7f863effd640 1 -- 172.21.15.64:0/4143665331 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f86440cfbd0 con 0x7f8648102d20 2024-09-12T03:31:16.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.453+0000 7f864d455640 1 -- 172.21.15.64:0/4274836846 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56297b37daf0 msgr2=0x56297b37ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:16.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.453+0000 7f864d455640 1 --2- 172.21.15.64:0/4274836846 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56297b37daf0 0x56297b37ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.453+0000 7f864d455640 1 -- 172.21.15.64:0/4274836846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56297b37e410 msgr2=0x56297b388770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:16.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.453+0000 7f864d455640 1 --2- 172.21.15.64:0/4274836846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56297b37e410 0x56297b388770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.453+0000 7f864d455640 1 -- 172.21.15.64:0/4274836846 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56297b289340 con 0x56297b388cb0 2024-09-12T03:31:16.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.453+0000 7f864d455640 1 --2- 172.21.15.64:0/4274836846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56297b388cb0 0x56297b38b0a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8638009c10 tx=0x7f8638011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3180611600cb086f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:16.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.453+0000 7f864e457640 1 --2- 172.21.15.64:0/4143665331 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f86340a2e20 0x7f86340a52e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8640009a60 tx=0x7f86400095c0 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:16.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.453+0000 7f8633fff640 1 -- 172.21.15.64:0/4274836846 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8638004230 con 0x56297b388cb0 2024-09-12T03:31:16.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.453+0000 7f8633fff640 1 -- 172.21.15.64:0/4274836846 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86380043d0 con 0x56297b388cb0 2024-09-12T03:31:16.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.453+0000 7f8633fff640 1 -- 172.21.15.64:0/4274836846 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8638004de0 con 0x56297b388cb0 2024-09-12T03:31:16.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.454+0000 7f8652c5c740 1 -- 172.21.15.64:0/4274836846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56297b388cb0 msgr2=0x56297b38b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.454+0000 7f8652c5c740 1 --2- 172.21.15.64:0/4274836846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56297b388cb0 0x56297b38b0a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8638009c10 tx=0x7f8638011810 comp rx=0 tx=0).stop 2024-09-12T03:31:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.454+0000 7f8652c5c740 1 -- 172.21.15.64:0/4274836846 shutdown_connections 2024-09-12T03:31:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.454+0000 7f8652c5c740 1 --2- 172.21.15.64:0/4274836846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56297b388cb0 0x56297b38b0a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.454+0000 7f8652c5c740 1 --2- 172.21.15.64:0/4274836846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56297b37e410 0x56297b388770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.454+0000 7f8652c5c740 1 --2- 172.21.15.64:0/4274836846 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56297b37daf0 0x56297b37ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.454+0000 7f8652c5c740 1 -- 172.21.15.64:0/4274836846 >> 172.21.15.64:0/4274836846 conn(0x56297b37a060 msgr2=0x56297b37a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:16.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.454+0000 7f8652c5c740 1 -- 172.21.15.64:0/4274836846 shutdown_connections 2024-09-12T03:31:16.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.454+0000 7f8652c5c740 1 -- 172.21.15.64:0/4274836846 wait complete. 2024-09-12T03:31:16.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.454+0000 7f8652c5c740 1 -- start start 2024-09-12T03:31:16.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.454+0000 7f8652c5c740 1 Processor -- start 2024-09-12T03:31:16.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.455+0000 7f8652c5c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56297b37daf0 0x56297b385ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:16.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.455+0000 7f8652c5c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56297b37e410 0x56297b3860e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:16.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.455+0000 7f8652c5c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56297b388cb0 0x56297b386620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:16.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.455+0000 7f8652c5c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56297b38b880 con 0x56297b388cb0 2024-09-12T03:31:16.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.455+0000 7f8652c5c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56297b26b290 con 0x56297b37e410 2024-09-12T03:31:16.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.455+0000 7f8652c5c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56297b38ba00 con 0x56297b37daf0 2024-09-12T03:31:16.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.455+0000 7f8652c5c740 10 client.0 _ll_register_callbacks cb 0x56297b29c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:31:16.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.455+0000 7f864e457640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56297b388cb0 0x56297b386620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:16.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.455+0000 7f864e457640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56297b388cb0 0x56297b386620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45822/0 (socket says 172.21.15.64:45822) 2024-09-12T03:31:16.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.455+0000 7f864e457640 1 -- 172.21.15.64:0/2236327401 learned_addr learned my addr 172.21.15.64:0/2236327401 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:31:16.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.455+0000 7f864dc56640 1 --2- 172.21.15.64:0/2236327401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56297b37daf0 0x56297b385ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:16.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.455+0000 7f864d455640 1 --2- 172.21.15.64:0/2236327401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56297b37e410 0x56297b3860e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:16.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.455+0000 7f864e457640 1 -- 172.21.15.64:0/2236327401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56297b37daf0 msgr2=0x56297b385ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:16.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.455+0000 7f864e457640 1 --2- 172.21.15.64:0/2236327401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56297b37daf0 0x56297b385ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.455+0000 7f864e457640 1 -- 172.21.15.64:0/2236327401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56297b37e410 msgr2=0x56297b3860e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:16.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.455+0000 7f864e457640 1 --2- 172.21.15.64:0/2236327401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56297b37e410 0x56297b3860e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.455+0000 7f864e457640 1 -- 172.21.15.64:0/2236327401 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56297aff5210 con 0x56297b388cb0 2024-09-12T03:31:16.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.455+0000 7f864dc56640 1 --2- 172.21.15.64:0/2236327401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56297b37daf0 0x56297b385ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:31:16.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.456+0000 7f864e457640 1 --2- 172.21.15.64:0/2236327401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56297b388cb0 0x56297b386620 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f8640003bf0 tx=0x7f864002a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dbc3d000ba25ace server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:16.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.456+0000 7f86317fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:31:16.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.456+0000 7f86317fa640 1 -- 172.21.15.64:0/2236327401 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8640050070 con 0x56297b388cb0 2024-09-12T03:31:16.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.456+0000 7f86317fa640 1 -- 172.21.15.64:0/2236327401 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f864002f870 con 0x56297b388cb0 2024-09-12T03:31:16.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.456+0000 7f8652c5c740 1 -- 172.21.15.64:0/2236327401 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86380043d0 con 0x56297b388cb0 2024-09-12T03:31:16.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.456+0000 7f86317fa640 1 -- 172.21.15.64:0/2236327401 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8640034040 con 0x56297b388cb0 2024-09-12T03:31:16.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.456+0000 7f8652c5c740 10 client.15504 fetch_fsmap learned FSMap version 25 2024-09-12T03:31:16.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.456+0000 7f8652c5c740 1 -- 172.21.15.64:0/2236327401 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56297b0c1620 con 0x56297b388cb0 2024-09-12T03:31:16.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.456+0000 7f86317fa640 1 -- 172.21.15.64:0/2236327401 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86380043d0 con 0x56297b388cb0 2024-09-12T03:31:16.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.456+0000 7f86317fa640 1 -- 172.21.15.64:0/2236327401 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f8640005020 con 0x56297b388cb0 2024-09-12T03:31:16.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.456+0000 7f8652c5c740 10 client.15504 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:31:16.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.456+0000 7f8652c5c740 10 client.15504 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:31:16.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.456+0000 7f8652c5c740 20 client.15504 populate_metadata read hostname 'smithi064' 2024-09-12T03:31:16.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.456+0000 7f8652c5c740 1 --2- 172.21.15.64:0/2236327401 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x56297b398bb0 0x56297b39b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:16.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.457+0000 7f8652c5c740 4 client.15504 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:31:16.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.457+0000 7f8652c5c740 1 -- 172.21.15.64:0/2236327401 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56297b2efa20 con 0x56297b398bb0 2024-09-12T03:31:16.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.457+0000 7f864dc56640 1 --2- 172.21.15.64:0/2236327401 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x56297b398bb0 0x56297b39b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:16.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.457+0000 7f86317fa640 10 client.15504 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:31:16.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.457+0000 7f864dc56640 1 --2- 172.21.15.64:0/2236327401 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x56297b398bb0 0x56297b39b020 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c450e13a22bc370 server_cookie=1f0f6cd0df24c8ed in_seq=0 out_seq=0 2024-09-12T03:31:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.458+0000 7f86317fa640 1 -- 172.21.15.64:0/2236327401 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56297b2efa20 con 0x56297b398bb0 2024-09-12T03:31:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.458+0000 7f86317fa640 10 client.15504 handle_command_reply: tid=0 2024-09-12T03:31:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.458+0000 7f8652c5c740 1 client.15504 shutdown 2024-09-12T03:31:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.458+0000 7f8652c5c740 1 -- 172.21.15.64:0/2236327401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56297b388cb0 msgr2=0x56297b386620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.458+0000 7f8652c5c740 1 --2- 172.21.15.64:0/2236327401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56297b388cb0 0x56297b386620 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f8640003bf0 tx=0x7f864002a950 comp rx=0 tx=0).stop 2024-09-12T03:31:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.458+0000 7f8652c5c740 1 -- 172.21.15.64:0/2236327401 shutdown_connections 2024-09-12T03:31:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.458+0000 7f8652c5c740 1 --2- 172.21.15.64:0/2236327401 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x56297b398bb0 0x56297b39b020 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.458+0000 7f8652c5c740 1 --2- 172.21.15.64:0/2236327401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56297b388cb0 0x56297b386620 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.458+0000 7f8652c5c740 1 --2- 172.21.15.64:0/2236327401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56297b37e410 0x56297b3860e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.458+0000 7f8652c5c740 1 --2- 172.21.15.64:0/2236327401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56297b37daf0 0x56297b385ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.458+0000 7f8652c5c740 1 -- 172.21.15.64:0/2236327401 >> 172.21.15.64:0/2236327401 conn(0x56297b37a060 msgr2=0x56297b37fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.458+0000 7f86317fa640 0 client.15504 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:31:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.458+0000 7f8652c5c740 1 -- 172.21.15.64:0/2236327401 shutdown_connections 2024-09-12T03:31:16.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.458+0000 7f8652c5c740 1 -- 172.21.15.64:0/2236327401 wait complete. 2024-09-12T03:31:16.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.458+0000 7f8652c5c740 20 client.15504 trim_cache size 0 max 16384 2024-09-12T03:31:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.473+0000 7f8652c5c740 1 Processor -- start 2024-09-12T03:31:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.473+0000 7f8652c5c740 1 -- start start 2024-09-12T03:31:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.473+0000 7f8652c5c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56297b37daf0 0x56297b386450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.473+0000 7f8652c5c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56297b37e410 0x56297b3a0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.473+0000 7f8652c5c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56297b388cb0 0x56297b3a0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.473+0000 7f8652c5c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f86481b39e0 con 0x56297b37e410 2024-09-12T03:31:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.473+0000 7f8652c5c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f86481b12b0 con 0x56297b388cb0 2024-09-12T03:31:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.473+0000 7f8652c5c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f86481af100 con 0x56297b37daf0 2024-09-12T03:31:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.473+0000 7f864d455640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56297b37e410 0x56297b3a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.473+0000 7f864d455640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56297b37e410 0x56297b3a0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45838/0 (socket says 172.21.15.64:45838) 2024-09-12T03:31:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.473+0000 7f864d455640 1 -- 172.21.15.64:0/165246769 learned_addr learned my addr 172.21.15.64:0/165246769 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:31:16.476 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.473+0000 7f864dc56640 1 --2- 172.21.15.64:0/165246769 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56297b37daf0 0x56297b386450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:16.476 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.473+0000 7f864d455640 1 -- 172.21.15.64:0/165246769 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56297b37daf0 msgr2=0x56297b386450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:16.476 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.473+0000 7f864d455640 1 --2- 172.21.15.64:0/165246769 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56297b37daf0 0x56297b386450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.476 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.473+0000 7f864d455640 1 -- 172.21.15.64:0/165246769 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56297b388cb0 msgr2=0x56297b3a0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:16.476 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.473+0000 7f864d455640 1 --2- 172.21.15.64:0/165246769 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56297b388cb0 0x56297b3a0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.476 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.473+0000 7f864d455640 1 -- 172.21.15.64:0/165246769 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56297b379b00 con 0x56297b37e410 2024-09-12T03:31:16.476 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.474+0000 7f864d455640 1 --2- 172.21.15.64:0/165246769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56297b37e410 0x56297b3a0980 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8638011d10 tx=0x7f86380049e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d86056b7d84e9b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:16.476 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.474+0000 7f86337fe640 1 -- 172.21.15.64:0/165246769 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86380097a0 con 0x56297b37e410 2024-09-12T03:31:16.476 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.474+0000 7f86337fe640 1 -- 172.21.15.64:0/165246769 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f863801f040 con 0x56297b37e410 2024-09-12T03:31:16.477 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.474+0000 7f86337fe640 1 -- 172.21.15.64:0/165246769 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8638003bc0 con 0x56297b37e410 2024-09-12T03:31:16.477 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.474+0000 7f8652c5c740 1 -- 172.21.15.64:0/165246769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56297b37e410 msgr2=0x56297b3a0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:16.477 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.474+0000 7f8652c5c740 1 --2- 172.21.15.64:0/165246769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56297b37e410 0x56297b3a0980 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8638011d10 tx=0x7f86380049e0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.477 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.474+0000 7f8652c5c740 1 -- 172.21.15.64:0/165246769 shutdown_connections 2024-09-12T03:31:16.477 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.474+0000 7f8652c5c740 1 --2- 172.21.15.64:0/165246769 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56297b388cb0 0x56297b3a0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.477 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.474+0000 7f8652c5c740 1 --2- 172.21.15.64:0/165246769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56297b37e410 0x56297b3a0980 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.477 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.474+0000 7f8652c5c740 1 --2- 172.21.15.64:0/165246769 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56297b37daf0 0x56297b386450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.477 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.474+0000 7f8652c5c740 1 -- 172.21.15.64:0/165246769 >> 172.21.15.64:0/165246769 conn(0x56297b37a060 msgr2=0x56297b387c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:16.477 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.474+0000 7f8652c5c740 1 -- 172.21.15.64:0/165246769 shutdown_connections 2024-09-12T03:31:16.477 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.474+0000 7f8652c5c740 1 -- 172.21.15.64:0/165246769 wait complete. 2024-09-12T03:31:16.477 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.475+0000 7f8652c5c740 1 -- start start 2024-09-12T03:31:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.475+0000 7f8652c5c740 1 Processor -- start 2024-09-12T03:31:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.475+0000 7f8652c5c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56297b37daf0 0x56297b391100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.475+0000 7f8652c5c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56297b37e410 0x56297b382650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.475+0000 7f8652c5c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56297b388cb0 0x56297b382b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.475+0000 7f8652c5c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f86481af770 con 0x56297b37daf0 2024-09-12T03:31:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.475+0000 7f8652c5c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f86481b0be0 con 0x56297b37e410 2024-09-12T03:31:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.475+0000 7f8652c5c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f86481afe40 con 0x56297b388cb0 2024-09-12T03:31:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.475+0000 7f8652c5c740 10 client.0 _ll_register_callbacks cb 0x7f8644005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:31:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.475+0000 7f864dc56640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56297b37daf0 0x56297b391100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.475+0000 7f864dc56640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56297b37daf0 0x56297b391100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45842/0 (socket says 172.21.15.64:45842) 2024-09-12T03:31:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.475+0000 7f864dc56640 1 -- 172.21.15.64:0/1188543552 learned_addr learned my addr 172.21.15.64:0/1188543552 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:31:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.476+0000 7f864dc56640 1 -- 172.21.15.64:0/1188543552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56297b388cb0 msgr2=0x56297b382b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.476+0000 7f864dc56640 1 --2- 172.21.15.64:0/1188543552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56297b388cb0 0x56297b382b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.476+0000 7f864dc56640 1 -- 172.21.15.64:0/1188543552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56297b37e410 msgr2=0x56297b382650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.476+0000 7f864d455640 1 --2- 172.21.15.64:0/1188543552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56297b37e410 0x56297b382650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.476+0000 7f864dc56640 1 --2- 172.21.15.64:0/1188543552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56297b37e410 0x56297b382650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.476+0000 7f864dc56640 1 -- 172.21.15.64:0/1188543552 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56297b383100 con 0x56297b37daf0 2024-09-12T03:31:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.476+0000 7f864dc56640 1 --2- 172.21.15.64:0/1188543552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56297b37daf0 0x56297b391100 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f86440986f0 tx=0x7f864406ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b82aa48b0e897b47 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.476+0000 7f86317fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:31:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.476+0000 7f86317fa640 1 -- 172.21.15.64:0/1188543552 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8644069070 con 0x56297b37daf0 2024-09-12T03:31:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.476+0000 7f86317fa640 1 -- 172.21.15.64:0/1188543552 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f86440ba6e0 con 0x56297b37daf0 2024-09-12T03:31:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.476+0000 7f8652c5c740 1 -- 172.21.15.64:0/1188543552 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f863801f040 con 0x56297b37daf0 2024-09-12T03:31:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.476+0000 7f86317fa640 1 -- 172.21.15.64:0/1188543552 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8644066e70 con 0x56297b37daf0 2024-09-12T03:31:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.476+0000 7f8652c5c740 10 client.15510 fetch_fsmap learned FSMap version 25 2024-09-12T03:31:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.476+0000 7f8652c5c740 1 -- 172.21.15.64:0/1188543552 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56297b403760 con 0x56297b37daf0 2024-09-12T03:31:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.476+0000 7f86317fa640 1 -- 172.21.15.64:0/1188543552 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f863801f040 con 0x56297b37daf0 2024-09-12T03:31:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.477+0000 7f86317fa640 1 -- 172.21.15.64:0/1188543552 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f86440669d0 con 0x56297b37daf0 2024-09-12T03:31:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.477+0000 7f8652c5c740 10 client.15510 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:31:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.477+0000 7f8652c5c740 10 client.15510 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:31:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.477+0000 7f8652c5c740 20 client.15510 populate_metadata read hostname 'smithi064' 2024-09-12T03:31:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.477+0000 7f8652c5c740 1 --2- 172.21.15.64:0/1188543552 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x56297b403e90 0x56297b4062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.477+0000 7f8652c5c740 4 client.15510 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:31:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.477+0000 7f8652c5c740 1 -- 172.21.15.64:0/1188543552 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56297b2e9d70 con 0x56297b403e90 2024-09-12T03:31:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.477+0000 7f864d455640 1 --2- 172.21.15.64:0/1188543552 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x56297b403e90 0x56297b4062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.477+0000 7f86317fa640 10 client.15510 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:31:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.477+0000 7f864d455640 1 --2- 172.21.15.64:0/1188543552 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x56297b403e90 0x56297b4062d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e1a6e2dda0615e3 server_cookie=626aa32e8455a423 in_seq=0 out_seq=0 2024-09-12T03:31:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.490+0000 7f86317fa640 1 -- 172.21.15.64:0/1188543552 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56297b2e9d70 con 0x56297b403e90 2024-09-12T03:31:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.490+0000 7f86317fa640 10 client.15510 handle_command_reply: tid=0 2024-09-12T03:31:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.490+0000 7f8652c5c740 1 client.15510 shutdown 2024-09-12T03:31:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.491+0000 7f8652c5c740 1 -- 172.21.15.64:0/1188543552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56297b37daf0 msgr2=0x56297b391100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.491+0000 7f8652c5c740 1 --2- 172.21.15.64:0/1188543552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56297b37daf0 0x56297b391100 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f86440986f0 tx=0x7f864406ad10 comp rx=0 tx=0).stop 2024-09-12T03:31:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.491+0000 7f8652c5c740 1 -- 172.21.15.64:0/1188543552 shutdown_connections 2024-09-12T03:31:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.491+0000 7f8652c5c740 1 --2- 172.21.15.64:0/1188543552 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x56297b403e90 0x56297b4062d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.491+0000 7f8652c5c740 1 --2- 172.21.15.64:0/1188543552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56297b388cb0 0x56297b382b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.491+0000 7f8652c5c740 1 --2- 172.21.15.64:0/1188543552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56297b37e410 0x56297b382650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.491+0000 7f86317fa640 0 client.15510 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:31:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.491+0000 7f8652c5c740 1 --2- 172.21.15.64:0/1188543552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56297b37daf0 0x56297b391100 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.491+0000 7f8652c5c740 1 -- 172.21.15.64:0/1188543552 >> 172.21.15.64:0/1188543552 conn(0x56297b37a060 msgr2=0x56297b384010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.491+0000 7f8652c5c740 1 -- 172.21.15.64:0/1188543552 shutdown_connections 2024-09-12T03:31:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.491+0000 7f8652c5c740 1 -- 172.21.15.64:0/1188543552 wait complete. 2024-09-12T03:31:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.491+0000 7f8652c5c740 20 client.15510 trim_cache size 0 max 16384 2024-09-12T03:31:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:31:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:31:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:31:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:31:16.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.491+0000 7f864feba640 1 -- 172.21.15.64:0/4143665331 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f86340a2e20 msgr2=0x7f86340a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:16.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.491+0000 7f864feba640 1 --2- 172.21.15.64:0/4143665331 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f86340a2e20 0x7f86340a52e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8640009a60 tx=0x7f86400095c0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.491+0000 7f864feba640 1 -- 172.21.15.64:0/4143665331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8648102d20 msgr2=0x7f8648100970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:16.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.491+0000 7f864feba640 1 --2- 172.21.15.64:0/4143665331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8648102d20 0x7f8648100970 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f864400e9f0 tx=0x7f864400edf0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.492+0000 7f864feba640 1 -- 172.21.15.64:0/4143665331 shutdown_connections 2024-09-12T03:31:16.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.492+0000 7f864feba640 1 --2- 172.21.15.64:0/4143665331 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f86340a2e20 0x7f86340a52e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.492+0000 7f864feba640 1 --2- 172.21.15.64:0/4143665331 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f864810c0f0 0x7f86481013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.492+0000 7f864feba640 1 --2- 172.21.15.64:0/4143665331 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8648103640 0x7f8648100eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.492+0000 7f864feba640 1 --2- 172.21.15.64:0/4143665331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8648102d20 0x7f8648100970 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:16.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.492+0000 7f864feba640 1 -- 172.21.15.64:0/4143665331 >> 172.21.15.64:0/4143665331 conn(0x7f8648077990 msgr2=0x7f86480fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:16.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.492+0000 7f864feba640 1 -- 172.21.15.64:0/4143665331 shutdown_connections 2024-09-12T03:31:16.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:16.492+0000 7f864feba640 1 -- 172.21.15.64:0/4143665331 wait complete. 2024-09-12T03:31:17.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:31:17 smithi115 ceph-mon[34801]: pgmap v63: 129 pgs: 129 active+clean; 2.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-09-12T03:31:17.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:31:17 smithi195 ceph-mon[34786]: pgmap v63: 129 pgs: 129 active+clean; 2.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-09-12T03:31:17.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:31:17 smithi064 ceph-mon[30952]: pgmap v63: 129 pgs: 129 active+clean; 2.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-09-12T03:31:19.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:31:19 smithi115 ceph-mon[34801]: pgmap v64: 129 pgs: 129 active+clean; 2.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 34 KiB/s wr, 20 op/s 2024-09-12T03:31:19.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:31:19 smithi195 ceph-mon[34786]: pgmap v64: 129 pgs: 129 active+clean; 2.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 34 KiB/s wr, 20 op/s 2024-09-12T03:31:19.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:31:19 smithi064 ceph-mon[30952]: pgmap v64: 129 pgs: 129 active+clean; 2.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 34 KiB/s wr, 20 op/s 2024-09-12T03:31:21.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:31:21 smithi115 ceph-mon[34801]: pgmap v65: 129 pgs: 129 active+clean; 2.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-09-12T03:31:21.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:31:21 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:31:21.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:31:21 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:31:21.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:31:21 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:31:21.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:31:21 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:31:21.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:31:21 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:31:21.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:31:21 smithi195 ceph-mon[34786]: pgmap v65: 129 pgs: 129 active+clean; 2.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-09-12T03:31:21.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:31:21 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:31:21.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:31:21 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:31:21.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:31:21 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:31:21.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:31:21 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:31:21.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:31:21 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:31:21.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:31:21 smithi064 ceph-mon[30952]: pgmap v65: 129 pgs: 129 active+clean; 2.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-09-12T03:31:21.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:31:21 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:31:21.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:31:21 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:31:21.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:31:21 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:31:21.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:31:21 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:31:21.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:31:21 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:31:22.476 INFO:tasks.workunit.client.0.smithi064.stdout:1/600: open fd = 3 2024-09-12T03:31:22.477 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.477 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.477 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.477 INFO:tasks.workunit.client.0.smithi064.stdout:2/600: open fd = 3 2024-09-12T03:31:22.477 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.477 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.478 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.478 INFO:tasks.workunit.client.0.smithi064.stdout:3/600: open fd = 3 2024-09-12T03:31:22.478 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.478 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.478 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.478 INFO:tasks.workunit.client.0.smithi064.stdout:4/600: open fd = 3 2024-09-12T03:31:22.478 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.478 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.478 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.479 INFO:tasks.workunit.client.0.smithi064.stdout:5/600: open fd = 3 2024-09-12T03:31:22.479 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.479 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.479 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.479 INFO:tasks.workunit.client.0.smithi064.stdout:6/600: open fd = 3 2024-09-12T03:31:22.479 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.479 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.479 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.479 INFO:tasks.workunit.client.0.smithi064.stdout:7/600: open fd = 3 2024-09-12T03:31:22.479 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.479 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.479 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.479 INFO:tasks.workunit.client.0.smithi064.stdout:8/600: open fd = 3 2024-09-12T03:31:22.480 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.480 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.480 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.480 INFO:tasks.workunit.client.0.smithi064.stdout:9/600: open fd = 3 2024-09-12T03:31:22.480 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.480 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.480 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.480 INFO:tasks.workunit.client.0.smithi064.stdout:10/600: open fd = 3 2024-09-12T03:31:22.481 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.481 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.481 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.481 INFO:tasks.workunit.client.0.smithi064.stdout:11/600: open fd = 3 2024-09-12T03:31:22.481 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.481 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.481 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.481 INFO:tasks.workunit.client.0.smithi064.stdout:12/600: open fd = 3 2024-09-12T03:31:22.481 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.481 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.481 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.481 INFO:tasks.workunit.client.0.smithi064.stdout:13/600: open fd = 3 2024-09-12T03:31:22.482 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.482 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.482 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.482 INFO:tasks.workunit.client.0.smithi064.stdout:14/600: open fd = 3 2024-09-12T03:31:22.482 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.482 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.482 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.482 INFO:tasks.workunit.client.0.smithi064.stdout:15/600: open fd = 3 2024-09-12T03:31:22.482 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.482 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.482 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.482 INFO:tasks.workunit.client.0.smithi064.stdout:16/600: open fd = 3 2024-09-12T03:31:22.482 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.482 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.483 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.483 INFO:tasks.workunit.client.0.smithi064.stdout:17/600: open fd = 3 2024-09-12T03:31:22.483 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.483 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.483 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.483 INFO:tasks.workunit.client.0.smithi064.stdout:18/600: open fd = 3 2024-09-12T03:31:22.483 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.483 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.483 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.483 INFO:tasks.workunit.client.0.smithi064.stdout:19/600: open fd = 3 2024-09-12T03:31:22.483 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.483 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.483 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.484 INFO:tasks.workunit.client.0.smithi064.stdout:20/600: open fd = 3 2024-09-12T03:31:22.484 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.484 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.484 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.484 INFO:tasks.workunit.client.0.smithi064.stdout:21/600: open fd = 3 2024-09-12T03:31:22.484 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.484 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.484 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.484 INFO:tasks.workunit.client.0.smithi064.stdout:22/600: open fd = 3 2024-09-12T03:31:22.484 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.484 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.484 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.484 INFO:tasks.workunit.client.0.smithi064.stdout:23/600: open fd = 3 2024-09-12T03:31:22.485 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.485 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.485 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.485 INFO:tasks.workunit.client.0.smithi064.stdout:24/600: open fd = 3 2024-09-12T03:31:22.485 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.485 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.485 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.485 INFO:tasks.workunit.client.0.smithi064.stdout:25/600: open fd = 3 2024-09-12T03:31:22.485 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.485 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.485 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.485 INFO:tasks.workunit.client.0.smithi064.stdout:26/600: open fd = 3 2024-09-12T03:31:22.485 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.485 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.486 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.486 INFO:tasks.workunit.client.0.smithi064.stdout:27/600: open fd = 3 2024-09-12T03:31:22.486 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.486 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.486 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.486 INFO:tasks.workunit.client.0.smithi064.stdout:28/600: open fd = 3 2024-09-12T03:31:22.486 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.486 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.486 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.486 INFO:tasks.workunit.client.0.smithi064.stdout:29/600: open fd = 3 2024-09-12T03:31:22.486 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.487 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.487 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.487 INFO:tasks.workunit.client.0.smithi064.stdout:30/600: open fd = 3 2024-09-12T03:31:22.487 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.487 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.487 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.487 INFO:tasks.workunit.client.0.smithi064.stdout:31/600: open fd = 3 2024-09-12T03:31:22.487 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.487 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.488 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.488 INFO:tasks.workunit.client.0.smithi064.stdout:32/600: open fd = 3 2024-09-12T03:31:22.488 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.488 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.488 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.488 INFO:tasks.workunit.client.0.smithi064.stdout:33/600: open fd = 3 2024-09-12T03:31:22.488 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.488 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.488 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.488 INFO:tasks.workunit.client.0.smithi064.stdout:34/600: open fd = 3 2024-09-12T03:31:22.488 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.489 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.489 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.489 INFO:tasks.workunit.client.0.smithi064.stdout:35/600: open fd = 3 2024-09-12T03:31:22.489 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.489 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.489 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.489 INFO:tasks.workunit.client.0.smithi064.stdout:36/600: open fd = 3 2024-09-12T03:31:22.489 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.489 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.489 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.490 INFO:tasks.workunit.client.0.smithi064.stdout:37/600: open fd = 3 2024-09-12T03:31:22.490 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.490 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.490 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.490 INFO:tasks.workunit.client.0.smithi064.stdout:38/600: open fd = 3 2024-09-12T03:31:22.490 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.490 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.490 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.490 INFO:tasks.workunit.client.0.smithi064.stdout:39/600: open fd = 3 2024-09-12T03:31:22.490 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.490 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.490 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.490 INFO:tasks.workunit.client.0.smithi064.stdout:40/600: open fd = 3 2024-09-12T03:31:22.490 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.491 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.491 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.491 INFO:tasks.workunit.client.0.smithi064.stdout:41/600: open fd = 3 2024-09-12T03:31:22.491 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.491 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.491 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.491 INFO:tasks.workunit.client.0.smithi064.stdout:42/600: open fd = 3 2024-09-12T03:31:22.491 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.491 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.491 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.491 INFO:tasks.workunit.client.0.smithi064.stdout:43/600: open fd = 3 2024-09-12T03:31:22.491 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.491 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.491 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.492 INFO:tasks.workunit.client.0.smithi064.stdout:44/600: open fd = 3 2024-09-12T03:31:22.492 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.492 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.492 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.492 INFO:tasks.workunit.client.0.smithi064.stdout:45/600: open fd = 3 2024-09-12T03:31:22.492 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.492 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.492 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.492 INFO:tasks.workunit.client.0.smithi064.stdout:46/600: open fd = 3 2024-09-12T03:31:22.492 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.492 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.492 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.492 INFO:tasks.workunit.client.0.smithi064.stdout:47/600: open fd = 3 2024-09-12T03:31:22.492 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.493 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.493 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.493 INFO:tasks.workunit.client.0.smithi064.stdout:48/600: open fd = 3 2024-09-12T03:31:22.493 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.493 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.493 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.493 INFO:tasks.workunit.client.0.smithi064.stdout:49/600: open fd = 3 2024-09-12T03:31:22.493 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.493 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.493 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.493 INFO:tasks.workunit.client.0.smithi064.stdout:50/600: open fd = 3 2024-09-12T03:31:22.493 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.493 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.493 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.494 INFO:tasks.workunit.client.0.smithi064.stdout:51/600: open fd = 3 2024-09-12T03:31:22.494 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.494 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.494 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.494 INFO:tasks.workunit.client.0.smithi064.stdout:52/600: open fd = 3 2024-09-12T03:31:22.494 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.494 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.494 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.494 INFO:tasks.workunit.client.0.smithi064.stdout:53/600: open fd = 3 2024-09-12T03:31:22.494 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.494 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.494 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.494 INFO:tasks.workunit.client.0.smithi064.stdout:54/600: open fd = 3 2024-09-12T03:31:22.495 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.495 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.495 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.495 INFO:tasks.workunit.client.0.smithi064.stdout:55/600: open fd = 3 2024-09-12T03:31:22.495 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.495 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.495 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.495 INFO:tasks.workunit.client.0.smithi064.stdout:56/600: open fd = 3 2024-09-12T03:31:22.495 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.495 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.495 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.495 INFO:tasks.workunit.client.0.smithi064.stdout:57/600: open fd = 3 2024-09-12T03:31:22.495 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.495 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.496 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.496 INFO:tasks.workunit.client.0.smithi064.stdout:58/600: open fd = 3 2024-09-12T03:31:22.496 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.496 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.496 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.496 INFO:tasks.workunit.client.0.smithi064.stdout:59/600: open fd = 3 2024-09-12T03:31:22.496 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.496 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.496 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.496 INFO:tasks.workunit.client.0.smithi064.stdout:60/600: open fd = 3 2024-09-12T03:31:22.496 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.496 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.496 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.497 INFO:tasks.workunit.client.0.smithi064.stdout:61/600: open fd = 3 2024-09-12T03:31:22.497 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.497 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.497 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.497 INFO:tasks.workunit.client.0.smithi064.stdout:62/600: open fd = 3 2024-09-12T03:31:22.497 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.497 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.497 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.497 INFO:tasks.workunit.client.0.smithi064.stdout:63/600: open fd = 3 2024-09-12T03:31:22.497 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.497 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:31:22.497 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:31:22.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:31:22 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:31:22.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:31:22 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:31:22.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:31:22 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:31:22.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:31:22 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:31:22.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:31:22 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:31:22.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:31:22 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:31:22.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:31:22 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:31:22.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:31:22 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:31:22.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:31:22 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:31:23.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:31:23 smithi115 ceph-mon[34801]: pgmap v66: 129 pgs: 129 active+clean; 2.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 KiB/s wr, 17 op/s 2024-09-12T03:31:23.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:31:23 smithi195 ceph-mon[34786]: pgmap v66: 129 pgs: 129 active+clean; 2.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 KiB/s wr, 17 op/s 2024-09-12T03:31:23.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:31:23 smithi064 ceph-mon[30952]: pgmap v66: 129 pgs: 129 active+clean; 2.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 KiB/s wr, 17 op/s 2024-09-12T03:31:25.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:31:25 smithi115 ceph-mon[34801]: pgmap v67: 129 pgs: 129 active+clean; 2.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-09-12T03:31:25.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:31:25 smithi195 ceph-mon[34786]: pgmap v67: 129 pgs: 129 active+clean; 2.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-09-12T03:31:25.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:31:25 smithi064 ceph-mon[30952]: pgmap v67: 129 pgs: 129 active+clean; 2.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-09-12T03:31:27.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:31:27 smithi115 ceph-mon[34801]: pgmap v68: 129 pgs: 129 active+clean; 2.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-09-12T03:31:27.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:31:27 smithi195 ceph-mon[34786]: pgmap v68: 129 pgs: 129 active+clean; 2.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-09-12T03:31:27.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:31:27 smithi064 ceph-mon[30952]: pgmap v68: 129 pgs: 129 active+clean; 2.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-09-12T03:31:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.592+0000 7fbfa6423640 1 -- 172.21.15.64:0/3287530407 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa01069c0 msgr2=0x7fbfa0106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.592+0000 7fbfa6423640 1 --2- 172.21.15.64:0/3287530407 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa01069c0 0x7fbfa0106da0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fbf90009930 tx=0x7fbf9002a420 comp rx=0 tx=0).stop 2024-09-12T03:31:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.592+0000 7fbfa6423640 1 -- 172.21.15.64:0/3287530407 shutdown_connections 2024-09-12T03:31:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.592+0000 7fbfa6423640 1 --2- 172.21.15.64:0/3287530407 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfa0101350 0x7fbfa010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.592+0000 7fbfa6423640 1 --2- 172.21.15.64:0/3287530407 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfa01009b0 0x7fbfa0100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.592+0000 7fbfa6423640 1 --2- 172.21.15.64:0/3287530407 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa01069c0 0x7fbfa0106da0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.592+0000 7fbfa6423640 1 -- 172.21.15.64:0/3287530407 >> 172.21.15.64:0/3287530407 conn(0x7fbfa0077990 msgr2=0x7fbfa00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.592+0000 7fbfa6423640 1 -- 172.21.15.64:0/3287530407 shutdown_connections 2024-09-12T03:31:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.592+0000 7fbfa6423640 1 -- 172.21.15.64:0/3287530407 wait complete. 2024-09-12T03:31:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.593+0000 7fbfa6423640 1 Processor -- start 2024-09-12T03:31:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.593+0000 7fbfa6423640 1 -- start start 2024-09-12T03:31:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.593+0000 7fbfa6423640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfa01009b0 0x7fbfa01982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.593+0000 7fbfa6423640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa0101350 0x7fbfa01987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.593+0000 7fbfa6423640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfa01069c0 0x7fbfa019cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.593+0000 7fbfa6423640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa0110cb0 con 0x7fbfa0101350 2024-09-12T03:31:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.593+0000 7fbfa6423640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa0110b30 con 0x7fbfa01069c0 2024-09-12T03:31:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.593+0000 7fbfa6423640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa0110e30 con 0x7fbfa01009b0 2024-09-12T03:31:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.593+0000 7fbf9f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa0101350 0x7fbfa01987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:29.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.593+0000 7fbf9f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa0101350 0x7fbfa01987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60824/0 (socket says 172.21.15.64:60824) 2024-09-12T03:31:29.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.593+0000 7fbf9f7fe640 1 -- 172.21.15.64:0/4071614129 learned_addr learned my addr 172.21.15.64:0/4071614129 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:31:29.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.593+0000 7fbf9ffff640 1 --2- 172.21.15.64:0/4071614129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfa01009b0 0x7fbfa01982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:29.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.594+0000 7fbfa49c0640 1 --2- 172.21.15.64:0/4071614129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfa01069c0 0x7fbfa019cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:29.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.594+0000 7fbf9f7fe640 1 -- 172.21.15.64:0/4071614129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfa01009b0 msgr2=0x7fbfa01982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:29.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.594+0000 7fbf9f7fe640 1 --2- 172.21.15.64:0/4071614129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfa01009b0 0x7fbfa01982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.594+0000 7fbf9f7fe640 1 -- 172.21.15.64:0/4071614129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfa01069c0 msgr2=0x7fbfa019cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:29.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.594+0000 7fbf9f7fe640 1 --2- 172.21.15.64:0/4071614129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfa01069c0 0x7fbfa019cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.594+0000 7fbf9f7fe640 1 -- 172.21.15.64:0/4071614129 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfa019d290 con 0x7fbfa0101350 2024-09-12T03:31:29.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.594+0000 7fbf9f7fe640 1 --2- 172.21.15.64:0/4071614129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa0101350 0x7fbfa01987e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fbf9400e9f0 tx=0x7fbf9400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:29.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.594+0000 7fbf9d7fa640 1 -- 172.21.15.64:0/4071614129 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9400cd80 con 0x7fbfa0101350 2024-09-12T03:31:29.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.594+0000 7fbfa6423640 1 -- 172.21.15.64:0/4071614129 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfa019d580 con 0x7fbfa0101350 2024-09-12T03:31:29.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.594+0000 7fbf9d7fa640 1 -- 172.21.15.64:0/4071614129 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf940108e0 con 0x7fbfa0101350 2024-09-12T03:31:29.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.594+0000 7fbf9d7fa640 1 -- 172.21.15.64:0/4071614129 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf94018920 con 0x7fbfa0101350 2024-09-12T03:31:29.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.594+0000 7fbfa6423640 1 -- 172.21.15.64:0/4071614129 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfa01a4e10 con 0x7fbfa0101350 2024-09-12T03:31:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.596+0000 7fbf9d7fa640 1 -- 172.21.15.64:0/4071614129 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fbf94010a90 con 0x7fbfa0101350 2024-09-12T03:31:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.597+0000 7fbfa91bd740 1 Processor -- start 2024-09-12T03:31:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.597+0000 7fbfa91bd740 1 -- start start 2024-09-12T03:31:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.597+0000 7fbfa91bd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561116012af0 0x561116012ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.597+0000 7fbfa91bd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561116013410 0x56111601d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.597+0000 7fbfa91bd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56111601dcb0 0x5611160200a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.597+0000 7fbfa91bd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561115f772f0 con 0x561116013410 2024-09-12T03:31:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.597+0000 7fbfa91bd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561115f1f270 con 0x561116012af0 2024-09-12T03:31:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.597+0000 7fbfa91bd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561115f732b0 con 0x56111601dcb0 2024-09-12T03:31:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.597+0000 7fbf9d7fa640 1 --2- 172.21.15.64:0/4071614129 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fbf880a2da0 0x7fbf880a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.597+0000 7fbf9ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561116013410 0x56111601d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.597+0000 7fbf9ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561116013410 0x56111601d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60836/0 (socket says 172.21.15.64:60836) 2024-09-12T03:31:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.597+0000 7fbf9ffff640 1 -- 172.21.15.64:0/617211719 learned_addr learned my addr 172.21.15.64:0/617211719 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:31:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.597+0000 7fbfa49c0640 1 --2- 172.21.15.64:0/4071614129 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fbf880a2da0 0x7fbf880a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.598+0000 7fbf9d7fa640 1 -- 172.21.15.64:0/4071614129 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7fbf940cfba0 con 0x7fbfa0101350 2024-09-12T03:31:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.598+0000 7fbfa49c0640 1 --2- 172.21.15.64:0/617211719 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561116012af0 0x561116012ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.598+0000 7fbf9f7fe640 1 --2- 172.21.15.64:0/617211719 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56111601dcb0 0x5611160200a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.598+0000 7fbf9ffff640 1 -- 172.21.15.64:0/617211719 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56111601dcb0 msgr2=0x5611160200a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.598+0000 7fbf9ffff640 1 --2- 172.21.15.64:0/617211719 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56111601dcb0 0x5611160200a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.598+0000 7fbf9ffff640 1 -- 172.21.15.64:0/617211719 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561116012af0 msgr2=0x561116012ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:29.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.598+0000 7fbf9ffff640 1 --2- 172.21.15.64:0/617211719 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561116012af0 0x561116012ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.598+0000 7fbf9ffff640 1 -- 172.21.15.64:0/617211719 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561115f1e340 con 0x561116013410 2024-09-12T03:31:29.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.598+0000 7fbfa49c0640 1 --2- 172.21.15.64:0/617211719 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561116012af0 0x561116012ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:31:29.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.598+0000 7fbf9f7fe640 1 --2- 172.21.15.64:0/617211719 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56111601dcb0 0x5611160200a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:31:29.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.598+0000 7fbfa49c0640 1 --2- 172.21.15.64:0/4071614129 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fbf880a2da0 0x7fbf880a5260 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbf98005cc0 tx=0x7fbf9800a340 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:29.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.598+0000 7fbf9ffff640 1 --2- 172.21.15.64:0/617211719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561116013410 0x56111601d770 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbf90009930 tx=0x7fbf90005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27368ea440c55ce server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:29.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.598+0000 7fbf867fc640 1 -- 172.21.15.64:0/617211719 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf90040070 con 0x561116013410 2024-09-12T03:31:29.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.598+0000 7fbf867fc640 1 -- 172.21.15.64:0/617211719 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf900316a0 con 0x561116013410 2024-09-12T03:31:29.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.598+0000 7fbf867fc640 1 -- 172.21.15.64:0/617211719 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf90035040 con 0x561116013410 2024-09-12T03:31:29.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.599+0000 7fbfa91bd740 1 -- 172.21.15.64:0/617211719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561116013410 msgr2=0x56111601d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:29.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.599+0000 7fbfa91bd740 1 --2- 172.21.15.64:0/617211719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561116013410 0x56111601d770 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbf90009930 tx=0x7fbf90005fd0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.599+0000 7fbfa91bd740 1 -- 172.21.15.64:0/617211719 shutdown_connections 2024-09-12T03:31:29.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.599+0000 7fbfa91bd740 1 --2- 172.21.15.64:0/617211719 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56111601dcb0 0x5611160200a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.599+0000 7fbfa91bd740 1 --2- 172.21.15.64:0/617211719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561116013410 0x56111601d770 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.599+0000 7fbfa91bd740 1 --2- 172.21.15.64:0/617211719 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561116012af0 0x561116012ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.599+0000 7fbfa91bd740 1 -- 172.21.15.64:0/617211719 >> 172.21.15.64:0/617211719 conn(0x56111600f060 msgr2=0x56111600f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:29.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.599+0000 7fbfa91bd740 1 -- 172.21.15.64:0/617211719 shutdown_connections 2024-09-12T03:31:29.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.599+0000 7fbfa91bd740 1 -- 172.21.15.64:0/617211719 wait complete. 2024-09-12T03:31:29.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.599+0000 7fbfa91bd740 1 -- start start 2024-09-12T03:31:29.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.600+0000 7fbfa91bd740 1 Processor -- start 2024-09-12T03:31:29.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.600+0000 7fbfa91bd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561116012af0 0x56111601aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:29.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.600+0000 7fbfa91bd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561116013410 0x56111601b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:29.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.600+0000 7fbfa91bd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56111601dcb0 0x56111601b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:29.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.600+0000 7fbfa91bd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561116020880 con 0x56111601dcb0 2024-09-12T03:31:29.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.600+0000 7fbfa91bd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561115f00290 con 0x561116012af0 2024-09-12T03:31:29.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.600+0000 7fbfa91bd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561116020a00 con 0x561116013410 2024-09-12T03:31:29.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.600+0000 7fbfa91bd740 10 client.0 _ll_register_callbacks cb 0x561115f31140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:31:29.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.600+0000 7fbfa49c0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56111601dcb0 0x56111601b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:29.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.600+0000 7fbfa49c0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56111601dcb0 0x56111601b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60850/0 (socket says 172.21.15.64:60850) 2024-09-12T03:31:29.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.600+0000 7fbfa49c0640 1 -- 172.21.15.64:0/518600326 learned_addr learned my addr 172.21.15.64:0/518600326 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:31:29.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.600+0000 7fbf9f7fe640 1 --2- 172.21.15.64:0/518600326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561116013410 0x56111601b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:29.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.600+0000 7fbf9ffff640 1 --2- 172.21.15.64:0/518600326 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561116012af0 0x56111601aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:29.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.600+0000 7fbfa49c0640 1 -- 172.21.15.64:0/518600326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561116013410 msgr2=0x56111601b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:29.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.600+0000 7fbfa49c0640 1 --2- 172.21.15.64:0/518600326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561116013410 0x56111601b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.600+0000 7fbfa49c0640 1 -- 172.21.15.64:0/518600326 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561116012af0 msgr2=0x56111601aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:29.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.600+0000 7fbfa49c0640 1 --2- 172.21.15.64:0/518600326 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561116012af0 0x56111601aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.601+0000 7fbfa49c0640 1 -- 172.21.15.64:0/518600326 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561115c8a210 con 0x56111601dcb0 2024-09-12T03:31:29.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.601+0000 7fbfa49c0640 1 --2- 172.21.15.64:0/518600326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56111601dcb0 0x56111601b620 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbf9800eac0 tx=0x7fbf98004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98cbfeed8e6da16d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:29.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.601+0000 7fbf7f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:31:29.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.601+0000 7fbf7f7fe640 1 -- 172.21.15.64:0/518600326 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf98011dd0 con 0x56111601dcb0 2024-09-12T03:31:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.601+0000 7fbfa91bd740 1 -- 172.21.15.64:0/518600326 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf900316a0 con 0x56111601dcb0 2024-09-12T03:31:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.601+0000 7fbf7f7fe640 1 -- 172.21.15.64:0/518600326 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7fbf98028480 con 0x56111601dcb0 2024-09-12T03:31:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.601+0000 7fbf7f7fe640 1 -- 172.21.15.64:0/518600326 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbf98031050 con 0x56111601dcb0 2024-09-12T03:31:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.601+0000 7fbfa91bd740 10 client.15522 fetch_fsmap learned FSMap version 25 2024-09-12T03:31:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.601+0000 7fbfa91bd740 1 -- 172.21.15.64:0/518600326 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561115d56620 con 0x56111601dcb0 2024-09-12T03:31:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.601+0000 7fbf7f7fe640 1 -- 172.21.15.64:0/518600326 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf900316a0 con 0x56111601dcb0 2024-09-12T03:31:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.602+0000 7fbf7f7fe640 1 -- 172.21.15.64:0/518600326 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fbf9801c520 con 0x56111601dcb0 2024-09-12T03:31:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.602+0000 7fbfa91bd740 10 client.15522 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:31:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.602+0000 7fbfa91bd740 10 client.15522 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:31:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.602+0000 7fbfa91bd740 20 client.15522 populate_metadata read hostname 'smithi064' 2024-09-12T03:31:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.602+0000 7fbfa91bd740 1 --2- 172.21.15.64:0/518600326 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x56111602dc20 0x561116030030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.602+0000 7fbfa91bd740 4 client.15522 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:31:29.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.602+0000 7fbfa91bd740 1 -- 172.21.15.64:0/518600326 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561115f84a20 con 0x56111602dc20 2024-09-12T03:31:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.602+0000 7fbf9ffff640 1 --2- 172.21.15.64:0/518600326 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x56111602dc20 0x561116030030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.602+0000 7fbf7f7fe640 10 client.15522 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:31:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.602+0000 7fbf9ffff640 1 --2- 172.21.15.64:0/518600326 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x56111602dc20 0x561116030030 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fcd0fc30058afd2 server_cookie=646d74468a255f23 in_seq=0 out_seq=0 2024-09-12T03:31:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.603+0000 7fbf7f7fe640 1 -- 172.21.15.64:0/518600326 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561115f84a20 con 0x56111602dc20 2024-09-12T03:31:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.603+0000 7fbf7f7fe640 10 client.15522 handle_command_reply: tid=0 2024-09-12T03:31:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.603+0000 7fbfa91bd740 1 client.15522 shutdown 2024-09-12T03:31:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.603+0000 7fbfa91bd740 1 -- 172.21.15.64:0/518600326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56111601dcb0 msgr2=0x56111601b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.603+0000 7fbfa91bd740 1 --2- 172.21.15.64:0/518600326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56111601dcb0 0x56111601b620 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbf9800eac0 tx=0x7fbf98004ab0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.603+0000 7fbfa91bd740 1 -- 172.21.15.64:0/518600326 shutdown_connections 2024-09-12T03:31:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.603+0000 7fbfa91bd740 1 --2- 172.21.15.64:0/518600326 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x56111602dc20 0x561116030030 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.603+0000 7fbfa91bd740 1 --2- 172.21.15.64:0/518600326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56111601dcb0 0x56111601b620 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.603+0000 7fbfa91bd740 1 --2- 172.21.15.64:0/518600326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561116013410 0x56111601b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.603+0000 7fbf7f7fe640 0 client.15522 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:31:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.603+0000 7fbfa91bd740 1 --2- 172.21.15.64:0/518600326 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561116012af0 0x56111601aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.603+0000 7fbfa91bd740 1 -- 172.21.15.64:0/518600326 >> 172.21.15.64:0/518600326 conn(0x56111600f060 msgr2=0x561116014dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.603+0000 7fbfa91bd740 1 -- 172.21.15.64:0/518600326 shutdown_connections 2024-09-12T03:31:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.603+0000 7fbfa91bd740 1 -- 172.21.15.64:0/518600326 wait complete. 2024-09-12T03:31:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.603+0000 7fbfa91bd740 20 client.15522 trim_cache size 0 max 16384 2024-09-12T03:31:29.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.618+0000 7fbfa91bd740 1 Processor -- start 2024-09-12T03:31:29.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.618+0000 7fbfa91bd740 1 -- start start 2024-09-12T03:31:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.618+0000 7fbfa91bd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561116012af0 0x561116015560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.618+0000 7fbfa91bd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561116013410 0x56111601b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.618+0000 7fbfa91bd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56111601dcb0 0x561116035940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.618+0000 7fbfa91bd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa01b1840 con 0x561116012af0 2024-09-12T03:31:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.618+0000 7fbfa91bd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa01af110 con 0x56111601dcb0 2024-09-12T03:31:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.618+0000 7fbfa91bd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa01acf60 con 0x561116013410 2024-09-12T03:31:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.618+0000 7fbf9ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561116012af0 0x561116015560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.618+0000 7fbf9ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561116012af0 0x561116015560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60864/0 (socket says 172.21.15.64:60864) 2024-09-12T03:31:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.618+0000 7fbf9ffff640 1 -- 172.21.15.64:0/1277344931 learned_addr learned my addr 172.21.15.64:0/1277344931 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:31:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.619+0000 7fbfa49c0640 1 --2- 172.21.15.64:0/1277344931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56111601dcb0 0x561116035940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.619+0000 7fbf9f7fe640 1 --2- 172.21.15.64:0/1277344931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561116013410 0x56111601b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.619+0000 7fbf9ffff640 1 -- 172.21.15.64:0/1277344931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561116013410 msgr2=0x56111601b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.619+0000 7fbf9ffff640 1 --2- 172.21.15.64:0/1277344931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561116013410 0x56111601b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.619+0000 7fbf9ffff640 1 -- 172.21.15.64:0/1277344931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56111601dcb0 msgr2=0x561116035940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.619+0000 7fbf9ffff640 1 --2- 172.21.15.64:0/1277344931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56111601dcb0 0x561116035940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.619+0000 7fbf9ffff640 1 -- 172.21.15.64:0/1277344931 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561116012920 con 0x561116012af0 2024-09-12T03:31:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.619+0000 7fbfa49c0640 1 --2- 172.21.15.64:0/1277344931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56111601dcb0 0x561116035940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:31:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.619+0000 7fbf9ffff640 1 --2- 172.21.15.64:0/1277344931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561116012af0 0x561116015560 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x56111600fec0 tx=0x7fbf900051c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3b9355839d87aa0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.619+0000 7fbf85ffb640 1 -- 172.21.15.64:0/1277344931 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf90035040 con 0x561116012af0 2024-09-12T03:31:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.619+0000 7fbf85ffb640 1 -- 172.21.15.64:0/1277344931 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf90054030 con 0x561116012af0 2024-09-12T03:31:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.619+0000 7fbf85ffb640 1 -- 172.21.15.64:0/1277344931 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf90040040 con 0x561116012af0 2024-09-12T03:31:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.620+0000 7fbfa91bd740 1 -- 172.21.15.64:0/1277344931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561116012af0 msgr2=0x561116015560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.620+0000 7fbfa91bd740 1 --2- 172.21.15.64:0/1277344931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561116012af0 0x561116015560 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x56111600fec0 tx=0x7fbf900051c0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.620+0000 7fbfa91bd740 1 -- 172.21.15.64:0/1277344931 shutdown_connections 2024-09-12T03:31:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.620+0000 7fbfa91bd740 1 --2- 172.21.15.64:0/1277344931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56111601dcb0 0x561116035940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.620+0000 7fbfa91bd740 1 --2- 172.21.15.64:0/1277344931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561116013410 0x56111601b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.620+0000 7fbfa91bd740 1 --2- 172.21.15.64:0/1277344931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561116012af0 0x561116015560 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.620+0000 7fbfa91bd740 1 -- 172.21.15.64:0/1277344931 >> 172.21.15.64:0/1277344931 conn(0x56111600f060 msgr2=0x56111601cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.620+0000 7fbfa91bd740 1 -- 172.21.15.64:0/1277344931 shutdown_connections 2024-09-12T03:31:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.620+0000 7fbfa91bd740 1 -- 172.21.15.64:0/1277344931 wait complete. 2024-09-12T03:31:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.620+0000 7fbfa91bd740 1 -- start start 2024-09-12T03:31:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.620+0000 7fbfa91bd740 1 Processor -- start 2024-09-12T03:31:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.620+0000 7fbfa91bd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561116012af0 0x5611160263d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.620+0000 7fbfa91bd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561116013410 0x5611160216f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.620+0000 7fbfa91bd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56111601dcb0 0x561116021c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.620+0000 7fbfa91bd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa01ad5d0 con 0x561116013410 2024-09-12T03:31:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.621+0000 7fbfa91bd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa01aea40 con 0x56111601dcb0 2024-09-12T03:31:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.621+0000 7fbfa91bd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa01adca0 con 0x561116012af0 2024-09-12T03:31:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.621+0000 7fbfa91bd740 10 client.0 _ll_register_callbacks cb 0x7fbf9800a5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:31:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.621+0000 7fbf9f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561116013410 0x5611160216f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.621+0000 7fbf9f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561116013410 0x5611160216f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60872/0 (socket says 172.21.15.64:60872) 2024-09-12T03:31:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.621+0000 7fbf9f7fe640 1 -- 172.21.15.64:0/1211122744 learned_addr learned my addr 172.21.15.64:0/1211122744 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:31:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.621+0000 7fbf9ffff640 1 --2- 172.21.15.64:0/1211122744 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561116012af0 0x5611160263d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.621+0000 7fbfa49c0640 1 --2- 172.21.15.64:0/1211122744 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56111601dcb0 0x561116021c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.621+0000 7fbf9f7fe640 1 -- 172.21.15.64:0/1211122744 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561116012af0 msgr2=0x5611160263d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.621+0000 7fbf9f7fe640 1 --2- 172.21.15.64:0/1211122744 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561116012af0 0x5611160263d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.621+0000 7fbf9f7fe640 1 -- 172.21.15.64:0/1211122744 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56111601dcb0 msgr2=0x561116021c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.621+0000 7fbf9f7fe640 1 --2- 172.21.15.64:0/1211122744 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56111601dcb0 0x561116021c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.621+0000 7fbf9f7fe640 1 -- 172.21.15.64:0/1211122744 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56111602c7c0 con 0x561116013410 2024-09-12T03:31:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.621+0000 7fbf9f7fe640 1 --2- 172.21.15.64:0/1211122744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561116013410 0x5611160216f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbf940ccfd0 tx=0x7fbf9409b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4864af41f2cabfef server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.621+0000 7fbf7f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:31:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.621+0000 7fbf7f7fe640 1 -- 172.21.15.64:0/1211122744 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf940b9060 con 0x561116013410 2024-09-12T03:31:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.621+0000 7fbf7f7fe640 1 -- 172.21.15.64:0/1211122744 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7fbf94010630 con 0x561116013410 2024-09-12T03:31:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.621+0000 7fbfa91bd740 1 -- 172.21.15.64:0/1211122744 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf90054030 con 0x561116013410 2024-09-12T03:31:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.621+0000 7fbf7f7fe640 1 -- 172.21.15.64:0/1211122744 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbf9409bcf0 con 0x561116013410 2024-09-12T03:31:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.622+0000 7fbfa91bd740 10 client.15528 fetch_fsmap learned FSMap version 25 2024-09-12T03:31:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.622+0000 7fbfa91bd740 1 -- 172.21.15.64:0/1211122744 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561116098720 con 0x561116013410 2024-09-12T03:31:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.622+0000 7fbf7f7fe640 1 -- 172.21.15.64:0/1211122744 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf90054030 con 0x561116013410 2024-09-12T03:31:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.622+0000 7fbf7f7fe640 1 -- 172.21.15.64:0/1211122744 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fbf94097a40 con 0x561116013410 2024-09-12T03:31:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.622+0000 7fbfa91bd740 10 client.15528 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:31:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.622+0000 7fbfa91bd740 10 client.15528 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:31:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.622+0000 7fbfa91bd740 20 client.15528 populate_metadata read hostname 'smithi064' 2024-09-12T03:31:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.622+0000 7fbfa91bd740 1 --2- 172.21.15.64:0/1211122744 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x561116098dd0 0x56111609b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.622+0000 7fbfa91bd740 4 client.15528 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:31:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.622+0000 7fbfa91bd740 1 -- 172.21.15.64:0/1211122744 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561115f7ed70 con 0x561116098dd0 2024-09-12T03:31:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.622+0000 7fbf9ffff640 1 --2- 172.21.15.64:0/1211122744 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x561116098dd0 0x56111609b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.623+0000 7fbf7f7fe640 10 client.15528 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:31:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.623+0000 7fbf9ffff640 1 --2- 172.21.15.64:0/1211122744 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x561116098dd0 0x56111609b1f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65a798cc25a43f7c server_cookie=f6f9d12a3a430542 in_seq=0 out_seq=0 2024-09-12T03:31:29.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.636+0000 7fbf7f7fe640 1 -- 172.21.15.64:0/1211122744 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561115f7ed70 con 0x561116098dd0 2024-09-12T03:31:29.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.636+0000 7fbf7f7fe640 10 client.15528 handle_command_reply: tid=0 2024-09-12T03:31:29.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.636+0000 7fbfa91bd740 1 client.15528 shutdown 2024-09-12T03:31:29.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.636+0000 7fbfa91bd740 1 -- 172.21.15.64:0/1211122744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561116013410 msgr2=0x5611160216f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:29.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.636+0000 7fbfa91bd740 1 --2- 172.21.15.64:0/1211122744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561116013410 0x5611160216f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbf940ccfd0 tx=0x7fbf9409b920 comp rx=0 tx=0).stop 2024-09-12T03:31:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.636+0000 7fbfa91bd740 1 -- 172.21.15.64:0/1211122744 shutdown_connections 2024-09-12T03:31:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.636+0000 7fbfa91bd740 1 --2- 172.21.15.64:0/1211122744 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x561116098dd0 0x56111609b1f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.636+0000 7fbfa91bd740 1 --2- 172.21.15.64:0/1211122744 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56111601dcb0 0x561116021c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.636+0000 7fbf7f7fe640 0 client.15528 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:31:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.636+0000 7fbfa91bd740 1 --2- 172.21.15.64:0/1211122744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561116013410 0x5611160216f0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.636+0000 7fbfa91bd740 1 --2- 172.21.15.64:0/1211122744 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561116012af0 0x5611160263d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.636+0000 7fbfa91bd740 1 -- 172.21.15.64:0/1211122744 >> 172.21.15.64:0/1211122744 conn(0x56111600f060 msgr2=0x56111601c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.636+0000 7fbfa91bd740 1 -- 172.21.15.64:0/1211122744 shutdown_connections 2024-09-12T03:31:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.636+0000 7fbfa91bd740 1 -- 172.21.15.64:0/1211122744 wait complete. 2024-09-12T03:31:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.636+0000 7fbfa91bd740 20 client.15528 trim_cache size 0 max 16384 2024-09-12T03:31:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:31:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:31:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:31:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:31:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.637+0000 7fbfa6423640 1 -- 172.21.15.64:0/4071614129 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fbf880a2da0 msgr2=0x7fbf880a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.637+0000 7fbfa6423640 1 --2- 172.21.15.64:0/4071614129 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fbf880a2da0 0x7fbf880a5260 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbf98005cc0 tx=0x7fbf9800a340 comp rx=0 tx=0).stop 2024-09-12T03:31:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.637+0000 7fbfa6423640 1 -- 172.21.15.64:0/4071614129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa0101350 msgr2=0x7fbfa01987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.637+0000 7fbfa6423640 1 --2- 172.21.15.64:0/4071614129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa0101350 0x7fbfa01987e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fbf9400e9f0 tx=0x7fbf9400eeb0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.637+0000 7fbfa6423640 1 -- 172.21.15.64:0/4071614129 shutdown_connections 2024-09-12T03:31:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.637+0000 7fbfa6423640 1 --2- 172.21.15.64:0/4071614129 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fbf880a2da0 0x7fbf880a5260 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.637+0000 7fbfa6423640 1 --2- 172.21.15.64:0/4071614129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfa01069c0 0x7fbfa019cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.637+0000 7fbfa6423640 1 --2- 172.21.15.64:0/4071614129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa0101350 0x7fbfa01987e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.637+0000 7fbfa6423640 1 --2- 172.21.15.64:0/4071614129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfa01009b0 0x7fbfa01982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.637+0000 7fbfa6423640 1 -- 172.21.15.64:0/4071614129 >> 172.21.15.64:0/4071614129 conn(0x7fbfa0077990 msgr2=0x7fbfa0069a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.637+0000 7fbfa6423640 1 -- 172.21.15.64:0/4071614129 shutdown_connections 2024-09-12T03:31:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:29.637+0000 7fbfa6423640 1 -- 172.21.15.64:0/4071614129 wait complete. 2024-09-12T03:31:29.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:31:29 smithi115 ceph-mon[34801]: pgmap v69: 129 pgs: 129 active+clean; 2.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 KiB/s wr, 18 op/s 2024-09-12T03:31:29.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:31:29 smithi195 ceph-mon[34786]: pgmap v69: 129 pgs: 129 active+clean; 2.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 KiB/s wr, 18 op/s 2024-09-12T03:31:29.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:31:29 smithi064 ceph-mon[30952]: pgmap v69: 129 pgs: 129 active+clean; 2.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 KiB/s wr, 18 op/s 2024-09-12T03:31:31.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:31:31 smithi115 ceph-mon[34801]: pgmap v70: 129 pgs: 129 active+clean; 2.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 10 op/s 2024-09-12T03:31:31.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:31:31 smithi195 ceph-mon[34786]: pgmap v70: 129 pgs: 129 active+clean; 2.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 10 op/s 2024-09-12T03:31:31.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:31:31 smithi064 ceph-mon[30952]: pgmap v70: 129 pgs: 129 active+clean; 2.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 10 op/s 2024-09-12T03:31:33.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:31:33 smithi115 ceph-mon[34801]: pgmap v71: 129 pgs: 129 active+clean; 2.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 18 op/s 2024-09-12T03:31:33.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:31:33 smithi195 ceph-mon[34786]: pgmap v71: 129 pgs: 129 active+clean; 2.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 18 op/s 2024-09-12T03:31:33.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:31:33 smithi064 ceph-mon[30952]: pgmap v71: 129 pgs: 129 active+clean; 2.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 18 op/s 2024-09-12T03:31:35.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:31:35 smithi115 ceph-mon[34801]: pgmap v72: 129 pgs: 129 active+clean; 2.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-09-12T03:31:35.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:31:35 smithi195 ceph-mon[34786]: pgmap v72: 129 pgs: 129 active+clean; 2.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-09-12T03:31:35.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:31:35 smithi064 ceph-mon[30952]: pgmap v72: 129 pgs: 129 active+clean; 2.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-09-12T03:31:37.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:31:37 smithi115 ceph-mon[34801]: pgmap v73: 129 pgs: 129 active+clean; 2.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-09-12T03:31:37.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:31:37 smithi195 ceph-mon[34786]: pgmap v73: 129 pgs: 129 active+clean; 2.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-09-12T03:31:37.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:31:37 smithi064 ceph-mon[30952]: pgmap v73: 129 pgs: 129 active+clean; 2.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-09-12T03:31:39.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:31:39 smithi115 ceph-mon[34801]: pgmap v74: 129 pgs: 129 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 18 op/s 2024-09-12T03:31:39.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:31:39 smithi195 ceph-mon[34786]: pgmap v74: 129 pgs: 129 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 18 op/s 2024-09-12T03:31:39.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:31:39 smithi064 ceph-mon[30952]: pgmap v74: 129 pgs: 129 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 18 op/s 2024-09-12T03:31:41.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:31:41 smithi115 ceph-mon[34801]: pgmap v75: 129 pgs: 129 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-09-12T03:31:41.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:31:41 smithi195 ceph-mon[34786]: pgmap v75: 129 pgs: 129 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-09-12T03:31:41.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:31:41 smithi064 ceph-mon[30952]: pgmap v75: 129 pgs: 129 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-09-12T03:31:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.735+0000 7fbfaff0a640 1 -- 172.21.15.64:0/3739647552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa8101390 msgr2=0x7fbfa810d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.735+0000 7fbfaff0a640 1 --2- 172.21.15.64:0/3739647552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa8101390 0x7fbfa810d8c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0009cd0 tx=0x7fbfa002a8e0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.735+0000 7fbfaff0a640 1 -- 172.21.15.64:0/3739647552 shutdown_connections 2024-09-12T03:31:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.735+0000 7fbfaff0a640 1 --2- 172.21.15.64:0/3739647552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa8101390 0x7fbfa810d8c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.735+0000 7fbfaff0a640 1 --2- 172.21.15.64:0/3739647552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfa81009f0 0x7fbfa8100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.735+0000 7fbfaff0a640 1 --2- 172.21.15.64:0/3739647552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfa8106a00 0x7fbfa8106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.735+0000 7fbfaff0a640 1 -- 172.21.15.64:0/3739647552 >> 172.21.15.64:0/3739647552 conn(0x7fbfa80fc6b0 msgr2=0x7fbfa80fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.735+0000 7fbfaff0a640 1 -- 172.21.15.64:0/3739647552 shutdown_connections 2024-09-12T03:31:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.736+0000 7fbfaff0a640 1 -- 172.21.15.64:0/3739647552 wait complete. 2024-09-12T03:31:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.736+0000 7fbfaff0a640 1 Processor -- start 2024-09-12T03:31:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.736+0000 7fbfaff0a640 1 -- start start 2024-09-12T03:31:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.736+0000 7fbfaff0a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfa81009f0 0x7fbfa81981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.736+0000 7fbfaff0a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa8101390 0x7fbfa81986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.736+0000 7fbfaff0a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfa8106a00 0x7fbfa819ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.736+0000 7fbfaff0a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa8110d40 con 0x7fbfa8101390 2024-09-12T03:31:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.736+0000 7fbfaff0a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa8110bc0 con 0x7fbfa81009f0 2024-09-12T03:31:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.736+0000 7fbfaff0a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa8110ec0 con 0x7fbfa8106a00 2024-09-12T03:31:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.737+0000 7fbfad4a5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa8101390 0x7fbfa81986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.737+0000 7fbfad4a5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa8101390 0x7fbfa81986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34956/0 (socket says 172.21.15.64:34956) 2024-09-12T03:31:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.737+0000 7fbfad4a5640 1 -- 172.21.15.64:0/339326706 learned_addr learned my addr 172.21.15.64:0/339326706 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:31:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.737+0000 7fbfadca6640 1 --2- 172.21.15.64:0/339326706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfa81009f0 0x7fbfa81981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.737+0000 7fbfae4a7640 1 --2- 172.21.15.64:0/339326706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfa8106a00 0x7fbfa819ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.737+0000 7fbfad4a5640 1 -- 172.21.15.64:0/339326706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfa8106a00 msgr2=0x7fbfa819ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.737+0000 7fbfad4a5640 1 --2- 172.21.15.64:0/339326706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfa8106a00 0x7fbfa819ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.737+0000 7fbfad4a5640 1 -- 172.21.15.64:0/339326706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfa81009f0 msgr2=0x7fbfa81981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:42.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.737+0000 7fbfad4a5640 1 --2- 172.21.15.64:0/339326706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfa81009f0 0x7fbfa81981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.737+0000 7fbfad4a5640 1 -- 172.21.15.64:0/339326706 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfa819d190 con 0x7fbfa8101390 2024-09-12T03:31:42.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.737+0000 7fbfae4a7640 1 --2- 172.21.15.64:0/339326706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfa8106a00 0x7fbfa819ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:31:42.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.737+0000 7fbfad4a5640 1 --2- 172.21.15.64:0/339326706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa8101390 0x7fbfa81986e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fbf9800d950 tx=0x7fbf9800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:42.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.737+0000 7fbf9effd640 1 -- 172.21.15.64:0/339326706 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf98017070 con 0x7fbfa8101390 2024-09-12T03:31:42.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.737+0000 7fbf9effd640 1 -- 172.21.15.64:0/339326706 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf98009c30 con 0x7fbfa8101390 2024-09-12T03:31:42.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.737+0000 7fbfaff0a640 1 -- 172.21.15.64:0/339326706 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfa819d480 con 0x7fbfa8101390 2024-09-12T03:31:42.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.737+0000 7fbf9effd640 1 -- 172.21.15.64:0/339326706 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9800f620 con 0x7fbfa8101390 2024-09-12T03:31:42.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.737+0000 7fbfaff0a640 1 -- 172.21.15.64:0/339326706 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfa81a4d10 con 0x7fbfa8101390 2024-09-12T03:31:42.746 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.740+0000 7fbf9effd640 1 -- 172.21.15.64:0/339326706 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fbf980109a0 con 0x7fbfa8101390 2024-09-12T03:31:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.740+0000 7fbfb2ca2740 1 Processor -- start 2024-09-12T03:31:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.740+0000 7fbfb2ca2740 1 -- start start 2024-09-12T03:31:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.740+0000 7fbfb2ca2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dc65c9af0 0x557dc65c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.740+0000 7fbfb2ca2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc65ca410 0x557dc65d4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.740+0000 7fbfb2ca2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dc65d4cb0 0x557dc65d70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.740+0000 7fbfb2ca2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x557dc652e2f0 con 0x557dc65ca410 2024-09-12T03:31:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.740+0000 7fbfb2ca2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557dc64d6270 con 0x557dc65c9af0 2024-09-12T03:31:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.740+0000 7fbfb2ca2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557dc652a2b0 con 0x557dc65d4cb0 2024-09-12T03:31:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.741+0000 7fbfadca6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc65ca410 0x557dc65d4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.741+0000 7fbfadca6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc65ca410 0x557dc65d4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34966/0 (socket says 172.21.15.64:34966) 2024-09-12T03:31:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.741+0000 7fbfadca6640 1 -- 172.21.15.64:0/184245638 learned_addr learned my addr 172.21.15.64:0/184245638 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:31:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.741+0000 7fbfae4a7640 1 --2- 172.21.15.64:0/184245638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dc65c9af0 0x557dc65c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.741+0000 7fbfad4a5640 1 --2- 172.21.15.64:0/184245638 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dc65d4cb0 0x557dc65d70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.741+0000 7fbf9effd640 1 --2- 172.21.15.64:0/339326706 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fbf940a2da0 0x7fbf940a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.741+0000 7fbfadca6640 1 -- 172.21.15.64:0/184245638 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dc65d4cb0 msgr2=0x557dc65d70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.741+0000 7fbfadca6640 1 --2- 172.21.15.64:0/184245638 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dc65d4cb0 0x557dc65d70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.741+0000 7fbfadca6640 1 -- 172.21.15.64:0/184245638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dc65c9af0 msgr2=0x557dc65c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.741+0000 7fbfadca6640 1 --2- 172.21.15.64:0/184245638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dc65c9af0 0x557dc65c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.741+0000 7fbfadca6640 1 -- 172.21.15.64:0/184245638 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dc64d5340 con 0x557dc65ca410 2024-09-12T03:31:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.741+0000 7fbfad4a5640 1 --2- 172.21.15.64:0/184245638 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dc65d4cb0 0x557dc65d70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:31:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.741+0000 7fbfae4a7640 1 --2- 172.21.15.64:0/339326706 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fbf940a2da0 0x7fbf940a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.741+0000 7fbf9effd640 1 -- 172.21.15.64:0/339326706 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7fbf980cf500 con 0x7fbfa8101390 2024-09-12T03:31:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.741+0000 7fbfadca6640 1 --2- 172.21.15.64:0/184245638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc65ca410 0x557dc65d4770 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fbfa400cbe0 tx=0x7fbfa400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe49d47b491b12e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.741+0000 7fbf93fff640 1 -- 172.21.15.64:0/184245638 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4005220 con 0x557dc65ca410 2024-09-12T03:31:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.741+0000 7fbf93fff640 1 -- 172.21.15.64:0/184245638 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbfa4004c50 con 0x557dc65ca410 2024-09-12T03:31:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.741+0000 7fbf93fff640 1 -- 172.21.15.64:0/184245638 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa40107b0 con 0x557dc65ca410 2024-09-12T03:31:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.742+0000 7fbfb2ca2740 1 -- 172.21.15.64:0/184245638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc65ca410 msgr2=0x557dc65d4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.742+0000 7fbfb2ca2740 1 --2- 172.21.15.64:0/184245638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc65ca410 0x557dc65d4770 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fbfa400cbe0 tx=0x7fbfa400e720 comp rx=0 tx=0).stop 2024-09-12T03:31:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.742+0000 7fbfae4a7640 1 --2- 172.21.15.64:0/339326706 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fbf940a2da0 0x7fbf940a5260 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0009a60 tx=0x7fbfa00095c0 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.742+0000 7fbfb2ca2740 1 -- 172.21.15.64:0/184245638 shutdown_connections 2024-09-12T03:31:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.742+0000 7fbfb2ca2740 1 --2- 172.21.15.64:0/184245638 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dc65d4cb0 0x557dc65d70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.742+0000 7fbfb2ca2740 1 --2- 172.21.15.64:0/184245638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc65ca410 0x557dc65d4770 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.742+0000 7fbfb2ca2740 1 --2- 172.21.15.64:0/184245638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dc65c9af0 0x557dc65c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.742+0000 7fbfb2ca2740 1 -- 172.21.15.64:0/184245638 >> 172.21.15.64:0/184245638 conn(0x557dc65c6060 msgr2=0x557dc65c6440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.742+0000 7fbfb2ca2740 1 -- 172.21.15.64:0/184245638 shutdown_connections 2024-09-12T03:31:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.742+0000 7fbfb2ca2740 1 -- 172.21.15.64:0/184245638 wait complete. 2024-09-12T03:31:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.743+0000 7fbfb2ca2740 1 -- start start 2024-09-12T03:31:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.743+0000 7fbfb2ca2740 1 Processor -- start 2024-09-12T03:31:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.743+0000 7fbfb2ca2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dc65c9af0 0x557dc65d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.743+0000 7fbfb2ca2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dc65ca410 0x557dc65d2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.743+0000 7fbfb2ca2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc65d4cb0 0x557dc65d2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.743+0000 7fbfb2ca2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x557dc65d7880 con 0x557dc65d4cb0 2024-09-12T03:31:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.743+0000 7fbfb2ca2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557dc64b7290 con 0x557dc65c9af0 2024-09-12T03:31:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.743+0000 7fbfb2ca2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557dc65d7a00 con 0x557dc65ca410 2024-09-12T03:31:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.743+0000 7fbfb2ca2740 10 client.0 _ll_register_callbacks cb 0x557dc64e8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:31:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.743+0000 7fbfae4a7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc65d4cb0 0x557dc65d2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.743+0000 7fbfae4a7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc65d4cb0 0x557dc65d2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34970/0 (socket says 172.21.15.64:34970) 2024-09-12T03:31:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.743+0000 7fbfae4a7640 1 -- 172.21.15.64:0/2475638077 learned_addr learned my addr 172.21.15.64:0/2475638077 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:31:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.743+0000 7fbfadca6640 1 --2- 172.21.15.64:0/2475638077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dc65c9af0 0x557dc65d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.744+0000 7fbfad4a5640 1 --2- 172.21.15.64:0/2475638077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dc65ca410 0x557dc65d2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.744+0000 7fbfae4a7640 1 -- 172.21.15.64:0/2475638077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dc65ca410 msgr2=0x557dc65d2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.744+0000 7fbfae4a7640 1 --2- 172.21.15.64:0/2475638077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dc65ca410 0x557dc65d2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.744+0000 7fbfae4a7640 1 -- 172.21.15.64:0/2475638077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dc65c9af0 msgr2=0x557dc65d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.744+0000 7fbfae4a7640 1 --2- 172.21.15.64:0/2475638077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dc65c9af0 0x557dc65d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.744+0000 7fbfae4a7640 1 -- 172.21.15.64:0/2475638077 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dc6241210 con 0x557dc65d4cb0 2024-09-12T03:31:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.744+0000 7fbfae4a7640 1 --2- 172.21.15.64:0/2475638077 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc65d4cb0 0x557dc65d2570 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0003bf0 tx=0x7fbfa002a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92013449521d1b0e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.744+0000 7fbf917fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:31:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.744+0000 7fbf917fa640 1 -- 172.21.15.64:0/2475638077 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa00047c0 con 0x557dc65d4cb0 2024-09-12T03:31:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.744+0000 7fbf917fa640 1 -- 172.21.15.64:0/2475638077 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7fbfa00305a0 con 0x557dc65d4cb0 2024-09-12T03:31:42.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.744+0000 7fbfb2ca2740 1 -- 172.21.15.64:0/2475638077 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfa4004c50 con 0x557dc65d4cb0 2024-09-12T03:31:42.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.744+0000 7fbf917fa640 1 -- 172.21.15.64:0/2475638077 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbfa005a050 con 0x557dc65d4cb0 2024-09-12T03:31:42.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.744+0000 7fbfb2ca2740 10 client.15540 fetch_fsmap learned FSMap version 25 2024-09-12T03:31:42.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.744+0000 7fbfb2ca2740 1 -- 172.21.15.64:0/2475638077 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557dc630d620 con 0x557dc65d4cb0 2024-09-12T03:31:42.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.745+0000 7fbf917fa640 1 -- 172.21.15.64:0/2475638077 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbfa4004c50 con 0x557dc65d4cb0 2024-09-12T03:31:42.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.745+0000 7fbf917fa640 1 -- 172.21.15.64:0/2475638077 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fbfa0030070 con 0x557dc65d4cb0 2024-09-12T03:31:42.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.745+0000 7fbfb2ca2740 10 client.15540 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:31:42.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.745+0000 7fbfb2ca2740 10 client.15540 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:31:42.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.745+0000 7fbfb2ca2740 20 client.15540 populate_metadata read hostname 'smithi064' 2024-09-12T03:31:42.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.745+0000 7fbfb2ca2740 1 --2- 172.21.15.64:0/2475638077 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x557dc65e4a70 0x557dc65e6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:42.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.745+0000 7fbfb2ca2740 4 client.15540 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:31:42.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.745+0000 7fbfb2ca2740 1 -- 172.21.15.64:0/2475638077 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557dc653ba20 con 0x557dc65e4a70 2024-09-12T03:31:42.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.745+0000 7fbfadca6640 1 --2- 172.21.15.64:0/2475638077 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x557dc65e4a70 0x557dc65e6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:42.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.746+0000 7fbfadca6640 1 --2- 172.21.15.64:0/2475638077 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x557dc65e4a70 0x557dc65e6e60 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d3955450935b70b server_cookie=763c553d8d10ddcd in_seq=0 out_seq=0 2024-09-12T03:31:42.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.746+0000 7fbf917fa640 10 client.15540 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:31:42.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.747+0000 7fbf917fa640 1 -- 172.21.15.64:0/2475638077 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557dc653ba20 con 0x557dc65e4a70 2024-09-12T03:31:42.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.747+0000 7fbf917fa640 10 client.15540 handle_command_reply: tid=0 2024-09-12T03:31:42.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.747+0000 7fbfb2ca2740 1 client.15540 shutdown 2024-09-12T03:31:42.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.747+0000 7fbfb2ca2740 1 -- 172.21.15.64:0/2475638077 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc65d4cb0 msgr2=0x557dc65d2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.747+0000 7fbfb2ca2740 1 --2- 172.21.15.64:0/2475638077 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc65d4cb0 0x557dc65d2570 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0003bf0 tx=0x7fbfa002a8e0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.747+0000 7fbfb2ca2740 1 -- 172.21.15.64:0/2475638077 shutdown_connections 2024-09-12T03:31:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.747+0000 7fbfb2ca2740 1 --2- 172.21.15.64:0/2475638077 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x557dc65e4a70 0x557dc65e6e60 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.747+0000 7fbfb2ca2740 1 --2- 172.21.15.64:0/2475638077 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc65d4cb0 0x557dc65d2570 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.747+0000 7fbfb2ca2740 1 --2- 172.21.15.64:0/2475638077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dc65ca410 0x557dc65d2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.747+0000 7fbf917fa640 0 client.15540 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:31:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.747+0000 7fbfb2ca2740 1 --2- 172.21.15.64:0/2475638077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dc65c9af0 0x557dc65d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.747+0000 7fbfb2ca2740 1 -- 172.21.15.64:0/2475638077 >> 172.21.15.64:0/2475638077 conn(0x557dc65c6060 msgr2=0x557dc65cbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.747+0000 7fbfb2ca2740 1 -- 172.21.15.64:0/2475638077 shutdown_connections 2024-09-12T03:31:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.747+0000 7fbfb2ca2740 1 -- 172.21.15.64:0/2475638077 wait complete. 2024-09-12T03:31:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.747+0000 7fbfb2ca2740 20 client.15540 trim_cache size 0 max 16384 2024-09-12T03:31:42.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.762+0000 7fbfb2ca2740 1 Processor -- start 2024-09-12T03:31:42.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.762+0000 7fbfb2ca2740 1 -- start start 2024-09-12T03:31:42.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.762+0000 7fbfb2ca2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc65c9af0 0x557dc65cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:42.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.762+0000 7fbfb2ca2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dc65ca410 0x557dc65d23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:42.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.762+0000 7fbfb2ca2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dc65d4cb0 0x557dc65ec770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:42.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.762+0000 7fbfb2ca2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa81b17f0 con 0x557dc65c9af0 2024-09-12T03:31:42.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.762+0000 7fbfb2ca2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa81af0c0 con 0x557dc65ca410 2024-09-12T03:31:42.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.762+0000 7fbfb2ca2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa81acf10 con 0x557dc65d4cb0 2024-09-12T03:31:42.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.762+0000 7fbfadca6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc65c9af0 0x557dc65cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:42.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.762+0000 7fbfadca6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc65c9af0 0x557dc65cc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34974/0 (socket says 172.21.15.64:34974) 2024-09-12T03:31:42.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.762+0000 7fbfadca6640 1 -- 172.21.15.64:0/1111029348 learned_addr learned my addr 172.21.15.64:0/1111029348 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:31:42.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.762+0000 7fbfad4a5640 1 --2- 172.21.15.64:0/1111029348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dc65ca410 0x557dc65d23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:42.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.762+0000 7fbfae4a7640 1 --2- 172.21.15.64:0/1111029348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dc65d4cb0 0x557dc65ec770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:42.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.762+0000 7fbfadca6640 1 -- 172.21.15.64:0/1111029348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dc65d4cb0 msgr2=0x557dc65ec770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:42.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.762+0000 7fbfadca6640 1 --2- 172.21.15.64:0/1111029348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dc65d4cb0 0x557dc65ec770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.762+0000 7fbfadca6640 1 -- 172.21.15.64:0/1111029348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dc65ca410 msgr2=0x557dc65d23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:42.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.762+0000 7fbfadca6640 1 --2- 172.21.15.64:0/1111029348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dc65ca410 0x557dc65d23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.762+0000 7fbfadca6640 1 -- 172.21.15.64:0/1111029348 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dc65e71d0 con 0x557dc65c9af0 2024-09-12T03:31:42.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.762+0000 7fbfae4a7640 1 --2- 172.21.15.64:0/1111029348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dc65d4cb0 0x557dc65ec770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:31:42.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.763+0000 7fbfadca6640 1 --2- 172.21.15.64:0/1111029348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc65c9af0 0x557dc65cc560 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fbfa400f010 tx=0x7fbfa400c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c740ca79c080c97 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:42.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.763+0000 7fbf937fe640 1 -- 172.21.15.64:0/1111029348 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa40052d0 con 0x557dc65c9af0 2024-09-12T03:31:42.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.763+0000 7fbf937fe640 1 -- 172.21.15.64:0/1111029348 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbfa4005470 con 0x557dc65c9af0 2024-09-12T03:31:42.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.763+0000 7fbf937fe640 1 -- 172.21.15.64:0/1111029348 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4028820 con 0x557dc65c9af0 2024-09-12T03:31:42.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.763+0000 7fbfb2ca2740 1 -- 172.21.15.64:0/1111029348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc65c9af0 msgr2=0x557dc65cc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:42.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.763+0000 7fbfb2ca2740 1 --2- 172.21.15.64:0/1111029348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc65c9af0 0x557dc65cc560 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fbfa400f010 tx=0x7fbfa400c4d0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.763+0000 7fbfb2ca2740 1 -- 172.21.15.64:0/1111029348 shutdown_connections 2024-09-12T03:31:42.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.763+0000 7fbfb2ca2740 1 --2- 172.21.15.64:0/1111029348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dc65d4cb0 0x557dc65ec770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.763+0000 7fbfb2ca2740 1 --2- 172.21.15.64:0/1111029348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dc65ca410 0x557dc65d23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.763+0000 7fbfb2ca2740 1 --2- 172.21.15.64:0/1111029348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc65c9af0 0x557dc65cc560 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.763+0000 7fbfb2ca2740 1 -- 172.21.15.64:0/1111029348 >> 172.21.15.64:0/1111029348 conn(0x557dc65c6060 msgr2=0x557dc65d3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.763+0000 7fbfb2ca2740 1 -- 172.21.15.64:0/1111029348 shutdown_connections 2024-09-12T03:31:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.763+0000 7fbfb2ca2740 1 -- 172.21.15.64:0/1111029348 wait complete. 2024-09-12T03:31:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.764+0000 7fbfb2ca2740 1 -- start start 2024-09-12T03:31:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.764+0000 7fbfb2ca2740 1 Processor -- start 2024-09-12T03:31:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.764+0000 7fbfb2ca2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dc65c9af0 0x557dc65dcf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.764+0000 7fbfb2ca2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc65ca410 0x557dc65dd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.764+0000 7fbfb2ca2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dc65d4cb0 0x557dc65d8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.764+0000 7fbfb2ca2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa81ad580 con 0x557dc65ca410 2024-09-12T03:31:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.764+0000 7fbfb2ca2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa81ae9f0 con 0x557dc65d4cb0 2024-09-12T03:31:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.764+0000 7fbfb2ca2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa81adc50 con 0x557dc65c9af0 2024-09-12T03:31:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.764+0000 7fbfb2ca2740 10 client.0 _ll_register_callbacks cb 0x7fbfa0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:31:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.764+0000 7fbfad4a5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc65ca410 0x557dc65dd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.764+0000 7fbfad4a5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc65ca410 0x557dc65dd4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34976/0 (socket says 172.21.15.64:34976) 2024-09-12T03:31:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.764+0000 7fbfad4a5640 1 -- 172.21.15.64:0/374607993 learned_addr learned my addr 172.21.15.64:0/374607993 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:31:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.764+0000 7fbfae4a7640 1 --2- 172.21.15.64:0/374607993 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dc65d4cb0 0x557dc65d8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.765+0000 7fbfadca6640 1 --2- 172.21.15.64:0/374607993 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dc65c9af0 0x557dc65dcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.765+0000 7fbfad4a5640 1 -- 172.21.15.64:0/374607993 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dc65c9af0 msgr2=0x557dc65dcf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.765+0000 7fbfad4a5640 1 --2- 172.21.15.64:0/374607993 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dc65c9af0 0x557dc65dcf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.765+0000 7fbfad4a5640 1 -- 172.21.15.64:0/374607993 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dc65d4cb0 msgr2=0x557dc65d8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.765+0000 7fbfad4a5640 1 --2- 172.21.15.64:0/374607993 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dc65d4cb0 0x557dc65d8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.765+0000 7fbfad4a5640 1 -- 172.21.15.64:0/374607993 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dc65d8b60 con 0x557dc65ca410 2024-09-12T03:31:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.765+0000 7fbfad4a5640 1 --2- 172.21.15.64:0/374607993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc65ca410 0x557dc65dd4d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fbf9809b5e0 tx=0x7fbf9809b070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=359200f532b2a1b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.765+0000 7fbf917fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:31:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.765+0000 7fbf917fa640 1 -- 172.21.15.64:0/374607993 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf98020050 con 0x557dc65ca410 2024-09-12T03:31:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.765+0000 7fbf917fa640 1 -- 172.21.15.64:0/374607993 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7fbf980cf150 con 0x557dc65ca410 2024-09-12T03:31:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.765+0000 7fbfb2ca2740 1 -- 172.21.15.64:0/374607993 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfa4005470 con 0x557dc65ca410 2024-09-12T03:31:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.765+0000 7fbf917fa640 1 -- 172.21.15.64:0/374607993 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbf980cf350 con 0x557dc65ca410 2024-09-12T03:31:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.765+0000 7fbfb2ca2740 10 client.15546 fetch_fsmap learned FSMap version 25 2024-09-12T03:31:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.765+0000 7fbfb2ca2740 1 -- 172.21.15.64:0/374607993 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557dc664f310 con 0x557dc65ca410 2024-09-12T03:31:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.765+0000 7fbf917fa640 1 -- 172.21.15.64:0/374607993 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbfa4005470 con 0x557dc65ca410 2024-09-12T03:31:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.766+0000 7fbf917fa640 1 -- 172.21.15.64:0/374607993 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fbf98084c20 con 0x557dc65ca410 2024-09-12T03:31:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.766+0000 7fbfb2ca2740 10 client.15546 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:31:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.766+0000 7fbfb2ca2740 10 client.15546 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:31:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.766+0000 7fbfb2ca2740 20 client.15546 populate_metadata read hostname 'smithi064' 2024-09-12T03:31:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.766+0000 7fbfb2ca2740 1 --2- 172.21.15.64:0/374607993 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x557dc664fb60 0x557dc6651f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:42.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.766+0000 7fbfb2ca2740 4 client.15546 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:31:42.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.766+0000 7fbfb2ca2740 1 -- 172.21.15.64:0/374607993 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557dc6535d70 con 0x557dc664fb60 2024-09-12T03:31:42.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.766+0000 7fbfadca6640 1 --2- 172.21.15.64:0/374607993 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x557dc664fb60 0x557dc6651f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:42.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.767+0000 7fbf917fa640 10 client.15546 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:31:42.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.767+0000 7fbfadca6640 1 --2- 172.21.15.64:0/374607993 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x557dc664fb60 0x557dc6651f50 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cec483f45457a6c2 server_cookie=4c822d19bab66af2 in_seq=0 out_seq=0 2024-09-12T03:31:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.780+0000 7fbf917fa640 1 -- 172.21.15.64:0/374607993 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557dc6535d70 con 0x557dc664fb60 2024-09-12T03:31:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.780+0000 7fbf917fa640 10 client.15546 handle_command_reply: tid=0 2024-09-12T03:31:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.780+0000 7fbfb2ca2740 1 client.15546 shutdown 2024-09-12T03:31:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.780+0000 7fbfb2ca2740 1 -- 172.21.15.64:0/374607993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc65ca410 msgr2=0x557dc65dd4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.780+0000 7fbfb2ca2740 1 --2- 172.21.15.64:0/374607993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc65ca410 0x557dc65dd4d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fbf9809b5e0 tx=0x7fbf9809b070 comp rx=0 tx=0).stop 2024-09-12T03:31:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.780+0000 7fbfb2ca2740 1 -- 172.21.15.64:0/374607993 shutdown_connections 2024-09-12T03:31:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.780+0000 7fbfb2ca2740 1 --2- 172.21.15.64:0/374607993 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x557dc664fb60 0x557dc6651f50 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.780+0000 7fbfb2ca2740 1 --2- 172.21.15.64:0/374607993 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dc65d4cb0 0x557dc65d8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.780+0000 7fbfb2ca2740 1 --2- 172.21.15.64:0/374607993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557dc65ca410 0x557dc65dd4d0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.780+0000 7fbfb2ca2740 1 --2- 172.21.15.64:0/374607993 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dc65c9af0 0x557dc65dcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.780+0000 7fbf917fa640 0 client.15546 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:31:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.780+0000 7fbfb2ca2740 1 -- 172.21.15.64:0/374607993 >> 172.21.15.64:0/374607993 conn(0x557dc65c6060 msgr2=0x557dc65c49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.780+0000 7fbfb2ca2740 1 -- 172.21.15.64:0/374607993 shutdown_connections 2024-09-12T03:31:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.780+0000 7fbfb2ca2740 1 -- 172.21.15.64:0/374607993 wait complete. 2024-09-12T03:31:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.780+0000 7fbfb2ca2740 20 client.15546 trim_cache size 0 max 16384 2024-09-12T03:31:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:31:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:31:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:31:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:31:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.781+0000 7fbfaff0a640 1 -- 172.21.15.64:0/339326706 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fbf940a2da0 msgr2=0x7fbf940a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.781+0000 7fbfaff0a640 1 --2- 172.21.15.64:0/339326706 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fbf940a2da0 0x7fbf940a5260 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0009a60 tx=0x7fbfa00095c0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.781+0000 7fbfaff0a640 1 -- 172.21.15.64:0/339326706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa8101390 msgr2=0x7fbfa81986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.781+0000 7fbfaff0a640 1 --2- 172.21.15.64:0/339326706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa8101390 0x7fbfa81986e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fbf9800d950 tx=0x7fbf9800de10 comp rx=0 tx=0).stop 2024-09-12T03:31:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.781+0000 7fbfaff0a640 1 -- 172.21.15.64:0/339326706 shutdown_connections 2024-09-12T03:31:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.781+0000 7fbfaff0a640 1 --2- 172.21.15.64:0/339326706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfa8106a00 0x7fbfa819ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.781+0000 7fbfaff0a640 1 --2- 172.21.15.64:0/339326706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa8101390 0x7fbfa81986e0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.781+0000 7fbfaff0a640 1 --2- 172.21.15.64:0/339326706 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fbf940a2da0 0x7fbf940a5260 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.781+0000 7fbfaff0a640 1 --2- 172.21.15.64:0/339326706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfa81009f0 0x7fbfa81981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.781+0000 7fbfaff0a640 1 -- 172.21.15.64:0/339326706 >> 172.21.15.64:0/339326706 conn(0x7fbfa80fc6b0 msgr2=0x7fbfa80fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.781+0000 7fbfaff0a640 1 -- 172.21.15.64:0/339326706 shutdown_connections 2024-09-12T03:31:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:42.781+0000 7fbfaff0a640 1 -- 172.21.15.64:0/339326706 wait complete. 2024-09-12T03:31:43.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:31:43 smithi115 ceph-mon[34801]: pgmap v76: 129 pgs: 129 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 17 op/s 2024-09-12T03:31:43.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:31:43 smithi195 ceph-mon[34786]: pgmap v76: 129 pgs: 129 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 17 op/s 2024-09-12T03:31:43.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:31:43 smithi064 ceph-mon[30952]: pgmap v76: 129 pgs: 129 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 17 op/s 2024-09-12T03:31:45.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:31:45 smithi115 ceph-mon[34801]: pgmap v77: 129 pgs: 129 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-09-12T03:31:45.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:31:45 smithi195 ceph-mon[34786]: pgmap v77: 129 pgs: 129 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-09-12T03:31:45.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:31:45 smithi064 ceph-mon[30952]: pgmap v77: 129 pgs: 129 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-09-12T03:31:47.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:31:47 smithi115 ceph-mon[34801]: pgmap v78: 129 pgs: 129 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-09-12T03:31:47.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:31:47 smithi195 ceph-mon[34786]: pgmap v78: 129 pgs: 129 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-09-12T03:31:47.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:31:47 smithi064 ceph-mon[30952]: pgmap v78: 129 pgs: 129 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-09-12T03:31:49.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:31:49 smithi115 ceph-mon[34801]: pgmap v79: 129 pgs: 129 active+clean; 2.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 17 op/s 2024-09-12T03:31:49.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:31:49 smithi195 ceph-mon[34786]: pgmap v79: 129 pgs: 129 active+clean; 2.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 17 op/s 2024-09-12T03:31:49.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:31:49 smithi064 ceph-mon[30952]: pgmap v79: 129 pgs: 129 active+clean; 2.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 17 op/s 2024-09-12T03:31:51.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:31:51 smithi115 ceph-mon[34801]: pgmap v80: 129 pgs: 129 active+clean; 2.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-09-12T03:31:51.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:31:51 smithi195 ceph-mon[34786]: pgmap v80: 129 pgs: 129 active+clean; 2.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-09-12T03:31:51.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:31:51 smithi064 ceph-mon[30952]: pgmap v80: 129 pgs: 129 active+clean; 2.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-09-12T03:31:53.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:31:53 smithi115 ceph-mon[34801]: pgmap v81: 129 pgs: 129 active+clean; 2.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 17 op/s 2024-09-12T03:31:53.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:31:53 smithi195 ceph-mon[34786]: pgmap v81: 129 pgs: 129 active+clean; 2.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 17 op/s 2024-09-12T03:31:53.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:31:53 smithi064 ceph-mon[30952]: pgmap v81: 129 pgs: 129 active+clean; 2.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 17 op/s 2024-09-12T03:31:55.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:31:55 smithi115 ceph-mon[34801]: pgmap v82: 129 pgs: 129 active+clean; 2.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-09-12T03:31:55.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:31:55 smithi195 ceph-mon[34786]: pgmap v82: 129 pgs: 129 active+clean; 2.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-09-12T03:31:55.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:31:55 smithi064 ceph-mon[30952]: pgmap v82: 129 pgs: 129 active+clean; 2.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-09-12T03:31:55.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.881+0000 7fc1ba7fd640 1 -- 172.21.15.64:0/3929839855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1b4100830 msgr2=0x7fc1b4100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:55.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.881+0000 7fc1ba7fd640 1 --2- 172.21.15.64:0/3929839855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1b4100830 0x7fc1b4100c10 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4009930 tx=0x7fc1a402a4d0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.881+0000 7fc1ba7fd640 1 -- 172.21.15.64:0/3929839855 shutdown_connections 2024-09-12T03:31:55.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.881+0000 7fc1ba7fd640 1 --2- 172.21.15.64:0/3929839855 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1b41080f0 0x7fc1b410ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.881+0000 7fc1ba7fd640 1 --2- 172.21.15.64:0/3929839855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1b4101150 0x7fc1b4107990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.881+0000 7fc1ba7fd640 1 --2- 172.21.15.64:0/3929839855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1b4100830 0x7fc1b4100c10 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.881+0000 7fc1ba7fd640 1 -- 172.21.15.64:0/3929839855 >> 172.21.15.64:0/3929839855 conn(0x7fc1b4075ec0 msgr2=0x7fc1b40762d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:55.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.881+0000 7fc1ba7fd640 1 -- 172.21.15.64:0/3929839855 shutdown_connections 2024-09-12T03:31:55.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.881+0000 7fc1ba7fd640 1 -- 172.21.15.64:0/3929839855 wait complete. 2024-09-12T03:31:55.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.882+0000 7fc1ba7fd640 1 Processor -- start 2024-09-12T03:31:55.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.882+0000 7fc1ba7fd640 1 -- start start 2024-09-12T03:31:55.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.882+0000 7fc1ba7fd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1b4100830 0x7fc1b41017a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:55.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.883+0000 7fc1ba7fd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1b4101150 0x7fc1b4101ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:55.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.883+0000 7fc1ba7fd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1b41080f0 0x7fc1b4102220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:55.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.883+0000 7fc1ba7fd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b4111dd0 con 0x7fc1b41080f0 2024-09-12T03:31:55.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.883+0000 7fc1ba7fd640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b4111c50 con 0x7fc1b4100830 2024-09-12T03:31:55.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.883+0000 7fc1ba7fd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b4111f50 con 0x7fc1b4101150 2024-09-12T03:31:55.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.883+0000 7fc1b8d9a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1b41080f0 0x7fc1b4102220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:55.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.883+0000 7fc1b8d9a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1b41080f0 0x7fc1b4102220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60060/0 (socket says 172.21.15.64:60060) 2024-09-12T03:31:55.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.883+0000 7fc1b8d9a640 1 -- 172.21.15.64:0/3698981538 learned_addr learned my addr 172.21.15.64:0/3698981538 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:31:55.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.883+0000 7fc1b3fff640 1 --2- 172.21.15.64:0/3698981538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1b4100830 0x7fc1b41017a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:55.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.883+0000 7fc1b37fe640 1 --2- 172.21.15.64:0/3698981538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1b4101150 0x7fc1b4101ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:55.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.883+0000 7fc1b8d9a640 1 -- 172.21.15.64:0/3698981538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1b4101150 msgr2=0x7fc1b4101ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:55.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.883+0000 7fc1b8d9a640 1 --2- 172.21.15.64:0/3698981538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1b4101150 0x7fc1b4101ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.883+0000 7fc1b8d9a640 1 -- 172.21.15.64:0/3698981538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1b4100830 msgr2=0x7fc1b41017a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:55.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.883+0000 7fc1b8d9a640 1 --2- 172.21.15.64:0/3698981538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1b4100830 0x7fc1b41017a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.883+0000 7fc1b8d9a640 1 -- 172.21.15.64:0/3698981538 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b41a6820 con 0x7fc1b41080f0 2024-09-12T03:31:55.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.883+0000 7fc1b8d9a640 1 --2- 172.21.15.64:0/3698981538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1b41080f0 0x7fc1b4102220 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac00eaa0 tx=0x7fc1ac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:55.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.883+0000 7fc1b17fa640 1 -- 172.21.15.64:0/3698981538 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac00cdb0 con 0x7fc1b41080f0 2024-09-12T03:31:55.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.883+0000 7fc1b17fa640 1 -- 172.21.15.64:0/3698981538 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1ac005020 con 0x7fc1b41080f0 2024-09-12T03:31:55.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.883+0000 7fc1b17fa640 1 -- 172.21.15.64:0/3698981538 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac0116d0 con 0x7fc1b41080f0 2024-09-12T03:31:55.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.883+0000 7fc1ba7fd640 1 -- 172.21.15.64:0/3698981538 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1b41a6b10 con 0x7fc1b41080f0 2024-09-12T03:31:55.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.884+0000 7fc1ba7fd640 1 -- 172.21.15.64:0/3698981538 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1b41a7060 con 0x7fc1b41080f0 2024-09-12T03:31:55.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.886+0000 7fc1b17fa640 1 -- 172.21.15.64:0/3698981538 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc1ac0051d0 con 0x7fc1b41080f0 2024-09-12T03:31:55.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.886+0000 7fc1bd67d740 1 Processor -- start 2024-09-12T03:31:55.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.886+0000 7fc1bd67d740 1 -- start start 2024-09-12T03:31:55.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.886+0000 7fc1bd67d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a6d1571af0 0x55a6d1571ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:55.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.886+0000 7fc1bd67d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a6d1572410 0x55a6d157c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:55.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.886+0000 7fc1bd67d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a6d157ccb0 0x55a6d157f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.886+0000 7fc1bd67d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a6d14d62f0 con 0x55a6d1571af0 2024-09-12T03:31:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.886+0000 7fc1bd67d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a6d147e270 con 0x55a6d157ccb0 2024-09-12T03:31:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.886+0000 7fc1bd67d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a6d14d22b0 con 0x55a6d1572410 2024-09-12T03:31:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.887+0000 7fc1b37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a6d1571af0 0x55a6d1571ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.887+0000 7fc1b37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a6d1571af0 0x55a6d1571ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60066/0 (socket says 172.21.15.64:60066) 2024-09-12T03:31:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.887+0000 7fc1b17fa640 1 --2- 172.21.15.64:0/3698981538 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fc19c0a2da0 0x7fc19c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.887+0000 7fc1b37fe640 1 -- 172.21.15.64:0/423170707 learned_addr learned my addr 172.21.15.64:0/423170707 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:31:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.887+0000 7fc1b3fff640 1 --2- 172.21.15.64:0/423170707 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a6d1572410 0x55a6d157c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.887+0000 7fc1b37fe640 1 --2- 172.21.15.64:0/423170707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a6d157ccb0 0x55a6d157f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.887+0000 7fc1b8d9a640 1 --2- 172.21.15.64:0/3698981538 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fc19c0a2da0 0x7fc19c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.887+0000 7fc1b37fe640 1 -- 172.21.15.64:0/423170707 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a6d1572410 msgr2=0x55a6d157c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.887+0000 7fc1b37fe640 1 --2- 172.21.15.64:0/423170707 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a6d1572410 0x55a6d157c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.887+0000 7fc1b37fe640 1 -- 172.21.15.64:0/423170707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a6d157ccb0 msgr2=0x55a6d157f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.887+0000 7fc1b37fe640 1 --2- 172.21.15.64:0/423170707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a6d157ccb0 0x55a6d157f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.887+0000 7fc1b37fe640 1 -- 172.21.15.64:0/423170707 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6d147d340 con 0x55a6d1571af0 2024-09-12T03:31:55.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.887+0000 7fc1b17fa640 1 -- 172.21.15.64:0/3698981538 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7fc1ac0cf870 con 0x7fc1b41080f0 2024-09-12T03:31:55.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.887+0000 7fc1b37fe640 1 --2- 172.21.15.64:0/423170707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a6d1571af0 0x55a6d1571ed0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc1a800afd0 tx=0x7fc1a800a740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb476fe9b12b709 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:55.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.887+0000 7fc1b8d9a640 1 --2- 172.21.15.64:0/3698981538 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fc19c0a2da0 0x7fc19c0a5260 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac097ed0 tx=0x7fc1ac06eae0 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:55.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.887+0000 7fc19a7fc640 1 -- 172.21.15.64:0/423170707 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8019070 con 0x55a6d1571af0 2024-09-12T03:31:55.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.888+0000 7fc19a7fc640 1 -- 172.21.15.64:0/423170707 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1a80054d0 con 0x55a6d1571af0 2024-09-12T03:31:55.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.888+0000 7fc19a7fc640 1 -- 172.21.15.64:0/423170707 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8004bf0 con 0x55a6d1571af0 2024-09-12T03:31:55.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.888+0000 7fc1bd67d740 1 -- 172.21.15.64:0/423170707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a6d1571af0 msgr2=0x55a6d1571ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:55.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.888+0000 7fc1bd67d740 1 --2- 172.21.15.64:0/423170707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a6d1571af0 0x55a6d1571ed0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc1a800afd0 tx=0x7fc1a800a740 comp rx=0 tx=0).stop 2024-09-12T03:31:55.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.888+0000 7fc1bd67d740 1 -- 172.21.15.64:0/423170707 shutdown_connections 2024-09-12T03:31:55.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.888+0000 7fc1bd67d740 1 --2- 172.21.15.64:0/423170707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a6d157ccb0 0x55a6d157f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.888+0000 7fc1bd67d740 1 --2- 172.21.15.64:0/423170707 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a6d1572410 0x55a6d157c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.888+0000 7fc1bd67d740 1 --2- 172.21.15.64:0/423170707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a6d1571af0 0x55a6d1571ed0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.888+0000 7fc1bd67d740 1 -- 172.21.15.64:0/423170707 >> 172.21.15.64:0/423170707 conn(0x55a6d156e060 msgr2=0x55a6d156e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:55.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.888+0000 7fc1bd67d740 1 -- 172.21.15.64:0/423170707 shutdown_connections 2024-09-12T03:31:55.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.888+0000 7fc1bd67d740 1 -- 172.21.15.64:0/423170707 wait complete. 2024-09-12T03:31:55.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.889+0000 7fc1bd67d740 1 -- start start 2024-09-12T03:31:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.889+0000 7fc1bd67d740 1 Processor -- start 2024-09-12T03:31:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.889+0000 7fc1bd67d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a6d1571af0 0x55a6d1579ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.889+0000 7fc1bd67d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a6d1572410 0x55a6d157a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.889+0000 7fc1bd67d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a6d157ccb0 0x55a6d157a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.889+0000 7fc1bd67d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a6d157f880 con 0x55a6d1571af0 2024-09-12T03:31:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.889+0000 7fc1bd67d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a6d145f290 con 0x55a6d157ccb0 2024-09-12T03:31:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.889+0000 7fc1bd67d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a6d157fa00 con 0x55a6d1572410 2024-09-12T03:31:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.889+0000 7fc1bd67d740 10 client.0 _ll_register_callbacks cb 0x55a6d1490140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:31:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.889+0000 7fc1b37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a6d1571af0 0x55a6d1579ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.889+0000 7fc1b37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a6d1571af0 0x55a6d1579ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60072/0 (socket says 172.21.15.64:60072) 2024-09-12T03:31:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.889+0000 7fc1b37fe640 1 -- 172.21.15.64:0/2724737308 learned_addr learned my addr 172.21.15.64:0/2724737308 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:31:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.890+0000 7fc1b3fff640 1 --2- 172.21.15.64:0/2724737308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a6d1572410 0x55a6d157a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.890+0000 7fc1b37fe640 1 -- 172.21.15.64:0/2724737308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a6d1572410 msgr2=0x55a6d157a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.890+0000 7fc1b37fe640 1 --2- 172.21.15.64:0/2724737308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a6d1572410 0x55a6d157a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.890+0000 7fc1b37fe640 1 -- 172.21.15.64:0/2724737308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a6d157ccb0 msgr2=0x55a6d157a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.890+0000 7fc1b37fe640 1 --2- 172.21.15.64:0/2724737308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a6d157ccb0 0x55a6d157a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.890+0000 7fc1b37fe640 1 -- 172.21.15.64:0/2724737308 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6d11e9210 con 0x55a6d1571af0 2024-09-12T03:31:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.890+0000 7fc1b37fe640 1 --2- 172.21.15.64:0/2724737308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a6d1571af0 0x55a6d1579ac0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc1a800a660 tx=0x7fc1a800ae00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ca2ca97f4f03f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.890+0000 7fc1937fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:31:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.890+0000 7fc1937fe640 1 -- 172.21.15.64:0/2724737308 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8019040 con 0x55a6d1571af0 2024-09-12T03:31:55.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.890+0000 7fc1937fe640 1 -- 172.21.15.64:0/2724737308 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7fc1a8004b90 con 0x55a6d1571af0 2024-09-12T03:31:55.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.890+0000 7fc1bd67d740 1 -- 172.21.15.64:0/2724737308 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1a80054d0 con 0x55a6d1571af0 2024-09-12T03:31:55.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.890+0000 7fc1937fe640 1 -- 172.21.15.64:0/2724737308 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc1a8004d90 con 0x55a6d1571af0 2024-09-12T03:31:55.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.890+0000 7fc1bd67d740 10 client.15558 fetch_fsmap learned FSMap version 25 2024-09-12T03:31:55.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.890+0000 7fc1bd67d740 1 -- 172.21.15.64:0/2724737308 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a6d12b5620 con 0x55a6d1571af0 2024-09-12T03:31:55.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.891+0000 7fc1937fe640 1 -- 172.21.15.64:0/2724737308 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1a80054d0 con 0x55a6d1571af0 2024-09-12T03:31:55.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.891+0000 7fc1937fe640 1 -- 172.21.15.64:0/2724737308 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc1a8003680 con 0x55a6d1571af0 2024-09-12T03:31:55.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.891+0000 7fc1bd67d740 10 client.15558 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:31:55.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.891+0000 7fc1bd67d740 10 client.15558 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:31:55.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.891+0000 7fc1bd67d740 20 client.15558 populate_metadata read hostname 'smithi064' 2024-09-12T03:31:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.891+0000 7fc1bd67d740 1 --2- 172.21.15.64:0/2724737308 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55a6d158ca90 0x55a6d158ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.891+0000 7fc1bd67d740 4 client.15558 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:31:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.891+0000 7fc1bd67d740 1 -- 172.21.15.64:0/2724737308 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6d14e3a20 con 0x55a6d158ca90 2024-09-12T03:31:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.891+0000 7fc1b3fff640 1 --2- 172.21.15.64:0/2724737308 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55a6d158ca90 0x55a6d158ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.892+0000 7fc1937fe640 10 client.15558 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:31:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.892+0000 7fc1b3fff640 1 --2- 172.21.15.64:0/2724737308 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55a6d158ca90 0x55a6d158ee80 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1c1d437f1a3783f server_cookie=c15fefec7d84b6b5 in_seq=0 out_seq=0 2024-09-12T03:31:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.892+0000 7fc1937fe640 1 -- 172.21.15.64:0/2724737308 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a6d14e3a20 con 0x55a6d158ca90 2024-09-12T03:31:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.892+0000 7fc1937fe640 10 client.15558 handle_command_reply: tid=0 2024-09-12T03:31:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.892+0000 7fc1bd67d740 1 client.15558 shutdown 2024-09-12T03:31:55.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.892+0000 7fc1bd67d740 1 -- 172.21.15.64:0/2724737308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a6d1571af0 msgr2=0x55a6d1579ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:55.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.892+0000 7fc1bd67d740 1 --2- 172.21.15.64:0/2724737308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a6d1571af0 0x55a6d1579ac0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc1a800a660 tx=0x7fc1a800ae00 comp rx=0 tx=0).stop 2024-09-12T03:31:55.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.892+0000 7fc1bd67d740 1 -- 172.21.15.64:0/2724737308 shutdown_connections 2024-09-12T03:31:55.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.892+0000 7fc1bd67d740 1 --2- 172.21.15.64:0/2724737308 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55a6d158ca90 0x55a6d158ee80 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.892+0000 7fc1bd67d740 1 --2- 172.21.15.64:0/2724737308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a6d157ccb0 0x55a6d157a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.892+0000 7fc1bd67d740 1 --2- 172.21.15.64:0/2724737308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a6d1572410 0x55a6d157a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.892+0000 7fc1937fe640 0 client.15558 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:31:55.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.892+0000 7fc1bd67d740 1 --2- 172.21.15.64:0/2724737308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a6d1571af0 0x55a6d1579ac0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.892+0000 7fc1bd67d740 1 -- 172.21.15.64:0/2724737308 >> 172.21.15.64:0/2724737308 conn(0x55a6d156e060 msgr2=0x55a6d1573d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:55.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.893+0000 7fc1bd67d740 1 -- 172.21.15.64:0/2724737308 shutdown_connections 2024-09-12T03:31:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.893+0000 7fc1bd67d740 1 -- 172.21.15.64:0/2724737308 wait complete. 2024-09-12T03:31:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.893+0000 7fc1bd67d740 20 client.15558 trim_cache size 0 max 16384 2024-09-12T03:31:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.907+0000 7fc1bd67d740 1 Processor -- start 2024-09-12T03:31:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.907+0000 7fc1bd67d740 1 -- start start 2024-09-12T03:31:55.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.907+0000 7fc1bd67d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a6d1571af0 0x55a6d157a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:55.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.907+0000 7fc1bd67d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a6d1572410 0x55a6d1591ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:55.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.907+0000 7fc1bd67d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a6d157ccb0 0x55a6d15923e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:55.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.907+0000 7fc1bd67d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b41b3ab0 con 0x55a6d1572410 2024-09-12T03:31:55.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.907+0000 7fc1bd67d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b41b1380 con 0x55a6d157ccb0 2024-09-12T03:31:55.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.907+0000 7fc1bd67d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b41af1d0 con 0x55a6d1571af0 2024-09-12T03:31:55.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.908+0000 7fc1b3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a6d1572410 0x55a6d1591ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:55.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.908+0000 7fc1b3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a6d1572410 0x55a6d1591ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60082/0 (socket says 172.21.15.64:60082) 2024-09-12T03:31:55.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.908+0000 7fc1b3fff640 1 -- 172.21.15.64:0/1714313710 learned_addr learned my addr 172.21.15.64:0/1714313710 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:31:55.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.908+0000 7fc1b3fff640 1 -- 172.21.15.64:0/1714313710 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a6d1571af0 msgr2=0x55a6d157a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:55.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.908+0000 7fc1b3fff640 1 --2- 172.21.15.64:0/1714313710 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a6d1571af0 0x55a6d157a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.908+0000 7fc1b3fff640 1 -- 172.21.15.64:0/1714313710 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a6d157ccb0 msgr2=0x55a6d15923e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:55.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.908+0000 7fc1b3fff640 1 --2- 172.21.15.64:0/1714313710 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a6d157ccb0 0x55a6d15923e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.908+0000 7fc1b3fff640 1 -- 172.21.15.64:0/1714313710 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6d156de90 con 0x55a6d1572410 2024-09-12T03:31:55.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.908+0000 7fc1b3fff640 1 --2- 172.21.15.64:0/1714313710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a6d1572410 0x55a6d1591ea0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4033000 tx=0x7fc1a4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8099ea816f801bbe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:55.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.908+0000 7fc199ffb640 1 -- 172.21.15.64:0/1714313710 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a4047030 con 0x55a6d1572410 2024-09-12T03:31:55.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.908+0000 7fc199ffb640 1 -- 172.21.15.64:0/1714313710 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1a40043c0 con 0x55a6d1572410 2024-09-12T03:31:55.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.908+0000 7fc199ffb640 1 -- 172.21.15.64:0/1714313710 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a4035040 con 0x55a6d1572410 2024-09-12T03:31:55.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.909+0000 7fc1bd67d740 1 -- 172.21.15.64:0/1714313710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a6d1572410 msgr2=0x55a6d1591ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:55.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.909+0000 7fc1bd67d740 1 --2- 172.21.15.64:0/1714313710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a6d1572410 0x55a6d1591ea0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4033000 tx=0x7fc1a4000bc0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.909+0000 7fc1bd67d740 1 -- 172.21.15.64:0/1714313710 shutdown_connections 2024-09-12T03:31:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.909+0000 7fc1bd67d740 1 --2- 172.21.15.64:0/1714313710 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a6d157ccb0 0x55a6d15923e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.909+0000 7fc1bd67d740 1 --2- 172.21.15.64:0/1714313710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a6d1572410 0x55a6d1591ea0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.909+0000 7fc1bd67d740 1 --2- 172.21.15.64:0/1714313710 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a6d1571af0 0x55a6d157a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.909+0000 7fc1bd67d740 1 -- 172.21.15.64:0/1714313710 >> 172.21.15.64:0/1714313710 conn(0x55a6d156e060 msgr2=0x55a6d157bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.909+0000 7fc1bd67d740 1 -- 172.21.15.64:0/1714313710 shutdown_connections 2024-09-12T03:31:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.909+0000 7fc1bd67d740 1 -- 172.21.15.64:0/1714313710 wait complete. 2024-09-12T03:31:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.909+0000 7fc1bd67d740 1 -- start start 2024-09-12T03:31:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.909+0000 7fc1bd67d740 1 Processor -- start 2024-09-12T03:31:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.910+0000 7fc1bd67d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a6d1571af0 0x55a6d158fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.910+0000 7fc1bd67d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a6d1572410 0x55a6d15901b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.910+0000 7fc1bd67d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a6d157ccb0 0x55a6d15906f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.910+0000 7fc1bd67d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b41af840 con 0x55a6d157ccb0 2024-09-12T03:31:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.910+0000 7fc1bd67d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b41b0cb0 con 0x55a6d1571af0 2024-09-12T03:31:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.910+0000 7fc1bd67d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b41aff10 con 0x55a6d1572410 2024-09-12T03:31:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.910+0000 7fc1bd67d740 10 client.0 _ll_register_callbacks cb 0x7fc1a800af00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:31:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.910+0000 7fc1b37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a6d157ccb0 0x55a6d15906f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.910+0000 7fc1b37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a6d157ccb0 0x55a6d15906f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60086/0 (socket says 172.21.15.64:60086) 2024-09-12T03:31:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.910+0000 7fc1b37fe640 1 -- 172.21.15.64:0/2335894651 learned_addr learned my addr 172.21.15.64:0/2335894651 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:31:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.910+0000 7fc1b37fe640 1 --2- 172.21.15.64:0/2335894651 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a6d1571af0 0x55a6d158fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.910+0000 7fc1b3fff640 1 --2- 172.21.15.64:0/2335894651 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a6d1572410 0x55a6d15901b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.910+0000 7fc1b37fe640 1 -- 172.21.15.64:0/2335894651 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a6d1572410 msgr2=0x55a6d15901b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.910+0000 7fc1b37fe640 1 --2- 172.21.15.64:0/2335894651 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a6d1572410 0x55a6d15901b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.910+0000 7fc1b37fe640 1 -- 172.21.15.64:0/2335894651 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a6d1571af0 msgr2=0x55a6d158fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.910+0000 7fc1b37fe640 1 --2- 172.21.15.64:0/2335894651 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a6d1571af0 0x55a6d158fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.910+0000 7fc1b37fe640 1 -- 172.21.15.64:0/2335894651 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6d15692d0 con 0x55a6d157ccb0 2024-09-12T03:31:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.910+0000 7fc1b37fe640 1 --2- 172.21.15.64:0/2335894651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a6d157ccb0 0x55a6d15906f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8004ff0 tx=0x7fc1a801f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d11fad4fa8dcfa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.910+0000 7fc1937fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:31:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.910+0000 7fc1937fe640 1 -- 172.21.15.64:0/2335894651 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8019030 con 0x55a6d157ccb0 2024-09-12T03:31:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.910+0000 7fc1bd67d740 1 -- 172.21.15.64:0/2335894651 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1a40043c0 con 0x55a6d157ccb0 2024-09-12T03:31:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.910+0000 7fc1937fe640 1 -- 172.21.15.64:0/2335894651 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7fc1a8005330 con 0x55a6d157ccb0 2024-09-12T03:31:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.911+0000 7fc1937fe640 1 -- 172.21.15.64:0/2335894651 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc1a8005530 con 0x55a6d157ccb0 2024-09-12T03:31:55.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.911+0000 7fc1bd67d740 10 client.15564 fetch_fsmap learned FSMap version 25 2024-09-12T03:31:55.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.911+0000 7fc1bd67d740 1 -- 172.21.15.64:0/2335894651 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a6d156f110 con 0x55a6d157ccb0 2024-09-12T03:31:55.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.911+0000 7fc1937fe640 1 -- 172.21.15.64:0/2335894651 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1a40043c0 con 0x55a6d157ccb0 2024-09-12T03:31:55.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.911+0000 7fc1937fe640 1 -- 172.21.15.64:0/2335894651 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc1a80117d0 con 0x55a6d157ccb0 2024-09-12T03:31:55.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.911+0000 7fc1bd67d740 10 client.15564 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:31:55.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.911+0000 7fc1bd67d740 10 client.15564 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:31:55.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.911+0000 7fc1bd67d740 20 client.15564 populate_metadata read hostname 'smithi064' 2024-09-12T03:31:55.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.911+0000 7fc1bd67d740 1 --2- 172.21.15.64:0/2335894651 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55a6d15f7980 0x55a6d15f9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:55.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.911+0000 7fc1bd67d740 4 client.15564 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:31:55.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.911+0000 7fc1bd67d740 1 -- 172.21.15.64:0/2335894651 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a6d14ddd70 con 0x55a6d15f7980 2024-09-12T03:31:55.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.911+0000 7fc1b3fff640 1 --2- 172.21.15.64:0/2335894651 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55a6d15f7980 0x55a6d15f9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:55.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.912+0000 7fc1b3fff640 1 --2- 172.21.15.64:0/2335894651 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55a6d15f7980 0x55a6d15f9da0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3379ad0224eeb54 server_cookie=474895f33fb800f5 in_seq=0 out_seq=0 2024-09-12T03:31:55.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.912+0000 7fc1937fe640 10 client.15564 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:31:55.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.925+0000 7fc1937fe640 1 -- 172.21.15.64:0/2335894651 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a6d14ddd70 con 0x55a6d15f7980 2024-09-12T03:31:55.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.925+0000 7fc1937fe640 10 client.15564 handle_command_reply: tid=0 2024-09-12T03:31:55.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.926+0000 7fc1bd67d740 1 client.15564 shutdown 2024-09-12T03:31:55.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.926+0000 7fc1bd67d740 1 -- 172.21.15.64:0/2335894651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a6d157ccb0 msgr2=0x55a6d15906f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:55.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.926+0000 7fc1bd67d740 1 --2- 172.21.15.64:0/2335894651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a6d157ccb0 0x55a6d15906f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8004ff0 tx=0x7fc1a801f040 comp rx=0 tx=0).stop 2024-09-12T03:31:55.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.926+0000 7fc1bd67d740 1 -- 172.21.15.64:0/2335894651 shutdown_connections 2024-09-12T03:31:55.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.926+0000 7fc1bd67d740 1 --2- 172.21.15.64:0/2335894651 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55a6d15f7980 0x55a6d15f9da0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.926+0000 7fc1bd67d740 1 --2- 172.21.15.64:0/2335894651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a6d157ccb0 0x55a6d15906f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.926+0000 7fc1bd67d740 1 --2- 172.21.15.64:0/2335894651 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a6d1572410 0x55a6d15901b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.926+0000 7fc1937fe640 0 client.15564 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:31:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.926+0000 7fc1bd67d740 1 --2- 172.21.15.64:0/2335894651 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a6d1571af0 0x55a6d158fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.926+0000 7fc1bd67d740 1 -- 172.21.15.64:0/2335894651 >> 172.21.15.64:0/2335894651 conn(0x55a6d156e060 msgr2=0x55a6d1587cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.926+0000 7fc1bd67d740 1 -- 172.21.15.64:0/2335894651 shutdown_connections 2024-09-12T03:31:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.926+0000 7fc1bd67d740 1 -- 172.21.15.64:0/2335894651 wait complete. 2024-09-12T03:31:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.926+0000 7fc1bd67d740 20 client.15564 trim_cache size 0 max 16384 2024-09-12T03:31:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:31:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:31:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:31:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:31:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.926+0000 7fc1ba7fd640 1 -- 172.21.15.64:0/3698981538 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fc19c0a2da0 msgr2=0x7fc19c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.926+0000 7fc1ba7fd640 1 --2- 172.21.15.64:0/3698981538 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fc19c0a2da0 0x7fc19c0a5260 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac097ed0 tx=0x7fc1ac06eae0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.926+0000 7fc1ba7fd640 1 -- 172.21.15.64:0/3698981538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1b41080f0 msgr2=0x7fc1b4102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.926+0000 7fc1ba7fd640 1 --2- 172.21.15.64:0/3698981538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1b41080f0 0x7fc1b4102220 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac00eaa0 tx=0x7fc1ac00ef60 comp rx=0 tx=0).stop 2024-09-12T03:31:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.927+0000 7fc1ba7fd640 1 -- 172.21.15.64:0/3698981538 shutdown_connections 2024-09-12T03:31:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.927+0000 7fc1ba7fd640 1 --2- 172.21.15.64:0/3698981538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1b41080f0 0x7fc1b4102220 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.927+0000 7fc1ba7fd640 1 --2- 172.21.15.64:0/3698981538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1b4101150 0x7fc1b4101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.927+0000 7fc1ba7fd640 1 --2- 172.21.15.64:0/3698981538 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fc19c0a2da0 0x7fc19c0a5260 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.927+0000 7fc1ba7fd640 1 --2- 172.21.15.64:0/3698981538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1b4100830 0x7fc1b41017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.927+0000 7fc1ba7fd640 1 -- 172.21.15.64:0/3698981538 >> 172.21.15.64:0/3698981538 conn(0x7fc1b4075ec0 msgr2=0x7fc1b4105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.927+0000 7fc1ba7fd640 1 -- 172.21.15.64:0/3698981538 shutdown_connections 2024-09-12T03:31:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:31:55.927+0000 7fc1ba7fd640 1 -- 172.21.15.64:0/3698981538 wait complete. 2024-09-12T03:31:57.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:31:57 smithi115 ceph-mon[34801]: pgmap v83: 129 pgs: 129 active+clean; 2.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-09-12T03:31:57.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:31:57 smithi195 ceph-mon[34786]: pgmap v83: 129 pgs: 129 active+clean; 2.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-09-12T03:31:57.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:31:57 smithi064 ceph-mon[30952]: pgmap v83: 129 pgs: 129 active+clean; 2.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-09-12T03:31:59.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:31:59 smithi115 ceph-mon[34801]: pgmap v84: 129 pgs: 129 active+clean; 3.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 33 KiB/s wr, 20 op/s 2024-09-12T03:31:59.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:31:59 smithi195 ceph-mon[34786]: pgmap v84: 129 pgs: 129 active+clean; 3.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 33 KiB/s wr, 20 op/s 2024-09-12T03:31:59.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:31:59 smithi064 ceph-mon[30952]: pgmap v84: 129 pgs: 129 active+clean; 3.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 33 KiB/s wr, 20 op/s 2024-09-12T03:32:01.646 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:32:01 smithi115 ceph-mon[34801]: pgmap v85: 129 pgs: 129 active+clean; 3.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-09-12T03:32:01.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:32:01 smithi195 ceph-mon[34786]: pgmap v85: 129 pgs: 129 active+clean; 3.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-09-12T03:32:01.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:32:01 smithi064 ceph-mon[30952]: pgmap v85: 129 pgs: 129 active+clean; 3.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-09-12T03:32:03.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:32:03 smithi115 ceph-mon[34801]: pgmap v86: 129 pgs: 129 active+clean; 3.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 52 KiB/s wr, 32 op/s 2024-09-12T03:32:03.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:32:03 smithi195 ceph-mon[34786]: pgmap v86: 129 pgs: 129 active+clean; 3.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 52 KiB/s wr, 32 op/s 2024-09-12T03:32:03.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:32:03 smithi064 ceph-mon[30952]: pgmap v86: 129 pgs: 129 active+clean; 3.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 52 KiB/s wr, 32 op/s 2024-09-12T03:32:05.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:32:05 smithi115 ceph-mon[34801]: pgmap v87: 129 pgs: 129 active+clean; 3.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 43 KiB/s wr, 27 op/s 2024-09-12T03:32:05.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:32:05 smithi195 ceph-mon[34786]: pgmap v87: 129 pgs: 129 active+clean; 3.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 43 KiB/s wr, 27 op/s 2024-09-12T03:32:05.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:32:05 smithi064 ceph-mon[30952]: pgmap v87: 129 pgs: 129 active+clean; 3.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 43 KiB/s wr, 27 op/s 2024-09-12T03:32:07.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:32:07 smithi115 ceph-mon[34801]: pgmap v88: 129 pgs: 129 active+clean; 3.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 43 KiB/s wr, 27 op/s 2024-09-12T03:32:07.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:32:07 smithi195 ceph-mon[34786]: pgmap v88: 129 pgs: 129 active+clean; 3.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 43 KiB/s wr, 27 op/s 2024-09-12T03:32:07.774 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:32:07 smithi064 ceph-mon[30952]: pgmap v88: 129 pgs: 129 active+clean; 3.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 43 KiB/s wr, 27 op/s 2024-09-12T03:32:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.028+0000 7fa85792a640 1 -- 172.21.15.64:0/1365111515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8501069c0 msgr2=0x7fa850106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.028+0000 7fa85792a640 1 --2- 172.21.15.64:0/1365111515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8501069c0 0x7fa850106da0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa84c009930 tx=0x7fa84c02a420 comp rx=0 tx=0).stop 2024-09-12T03:32:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.028+0000 7fa85792a640 1 -- 172.21.15.64:0/1365111515 shutdown_connections 2024-09-12T03:32:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.028+0000 7fa85792a640 1 --2- 172.21.15.64:0/1365111515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa850101350 0x7fa85010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.028+0000 7fa85792a640 1 --2- 172.21.15.64:0/1365111515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa8501009b0 0x7fa850100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.028+0000 7fa85792a640 1 --2- 172.21.15.64:0/1365111515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8501069c0 0x7fa850106da0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.028+0000 7fa85792a640 1 -- 172.21.15.64:0/1365111515 >> 172.21.15.64:0/1365111515 conn(0x7fa850077990 msgr2=0x7fa8500fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.029+0000 7fa85792a640 1 -- 172.21.15.64:0/1365111515 shutdown_connections 2024-09-12T03:32:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.029+0000 7fa85792a640 1 -- 172.21.15.64:0/1365111515 wait complete. 2024-09-12T03:32:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.029+0000 7fa85792a640 1 Processor -- start 2024-09-12T03:32:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.029+0000 7fa85792a640 1 -- start start 2024-09-12T03:32:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.030+0000 7fa85792a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8501009b0 0x7fa8501982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.030+0000 7fa85792a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa850101350 0x7fa850198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.030+0000 7fa85792a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa8501069c0 0x7fa85019cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.030+0000 7fa85792a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa850110d00 con 0x7fa8501009b0 2024-09-12T03:32:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.030+0000 7fa85792a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa850110b80 con 0x7fa850101350 2024-09-12T03:32:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.030+0000 7fa85792a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa850110e80 con 0x7fa8501069c0 2024-09-12T03:32:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.030+0000 7fa8556c6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8501009b0 0x7fa8501982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.030+0000 7fa8556c6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8501009b0 0x7fa8501982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49218/0 (socket says 172.21.15.64:49218) 2024-09-12T03:32:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.030+0000 7fa8556c6640 1 -- 172.21.15.64:0/316431042 learned_addr learned my addr 172.21.15.64:0/316431042 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:32:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.030+0000 7fa854ec5640 1 --2- 172.21.15.64:0/316431042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa850101350 0x7fa850198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.030+0000 7fa855ec7640 1 --2- 172.21.15.64:0/316431042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa8501069c0 0x7fa85019cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.030+0000 7fa8556c6640 1 -- 172.21.15.64:0/316431042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa8501069c0 msgr2=0x7fa85019cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.030+0000 7fa8556c6640 1 --2- 172.21.15.64:0/316431042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa8501069c0 0x7fa85019cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.030+0000 7fa8556c6640 1 -- 172.21.15.64:0/316431042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa850101350 msgr2=0x7fa850198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.030+0000 7fa8556c6640 1 --2- 172.21.15.64:0/316431042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa850101350 0x7fa850198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.030+0000 7fa8556c6640 1 -- 172.21.15.64:0/316431042 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa85019d2d0 con 0x7fa8501009b0 2024-09-12T03:32:09.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.030+0000 7fa8556c6640 1 --2- 172.21.15.64:0/316431042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8501009b0 0x7fa8501982e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa84c005ab0 tx=0x7fa84c003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:09.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.030+0000 7fa8427fc640 1 -- 172.21.15.64:0/316431042 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c03f070 con 0x7fa8501009b0 2024-09-12T03:32:09.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.030+0000 7fa8427fc640 1 -- 172.21.15.64:0/316431042 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa84c031d60 con 0x7fa8501009b0 2024-09-12T03:32:09.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.030+0000 7fa85792a640 1 -- 172.21.15.64:0/316431042 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa85019d560 con 0x7fa8501009b0 2024-09-12T03:32:09.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.031+0000 7fa8427fc640 1 -- 172.21.15.64:0/316431042 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c035030 con 0x7fa8501009b0 2024-09-12T03:32:09.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.031+0000 7fa85792a640 1 -- 172.21.15.64:0/316431042 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8501a4e50 con 0x7fa8501009b0 2024-09-12T03:32:09.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.033+0000 7fa8427fc640 1 -- 172.21.15.64:0/316431042 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fa84c042070 con 0x7fa8501009b0 2024-09-12T03:32:09.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.033+0000 7fa85a6c5740 1 Processor -- start 2024-09-12T03:32:09.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.033+0000 7fa85a6c5740 1 -- start start 2024-09-12T03:32:09.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.034+0000 7fa85a6c5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6ab28a9d0 0x55f6ab28adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:09.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.034+0000 7fa85a6c5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ab28b2f0 0x55f6ab299b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:09.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.034+0000 7fa85a6c5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6ab29a0c0 0x55f6ab29c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:09.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.034+0000 7fa85a6c5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f6ab1ef2f0 con 0x55f6ab29a0c0 2024-09-12T03:32:09.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.034+0000 7fa85a6c5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f6ab197270 con 0x55f6ab28b2f0 2024-09-12T03:32:09.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.034+0000 7fa85a6c5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f6ab1eb2b0 con 0x55f6ab28a9d0 2024-09-12T03:32:09.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.034+0000 7fa8427fc640 1 --2- 172.21.15.64:0/316431042 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fa83c0a2e20 0x7fa83c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:09.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.034+0000 7fa854ec5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6ab29a0c0 0x55f6ab29c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:09.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.034+0000 7fa854ec5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6ab29a0c0 0x55f6ab29c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49226/0 (socket says 172.21.15.64:49226) 2024-09-12T03:32:09.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.034+0000 7fa854ec5640 1 -- 172.21.15.64:0/1795308933 learned_addr learned my addr 172.21.15.64:0/1795308933 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:32:09.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.034+0000 7fa855ec7640 1 --2- 172.21.15.64:0/316431042 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fa83c0a2e20 0x7fa83c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:09.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.034+0000 7fa855ec7640 1 --2- 172.21.15.64:0/1795308933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6ab28a9d0 0x55f6ab28adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:09.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.034+0000 7fa8556c6640 1 --2- 172.21.15.64:0/1795308933 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ab28b2f0 0x55f6ab299b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:09.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.034+0000 7fa8427fc640 1 -- 172.21.15.64:0/316431042 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7fa84c0efee0 con 0x7fa8501009b0 2024-09-12T03:32:09.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.034+0000 7fa854ec5640 1 -- 172.21.15.64:0/1795308933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6ab28a9d0 msgr2=0x55f6ab28adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.034+0000 7fa854ec5640 1 --2- 172.21.15.64:0/1795308933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6ab28a9d0 0x55f6ab28adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.034+0000 7fa854ec5640 1 -- 172.21.15.64:0/1795308933 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ab28b2f0 msgr2=0x55f6ab299b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.034+0000 7fa854ec5640 1 --2- 172.21.15.64:0/1795308933 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ab28b2f0 0x55f6ab299b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.034+0000 7fa854ec5640 1 -- 172.21.15.64:0/1795308933 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6ab196340 con 0x55f6ab29a0c0 2024-09-12T03:32:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.035+0000 7fa855ec7640 1 --2- 172.21.15.64:0/316431042 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fa83c0a2e20 0x7fa83c0a52e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa848005cc0 tx=0x7fa84800ad50 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.035+0000 7fa854ec5640 1 --2- 172.21.15.64:0/1795308933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6ab29a0c0 0x55f6ab29c4b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa84400b740 tx=0x7fa84400bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf285dd989ecfb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.035+0000 7fa83b7fe640 1 -- 172.21.15.64:0/1795308933 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa844003dd0 con 0x55f6ab29a0c0 2024-09-12T03:32:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.035+0000 7fa83b7fe640 1 -- 172.21.15.64:0/1795308933 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa844015070 con 0x55f6ab29a0c0 2024-09-12T03:32:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.035+0000 7fa83b7fe640 1 -- 172.21.15.64:0/1795308933 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa844004ce0 con 0x55f6ab29a0c0 2024-09-12T03:32:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.035+0000 7fa85a6c5740 1 -- 172.21.15.64:0/1795308933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6ab29a0c0 msgr2=0x55f6ab29c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.035+0000 7fa85a6c5740 1 --2- 172.21.15.64:0/1795308933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6ab29a0c0 0x55f6ab29c4b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa84400b740 tx=0x7fa84400bc00 comp rx=0 tx=0).stop 2024-09-12T03:32:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.036+0000 7fa85a6c5740 1 -- 172.21.15.64:0/1795308933 shutdown_connections 2024-09-12T03:32:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.036+0000 7fa85a6c5740 1 --2- 172.21.15.64:0/1795308933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6ab29a0c0 0x55f6ab29c4b0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.036+0000 7fa85a6c5740 1 --2- 172.21.15.64:0/1795308933 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ab28b2f0 0x55f6ab299b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.036+0000 7fa85a6c5740 1 --2- 172.21.15.64:0/1795308933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6ab28a9d0 0x55f6ab28adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.036+0000 7fa85a6c5740 1 -- 172.21.15.64:0/1795308933 >> 172.21.15.64:0/1795308933 conn(0x55f6ab287060 msgr2=0x55f6ab287440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.036+0000 7fa85a6c5740 1 -- 172.21.15.64:0/1795308933 shutdown_connections 2024-09-12T03:32:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.036+0000 7fa85a6c5740 1 -- 172.21.15.64:0/1795308933 wait complete. 2024-09-12T03:32:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.036+0000 7fa85a6c5740 1 -- start start 2024-09-12T03:32:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.036+0000 7fa85a6c5740 1 Processor -- start 2024-09-12T03:32:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.037+0000 7fa85a6c5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ab28a9d0 0x55f6ab297040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.037+0000 7fa85a6c5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6ab28b2f0 0x55f6ab297580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.037+0000 7fa85a6c5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6ab29a0c0 0x55f6ab297ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.037+0000 7fa85a6c5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f6ab29ccf0 con 0x55f6ab29a0c0 2024-09-12T03:32:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.037+0000 7fa85a6c5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f6ab178290 con 0x55f6ab28a9d0 2024-09-12T03:32:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.037+0000 7fa85a6c5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f6ab29ce70 con 0x55f6ab28b2f0 2024-09-12T03:32:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.037+0000 7fa85a6c5740 10 client.0 _ll_register_callbacks cb 0x55f6ab1a9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:32:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.037+0000 7fa855ec7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6ab29a0c0 0x55f6ab297ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.037+0000 7fa855ec7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6ab29a0c0 0x55f6ab297ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49236/0 (socket says 172.21.15.64:49236) 2024-09-12T03:32:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.037+0000 7fa855ec7640 1 -- 172.21.15.64:0/2583493923 learned_addr learned my addr 172.21.15.64:0/2583493923 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:32:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.037+0000 7fa8556c6640 1 --2- 172.21.15.64:0/2583493923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ab28a9d0 0x55f6ab297040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.037+0000 7fa854ec5640 1 --2- 172.21.15.64:0/2583493923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6ab28b2f0 0x55f6ab297580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.037+0000 7fa855ec7640 1 -- 172.21.15.64:0/2583493923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6ab28b2f0 msgr2=0x55f6ab297580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.037+0000 7fa855ec7640 1 --2- 172.21.15.64:0/2583493923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6ab28b2f0 0x55f6ab297580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.037+0000 7fa855ec7640 1 -- 172.21.15.64:0/2583493923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ab28a9d0 msgr2=0x55f6ab297040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.037+0000 7fa855ec7640 1 --2- 172.21.15.64:0/2583493923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ab28a9d0 0x55f6ab297040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.037+0000 7fa855ec7640 1 -- 172.21.15.64:0/2583493923 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6aaf02210 con 0x55f6ab29a0c0 2024-09-12T03:32:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.037+0000 7fa855ec7640 1 --2- 172.21.15.64:0/2583493923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6ab29a0c0 0x55f6ab297ac0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa848012000 tx=0x7fa84800a4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ffad6c8da6cbc9f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.037+0000 7fa838ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:32:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.037+0000 7fa838ff9640 1 -- 172.21.15.64:0/2583493923 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa848026070 con 0x55f6ab29a0c0 2024-09-12T03:32:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.037+0000 7fa838ff9640 1 -- 172.21.15.64:0/2583493923 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7fa848020860 con 0x55f6ab29a0c0 2024-09-12T03:32:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.037+0000 7fa85a6c5740 1 -- 172.21.15.64:0/2583493923 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa844015070 con 0x55f6ab29a0c0 2024-09-12T03:32:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.038+0000 7fa838ff9640 1 -- 172.21.15.64:0/2583493923 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa848032050 con 0x55f6ab29a0c0 2024-09-12T03:32:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.038+0000 7fa85a6c5740 10 client.15576 fetch_fsmap learned FSMap version 25 2024-09-12T03:32:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.038+0000 7fa85a6c5740 1 -- 172.21.15.64:0/2583493923 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f6aafce620 con 0x55f6ab29a0c0 2024-09-12T03:32:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.038+0000 7fa838ff9640 1 -- 172.21.15.64:0/2583493923 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa844015070 con 0x55f6ab29a0c0 2024-09-12T03:32:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.038+0000 7fa838ff9640 1 -- 172.21.15.64:0/2583493923 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa84800f8d0 con 0x55f6ab29a0c0 2024-09-12T03:32:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.038+0000 7fa85a6c5740 10 client.15576 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:32:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.038+0000 7fa85a6c5740 10 client.15576 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:32:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.038+0000 7fa85a6c5740 20 client.15576 populate_metadata read hostname 'smithi064' 2024-09-12T03:32:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.038+0000 7fa85a6c5740 1 --2- 172.21.15.64:0/2583493923 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55f6ab2aa0b0 0x55f6ab2ac520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.038+0000 7fa85a6c5740 4 client.15576 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:32:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.038+0000 7fa85a6c5740 1 -- 172.21.15.64:0/2583493923 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6ab1fca20 con 0x55f6ab2aa0b0 2024-09-12T03:32:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.038+0000 7fa8556c6640 1 --2- 172.21.15.64:0/2583493923 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55f6ab2aa0b0 0x55f6ab2ac520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.039+0000 7fa8556c6640 1 --2- 172.21.15.64:0/2583493923 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55f6ab2aa0b0 0x55f6ab2ac520 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=397a5d7b3b35f675 server_cookie=3947732c1e97ce6b in_seq=0 out_seq=0 2024-09-12T03:32:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.039+0000 7fa838ff9640 10 client.15576 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:32:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.039+0000 7fa838ff9640 1 -- 172.21.15.64:0/2583493923 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f6ab1fca20 con 0x55f6ab2aa0b0 2024-09-12T03:32:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.039+0000 7fa838ff9640 10 client.15576 handle_command_reply: tid=0 2024-09-12T03:32:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.040+0000 7fa85a6c5740 1 client.15576 shutdown 2024-09-12T03:32:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.040+0000 7fa85a6c5740 1 -- 172.21.15.64:0/2583493923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6ab29a0c0 msgr2=0x55f6ab297ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.040+0000 7fa85a6c5740 1 --2- 172.21.15.64:0/2583493923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6ab29a0c0 0x55f6ab297ac0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa848012000 tx=0x7fa84800a4d0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.040+0000 7fa85a6c5740 1 -- 172.21.15.64:0/2583493923 shutdown_connections 2024-09-12T03:32:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.040+0000 7fa85a6c5740 1 --2- 172.21.15.64:0/2583493923 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55f6ab2aa0b0 0x55f6ab2ac520 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.040+0000 7fa85a6c5740 1 --2- 172.21.15.64:0/2583493923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6ab29a0c0 0x55f6ab297ac0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.040+0000 7fa85a6c5740 1 --2- 172.21.15.64:0/2583493923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6ab28b2f0 0x55f6ab297580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.040+0000 7fa838ff9640 0 client.15576 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:32:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.040+0000 7fa85a6c5740 1 --2- 172.21.15.64:0/2583493923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ab28a9d0 0x55f6ab297040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.040+0000 7fa85a6c5740 1 -- 172.21.15.64:0/2583493923 >> 172.21.15.64:0/2583493923 conn(0x55f6ab287060 msgr2=0x55f6ab28cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.040+0000 7fa85a6c5740 1 -- 172.21.15.64:0/2583493923 shutdown_connections 2024-09-12T03:32:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.040+0000 7fa85a6c5740 1 -- 172.21.15.64:0/2583493923 wait complete. 2024-09-12T03:32:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.040+0000 7fa85a6c5740 20 client.15576 trim_cache size 0 max 16384 2024-09-12T03:32:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.055+0000 7fa85a6c5740 1 Processor -- start 2024-09-12T03:32:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.055+0000 7fa85a6c5740 1 -- start start 2024-09-12T03:32:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.055+0000 7fa85a6c5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6ab28a9d0 0x55f6ab298fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.055+0000 7fa85a6c5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6ab28b2f0 0x55f6ab2b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.055+0000 7fa85a6c5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ab29a0c0 0x55f6ab2b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.055+0000 7fa85a6c5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa8501af5c0 con 0x55f6ab28b2f0 2024-09-12T03:32:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.055+0000 7fa85a6c5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa8501ace90 con 0x55f6ab29a0c0 2024-09-12T03:32:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.055+0000 7fa85a6c5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa8501aace0 con 0x55f6ab28a9d0 2024-09-12T03:32:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.055+0000 7fa854ec5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6ab28b2f0 0x55f6ab2b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.055+0000 7fa854ec5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6ab28b2f0 0x55f6ab2b1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49250/0 (socket says 172.21.15.64:49250) 2024-09-12T03:32:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.055+0000 7fa854ec5640 1 -- 172.21.15.64:0/4268424859 learned_addr learned my addr 172.21.15.64:0/4268424859 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:32:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.055+0000 7fa855ec7640 1 --2- 172.21.15.64:0/4268424859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ab29a0c0 0x55f6ab2b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.055+0000 7fa8556c6640 1 --2- 172.21.15.64:0/4268424859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6ab28a9d0 0x55f6ab298fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.055+0000 7fa854ec5640 1 -- 172.21.15.64:0/4268424859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6ab28a9d0 msgr2=0x55f6ab298fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.055+0000 7fa854ec5640 1 --2- 172.21.15.64:0/4268424859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6ab28a9d0 0x55f6ab298fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.055+0000 7fa854ec5640 1 -- 172.21.15.64:0/4268424859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ab29a0c0 msgr2=0x55f6ab2b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.056+0000 7fa854ec5640 1 --2- 172.21.15.64:0/4268424859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ab29a0c0 0x55f6ab2b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.056+0000 7fa854ec5640 1 -- 172.21.15.64:0/4268424859 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6ab28b120 con 0x55f6ab28b2f0 2024-09-12T03:32:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.056+0000 7fa855ec7640 1 --2- 172.21.15.64:0/4268424859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ab29a0c0 0x55f6ab2b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:32:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.056+0000 7fa854ec5640 1 --2- 172.21.15.64:0/4268424859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6ab28b2f0 0x55f6ab2b1e40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa8440039c0 tx=0x7fa844005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88674f8bcb93b848 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.056+0000 7fa83affd640 1 -- 172.21.15.64:0/4268424859 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa844029070 con 0x55f6ab28b2f0 2024-09-12T03:32:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.056+0000 7fa83affd640 1 -- 172.21.15.64:0/4268424859 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa844015070 con 0x55f6ab28b2f0 2024-09-12T03:32:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.056+0000 7fa83affd640 1 -- 172.21.15.64:0/4268424859 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa844011db0 con 0x55f6ab28b2f0 2024-09-12T03:32:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.056+0000 7fa85a6c5740 1 -- 172.21.15.64:0/4268424859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6ab28b2f0 msgr2=0x55f6ab2b1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.056+0000 7fa85a6c5740 1 --2- 172.21.15.64:0/4268424859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6ab28b2f0 0x55f6ab2b1e40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa8440039c0 tx=0x7fa844005fd0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.057+0000 7fa85a6c5740 1 -- 172.21.15.64:0/4268424859 shutdown_connections 2024-09-12T03:32:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.057+0000 7fa85a6c5740 1 --2- 172.21.15.64:0/4268424859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ab29a0c0 0x55f6ab2b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.057+0000 7fa85a6c5740 1 --2- 172.21.15.64:0/4268424859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6ab28b2f0 0x55f6ab2b1e40 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.057+0000 7fa85a6c5740 1 --2- 172.21.15.64:0/4268424859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6ab28a9d0 0x55f6ab298fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.057+0000 7fa85a6c5740 1 -- 172.21.15.64:0/4268424859 >> 172.21.15.64:0/4268424859 conn(0x55f6ab287060 msgr2=0x55f6ab2988a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.057+0000 7fa85a6c5740 1 -- 172.21.15.64:0/4268424859 shutdown_connections 2024-09-12T03:32:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.057+0000 7fa85a6c5740 1 -- 172.21.15.64:0/4268424859 wait complete. 2024-09-12T03:32:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.057+0000 7fa85a6c5740 1 -- start start 2024-09-12T03:32:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.057+0000 7fa85a6c5740 1 Processor -- start 2024-09-12T03:32:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.057+0000 7fa85a6c5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6ab28a9d0 0x55f6ab2a2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.057+0000 7fa85a6c5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ab28b2f0 0x55f6ab29dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.057+0000 7fa85a6c5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6ab29a0c0 0x55f6ab29e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.057+0000 7fa85a6c5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa8501ab350 con 0x55f6ab28a9d0 2024-09-12T03:32:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.057+0000 7fa85a6c5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa8501ac7c0 con 0x55f6ab28b2f0 2024-09-12T03:32:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.057+0000 7fa85a6c5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa8501aba20 con 0x55f6ab29a0c0 2024-09-12T03:32:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.058+0000 7fa85a6c5740 10 client.0 _ll_register_callbacks cb 0x7fa84800af60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:32:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.058+0000 7fa8556c6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6ab28a9d0 0x55f6ab2a2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.058+0000 7fa8556c6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6ab28a9d0 0x55f6ab2a2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49256/0 (socket says 172.21.15.64:49256) 2024-09-12T03:32:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.058+0000 7fa8556c6640 1 -- 172.21.15.64:0/1632729933 learned_addr learned my addr 172.21.15.64:0/1632729933 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:32:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.058+0000 7fa854ec5640 1 --2- 172.21.15.64:0/1632729933 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ab28b2f0 0x55f6ab29dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.058+0000 7fa855ec7640 1 --2- 172.21.15.64:0/1632729933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6ab29a0c0 0x55f6ab29e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.058+0000 7fa8556c6640 1 -- 172.21.15.64:0/1632729933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6ab29a0c0 msgr2=0x55f6ab29e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.058+0000 7fa8556c6640 1 --2- 172.21.15.64:0/1632729933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6ab29a0c0 0x55f6ab29e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.058+0000 7fa8556c6640 1 -- 172.21.15.64:0/1632729933 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ab28b2f0 msgr2=0x55f6ab29dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.058+0000 7fa8556c6640 1 --2- 172.21.15.64:0/1632729933 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ab28b2f0 0x55f6ab29dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.058+0000 7fa8556c6640 1 -- 172.21.15.64:0/1632729933 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6ab29ea60 con 0x55f6ab28a9d0 2024-09-12T03:32:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.058+0000 7fa854ec5640 1 --2- 172.21.15.64:0/1632729933 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ab28b2f0 0x55f6ab29dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:32:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.058+0000 7fa8556c6640 1 --2- 172.21.15.64:0/1632729933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6ab28a9d0 0x55f6ab2a2920 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa84c0bc070 tx=0x7fa84c0414b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af15b239b67bb1e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.058+0000 7fa838ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:32:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.058+0000 7fa838ff9640 1 -- 172.21.15.64:0/1632729933 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c049040 con 0x55f6ab28a9d0 2024-09-12T03:32:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.058+0000 7fa838ff9640 1 -- 172.21.15.64:0/1632729933 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7fa84c0ed4e0 con 0x55f6ab28a9d0 2024-09-12T03:32:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.058+0000 7fa85a6c5740 1 -- 172.21.15.64:0/1632729933 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa844015070 con 0x55f6ab28a9d0 2024-09-12T03:32:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.058+0000 7fa838ff9640 1 -- 172.21.15.64:0/1632729933 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa84c0a2050 con 0x55f6ab28a9d0 2024-09-12T03:32:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.058+0000 7fa85a6c5740 10 client.15582 fetch_fsmap learned FSMap version 25 2024-09-12T03:32:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.059+0000 7fa85a6c5740 1 -- 172.21.15.64:0/1632729933 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f6ab314f50 con 0x55f6ab28a9d0 2024-09-12T03:32:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.059+0000 7fa838ff9640 1 -- 172.21.15.64:0/1632729933 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa844015070 con 0x55f6ab28a9d0 2024-09-12T03:32:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.059+0000 7fa838ff9640 1 -- 172.21.15.64:0/1632729933 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa84c0b8750 con 0x55f6ab28a9d0 2024-09-12T03:32:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.059+0000 7fa85a6c5740 10 client.15582 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:32:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.059+0000 7fa85a6c5740 10 client.15582 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:32:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.059+0000 7fa85a6c5740 20 client.15582 populate_metadata read hostname 'smithi064' 2024-09-12T03:32:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.059+0000 7fa85a6c5740 1 --2- 172.21.15.64:0/1632729933 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55f6ab315620 0x55f6ab317a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.059+0000 7fa85a6c5740 4 client.15582 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:32:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.059+0000 7fa85a6c5740 1 -- 172.21.15.64:0/1632729933 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f6ab1f6d70 con 0x55f6ab315620 2024-09-12T03:32:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.059+0000 7fa854ec5640 1 --2- 172.21.15.64:0/1632729933 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55f6ab315620 0x55f6ab317a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.060+0000 7fa838ff9640 10 client.15582 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:32:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.060+0000 7fa854ec5640 1 --2- 172.21.15.64:0/1632729933 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55f6ab315620 0x55f6ab317a10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9add517004d18e3 server_cookie=1fc17b72b019ea78 in_seq=0 out_seq=0 2024-09-12T03:32:09.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.073+0000 7fa838ff9640 1 -- 172.21.15.64:0/1632729933 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f6ab1f6d70 con 0x55f6ab315620 2024-09-12T03:32:09.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.073+0000 7fa838ff9640 10 client.15582 handle_command_reply: tid=0 2024-09-12T03:32:09.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.073+0000 7fa85a6c5740 1 client.15582 shutdown 2024-09-12T03:32:09.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.073+0000 7fa85a6c5740 1 -- 172.21.15.64:0/1632729933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6ab28a9d0 msgr2=0x55f6ab2a2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:09.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.073+0000 7fa85a6c5740 1 --2- 172.21.15.64:0/1632729933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6ab28a9d0 0x55f6ab2a2920 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa84c0bc070 tx=0x7fa84c0414b0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.073+0000 7fa85a6c5740 1 -- 172.21.15.64:0/1632729933 shutdown_connections 2024-09-12T03:32:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.073+0000 7fa85a6c5740 1 --2- 172.21.15.64:0/1632729933 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55f6ab315620 0x55f6ab317a10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.073+0000 7fa85a6c5740 1 --2- 172.21.15.64:0/1632729933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6ab29a0c0 0x55f6ab29e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.073+0000 7fa85a6c5740 1 --2- 172.21.15.64:0/1632729933 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f6ab28b2f0 0x55f6ab29dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.073+0000 7fa85a6c5740 1 --2- 172.21.15.64:0/1632729933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f6ab28a9d0 0x55f6ab2a2920 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.073+0000 7fa838ff9640 0 client.15582 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:32:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.073+0000 7fa85a6c5740 1 -- 172.21.15.64:0/1632729933 >> 172.21.15.64:0/1632729933 conn(0x55f6ab287060 msgr2=0x55f6ab297820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.074+0000 7fa85a6c5740 1 -- 172.21.15.64:0/1632729933 shutdown_connections 2024-09-12T03:32:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.074+0000 7fa85a6c5740 1 -- 172.21.15.64:0/1632729933 wait complete. 2024-09-12T03:32:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.074+0000 7fa85a6c5740 20 client.15582 trim_cache size 0 max 16384 2024-09-12T03:32:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:32:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:32:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:32:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:32:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.074+0000 7fa85792a640 1 -- 172.21.15.64:0/316431042 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fa83c0a2e20 msgr2=0x7fa83c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.074+0000 7fa85792a640 1 --2- 172.21.15.64:0/316431042 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fa83c0a2e20 0x7fa83c0a52e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa848005cc0 tx=0x7fa84800ad50 comp rx=0 tx=0).stop 2024-09-12T03:32:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.074+0000 7fa85792a640 1 -- 172.21.15.64:0/316431042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8501009b0 msgr2=0x7fa8501982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.074+0000 7fa85792a640 1 --2- 172.21.15.64:0/316431042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8501009b0 0x7fa8501982e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa84c005ab0 tx=0x7fa84c003cd0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.074+0000 7fa85792a640 1 -- 172.21.15.64:0/316431042 shutdown_connections 2024-09-12T03:32:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.074+0000 7fa85792a640 1 --2- 172.21.15.64:0/316431042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa8501069c0 0x7fa85019cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.074+0000 7fa85792a640 1 --2- 172.21.15.64:0/316431042 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fa83c0a2e20 0x7fa83c0a52e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.074+0000 7fa85792a640 1 --2- 172.21.15.64:0/316431042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa850101350 0x7fa850198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.074+0000 7fa85792a640 1 --2- 172.21.15.64:0/316431042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8501009b0 0x7fa8501982e0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.074+0000 7fa85792a640 1 -- 172.21.15.64:0/316431042 >> 172.21.15.64:0/316431042 conn(0x7fa850077990 msgr2=0x7fa850069b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:09.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.074+0000 7fa85792a640 1 -- 172.21.15.64:0/316431042 shutdown_connections 2024-09-12T03:32:09.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:09.074+0000 7fa85792a640 1 -- 172.21.15.64:0/316431042 wait complete. 2024-09-12T03:32:09.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:32:09 smithi115 ceph-mon[34801]: pgmap v89: 129 pgs: 129 active+clean; 3.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 54 KiB/s wr, 33 op/s 2024-09-12T03:32:09.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:32:09 smithi195 ceph-mon[34786]: pgmap v89: 129 pgs: 129 active+clean; 3.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 54 KiB/s wr, 33 op/s 2024-09-12T03:32:09.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:32:09 smithi064 ceph-mon[30952]: pgmap v89: 129 pgs: 129 active+clean; 3.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 54 KiB/s wr, 33 op/s 2024-09-12T03:32:11.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:32:11 smithi195 ceph-mon[34786]: pgmap v90: 129 pgs: 129 active+clean; 3.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 42 KiB/s wr, 25 op/s 2024-09-12T03:32:11.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:32:11 smithi064 ceph-mon[30952]: pgmap v90: 129 pgs: 129 active+clean; 3.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 42 KiB/s wr, 25 op/s 2024-09-12T03:32:11.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:32:11 smithi115 ceph-mon[34801]: pgmap v90: 129 pgs: 129 active+clean; 3.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 42 KiB/s wr, 25 op/s 2024-09-12T03:32:13.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:32:13 smithi195 ceph-mon[34786]: pgmap v91: 129 pgs: 129 active+clean; 3.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 57 KiB/s wr, 33 op/s 2024-09-12T03:32:13.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:32:13 smithi064 ceph-mon[30952]: pgmap v91: 129 pgs: 129 active+clean; 3.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 57 KiB/s wr, 33 op/s 2024-09-12T03:32:13.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:32:13 smithi115 ceph-mon[34801]: pgmap v91: 129 pgs: 129 active+clean; 3.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 57 KiB/s wr, 33 op/s 2024-09-12T03:32:15.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:32:15 smithi195 ceph-mon[34786]: pgmap v92: 129 pgs: 129 active+clean; 3.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-09-12T03:32:15.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:32:15 smithi064 ceph-mon[30952]: pgmap v92: 129 pgs: 129 active+clean; 3.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-09-12T03:32:15.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:32:15 smithi115 ceph-mon[34801]: pgmap v92: 129 pgs: 129 active+clean; 3.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-09-12T03:32:17.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:32:17 smithi195 ceph-mon[34786]: pgmap v93: 129 pgs: 129 active+clean; 3.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-09-12T03:32:17.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:32:17 smithi064 ceph-mon[30952]: pgmap v93: 129 pgs: 129 active+clean; 3.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-09-12T03:32:17.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:32:17 smithi115 ceph-mon[34801]: pgmap v93: 129 pgs: 129 active+clean; 3.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-09-12T03:32:19.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:32:19 smithi195 ceph-mon[34786]: pgmap v94: 129 pgs: 129 active+clean; 3.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-09-12T03:32:19.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:32:19 smithi064 ceph-mon[30952]: pgmap v94: 129 pgs: 129 active+clean; 3.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-09-12T03:32:19.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:32:19 smithi115 ceph-mon[34801]: pgmap v94: 129 pgs: 129 active+clean; 3.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-09-12T03:32:21.694 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:32:21 smithi115 ceph-mon[34801]: pgmap v95: 129 pgs: 129 active+clean; 3.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-09-12T03:32:21.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:32:21 smithi195 ceph-mon[34786]: pgmap v95: 129 pgs: 129 active+clean; 3.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-09-12T03:32:21.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:32:21 smithi064 ceph-mon[30952]: pgmap v95: 129 pgs: 129 active+clean; 3.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-09-12T03:32:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.164+0000 7fb299d31640 1 -- 172.21.15.64:0/1523166919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2941009b0 msgr2=0x7fb294100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.164+0000 7fb299d31640 1 --2- 172.21.15.64:0/1523166919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2941009b0 0x7fb294100e10 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb288009930 tx=0x7fb28802a420 comp rx=0 tx=0).stop 2024-09-12T03:32:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.164+0000 7fb299d31640 1 -- 172.21.15.64:0/1523166919 shutdown_connections 2024-09-12T03:32:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.164+0000 7fb299d31640 1 --2- 172.21.15.64:0/1523166919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb294101350 0x7fb29410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.164+0000 7fb299d31640 1 --2- 172.21.15.64:0/1523166919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2941009b0 0x7fb294100e10 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.164+0000 7fb299d31640 1 --2- 172.21.15.64:0/1523166919 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2941069c0 0x7fb294106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.164+0000 7fb299d31640 1 -- 172.21.15.64:0/1523166919 >> 172.21.15.64:0/1523166919 conn(0x7fb294077990 msgr2=0x7fb2940fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.164+0000 7fb299d31640 1 -- 172.21.15.64:0/1523166919 shutdown_connections 2024-09-12T03:32:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.164+0000 7fb299d31640 1 -- 172.21.15.64:0/1523166919 wait complete. 2024-09-12T03:32:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.165+0000 7fb299d31640 1 Processor -- start 2024-09-12T03:32:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.165+0000 7fb299d31640 1 -- start start 2024-09-12T03:32:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.165+0000 7fb299d31640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2941009b0 0x7fb2940fcde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.165+0000 7fb299d31640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb294101350 0x7fb2940fd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.165+0000 7fb299d31640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2941069c0 0x7fb2940fd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.165+0000 7fb299d31640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb294110c70 con 0x7fb294101350 2024-09-12T03:32:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.165+0000 7fb299d31640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb294110af0 con 0x7fb2941069c0 2024-09-12T03:32:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.165+0000 7fb299d31640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb294110df0 con 0x7fb2941009b0 2024-09-12T03:32:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.165+0000 7fb292ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb294101350 0x7fb2940fd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.165+0000 7fb292ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb294101350 0x7fb2940fd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58800/0 (socket says 172.21.15.64:58800) 2024-09-12T03:32:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.165+0000 7fb292ffd640 1 -- 172.21.15.64:0/2311981840 learned_addr learned my addr 172.21.15.64:0/2311981840 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:32:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.165+0000 7fb2937fe640 1 --2- 172.21.15.64:0/2311981840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2941009b0 0x7fb2940fcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.165+0000 7fb293fff640 1 --2- 172.21.15.64:0/2311981840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2941069c0 0x7fb2940fd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.166+0000 7fb292ffd640 1 -- 172.21.15.64:0/2311981840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2941009b0 msgr2=0x7fb2940fcde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.166+0000 7fb292ffd640 1 --2- 172.21.15.64:0/2311981840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2941009b0 0x7fb2940fcde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.166+0000 7fb292ffd640 1 -- 172.21.15.64:0/2311981840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2941069c0 msgr2=0x7fb2940fd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.166+0000 7fb292ffd640 1 --2- 172.21.15.64:0/2311981840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2941069c0 0x7fb2940fd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.166+0000 7fb292ffd640 1 -- 172.21.15.64:0/2311981840 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2941a67c0 con 0x7fb294101350 2024-09-12T03:32:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.166+0000 7fb292ffd640 1 --2- 172.21.15.64:0/2311981840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb294101350 0x7fb2940fd320 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb288005940 tx=0x7fb2880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.166+0000 7fb290ff9640 1 -- 172.21.15.64:0/2311981840 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28803f030 con 0x7fb294101350 2024-09-12T03:32:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.166+0000 7fb290ff9640 1 -- 172.21.15.64:0/2311981840 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2880318a0 con 0x7fb294101350 2024-09-12T03:32:22.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.166+0000 7fb299d31640 1 -- 172.21.15.64:0/2311981840 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2941a6a50 con 0x7fb294101350 2024-09-12T03:32:22.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.166+0000 7fb290ff9640 1 -- 172.21.15.64:0/2311981840 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288035070 con 0x7fb294101350 2024-09-12T03:32:22.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.166+0000 7fb299d31640 1 -- 172.21.15.64:0/2311981840 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2941a6f90 con 0x7fb294101350 2024-09-12T03:32:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.168+0000 7fb290ff9640 1 -- 172.21.15.64:0/2311981840 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fb288042030 con 0x7fb294101350 2024-09-12T03:32:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.169+0000 7fb29cacd740 1 Processor -- start 2024-09-12T03:32:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.169+0000 7fb29cacd740 1 -- start start 2024-09-12T03:32:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.169+0000 7fb29cacd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56231f139af0 0x56231f139ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.169+0000 7fb29cacd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231f13a410 0x56231f144770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.169+0000 7fb29cacd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231f144cb0 0x56231f1470a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.169+0000 7fb29cacd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56231f09e2f0 con 0x56231f139af0 2024-09-12T03:32:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.169+0000 7fb29cacd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56231f046270 con 0x56231f144cb0 2024-09-12T03:32:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.169+0000 7fb29cacd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56231f09a2b0 con 0x56231f13a410 2024-09-12T03:32:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.170+0000 7fb293fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56231f139af0 0x56231f139ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.170+0000 7fb290ff9640 1 --2- 172.21.15.64:0/2311981840 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fb27c0a2e20 0x7fb27c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.170+0000 7fb293fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56231f139af0 0x56231f139ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58808/0 (socket says 172.21.15.64:58808) 2024-09-12T03:32:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.170+0000 7fb293fff640 1 -- 172.21.15.64:0/725789671 learned_addr learned my addr 172.21.15.64:0/725789671 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:32:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.170+0000 7fb292ffd640 1 --2- 172.21.15.64:0/725789671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231f144cb0 0x56231f1470a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.170+0000 7fb2937fe640 1 --2- 172.21.15.64:0/725789671 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231f13a410 0x56231f144770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.170+0000 7fb293fff640 1 --2- 172.21.15.64:0/2311981840 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fb27c0a2e20 0x7fb27c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.170+0000 7fb293fff640 1 -- 172.21.15.64:0/725789671 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231f13a410 msgr2=0x56231f144770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.170+0000 7fb290ff9640 1 -- 172.21.15.64:0/2311981840 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7fb2880eff00 con 0x7fb294101350 2024-09-12T03:32:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.170+0000 7fb293fff640 1 --2- 172.21.15.64:0/725789671 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231f13a410 0x56231f144770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.170+0000 7fb293fff640 1 -- 172.21.15.64:0/725789671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231f144cb0 msgr2=0x56231f1470a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.170+0000 7fb293fff640 1 --2- 172.21.15.64:0/725789671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231f144cb0 0x56231f1470a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.170+0000 7fb293fff640 1 -- 172.21.15.64:0/725789671 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56231f045340 con 0x56231f139af0 2024-09-12T03:32:22.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.170+0000 7fb293fff640 1 --2- 172.21.15.64:0/725789671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56231f139af0 0x56231f139ed0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb28c005c40 tx=0x7fb28c00f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27b812ba2ccb43ce server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:22.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.170+0000 7fb279ffb640 1 -- 172.21.15.64:0/725789671 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c022020 con 0x56231f139af0 2024-09-12T03:32:22.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.170+0000 7fb279ffb640 1 -- 172.21.15.64:0/725789671 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb28c013170 con 0x56231f139af0 2024-09-12T03:32:22.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.170+0000 7fb279ffb640 1 -- 172.21.15.64:0/725789671 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c010690 con 0x56231f139af0 2024-09-12T03:32:22.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.171+0000 7fb29cacd740 1 -- 172.21.15.64:0/725789671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56231f139af0 msgr2=0x56231f139ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:22.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.171+0000 7fb293fff640 1 --2- 172.21.15.64:0/2311981840 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fb27c0a2e20 0x7fb27c0a52e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb28c023000 tx=0x7fb28c021000 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:22.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.171+0000 7fb29cacd740 1 --2- 172.21.15.64:0/725789671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56231f139af0 0x56231f139ed0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb28c005c40 tx=0x7fb28c00f3c0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.171+0000 7fb29cacd740 1 -- 172.21.15.64:0/725789671 shutdown_connections 2024-09-12T03:32:22.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.171+0000 7fb29cacd740 1 --2- 172.21.15.64:0/725789671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231f144cb0 0x56231f1470a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.171+0000 7fb29cacd740 1 --2- 172.21.15.64:0/725789671 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231f13a410 0x56231f144770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.171+0000 7fb29cacd740 1 --2- 172.21.15.64:0/725789671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56231f139af0 0x56231f139ed0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.171+0000 7fb29cacd740 1 -- 172.21.15.64:0/725789671 >> 172.21.15.64:0/725789671 conn(0x56231f136060 msgr2=0x56231f136440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:22.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.171+0000 7fb29cacd740 1 -- 172.21.15.64:0/725789671 shutdown_connections 2024-09-12T03:32:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.171+0000 7fb29cacd740 1 -- 172.21.15.64:0/725789671 wait complete. 2024-09-12T03:32:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.171+0000 7fb29cacd740 1 -- start start 2024-09-12T03:32:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.172+0000 7fb29cacd740 1 Processor -- start 2024-09-12T03:32:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.172+0000 7fb29cacd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231f139af0 0x56231f141bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.172+0000 7fb29cacd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56231f13a410 0x56231f1420f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.172+0000 7fb29cacd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231f144cb0 0x56231f142630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.172+0000 7fb29cacd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56231f147880 con 0x56231f13a410 2024-09-12T03:32:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.172+0000 7fb29cacd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56231f027290 con 0x56231f144cb0 2024-09-12T03:32:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.172+0000 7fb29cacd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56231f147a00 con 0x56231f139af0 2024-09-12T03:32:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.172+0000 7fb29cacd740 10 client.0 _ll_register_callbacks cb 0x56231f058140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:32:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.172+0000 7fb292ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56231f13a410 0x56231f1420f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.172+0000 7fb292ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56231f13a410 0x56231f1420f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58820/0 (socket says 172.21.15.64:58820) 2024-09-12T03:32:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.172+0000 7fb292ffd640 1 -- 172.21.15.64:0/3009342423 learned_addr learned my addr 172.21.15.64:0/3009342423 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:32:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.172+0000 7fb293fff640 1 --2- 172.21.15.64:0/3009342423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231f144cb0 0x56231f142630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.172+0000 7fb2937fe640 1 --2- 172.21.15.64:0/3009342423 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231f139af0 0x56231f141bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.172+0000 7fb292ffd640 1 -- 172.21.15.64:0/3009342423 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231f139af0 msgr2=0x56231f141bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.172+0000 7fb292ffd640 1 --2- 172.21.15.64:0/3009342423 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231f139af0 0x56231f141bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.172+0000 7fb292ffd640 1 -- 172.21.15.64:0/3009342423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231f144cb0 msgr2=0x56231f142630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.172+0000 7fb292ffd640 1 --2- 172.21.15.64:0/3009342423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231f144cb0 0x56231f142630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.172+0000 7fb292ffd640 1 -- 172.21.15.64:0/3009342423 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56231edb1210 con 0x56231f13a410 2024-09-12T03:32:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.172+0000 7fb293fff640 1 --2- 172.21.15.64:0/3009342423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231f144cb0 0x56231f142630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:32:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.172+0000 7fb292ffd640 1 --2- 172.21.15.64:0/3009342423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56231f13a410 0x56231f1420f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb2880b8500 tx=0x7fb28808bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7130dabeb81072ea server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.172+0000 7fb272ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:32:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.173+0000 7fb272ffd640 1 -- 172.21.15.64:0/3009342423 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2880dc0e0 con 0x56231f13a410 2024-09-12T03:32:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.173+0000 7fb272ffd640 1 -- 172.21.15.64:0/3009342423 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7fb2880bc300 con 0x56231f13a410 2024-09-12T03:32:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.173+0000 7fb29cacd740 1 -- 172.21.15.64:0/3009342423 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb28c013170 con 0x56231f13a410 2024-09-12T03:32:22.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.173+0000 7fb272ffd640 1 -- 172.21.15.64:0/3009342423 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb2880ed6a0 con 0x56231f13a410 2024-09-12T03:32:22.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.173+0000 7fb29cacd740 10 client.15594 fetch_fsmap learned FSMap version 25 2024-09-12T03:32:22.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.173+0000 7fb29cacd740 1 -- 172.21.15.64:0/3009342423 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56231ee7d620 con 0x56231f13a410 2024-09-12T03:32:22.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.173+0000 7fb272ffd640 1 -- 172.21.15.64:0/3009342423 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb28c013170 con 0x56231f13a410 2024-09-12T03:32:22.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.173+0000 7fb272ffd640 1 -- 172.21.15.64:0/3009342423 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb2880c47d0 con 0x56231f13a410 2024-09-12T03:32:22.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.173+0000 7fb29cacd740 10 client.15594 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:32:22.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.173+0000 7fb29cacd740 10 client.15594 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:32:22.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.173+0000 7fb29cacd740 20 client.15594 populate_metadata read hostname 'smithi064' 2024-09-12T03:32:22.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.173+0000 7fb29cacd740 1 --2- 172.21.15.64:0/3009342423 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x56231f154c60 0x56231f157050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.173+0000 7fb29cacd740 4 client.15594 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:32:22.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.173+0000 7fb29cacd740 1 -- 172.21.15.64:0/3009342423 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56231f0aba20 con 0x56231f154c60 2024-09-12T03:32:22.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.173+0000 7fb2937fe640 1 --2- 172.21.15.64:0/3009342423 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x56231f154c60 0x56231f157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.174+0000 7fb2937fe640 1 --2- 172.21.15.64:0/3009342423 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x56231f154c60 0x56231f157050 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13aeeaa3b2a21540 server_cookie=d1826cf7cd79cf15 in_seq=0 out_seq=0 2024-09-12T03:32:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.174+0000 7fb272ffd640 10 client.15594 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:32:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.174+0000 7fb272ffd640 1 -- 172.21.15.64:0/3009342423 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56231f0aba20 con 0x56231f154c60 2024-09-12T03:32:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.174+0000 7fb272ffd640 10 client.15594 handle_command_reply: tid=0 2024-09-12T03:32:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.174+0000 7fb29cacd740 1 client.15594 shutdown 2024-09-12T03:32:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.175+0000 7fb29cacd740 1 -- 172.21.15.64:0/3009342423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56231f13a410 msgr2=0x56231f1420f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.175+0000 7fb29cacd740 1 --2- 172.21.15.64:0/3009342423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56231f13a410 0x56231f1420f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb2880b8500 tx=0x7fb28808bf00 comp rx=0 tx=0).stop 2024-09-12T03:32:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.175+0000 7fb29cacd740 1 -- 172.21.15.64:0/3009342423 shutdown_connections 2024-09-12T03:32:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.175+0000 7fb29cacd740 1 --2- 172.21.15.64:0/3009342423 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x56231f154c60 0x56231f157050 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.175+0000 7fb29cacd740 1 --2- 172.21.15.64:0/3009342423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231f144cb0 0x56231f142630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.175+0000 7fb29cacd740 1 --2- 172.21.15.64:0/3009342423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56231f13a410 0x56231f1420f0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.175+0000 7fb272ffd640 0 client.15594 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:32:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.175+0000 7fb29cacd740 1 --2- 172.21.15.64:0/3009342423 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231f139af0 0x56231f141bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.175+0000 7fb29cacd740 1 -- 172.21.15.64:0/3009342423 >> 172.21.15.64:0/3009342423 conn(0x56231f136060 msgr2=0x56231f13bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.175+0000 7fb29cacd740 1 -- 172.21.15.64:0/3009342423 shutdown_connections 2024-09-12T03:32:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.175+0000 7fb29cacd740 1 -- 172.21.15.64:0/3009342423 wait complete. 2024-09-12T03:32:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.175+0000 7fb29cacd740 20 client.15594 trim_cache size 0 max 16384 2024-09-12T03:32:22.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.189+0000 7fb29cacd740 1 Processor -- start 2024-09-12T03:32:22.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.189+0000 7fb29cacd740 1 -- start start 2024-09-12T03:32:22.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.189+0000 7fb29cacd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231f139af0 0x56231f142460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.189+0000 7fb29cacd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231f13a410 0x56231f15c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.189+0000 7fb29cacd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56231f144cb0 0x56231f15cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.189+0000 7fb29cacd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb2941b3ba0 con 0x56231f144cb0 2024-09-12T03:32:22.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.189+0000 7fb29cacd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb2941b1470 con 0x56231f139af0 2024-09-12T03:32:22.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.189+0000 7fb29cacd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb2941af2c0 con 0x56231f13a410 2024-09-12T03:32:22.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.190+0000 7fb293fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56231f144cb0 0x56231f15cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.190+0000 7fb293fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56231f144cb0 0x56231f15cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58824/0 (socket says 172.21.15.64:58824) 2024-09-12T03:32:22.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.190+0000 7fb293fff640 1 -- 172.21.15.64:0/3005581427 learned_addr learned my addr 172.21.15.64:0/3005581427 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:32:22.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.190+0000 7fb2937fe640 1 --2- 172.21.15.64:0/3005581427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231f139af0 0x56231f142460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.190+0000 7fb292ffd640 1 --2- 172.21.15.64:0/3005581427 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231f13a410 0x56231f15c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.190+0000 7fb293fff640 1 -- 172.21.15.64:0/3005581427 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231f13a410 msgr2=0x56231f15c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:22.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.190+0000 7fb293fff640 1 --2- 172.21.15.64:0/3005581427 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231f13a410 0x56231f15c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.190+0000 7fb293fff640 1 -- 172.21.15.64:0/3005581427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231f139af0 msgr2=0x56231f142460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:22.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.190+0000 7fb293fff640 1 --2- 172.21.15.64:0/3005581427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231f139af0 0x56231f142460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.190+0000 7fb293fff640 1 -- 172.21.15.64:0/3005581427 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56231f1576d0 con 0x56231f144cb0 2024-09-12T03:32:22.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.190+0000 7fb292ffd640 1 --2- 172.21.15.64:0/3005581427 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231f13a410 0x56231f15c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:32:22.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.190+0000 7fb293fff640 1 --2- 172.21.15.64:0/3005581427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56231f144cb0 0x56231f15cea0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb28c00f8c0 tx=0x7fb28c009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f88e7868c0f1f27 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:22.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.190+0000 7fb2797fa640 1 -- 172.21.15.64:0/3005581427 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c022070 con 0x56231f144cb0 2024-09-12T03:32:22.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.190+0000 7fb2797fa640 1 -- 172.21.15.64:0/3005581427 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb28c013170 con 0x56231f144cb0 2024-09-12T03:32:22.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.190+0000 7fb2797fa640 1 -- 172.21.15.64:0/3005581427 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c010690 con 0x56231f144cb0 2024-09-12T03:32:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.191+0000 7fb29cacd740 1 -- 172.21.15.64:0/3005581427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56231f144cb0 msgr2=0x56231f15cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.191+0000 7fb29cacd740 1 --2- 172.21.15.64:0/3005581427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56231f144cb0 0x56231f15cea0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb28c00f8c0 tx=0x7fb28c009200 comp rx=0 tx=0).stop 2024-09-12T03:32:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.191+0000 7fb29cacd740 1 -- 172.21.15.64:0/3005581427 shutdown_connections 2024-09-12T03:32:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.191+0000 7fb29cacd740 1 --2- 172.21.15.64:0/3005581427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56231f144cb0 0x56231f15cea0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.191+0000 7fb29cacd740 1 --2- 172.21.15.64:0/3005581427 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231f13a410 0x56231f15c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.191+0000 7fb29cacd740 1 --2- 172.21.15.64:0/3005581427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231f139af0 0x56231f142460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.191+0000 7fb29cacd740 1 -- 172.21.15.64:0/3005581427 >> 172.21.15.64:0/3005581427 conn(0x56231f136060 msgr2=0x56231f143c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.191+0000 7fb29cacd740 1 -- 172.21.15.64:0/3005581427 shutdown_connections 2024-09-12T03:32:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.191+0000 7fb29cacd740 1 -- 172.21.15.64:0/3005581427 wait complete. 2024-09-12T03:32:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.191+0000 7fb29cacd740 1 -- start start 2024-09-12T03:32:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.191+0000 7fb29cacd740 1 Processor -- start 2024-09-12T03:32:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.192+0000 7fb29cacd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231f139af0 0x56231f14d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.192+0000 7fb29cacd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56231f13a410 0x56231f13e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.192+0000 7fb29cacd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231f144cb0 0x56231f13ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.192+0000 7fb29cacd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb2941af930 con 0x56231f13a410 2024-09-12T03:32:22.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.192+0000 7fb29cacd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb2941b0da0 con 0x56231f139af0 2024-09-12T03:32:22.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.192+0000 7fb29cacd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb2941b0000 con 0x56231f144cb0 2024-09-12T03:32:22.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.192+0000 7fb29cacd740 10 client.0 _ll_register_callbacks cb 0x7fb28804bfa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:32:22.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.192+0000 7fb292ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56231f13a410 0x56231f13e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.192+0000 7fb292ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56231f13a410 0x56231f13e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58834/0 (socket says 172.21.15.64:58834) 2024-09-12T03:32:22.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.192+0000 7fb292ffd640 1 -- 172.21.15.64:0/424477860 learned_addr learned my addr 172.21.15.64:0/424477860 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:32:22.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.192+0000 7fb2937fe640 1 --2- 172.21.15.64:0/424477860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231f139af0 0x56231f14d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.192+0000 7fb293fff640 1 --2- 172.21.15.64:0/424477860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231f144cb0 0x56231f13ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.192+0000 7fb292ffd640 1 -- 172.21.15.64:0/424477860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231f144cb0 msgr2=0x56231f13ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:22.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.192+0000 7fb292ffd640 1 --2- 172.21.15.64:0/424477860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231f144cb0 0x56231f13ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.192+0000 7fb292ffd640 1 -- 172.21.15.64:0/424477860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231f139af0 msgr2=0x56231f14d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.192+0000 7fb292ffd640 1 --2- 172.21.15.64:0/424477860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231f139af0 0x56231f14d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.192+0000 7fb292ffd640 1 -- 172.21.15.64:0/424477860 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56231f13f6c0 con 0x56231f13a410 2024-09-12T03:32:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.192+0000 7fb292ffd640 1 --2- 172.21.15.64:0/424477860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56231f13a410 0x56231f13e950 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb288004650 tx=0x7fb2880bbc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dc20063e047a915 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.192+0000 7fb272ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:32:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.192+0000 7fb272ffd640 1 -- 172.21.15.64:0/424477860 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288049040 con 0x56231f13a410 2024-09-12T03:32:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.192+0000 7fb29cacd740 1 -- 172.21.15.64:0/424477860 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb28c013170 con 0x56231f13a410 2024-09-12T03:32:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.192+0000 7fb272ffd640 1 -- 172.21.15.64:0/424477860 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7fb2880bc500 con 0x56231f13a410 2024-09-12T03:32:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.193+0000 7fb272ffd640 1 -- 172.21.15.64:0/424477860 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb2880eb050 con 0x56231f13a410 2024-09-12T03:32:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.193+0000 7fb29cacd740 10 client.15600 fetch_fsmap learned FSMap version 25 2024-09-12T03:32:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.193+0000 7fb29cacd740 1 -- 172.21.15.64:0/424477860 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56231f1bf910 con 0x56231f13a410 2024-09-12T03:32:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.193+0000 7fb272ffd640 1 -- 172.21.15.64:0/424477860 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb28c013170 con 0x56231f13a410 2024-09-12T03:32:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.193+0000 7fb272ffd640 1 -- 172.21.15.64:0/424477860 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb2880d8070 con 0x56231f13a410 2024-09-12T03:32:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.193+0000 7fb29cacd740 10 client.15600 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:32:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.193+0000 7fb29cacd740 10 client.15600 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:32:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.193+0000 7fb29cacd740 20 client.15600 populate_metadata read hostname 'smithi064' 2024-09-12T03:32:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.193+0000 7fb29cacd740 1 --2- 172.21.15.64:0/424477860 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x56231f1bfe40 0x56231f1c2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.193+0000 7fb29cacd740 4 client.15600 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:32:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.193+0000 7fb29cacd740 1 -- 172.21.15.64:0/424477860 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56231f0a5d70 con 0x56231f1bfe40 2024-09-12T03:32:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.193+0000 7fb2937fe640 1 --2- 172.21.15.64:0/424477860 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x56231f1bfe40 0x56231f1c2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.193+0000 7fb272ffd640 10 client.15600 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:32:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.193+0000 7fb2937fe640 1 --2- 172.21.15.64:0/424477860 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x56231f1bfe40 0x56231f1c2230 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6a1b6a8cb3ab480 server_cookie=5ff1d3937b2d0c26 in_seq=0 out_seq=0 2024-09-12T03:32:22.206 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.205+0000 7fb272ffd640 1 -- 172.21.15.64:0/424477860 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56231f0a5d70 con 0x56231f1bfe40 2024-09-12T03:32:22.206 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.205+0000 7fb272ffd640 10 client.15600 handle_command_reply: tid=0 2024-09-12T03:32:22.206 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.205+0000 7fb29cacd740 1 client.15600 shutdown 2024-09-12T03:32:22.207 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.205+0000 7fb29cacd740 1 -- 172.21.15.64:0/424477860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56231f13a410 msgr2=0x56231f13e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:22.207 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.205+0000 7fb29cacd740 1 --2- 172.21.15.64:0/424477860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56231f13a410 0x56231f13e950 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb288004650 tx=0x7fb2880bbc60 comp rx=0 tx=0).stop 2024-09-12T03:32:22.207 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.205+0000 7fb29cacd740 1 -- 172.21.15.64:0/424477860 shutdown_connections 2024-09-12T03:32:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.205+0000 7fb29cacd740 1 --2- 172.21.15.64:0/424477860 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x56231f1bfe40 0x56231f1c2230 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.206+0000 7fb29cacd740 1 --2- 172.21.15.64:0/424477860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231f144cb0 0x56231f13ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.206+0000 7fb29cacd740 1 --2- 172.21.15.64:0/424477860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56231f13a410 0x56231f13e950 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.206+0000 7fb272ffd640 0 client.15600 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:32:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.206+0000 7fb29cacd740 1 --2- 172.21.15.64:0/424477860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231f139af0 0x56231f14d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.206+0000 7fb29cacd740 1 -- 172.21.15.64:0/424477860 >> 172.21.15.64:0/424477860 conn(0x56231f136060 msgr2=0x56231f1349a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.206+0000 7fb29cacd740 1 -- 172.21.15.64:0/424477860 shutdown_connections 2024-09-12T03:32:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.206+0000 7fb29cacd740 1 -- 172.21.15.64:0/424477860 wait complete. 2024-09-12T03:32:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.206+0000 7fb29cacd740 20 client.15600 trim_cache size 0 max 16384 2024-09-12T03:32:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:32:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:32:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:32:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:32:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.206+0000 7fb299d31640 1 -- 172.21.15.64:0/2311981840 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fb27c0a2e20 msgr2=0x7fb27c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.206+0000 7fb299d31640 1 --2- 172.21.15.64:0/2311981840 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fb27c0a2e20 0x7fb27c0a52e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb28c023000 tx=0x7fb28c021000 comp rx=0 tx=0).stop 2024-09-12T03:32:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.206+0000 7fb299d31640 1 -- 172.21.15.64:0/2311981840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb294101350 msgr2=0x7fb2940fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.206+0000 7fb299d31640 1 --2- 172.21.15.64:0/2311981840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb294101350 0x7fb2940fd320 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb288005940 tx=0x7fb2880042f0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.206+0000 7fb299d31640 1 -- 172.21.15.64:0/2311981840 shutdown_connections 2024-09-12T03:32:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.206+0000 7fb299d31640 1 --2- 172.21.15.64:0/2311981840 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fb27c0a2e20 0x7fb27c0a52e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.206+0000 7fb299d31640 1 --2- 172.21.15.64:0/2311981840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2941069c0 0x7fb2940fd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.206+0000 7fb299d31640 1 --2- 172.21.15.64:0/2311981840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb294101350 0x7fb2940fd320 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.206+0000 7fb299d31640 1 --2- 172.21.15.64:0/2311981840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2941009b0 0x7fb2940fcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.206+0000 7fb299d31640 1 -- 172.21.15.64:0/2311981840 >> 172.21.15.64:0/2311981840 conn(0x7fb294077990 msgr2=0x7fb294069a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.206+0000 7fb299d31640 1 -- 172.21.15.64:0/2311981840 shutdown_connections 2024-09-12T03:32:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:22.206+0000 7fb299d31640 1 -- 172.21.15.64:0/2311981840 wait complete. 2024-09-12T03:32:22.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:32:22 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:32:22.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:32:22 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:32:22.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:32:22 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:32:22.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:32:22 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:32:22.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:32:22 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:32:22.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:32:22 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:32:22.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:32:22 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:32:22.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:32:22 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:32:22.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:32:22 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:32:22.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:32:22 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:32:22.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:32:22 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:32:22.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:32:22 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:32:23.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:32:23 smithi195 ceph-mon[34786]: pgmap v96: 129 pgs: 129 active+clean; 3.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-09-12T03:32:23.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:32:23 smithi064 ceph-mon[30952]: pgmap v96: 129 pgs: 129 active+clean; 3.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-09-12T03:32:23.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:32:23 smithi115 ceph-mon[34801]: pgmap v96: 129 pgs: 129 active+clean; 3.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-09-12T03:32:25.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:32:25 smithi195 ceph-mon[34786]: pgmap v97: 129 pgs: 129 active+clean; 3.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-09-12T03:32:25.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:32:25 smithi064 ceph-mon[30952]: pgmap v97: 129 pgs: 129 active+clean; 3.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-09-12T03:32:25.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:32:25 smithi115 ceph-mon[34801]: pgmap v97: 129 pgs: 129 active+clean; 3.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-09-12T03:32:25.996 INFO:tasks.workunit.client.0.smithi064.stdout:64/600: open fd = 3 2024-09-12T03:32:25.996 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:25.996 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:25.996 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:25.997 INFO:tasks.workunit.client.0.smithi064.stdout:65/600: open fd = 3 2024-09-12T03:32:25.997 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:25.997 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:25.997 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:25.997 INFO:tasks.workunit.client.0.smithi064.stdout:66/600: open fd = 3 2024-09-12T03:32:25.997 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:25.997 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:25.997 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:25.997 INFO:tasks.workunit.client.0.smithi064.stdout:67/600: open fd = 3 2024-09-12T03:32:25.997 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:25.997 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:25.997 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:25.997 INFO:tasks.workunit.client.0.smithi064.stdout:68/600: open fd = 3 2024-09-12T03:32:25.998 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:25.998 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:25.998 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:25.998 INFO:tasks.workunit.client.0.smithi064.stdout:69/600: open fd = 3 2024-09-12T03:32:25.998 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:25.998 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:25.998 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:25.998 INFO:tasks.workunit.client.0.smithi064.stdout:70/600: open fd = 3 2024-09-12T03:32:25.998 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:25.998 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:25.998 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:25.998 INFO:tasks.workunit.client.0.smithi064.stdout:71/600: open fd = 3 2024-09-12T03:32:25.998 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:25.998 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:25.999 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:25.999 INFO:tasks.workunit.client.0.smithi064.stdout:72/600: open fd = 3 2024-09-12T03:32:25.999 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:25.999 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:25.999 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:25.999 INFO:tasks.workunit.client.0.smithi064.stdout:73/600: open fd = 3 2024-09-12T03:32:25.999 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:25.999 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:25.999 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:25.999 INFO:tasks.workunit.client.0.smithi064.stdout:74/600: open fd = 3 2024-09-12T03:32:25.999 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:25.999 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:25.999 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.000 INFO:tasks.workunit.client.0.smithi064.stdout:75/600: open fd = 3 2024-09-12T03:32:26.000 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.000 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.000 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.000 INFO:tasks.workunit.client.0.smithi064.stdout:76/600: open fd = 3 2024-09-12T03:32:26.000 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.000 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.000 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.000 INFO:tasks.workunit.client.0.smithi064.stdout:77/600: open fd = 3 2024-09-12T03:32:26.000 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.000 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.001 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.001 INFO:tasks.workunit.client.0.smithi064.stdout:78/600: open fd = 3 2024-09-12T03:32:26.001 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.001 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.001 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.001 INFO:tasks.workunit.client.0.smithi064.stdout:79/600: open fd = 3 2024-09-12T03:32:26.001 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.001 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.001 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.002 INFO:tasks.workunit.client.0.smithi064.stdout:80/600: open fd = 3 2024-09-12T03:32:26.002 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.002 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.002 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.002 INFO:tasks.workunit.client.0.smithi064.stdout:81/600: open fd = 3 2024-09-12T03:32:26.002 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.002 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.003 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.003 INFO:tasks.workunit.client.0.smithi064.stdout:82/600: open fd = 3 2024-09-12T03:32:26.003 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.003 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.003 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.003 INFO:tasks.workunit.client.0.smithi064.stdout:83/600: open fd = 3 2024-09-12T03:32:26.003 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.003 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.003 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.003 INFO:tasks.workunit.client.0.smithi064.stdout:84/600: open fd = 3 2024-09-12T03:32:26.003 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.003 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.003 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.003 INFO:tasks.workunit.client.0.smithi064.stdout:85/600: open fd = 3 2024-09-12T03:32:26.004 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.004 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.004 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.004 INFO:tasks.workunit.client.0.smithi064.stdout:86/600: open fd = 3 2024-09-12T03:32:26.004 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.004 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.004 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.004 INFO:tasks.workunit.client.0.smithi064.stdout:87/600: open fd = 3 2024-09-12T03:32:26.005 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.005 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.005 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.005 INFO:tasks.workunit.client.0.smithi064.stdout:88/600: open fd = 3 2024-09-12T03:32:26.005 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.005 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.005 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.005 INFO:tasks.workunit.client.0.smithi064.stdout:89/600: open fd = 3 2024-09-12T03:32:26.005 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.005 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.005 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.005 INFO:tasks.workunit.client.0.smithi064.stdout:90/600: open fd = 3 2024-09-12T03:32:26.005 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.005 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.006 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.006 INFO:tasks.workunit.client.0.smithi064.stdout:91/600: open fd = 3 2024-09-12T03:32:26.006 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.006 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.006 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.006 INFO:tasks.workunit.client.0.smithi064.stdout:92/600: open fd = 3 2024-09-12T03:32:26.006 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.006 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.006 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.006 INFO:tasks.workunit.client.0.smithi064.stdout:93/600: open fd = 3 2024-09-12T03:32:26.006 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.007 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.007 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.007 INFO:tasks.workunit.client.0.smithi064.stdout:94/600: open fd = 3 2024-09-12T03:32:26.007 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.007 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.007 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.007 INFO:tasks.workunit.client.0.smithi064.stdout:95/600: open fd = 3 2024-09-12T03:32:26.007 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.007 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.007 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.007 INFO:tasks.workunit.client.0.smithi064.stdout:96/600: open fd = 3 2024-09-12T03:32:26.007 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.007 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.007 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.008 INFO:tasks.workunit.client.0.smithi064.stdout:97/600: open fd = 3 2024-09-12T03:32:26.008 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.008 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.008 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.008 INFO:tasks.workunit.client.0.smithi064.stdout:98/600: open fd = 3 2024-09-12T03:32:26.008 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.008 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.008 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.008 INFO:tasks.workunit.client.0.smithi064.stdout:99/600: open fd = 3 2024-09-12T03:32:26.008 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.008 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.008 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.008 INFO:tasks.workunit.client.0.smithi064.stdout:100/600: open fd = 3 2024-09-12T03:32:26.008 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.009 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.009 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.009 INFO:tasks.workunit.client.0.smithi064.stdout:101/600: open fd = 3 2024-09-12T03:32:26.009 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.009 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.012 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.012 INFO:tasks.workunit.client.0.smithi064.stdout:102/600: open fd = 3 2024-09-12T03:32:26.012 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.013 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.013 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.013 INFO:tasks.workunit.client.0.smithi064.stdout:103/600: open fd = 3 2024-09-12T03:32:26.013 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.013 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.013 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.013 INFO:tasks.workunit.client.0.smithi064.stdout:104/600: open fd = 3 2024-09-12T03:32:26.013 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.013 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.014 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.014 INFO:tasks.workunit.client.0.smithi064.stdout:105/600: open fd = 3 2024-09-12T03:32:26.014 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.014 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.014 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.014 INFO:tasks.workunit.client.0.smithi064.stdout:106/600: open fd = 3 2024-09-12T03:32:26.014 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.014 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.014 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.014 INFO:tasks.workunit.client.0.smithi064.stdout:107/600: open fd = 3 2024-09-12T03:32:26.015 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.015 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.015 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.015 INFO:tasks.workunit.client.0.smithi064.stdout:108/600: open fd = 3 2024-09-12T03:32:26.015 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.015 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.015 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.015 INFO:tasks.workunit.client.0.smithi064.stdout:109/600: open fd = 3 2024-09-12T03:32:26.015 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.015 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.015 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.016 INFO:tasks.workunit.client.0.smithi064.stdout:110/600: open fd = 3 2024-09-12T03:32:26.016 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.016 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.016 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.016 INFO:tasks.workunit.client.0.smithi064.stdout:111/600: open fd = 3 2024-09-12T03:32:26.016 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.016 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.016 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.016 INFO:tasks.workunit.client.0.smithi064.stdout:112/600: open fd = 3 2024-09-12T03:32:26.016 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.016 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.017 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.017 INFO:tasks.workunit.client.0.smithi064.stdout:113/600: open fd = 3 2024-09-12T03:32:26.017 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.017 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.017 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.017 INFO:tasks.workunit.client.0.smithi064.stdout:114/600: open fd = 3 2024-09-12T03:32:26.017 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.017 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.017 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.018 INFO:tasks.workunit.client.0.smithi064.stdout:115/600: open fd = 3 2024-09-12T03:32:26.018 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.018 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.018 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.018 INFO:tasks.workunit.client.0.smithi064.stdout:116/600: open fd = 3 2024-09-12T03:32:26.018 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.018 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.019 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.019 INFO:tasks.workunit.client.0.smithi064.stdout:117/600: open fd = 3 2024-09-12T03:32:26.019 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.019 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.019 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.019 INFO:tasks.workunit.client.0.smithi064.stdout:118/600: open fd = 3 2024-09-12T03:32:26.019 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.019 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.019 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.019 INFO:tasks.workunit.client.0.smithi064.stdout:119/600: open fd = 3 2024-09-12T03:32:26.019 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.019 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.019 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.020 INFO:tasks.workunit.client.0.smithi064.stdout:120/600: open fd = 3 2024-09-12T03:32:26.020 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.020 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.020 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.020 INFO:tasks.workunit.client.0.smithi064.stdout:121/600: open fd = 3 2024-09-12T03:32:26.020 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.020 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.020 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.020 INFO:tasks.workunit.client.0.smithi064.stdout:122/600: open fd = 3 2024-09-12T03:32:26.020 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.020 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.020 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.021 INFO:tasks.workunit.client.0.smithi064.stdout:123/600: open fd = 3 2024-09-12T03:32:26.021 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.021 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.021 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.021 INFO:tasks.workunit.client.0.smithi064.stdout:124/600: open fd = 3 2024-09-12T03:32:26.021 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.021 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.021 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.021 INFO:tasks.workunit.client.0.smithi064.stdout:125/600: open fd = 3 2024-09-12T03:32:26.021 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.021 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:26.021 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:32:26.022 INFO:tasks.workunit.client.0.smithi064.stdout:126/600: open fd = 3 2024-09-12T03:32:26.022 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:32:27.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:32:27 smithi195 ceph-mon[34786]: pgmap v98: 129 pgs: 129 active+clean; 3.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-09-12T03:32:27.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:32:27 smithi064 ceph-mon[30952]: pgmap v98: 129 pgs: 129 active+clean; 3.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-09-12T03:32:27.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:32:27 smithi115 ceph-mon[34801]: pgmap v98: 129 pgs: 129 active+clean; 3.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-09-12T03:32:29.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:32:29 smithi195 ceph-mon[34786]: pgmap v99: 129 pgs: 129 active+clean; 3.8 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-09-12T03:32:29.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:32:29 smithi064 ceph-mon[30952]: pgmap v99: 129 pgs: 129 active+clean; 3.8 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-09-12T03:32:29.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:32:29 smithi115 ceph-mon[34801]: pgmap v99: 129 pgs: 129 active+clean; 3.8 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-09-12T03:32:31.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:32:31 smithi195 ceph-mon[34786]: pgmap v100: 129 pgs: 129 active+clean; 3.8 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-09-12T03:32:31.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:32:31 smithi064 ceph-mon[30952]: pgmap v100: 129 pgs: 129 active+clean; 3.8 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-09-12T03:32:31.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:32:31 smithi115 ceph-mon[34801]: pgmap v100: 129 pgs: 129 active+clean; 3.8 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-09-12T03:32:33.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:32:33 smithi195 ceph-mon[34786]: pgmap v101: 129 pgs: 129 active+clean; 3.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 KiB/s wr, 18 op/s 2024-09-12T03:32:33.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:32:33 smithi064 ceph-mon[30952]: pgmap v101: 129 pgs: 129 active+clean; 3.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 KiB/s wr, 18 op/s 2024-09-12T03:32:33.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:32:33 smithi115 ceph-mon[34801]: pgmap v101: 129 pgs: 129 active+clean; 3.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 KiB/s wr, 18 op/s 2024-09-12T03:32:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.305+0000 7f5d91eb7640 1 -- 172.21.15.64:0/4232561491 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d8c10c110 msgr2=0x7f5d8c10e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.305+0000 7f5d91eb7640 1 --2- 172.21.15.64:0/4232561491 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d8c10c110 0x7f5d8c10e590 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5d84009ca0 tx=0x7f5d8402a950 comp rx=0 tx=0).stop 2024-09-12T03:32:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.305+0000 7f5d91eb7640 1 -- 172.21.15.64:0/4232561491 shutdown_connections 2024-09-12T03:32:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.305+0000 7f5d91eb7640 1 --2- 172.21.15.64:0/4232561491 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d8c10c110 0x7f5d8c10e590 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.305+0000 7f5d91eb7640 1 --2- 172.21.15.64:0/4232561491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d8c1035e0 0x7f5d8c10bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.305+0000 7f5d91eb7640 1 --2- 172.21.15.64:0/4232561491 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d8c102cc0 0x7f5d8c1030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.305+0000 7f5d91eb7640 1 -- 172.21.15.64:0/4232561491 >> 172.21.15.64:0/4232561491 conn(0x7f5d8c0fc6b0 msgr2=0x7f5d8c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.305+0000 7f5d91eb7640 1 -- 172.21.15.64:0/4232561491 shutdown_connections 2024-09-12T03:32:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.305+0000 7f5d91eb7640 1 -- 172.21.15.64:0/4232561491 wait complete. 2024-09-12T03:32:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.305+0000 7f5d91eb7640 1 Processor -- start 2024-09-12T03:32:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.306+0000 7f5d91eb7640 1 -- start start 2024-09-12T03:32:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.306+0000 7f5d91eb7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d8c102cc0 0x7f5d8c100910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.306+0000 7f5d91eb7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d8c1035e0 0x7f5d8c100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.306+0000 7f5d91eb7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d8c10c110 0x7f5d8c101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.306+0000 7f5d91eb7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8c111da0 con 0x7f5d8c1035e0 2024-09-12T03:32:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.306+0000 7f5d91eb7640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8c111c20 con 0x7f5d8c102cc0 2024-09-12T03:32:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.306+0000 7f5d91eb7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8c111f20 con 0x7f5d8c10c110 2024-09-12T03:32:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.306+0000 7f5d82ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d8c1035e0 0x7f5d8c100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.306+0000 7f5d82ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d8c1035e0 0x7f5d8c100e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39310/0 (socket says 172.21.15.64:39310) 2024-09-12T03:32:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.306+0000 7f5d82ffd640 1 -- 172.21.15.64:0/155698263 learned_addr learned my addr 172.21.15.64:0/155698263 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:32:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.306+0000 7f5d8bfff640 1 --2- 172.21.15.64:0/155698263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d8c10c110 0x7f5d8c101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.306+0000 7f5d8b7fe640 1 --2- 172.21.15.64:0/155698263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d8c102cc0 0x7f5d8c100910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.307+0000 7f5d82ffd640 1 -- 172.21.15.64:0/155698263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d8c10c110 msgr2=0x7f5d8c101390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.307+0000 7f5d82ffd640 1 --2- 172.21.15.64:0/155698263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d8c10c110 0x7f5d8c101390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.307+0000 7f5d82ffd640 1 -- 172.21.15.64:0/155698263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d8c102cc0 msgr2=0x7f5d8c100910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.307+0000 7f5d82ffd640 1 --2- 172.21.15.64:0/155698263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d8c102cc0 0x7f5d8c100910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.307+0000 7f5d82ffd640 1 -- 172.21.15.64:0/155698263 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d8c1a6770 con 0x7f5d8c1035e0 2024-09-12T03:32:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.307+0000 7f5d8b7fe640 1 --2- 172.21.15.64:0/155698263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d8c102cc0 0x7f5d8c100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:32:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.307+0000 7f5d82ffd640 1 --2- 172.21.15.64:0/155698263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d8c1035e0 0x7f5d8c100e50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5d7400e9f0 tx=0x7f5d7400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.307+0000 7f5d897fa640 1 -- 172.21.15.64:0/155698263 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7400ccc0 con 0x7f5d8c1035e0 2024-09-12T03:32:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.307+0000 7f5d91eb7640 1 -- 172.21.15.64:0/155698263 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d8c1a6a60 con 0x7f5d8c1035e0 2024-09-12T03:32:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.307+0000 7f5d897fa640 1 -- 172.21.15.64:0/155698263 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d740108e0 con 0x7f5d8c1035e0 2024-09-12T03:32:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.307+0000 7f5d897fa640 1 -- 172.21.15.64:0/155698263 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d740187c0 con 0x7f5d8c1035e0 2024-09-12T03:32:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.307+0000 7f5d91eb7640 1 -- 172.21.15.64:0/155698263 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d8c1a6fb0 con 0x7f5d8c1035e0 2024-09-12T03:32:35.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.310+0000 7f5d897fa640 1 -- 172.21.15.64:0/155698263 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f5d74010430 con 0x7f5d8c1035e0 2024-09-12T03:32:35.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.310+0000 7f5d94c50740 1 Processor -- start 2024-09-12T03:32:35.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.310+0000 7f5d94c50740 1 -- start start 2024-09-12T03:32:35.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.310+0000 7f5d94c50740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c3c602af0 0x564c3c602ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:35.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.310+0000 7f5d94c50740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c3c603410 0x564c3c60d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:35.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.310+0000 7f5d94c50740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c3c60dcb0 0x564c3c6100a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:35.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.310+0000 7f5d94c50740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564c3c5672f0 con 0x564c3c60dcb0 2024-09-12T03:32:35.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.310+0000 7f5d94c50740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564c3c50f270 con 0x564c3c602af0 2024-09-12T03:32:35.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.310+0000 7f5d94c50740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564c3c5632b0 con 0x564c3c603410 2024-09-12T03:32:35.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.310+0000 7f5d82ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c3c60dcb0 0x564c3c6100a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:35.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.310+0000 7f5d82ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c3c60dcb0 0x564c3c6100a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39322/0 (socket says 172.21.15.64:39322) 2024-09-12T03:32:35.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.310+0000 7f5d82ffd640 1 -- 172.21.15.64:0/2912506766 learned_addr learned my addr 172.21.15.64:0/2912506766 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:32:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.310+0000 7f5d8bfff640 1 --2- 172.21.15.64:0/2912506766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c3c602af0 0x564c3c602ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.310+0000 7f5d897fa640 1 --2- 172.21.15.64:0/155698263 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f5d780a2da0 0x7f5d780a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.311+0000 7f5d8b7fe640 1 --2- 172.21.15.64:0/2912506766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c3c603410 0x564c3c60d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.311+0000 7f5d8bfff640 1 --2- 172.21.15.64:0/155698263 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f5d780a2da0 0x7f5d780a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.311+0000 7f5d897fa640 1 -- 172.21.15.64:0/155698263 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f5d740cfb10 con 0x7f5d8c1035e0 2024-09-12T03:32:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.311+0000 7f5d82ffd640 1 -- 172.21.15.64:0/2912506766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c3c603410 msgr2=0x564c3c60d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.311+0000 7f5d82ffd640 1 --2- 172.21.15.64:0/2912506766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c3c603410 0x564c3c60d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.311+0000 7f5d82ffd640 1 -- 172.21.15.64:0/2912506766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c3c602af0 msgr2=0x564c3c602ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.311+0000 7f5d82ffd640 1 --2- 172.21.15.64:0/2912506766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c3c602af0 0x564c3c602ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.311+0000 7f5d82ffd640 1 -- 172.21.15.64:0/2912506766 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c3c50e340 con 0x564c3c60dcb0 2024-09-12T03:32:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.311+0000 7f5d8bfff640 1 --2- 172.21.15.64:0/2912506766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c3c602af0 0x564c3c602ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:32:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.311+0000 7f5d82ffd640 1 --2- 172.21.15.64:0/2912506766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c3c60dcb0 0x564c3c6100a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5d740cce30 tx=0x7f5d7400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=943a89e24d7d2690 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:35.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.311+0000 7f5d81ffb640 1 -- 172.21.15.64:0/2912506766 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d740d4040 con 0x564c3c60dcb0 2024-09-12T03:32:35.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.311+0000 7f5d8bfff640 1 --2- 172.21.15.64:0/155698263 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f5d780a2da0 0x7f5d780a5260 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5d84004ad0 tx=0x7f5d8402a950 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:35.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.311+0000 7f5d81ffb640 1 -- 172.21.15.64:0/2912506766 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d7409c360 con 0x564c3c60dcb0 2024-09-12T03:32:35.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.311+0000 7f5d81ffb640 1 -- 172.21.15.64:0/2912506766 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d74010bb0 con 0x564c3c60dcb0 2024-09-12T03:32:35.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.312+0000 7f5d94c50740 1 -- 172.21.15.64:0/2912506766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c3c60dcb0 msgr2=0x564c3c6100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:35.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.312+0000 7f5d94c50740 1 --2- 172.21.15.64:0/2912506766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c3c60dcb0 0x564c3c6100a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5d740cce30 tx=0x7f5d7400ce60 comp rx=0 tx=0).stop 2024-09-12T03:32:35.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.312+0000 7f5d94c50740 1 -- 172.21.15.64:0/2912506766 shutdown_connections 2024-09-12T03:32:35.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.312+0000 7f5d94c50740 1 --2- 172.21.15.64:0/2912506766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c3c60dcb0 0x564c3c6100a0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:35.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.312+0000 7f5d94c50740 1 --2- 172.21.15.64:0/2912506766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c3c603410 0x564c3c60d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:35.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.312+0000 7f5d94c50740 1 --2- 172.21.15.64:0/2912506766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c3c602af0 0x564c3c602ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:35.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.312+0000 7f5d94c50740 1 -- 172.21.15.64:0/2912506766 >> 172.21.15.64:0/2912506766 conn(0x564c3c5ff060 msgr2=0x564c3c5ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:35.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.312+0000 7f5d94c50740 1 -- 172.21.15.64:0/2912506766 shutdown_connections 2024-09-12T03:32:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.312+0000 7f5d94c50740 1 -- 172.21.15.64:0/2912506766 wait complete. 2024-09-12T03:32:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.312+0000 7f5d94c50740 1 -- start start 2024-09-12T03:32:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.313+0000 7f5d94c50740 1 Processor -- start 2024-09-12T03:32:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.313+0000 7f5d94c50740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c3c602af0 0x564c3c60ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.313+0000 7f5d94c50740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c3c603410 0x564c3c60b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.313+0000 7f5d94c50740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c3c60dcb0 0x564c3c60b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.313+0000 7f5d94c50740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564c3c610820 con 0x564c3c603410 2024-09-12T03:32:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.313+0000 7f5d94c50740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564c3c4f0290 con 0x564c3c602af0 2024-09-12T03:32:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.313+0000 7f5d94c50740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564c3c6109a0 con 0x564c3c60dcb0 2024-09-12T03:32:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.313+0000 7f5d94c50740 10 client.0 _ll_register_callbacks cb 0x564c3c521140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:32:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.313+0000 7f5d82ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c3c603410 0x564c3c60b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.313+0000 7f5d82ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c3c603410 0x564c3c60b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39330/0 (socket says 172.21.15.64:39330) 2024-09-12T03:32:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.313+0000 7f5d82ffd640 1 -- 172.21.15.64:0/2114335987 learned_addr learned my addr 172.21.15.64:0/2114335987 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:32:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.313+0000 7f5d8bfff640 1 --2- 172.21.15.64:0/2114335987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c3c60dcb0 0x564c3c60b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.313+0000 7f5d82ffd640 1 -- 172.21.15.64:0/2114335987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c3c60dcb0 msgr2=0x564c3c60b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.313+0000 7f5d82ffd640 1 --2- 172.21.15.64:0/2114335987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c3c60dcb0 0x564c3c60b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.313+0000 7f5d8b7fe640 1 --2- 172.21.15.64:0/2114335987 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c3c602af0 0x564c3c60ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.313+0000 7f5d82ffd640 1 -- 172.21.15.64:0/2114335987 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c3c602af0 msgr2=0x564c3c60ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.313+0000 7f5d82ffd640 1 --2- 172.21.15.64:0/2114335987 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c3c602af0 0x564c3c60ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.313+0000 7f5d82ffd640 1 -- 172.21.15.64:0/2114335987 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c3c27a210 con 0x564c3c603410 2024-09-12T03:32:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.314+0000 7f5d82ffd640 1 --2- 172.21.15.64:0/2114335987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c3c603410 0x564c3c60b070 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5d74000c00 tx=0x7f5d74098500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb0255ed20a8e0b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.314+0000 7f5d6affd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:32:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.314+0000 7f5d6affd640 1 -- 172.21.15.64:0/2114335987 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d74097f00 con 0x564c3c603410 2024-09-12T03:32:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.314+0000 7f5d6affd640 1 -- 172.21.15.64:0/2114335987 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f5d7409b0b0 con 0x564c3c603410 2024-09-12T03:32:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.314+0000 7f5d94c50740 1 -- 172.21.15.64:0/2114335987 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d7409c360 con 0x564c3c603410 2024-09-12T03:32:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.314+0000 7f5d6affd640 1 -- 172.21.15.64:0/2114335987 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5d740b4e60 con 0x564c3c603410 2024-09-12T03:32:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.314+0000 7f5d94c50740 10 client.15612 fetch_fsmap learned FSMap version 25 2024-09-12T03:32:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.314+0000 7f5d94c50740 1 -- 172.21.15.64:0/2114335987 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564c3c346620 con 0x564c3c603410 2024-09-12T03:32:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.314+0000 7f5d6affd640 1 -- 172.21.15.64:0/2114335987 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d740b1020 con 0x564c3c603410 2024-09-12T03:32:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.314+0000 7f5d6affd640 1 -- 172.21.15.64:0/2114335987 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5d7409ab20 con 0x564c3c603410 2024-09-12T03:32:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.314+0000 7f5d94c50740 10 client.15612 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:32:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.314+0000 7f5d94c50740 10 client.15612 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:32:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.314+0000 7f5d94c50740 20 client.15612 populate_metadata read hostname 'smithi064' 2024-09-12T03:32:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.314+0000 7f5d94c50740 1 --2- 172.21.15.64:0/2114335987 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x564c3c61d9b0 0x564c3c61fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:35.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.314+0000 7f5d94c50740 4 client.15612 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:32:35.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.314+0000 7f5d94c50740 1 -- 172.21.15.64:0/2114335987 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c3c574a20 con 0x564c3c61d9b0 2024-09-12T03:32:35.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.315+0000 7f5d8b7fe640 1 --2- 172.21.15.64:0/2114335987 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x564c3c61d9b0 0x564c3c61fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.315+0000 7f5d6affd640 10 client.15612 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:32:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.315+0000 7f5d8b7fe640 1 --2- 172.21.15.64:0/2114335987 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x564c3c61d9b0 0x564c3c61fda0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78b9eb8a8947b99d server_cookie=a68257b1548fa426 in_seq=0 out_seq=0 2024-09-12T03:32:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.316+0000 7f5d6affd640 1 -- 172.21.15.64:0/2114335987 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564c3c574a20 con 0x564c3c61d9b0 2024-09-12T03:32:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.316+0000 7f5d6affd640 10 client.15612 handle_command_reply: tid=0 2024-09-12T03:32:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.316+0000 7f5d94c50740 1 client.15612 shutdown 2024-09-12T03:32:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.316+0000 7f5d94c50740 1 -- 172.21.15.64:0/2114335987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c3c603410 msgr2=0x564c3c60b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.316+0000 7f5d94c50740 1 --2- 172.21.15.64:0/2114335987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c3c603410 0x564c3c60b070 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5d74000c00 tx=0x7f5d74098500 comp rx=0 tx=0).stop 2024-09-12T03:32:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.316+0000 7f5d94c50740 1 -- 172.21.15.64:0/2114335987 shutdown_connections 2024-09-12T03:32:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.316+0000 7f5d94c50740 1 --2- 172.21.15.64:0/2114335987 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x564c3c61d9b0 0x564c3c61fda0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.316+0000 7f5d94c50740 1 --2- 172.21.15.64:0/2114335987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c3c60dcb0 0x564c3c60b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.316+0000 7f5d94c50740 1 --2- 172.21.15.64:0/2114335987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c3c603410 0x564c3c60b070 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.316+0000 7f5d6affd640 0 client.15612 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:32:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.316+0000 7f5d94c50740 1 --2- 172.21.15.64:0/2114335987 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c3c602af0 0x564c3c60ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.316+0000 7f5d94c50740 1 -- 172.21.15.64:0/2114335987 >> 172.21.15.64:0/2114335987 conn(0x564c3c5ff060 msgr2=0x564c3c5fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.316+0000 7f5d94c50740 1 -- 172.21.15.64:0/2114335987 shutdown_connections 2024-09-12T03:32:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.316+0000 7f5d94c50740 1 -- 172.21.15.64:0/2114335987 wait complete. 2024-09-12T03:32:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.316+0000 7f5d94c50740 20 client.15612 trim_cache size 0 max 16384 2024-09-12T03:32:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.331+0000 7f5d94c50740 1 Processor -- start 2024-09-12T03:32:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.331+0000 7f5d94c50740 1 -- start start 2024-09-12T03:32:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.331+0000 7f5d94c50740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c3c602af0 0x564c3c605560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.331+0000 7f5d94c50740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c3c603410 0x564c3c60b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.331+0000 7f5d94c50740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c3c60dcb0 0x564c3c622dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.331+0000 7f5d94c50740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8c1b3b40 con 0x564c3c603410 2024-09-12T03:32:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.331+0000 7f5d94c50740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8c1b1410 con 0x564c3c60dcb0 2024-09-12T03:32:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.331+0000 7f5d94c50740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8c1af260 con 0x564c3c602af0 2024-09-12T03:32:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.331+0000 7f5d82ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c3c603410 0x564c3c60b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.331+0000 7f5d82ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c3c603410 0x564c3c60b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39344/0 (socket says 172.21.15.64:39344) 2024-09-12T03:32:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.331+0000 7f5d82ffd640 1 -- 172.21.15.64:0/2950707049 learned_addr learned my addr 172.21.15.64:0/2950707049 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:32:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.331+0000 7f5d8bfff640 1 --2- 172.21.15.64:0/2950707049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c3c60dcb0 0x564c3c622dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.331+0000 7f5d8b7fe640 1 --2- 172.21.15.64:0/2950707049 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c3c602af0 0x564c3c605560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.332+0000 7f5d82ffd640 1 -- 172.21.15.64:0/2950707049 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c3c602af0 msgr2=0x564c3c605560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.332+0000 7f5d82ffd640 1 --2- 172.21.15.64:0/2950707049 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c3c602af0 0x564c3c605560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.332+0000 7f5d82ffd640 1 -- 172.21.15.64:0/2950707049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c3c60dcb0 msgr2=0x564c3c622dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.332+0000 7f5d82ffd640 1 --2- 172.21.15.64:0/2950707049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c3c60dcb0 0x564c3c622dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.332+0000 7f5d82ffd640 1 -- 172.21.15.64:0/2950707049 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c3c60d680 con 0x564c3c603410 2024-09-12T03:32:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.332+0000 7f5d82ffd640 1 --2- 172.21.15.64:0/2950707049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c3c603410 0x564c3c60b3e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5d7400e9c0 tx=0x7f5d7400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9427bf082a40b3d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.332+0000 7f5d817fa640 1 -- 172.21.15.64:0/2950707049 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d74010d90 con 0x564c3c603410 2024-09-12T03:32:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.332+0000 7f5d817fa640 1 -- 172.21.15.64:0/2950707049 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d740b1030 con 0x564c3c603410 2024-09-12T03:32:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.332+0000 7f5d817fa640 1 -- 172.21.15.64:0/2950707049 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d74097bc0 con 0x564c3c603410 2024-09-12T03:32:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.333+0000 7f5d94c50740 1 -- 172.21.15.64:0/2950707049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c3c603410 msgr2=0x564c3c60b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.333+0000 7f5d94c50740 1 --2- 172.21.15.64:0/2950707049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c3c603410 0x564c3c60b3e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5d7400e9c0 tx=0x7f5d7400ce60 comp rx=0 tx=0).stop 2024-09-12T03:32:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.333+0000 7f5d94c50740 1 -- 172.21.15.64:0/2950707049 shutdown_connections 2024-09-12T03:32:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.333+0000 7f5d94c50740 1 --2- 172.21.15.64:0/2950707049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c3c60dcb0 0x564c3c622dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.333+0000 7f5d94c50740 1 --2- 172.21.15.64:0/2950707049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c3c603410 0x564c3c60b3e0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.333+0000 7f5d94c50740 1 --2- 172.21.15.64:0/2950707049 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c3c602af0 0x564c3c605560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.333+0000 7f5d94c50740 1 -- 172.21.15.64:0/2950707049 >> 172.21.15.64:0/2950707049 conn(0x564c3c5ff060 msgr2=0x564c3c60cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.333+0000 7f5d94c50740 1 -- 172.21.15.64:0/2950707049 shutdown_connections 2024-09-12T03:32:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.333+0000 7f5d94c50740 1 -- 172.21.15.64:0/2950707049 wait complete. 2024-09-12T03:32:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.333+0000 7f5d94c50740 1 -- start start 2024-09-12T03:32:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.333+0000 7f5d94c50740 1 Processor -- start 2024-09-12T03:32:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.334+0000 7f5d94c50740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c3c602af0 0x564c3c6212a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.334+0000 7f5d94c50740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c3c603410 0x564c3c6217e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.334+0000 7f5d94c50740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c3c60dcb0 0x564c3c6872f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.334+0000 7f5d94c50740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8c1af8d0 con 0x564c3c603410 2024-09-12T03:32:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.334+0000 7f5d94c50740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8c1b0d40 con 0x564c3c60dcb0 2024-09-12T03:32:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.334+0000 7f5d94c50740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8c1affa0 con 0x564c3c602af0 2024-09-12T03:32:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.334+0000 7f5d94c50740 10 client.0 _ll_register_callbacks cb 0x7f5d84009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:32:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.334+0000 7f5d82ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c3c603410 0x564c3c6217e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.334+0000 7f5d82ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c3c603410 0x564c3c6217e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39352/0 (socket says 172.21.15.64:39352) 2024-09-12T03:32:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.334+0000 7f5d82ffd640 1 -- 172.21.15.64:0/1995374738 learned_addr learned my addr 172.21.15.64:0/1995374738 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:32:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.334+0000 7f5d8b7fe640 1 --2- 172.21.15.64:0/1995374738 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c3c602af0 0x564c3c6212a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.334+0000 7f5d8bfff640 1 --2- 172.21.15.64:0/1995374738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c3c60dcb0 0x564c3c6872f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.334+0000 7f5d82ffd640 1 -- 172.21.15.64:0/1995374738 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c3c602af0 msgr2=0x564c3c6212a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.334+0000 7f5d82ffd640 1 --2- 172.21.15.64:0/1995374738 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c3c602af0 0x564c3c6212a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.334+0000 7f5d82ffd640 1 -- 172.21.15.64:0/1995374738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c3c60dcb0 msgr2=0x564c3c6872f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.334+0000 7f5d82ffd640 1 --2- 172.21.15.64:0/1995374738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c3c60dcb0 0x564c3c6872f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.334+0000 7f5d82ffd640 1 -- 172.21.15.64:0/1995374738 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c3c687830 con 0x564c3c603410 2024-09-12T03:32:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.334+0000 7f5d82ffd640 1 --2- 172.21.15.64:0/1995374738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c3c603410 0x564c3c6217e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5d7409c510 tx=0x7f5d74010d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c0443efa800864d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.334+0000 7f5d6affd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:32:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.334+0000 7f5d6affd640 1 -- 172.21.15.64:0/1995374738 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d740a0010 con 0x564c3c603410 2024-09-12T03:32:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.334+0000 7f5d6affd640 1 -- 172.21.15.64:0/1995374738 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f5d740a12e0 con 0x564c3c603410 2024-09-12T03:32:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.334+0000 7f5d94c50740 1 -- 172.21.15.64:0/1995374738 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d740b1030 con 0x564c3c603410 2024-09-12T03:32:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.335+0000 7f5d6affd640 1 -- 172.21.15.64:0/1995374738 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5d74097bc0 con 0x564c3c603410 2024-09-12T03:32:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.335+0000 7f5d94c50740 10 client.15618 fetch_fsmap learned FSMap version 25 2024-09-12T03:32:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.335+0000 7f5d94c50740 1 -- 172.21.15.64:0/1995374738 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564c3c687830 con 0x564c3c603410 2024-09-12T03:32:35.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.335+0000 7f5d6affd640 1 -- 172.21.15.64:0/1995374738 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d740b1030 con 0x564c3c603410 2024-09-12T03:32:35.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.335+0000 7f5d6affd640 1 -- 172.21.15.64:0/1995374738 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5d740a4b90 con 0x564c3c603410 2024-09-12T03:32:35.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.335+0000 7f5d94c50740 10 client.15618 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:32:35.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.335+0000 7f5d94c50740 10 client.15618 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:32:35.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.335+0000 7f5d94c50740 20 client.15618 populate_metadata read hostname 'smithi064' 2024-09-12T03:32:35.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.335+0000 7f5d94c50740 1 --2- 172.21.15.64:0/1995374738 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x564c3c688e90 0x564c3c68b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:35.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.335+0000 7f5d94c50740 4 client.15618 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:32:35.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.335+0000 7f5d94c50740 1 -- 172.21.15.64:0/1995374738 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564c3c56ed70 con 0x564c3c688e90 2024-09-12T03:32:35.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.335+0000 7f5d8b7fe640 1 --2- 172.21.15.64:0/1995374738 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x564c3c688e90 0x564c3c68b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:35.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.336+0000 7f5d6affd640 10 client.15618 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:32:35.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.336+0000 7f5d8b7fe640 1 --2- 172.21.15.64:0/1995374738 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x564c3c688e90 0x564c3c68b280 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbaa0fbf453a598b server_cookie=46febdbb60bad879 in_seq=0 out_seq=0 2024-09-12T03:32:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.350+0000 7f5d6affd640 1 -- 172.21.15.64:0/1995374738 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564c3c56ed70 con 0x564c3c688e90 2024-09-12T03:32:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.350+0000 7f5d6affd640 10 client.15618 handle_command_reply: tid=0 2024-09-12T03:32:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.350+0000 7f5d94c50740 1 client.15618 shutdown 2024-09-12T03:32:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.350+0000 7f5d94c50740 1 -- 172.21.15.64:0/1995374738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c3c603410 msgr2=0x564c3c6217e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.350+0000 7f5d94c50740 1 --2- 172.21.15.64:0/1995374738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c3c603410 0x564c3c6217e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5d7409c510 tx=0x7f5d74010d90 comp rx=0 tx=0).stop 2024-09-12T03:32:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.350+0000 7f5d94c50740 1 -- 172.21.15.64:0/1995374738 shutdown_connections 2024-09-12T03:32:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.350+0000 7f5d94c50740 1 --2- 172.21.15.64:0/1995374738 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x564c3c688e90 0x564c3c68b280 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.350+0000 7f5d94c50740 1 --2- 172.21.15.64:0/1995374738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c3c60dcb0 0x564c3c6872f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.350+0000 7f5d6affd640 0 client.15618 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:32:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.350+0000 7f5d94c50740 1 --2- 172.21.15.64:0/1995374738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c3c603410 0x564c3c6217e0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.350+0000 7f5d94c50740 1 --2- 172.21.15.64:0/1995374738 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c3c602af0 0x564c3c6212a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.350+0000 7f5d94c50740 1 -- 172.21.15.64:0/1995374738 >> 172.21.15.64:0/1995374738 conn(0x564c3c5ff060 msgr2=0x564c3c5fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.350+0000 7f5d94c50740 1 -- 172.21.15.64:0/1995374738 shutdown_connections 2024-09-12T03:32:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.350+0000 7f5d94c50740 1 -- 172.21.15.64:0/1995374738 wait complete. 2024-09-12T03:32:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.350+0000 7f5d94c50740 20 client.15618 trim_cache size 0 max 16384 2024-09-12T03:32:35.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:32:35.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:32:35.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:32:35.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:32:35.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.351+0000 7f5d91eb7640 1 -- 172.21.15.64:0/155698263 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f5d780a2da0 msgr2=0x7f5d780a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:35.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.351+0000 7f5d91eb7640 1 --2- 172.21.15.64:0/155698263 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f5d780a2da0 0x7f5d780a5260 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5d84004ad0 tx=0x7f5d8402a950 comp rx=0 tx=0).stop 2024-09-12T03:32:35.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.351+0000 7f5d91eb7640 1 -- 172.21.15.64:0/155698263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d8c1035e0 msgr2=0x7f5d8c100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:35.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.351+0000 7f5d91eb7640 1 --2- 172.21.15.64:0/155698263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d8c1035e0 0x7f5d8c100e50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5d7400e9f0 tx=0x7f5d7400eeb0 comp rx=0 tx=0).stop 2024-09-12T03:32:35.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.351+0000 7f5d91eb7640 1 -- 172.21.15.64:0/155698263 shutdown_connections 2024-09-12T03:32:35.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.351+0000 7f5d91eb7640 1 --2- 172.21.15.64:0/155698263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d8c10c110 0x7f5d8c101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:35.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.351+0000 7f5d91eb7640 1 --2- 172.21.15.64:0/155698263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d8c1035e0 0x7f5d8c100e50 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:35.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.351+0000 7f5d91eb7640 1 --2- 172.21.15.64:0/155698263 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f5d780a2da0 0x7f5d780a5260 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:35.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.351+0000 7f5d91eb7640 1 --2- 172.21.15.64:0/155698263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d8c102cc0 0x7f5d8c100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:35.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.351+0000 7f5d91eb7640 1 -- 172.21.15.64:0/155698263 >> 172.21.15.64:0/155698263 conn(0x7f5d8c0fc6b0 msgr2=0x7f5d8c0fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:35.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.351+0000 7f5d91eb7640 1 -- 172.21.15.64:0/155698263 shutdown_connections 2024-09-12T03:32:35.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:35.351+0000 7f5d91eb7640 1 -- 172.21.15.64:0/155698263 wait complete. 2024-09-12T03:32:35.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:32:35 smithi195 ceph-mon[34786]: pgmap v102: 129 pgs: 129 active+clean; 3.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-09-12T03:32:35.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:32:35 smithi064 ceph-mon[30952]: pgmap v102: 129 pgs: 129 active+clean; 3.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-09-12T03:32:35.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:32:35 smithi115 ceph-mon[34801]: pgmap v102: 129 pgs: 129 active+clean; 3.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-09-12T03:32:37.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:32:37 smithi195 ceph-mon[34786]: pgmap v103: 129 pgs: 129 active+clean; 3.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-09-12T03:32:37.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:32:37 smithi064 ceph-mon[30952]: pgmap v103: 129 pgs: 129 active+clean; 3.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-09-12T03:32:37.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:32:37 smithi115 ceph-mon[34801]: pgmap v103: 129 pgs: 129 active+clean; 3.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-09-12T03:32:39.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:32:39 smithi195 ceph-mon[34786]: pgmap v104: 129 pgs: 129 active+clean; 4.0 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 20 op/s 2024-09-12T03:32:39.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:32:39 smithi064 ceph-mon[30952]: pgmap v104: 129 pgs: 129 active+clean; 4.0 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 20 op/s 2024-09-12T03:32:39.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:32:39 smithi115 ceph-mon[34801]: pgmap v104: 129 pgs: 129 active+clean; 4.0 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 20 op/s 2024-09-12T03:32:41.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:32:41 smithi195 ceph-mon[34786]: pgmap v105: 129 pgs: 129 active+clean; 4.0 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-09-12T03:32:41.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:32:41 smithi064 ceph-mon[30952]: pgmap v105: 129 pgs: 129 active+clean; 4.0 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-09-12T03:32:41.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:32:41 smithi115 ceph-mon[34801]: pgmap v105: 129 pgs: 129 active+clean; 4.0 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-09-12T03:32:43.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:32:43 smithi195 ceph-mon[34786]: pgmap v106: 129 pgs: 129 active+clean; 4.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-09-12T03:32:43.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:32:43 smithi064 ceph-mon[30952]: pgmap v106: 129 pgs: 129 active+clean; 4.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-09-12T03:32:43.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:32:43 smithi115 ceph-mon[34801]: pgmap v106: 129 pgs: 129 active+clean; 4.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-09-12T03:32:45.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:32:45 smithi195 ceph-mon[34786]: pgmap v107: 129 pgs: 129 active+clean; 4.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-09-12T03:32:45.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:32:45 smithi064 ceph-mon[30952]: pgmap v107: 129 pgs: 129 active+clean; 4.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-09-12T03:32:45.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:32:45 smithi115 ceph-mon[34801]: pgmap v107: 129 pgs: 129 active+clean; 4.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-09-12T03:32:47.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:32:47 smithi195 ceph-mon[34786]: pgmap v108: 129 pgs: 129 active+clean; 4.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-09-12T03:32:47.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:32:47 smithi064 ceph-mon[30952]: pgmap v108: 129 pgs: 129 active+clean; 4.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-09-12T03:32:47.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:32:47 smithi115 ceph-mon[34801]: pgmap v108: 129 pgs: 129 active+clean; 4.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-09-12T03:32:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.451+0000 7f00c0b42640 1 -- 172.21.15.64:0/3317892738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00bc1009b0 msgr2=0x7f00bc100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.451+0000 7f00c0b42640 1 --2- 172.21.15.64:0/3317892738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00bc1009b0 0x7f00bc100e10 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f00a4009930 tx=0x7f00a402a420 comp rx=0 tx=0).stop 2024-09-12T03:32:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.451+0000 7f00c0b42640 1 -- 172.21.15.64:0/3317892738 shutdown_connections 2024-09-12T03:32:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.451+0000 7f00c0b42640 1 --2- 172.21.15.64:0/3317892738 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00bc101350 0x7f00bc10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.451+0000 7f00c0b42640 1 --2- 172.21.15.64:0/3317892738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00bc1009b0 0x7f00bc100e10 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.451+0000 7f00c0b42640 1 --2- 172.21.15.64:0/3317892738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00bc1069c0 0x7f00bc106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.451+0000 7f00c0b42640 1 -- 172.21.15.64:0/3317892738 >> 172.21.15.64:0/3317892738 conn(0x7f00bc077990 msgr2=0x7f00bc0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.451+0000 7f00c0b42640 1 -- 172.21.15.64:0/3317892738 shutdown_connections 2024-09-12T03:32:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.451+0000 7f00c0b42640 1 -- 172.21.15.64:0/3317892738 wait complete. 2024-09-12T03:32:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.452+0000 7f00c0b42640 1 Processor -- start 2024-09-12T03:32:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.452+0000 7f00c0b42640 1 -- start start 2024-09-12T03:32:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.452+0000 7f00c0b42640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00bc1009b0 0x7f00bc1982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.452+0000 7f00c0b42640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00bc101350 0x7f00bc198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.452+0000 7f00c0b42640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00bc1069c0 0x7f00bc19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.452+0000 7f00c0b42640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc111da0 con 0x7f00bc101350 2024-09-12T03:32:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.452+0000 7f00c0b42640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc111c20 con 0x7f00bc1069c0 2024-09-12T03:32:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.452+0000 7f00c0b42640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc111f20 con 0x7f00bc1009b0 2024-09-12T03:32:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.452+0000 7f00b9d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00bc101350 0x7f00bc198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.452+0000 7f00b9d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00bc101350 0x7f00bc198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38036/0 (socket says 172.21.15.64:38036) 2024-09-12T03:32:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.452+0000 7f00b9d9b640 1 -- 172.21.15.64:0/1604871899 learned_addr learned my addr 172.21.15.64:0/1604871899 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:32:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.452+0000 7f00bad9d640 1 --2- 172.21.15.64:0/1604871899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00bc1069c0 0x7f00bc19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.452+0000 7f00ba59c640 1 --2- 172.21.15.64:0/1604871899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00bc1009b0 0x7f00bc1982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.453+0000 7f00b9d9b640 1 -- 172.21.15.64:0/1604871899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00bc1009b0 msgr2=0x7f00bc1982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.453+0000 7f00b9d9b640 1 --2- 172.21.15.64:0/1604871899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00bc1009b0 0x7f00bc1982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.453+0000 7f00b9d9b640 1 -- 172.21.15.64:0/1604871899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00bc1069c0 msgr2=0x7f00bc19cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.453+0000 7f00b9d9b640 1 --2- 172.21.15.64:0/1604871899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00bc1069c0 0x7f00bc19cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.453+0000 7f00b9d9b640 1 -- 172.21.15.64:0/1604871899 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00bc19d2c0 con 0x7f00bc101350 2024-09-12T03:32:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.453+0000 7f00ba59c640 1 --2- 172.21.15.64:0/1604871899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00bc1009b0 0x7f00bc1982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:32:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.453+0000 7f00b9d9b640 1 --2- 172.21.15.64:0/1604871899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00bc101350 0x7f00bc198810 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f00a4004770 tx=0x7f00a40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.453+0000 7f00b37fe640 1 -- 172.21.15.64:0/1604871899 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a403f070 con 0x7f00bc101350 2024-09-12T03:32:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.453+0000 7f00b37fe640 1 -- 172.21.15.64:0/1604871899 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00a403b460 con 0x7f00bc101350 2024-09-12T03:32:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.453+0000 7f00c0b42640 1 -- 172.21.15.64:0/1604871899 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00bc19d550 con 0x7f00bc101350 2024-09-12T03:32:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.453+0000 7f00b37fe640 1 -- 172.21.15.64:0/1604871899 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a4035070 con 0x7f00bc101350 2024-09-12T03:32:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.453+0000 7f00c0b42640 1 -- 172.21.15.64:0/1604871899 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00bc1a4e40 con 0x7f00bc101350 2024-09-12T03:32:48.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.456+0000 7f00b37fe640 1 -- 172.21.15.64:0/1604871899 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f00a403b610 con 0x7f00bc101350 2024-09-12T03:32:48.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.456+0000 7f00c38dc740 1 Processor -- start 2024-09-12T03:32:48.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.457+0000 7f00c38dc740 1 -- start start 2024-09-12T03:32:48.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.457+0000 7f00c38dc740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584230629d0 0x558423062db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:48.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.457+0000 7f00c38dc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584230632f0 0x558423071b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:48.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.457+0000 7f00c38dc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584230720c0 0x5584230744b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:48.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.457+0000 7f00c38dc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558422fc72f0 con 0x5584230632f0 2024-09-12T03:32:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.457+0000 7f00c38dc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558422f6f270 con 0x5584230720c0 2024-09-12T03:32:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.457+0000 7f00c38dc740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558422fc32b0 con 0x5584230629d0 2024-09-12T03:32:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.457+0000 7f00ba59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584230632f0 0x558423071b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.457+0000 7f00ba59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584230632f0 0x558423071b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38052/0 (socket says 172.21.15.64:38052) 2024-09-12T03:32:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.457+0000 7f00ba59c640 1 -- 172.21.15.64:0/4281352687 learned_addr learned my addr 172.21.15.64:0/4281352687 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:32:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.457+0000 7f00bad9d640 1 --2- 172.21.15.64:0/4281352687 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584230629d0 0x558423062db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.457+0000 7f00b9d9b640 1 --2- 172.21.15.64:0/4281352687 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584230720c0 0x5584230744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.457+0000 7f00b37fe640 1 --2- 172.21.15.64:0/1604871899 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f00a80a2da0 0x7f00a80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.457+0000 7f00bad9d640 1 --2- 172.21.15.64:0/1604871899 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f00a80a2da0 0x7f00a80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.458+0000 7f00ba59c640 1 -- 172.21.15.64:0/4281352687 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584230629d0 msgr2=0x558423062db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.458+0000 7f00ba59c640 1 --2- 172.21.15.64:0/4281352687 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584230629d0 0x558423062db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.458+0000 7f00ba59c640 1 -- 172.21.15.64:0/4281352687 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584230720c0 msgr2=0x5584230744b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.458+0000 7f00ba59c640 1 --2- 172.21.15.64:0/4281352687 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584230720c0 0x5584230744b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.458+0000 7f00ba59c640 1 -- 172.21.15.64:0/4281352687 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558422f6e340 con 0x5584230632f0 2024-09-12T03:32:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.458+0000 7f00b37fe640 1 -- 172.21.15.64:0/1604871899 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f00a40efbb0 con 0x7f00bc101350 2024-09-12T03:32:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.458+0000 7f00ba59c640 1 --2- 172.21.15.64:0/4281352687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584230632f0 0x558423071b80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f00b4010370 tx=0x7f00b4010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e80280e058b710b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.458+0000 7f00bad9d640 1 --2- 172.21.15.64:0/1604871899 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f00a80a2da0 0x7f00a80a5260 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f00ac005d00 tx=0x7f00ac004730 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.458+0000 7f00b0ff9640 1 -- 172.21.15.64:0/4281352687 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b4016070 con 0x5584230632f0 2024-09-12T03:32:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.458+0000 7f00b0ff9640 1 -- 172.21.15.64:0/4281352687 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00b4012bc0 con 0x5584230632f0 2024-09-12T03:32:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.458+0000 7f00b0ff9640 1 -- 172.21.15.64:0/4281352687 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b401bc30 con 0x5584230632f0 2024-09-12T03:32:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.459+0000 7f00c38dc740 1 -- 172.21.15.64:0/4281352687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584230632f0 msgr2=0x558423071b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.459+0000 7f00c38dc740 1 --2- 172.21.15.64:0/4281352687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584230632f0 0x558423071b80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f00b4010370 tx=0x7f00b4010830 comp rx=0 tx=0).stop 2024-09-12T03:32:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.459+0000 7f00c38dc740 1 -- 172.21.15.64:0/4281352687 shutdown_connections 2024-09-12T03:32:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.459+0000 7f00c38dc740 1 --2- 172.21.15.64:0/4281352687 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584230720c0 0x5584230744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.459+0000 7f00c38dc740 1 --2- 172.21.15.64:0/4281352687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584230632f0 0x558423071b80 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.459+0000 7f00c38dc740 1 --2- 172.21.15.64:0/4281352687 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584230629d0 0x558423062db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.459+0000 7f00c38dc740 1 -- 172.21.15.64:0/4281352687 >> 172.21.15.64:0/4281352687 conn(0x55842305f060 msgr2=0x55842305f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.459+0000 7f00c38dc740 1 -- 172.21.15.64:0/4281352687 shutdown_connections 2024-09-12T03:32:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.459+0000 7f00c38dc740 1 -- 172.21.15.64:0/4281352687 wait complete. 2024-09-12T03:32:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.459+0000 7f00c38dc740 1 -- start start 2024-09-12T03:32:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.459+0000 7f00c38dc740 1 Processor -- start 2024-09-12T03:32:48.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.460+0000 7f00c38dc740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584230629d0 0x55842306f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:48.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.460+0000 7f00c38dc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584230632f0 0x55842306f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:48.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.460+0000 7f00c38dc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584230720c0 0x55842306fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:48.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.460+0000 7f00c38dc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558423074cf0 con 0x5584230720c0 2024-09-12T03:32:48.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.460+0000 7f00c38dc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558422f50290 con 0x5584230632f0 2024-09-12T03:32:48.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.460+0000 7f00c38dc740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558423074e70 con 0x5584230629d0 2024-09-12T03:32:48.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.460+0000 7f00c38dc740 10 client.0 _ll_register_callbacks cb 0x558422f81140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:32:48.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.460+0000 7f00bad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584230720c0 0x55842306fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:48.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.460+0000 7f00bad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584230720c0 0x55842306fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38062/0 (socket says 172.21.15.64:38062) 2024-09-12T03:32:48.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.460+0000 7f00bad9d640 1 -- 172.21.15.64:0/2248309668 learned_addr learned my addr 172.21.15.64:0/2248309668 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:32:48.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.460+0000 7f00ba59c640 1 --2- 172.21.15.64:0/2248309668 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584230629d0 0x55842306f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:48.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.460+0000 7f00b9d9b640 1 --2- 172.21.15.64:0/2248309668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584230632f0 0x55842306f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:48.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.460+0000 7f00bad9d640 1 -- 172.21.15.64:0/2248309668 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584230629d0 msgr2=0x55842306f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:48.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.460+0000 7f00bad9d640 1 --2- 172.21.15.64:0/2248309668 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584230629d0 0x55842306f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.460+0000 7f00bad9d640 1 -- 172.21.15.64:0/2248309668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584230632f0 msgr2=0x55842306f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:48.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.460+0000 7f00bad9d640 1 --2- 172.21.15.64:0/2248309668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584230632f0 0x55842306f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.460+0000 7f00bad9d640 1 -- 172.21.15.64:0/2248309668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558422cda210 con 0x5584230720c0 2024-09-12T03:32:48.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.460+0000 7f00ba59c640 1 --2- 172.21.15.64:0/2248309668 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584230629d0 0x55842306f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:32:48.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.460+0000 7f00bad9d640 1 --2- 172.21.15.64:0/2248309668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584230720c0 0x55842306fa80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0124a0 tx=0x7f00ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f47c9f8871fed4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:48.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.460+0000 7f00a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:32:48.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.461+0000 7f00a1ffb640 1 -- 172.21.15.64:0/2248309668 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac025020 con 0x5584230720c0 2024-09-12T03:32:48.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.461+0000 7f00a1ffb640 1 -- 172.21.15.64:0/2248309668 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f00ac021950 con 0x5584230720c0 2024-09-12T03:32:48.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.461+0000 7f00c38dc740 1 -- 172.21.15.64:0/2248309668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00b4012bc0 con 0x5584230720c0 2024-09-12T03:32:48.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.461+0000 7f00a1ffb640 1 -- 172.21.15.64:0/2248309668 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f00ac030050 con 0x5584230720c0 2024-09-12T03:32:48.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.461+0000 7f00c38dc740 10 client.15630 fetch_fsmap learned FSMap version 25 2024-09-12T03:32:48.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.461+0000 7f00c38dc740 1 -- 172.21.15.64:0/2248309668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558422da6620 con 0x5584230720c0 2024-09-12T03:32:48.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.461+0000 7f00a1ffb640 1 -- 172.21.15.64:0/2248309668 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00b4012bc0 con 0x5584230720c0 2024-09-12T03:32:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.461+0000 7f00a1ffb640 1 -- 172.21.15.64:0/2248309668 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f00ac01e070 con 0x5584230720c0 2024-09-12T03:32:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.461+0000 7f00c38dc740 10 client.15630 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:32:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.461+0000 7f00c38dc740 10 client.15630 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:32:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.461+0000 7f00c38dc740 20 client.15630 populate_metadata read hostname 'smithi064' 2024-09-12T03:32:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.461+0000 7f00c38dc740 1 --2- 172.21.15.64:0/2248309668 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x558423082170 0x5584230845a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.461+0000 7f00c38dc740 4 client.15630 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:32:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.461+0000 7f00c38dc740 1 -- 172.21.15.64:0/2248309668 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558422fd4a20 con 0x558423082170 2024-09-12T03:32:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.462+0000 7f00ba59c640 1 --2- 172.21.15.64:0/2248309668 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x558423082170 0x5584230845a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.462+0000 7f00ba59c640 1 --2- 172.21.15.64:0/2248309668 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x558423082170 0x5584230845a0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b67e0c5b40cb969 server_cookie=d29bf94ab1aad94 in_seq=0 out_seq=0 2024-09-12T03:32:48.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.462+0000 7f00a1ffb640 10 client.15630 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:32:48.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.463+0000 7f00a1ffb640 1 -- 172.21.15.64:0/2248309668 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558422fd4a20 con 0x558423082170 2024-09-12T03:32:48.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.463+0000 7f00a1ffb640 10 client.15630 handle_command_reply: tid=0 2024-09-12T03:32:48.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.463+0000 7f00c38dc740 1 client.15630 shutdown 2024-09-12T03:32:48.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.463+0000 7f00c38dc740 1 -- 172.21.15.64:0/2248309668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584230720c0 msgr2=0x55842306fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:48.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.463+0000 7f00c38dc740 1 --2- 172.21.15.64:0/2248309668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584230720c0 0x55842306fa80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0124a0 tx=0x7f00ac0046c0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.463+0000 7f00c38dc740 1 -- 172.21.15.64:0/2248309668 shutdown_connections 2024-09-12T03:32:48.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.463+0000 7f00c38dc740 1 --2- 172.21.15.64:0/2248309668 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x558423082170 0x5584230845a0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.463+0000 7f00c38dc740 1 --2- 172.21.15.64:0/2248309668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584230720c0 0x55842306fa80 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.463+0000 7f00a1ffb640 0 client.15630 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:32:48.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.463+0000 7f00c38dc740 1 --2- 172.21.15.64:0/2248309668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584230632f0 0x55842306f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.463+0000 7f00c38dc740 1 --2- 172.21.15.64:0/2248309668 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584230629d0 0x55842306f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.463+0000 7f00c38dc740 1 -- 172.21.15.64:0/2248309668 >> 172.21.15.64:0/2248309668 conn(0x55842305f060 msgr2=0x558423064d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:48.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.463+0000 7f00c38dc740 1 -- 172.21.15.64:0/2248309668 shutdown_connections 2024-09-12T03:32:48.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.463+0000 7f00c38dc740 1 -- 172.21.15.64:0/2248309668 wait complete. 2024-09-12T03:32:48.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.463+0000 7f00c38dc740 20 client.15630 trim_cache size 0 max 16384 2024-09-12T03:32:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.478+0000 7f00c38dc740 1 Processor -- start 2024-09-12T03:32:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.478+0000 7f00c38dc740 1 -- start start 2024-09-12T03:32:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.478+0000 7f00c38dc740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584230629d0 0x558423089eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.478+0000 7f00c38dc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584230632f0 0x55842308a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.478+0000 7f00c38dc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584230720c0 0x55842306c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.478+0000 7f00c38dc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc1af6d0 con 0x5584230720c0 2024-09-12T03:32:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.478+0000 7f00c38dc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc1acfa0 con 0x5584230632f0 2024-09-12T03:32:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.478+0000 7f00c38dc740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc1aadf0 con 0x5584230629d0 2024-09-12T03:32:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.478+0000 7f00bad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584230720c0 0x55842306c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.478+0000 7f00bad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584230720c0 0x55842306c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38072/0 (socket says 172.21.15.64:38072) 2024-09-12T03:32:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.478+0000 7f00bad9d640 1 -- 172.21.15.64:0/1132357323 learned_addr learned my addr 172.21.15.64:0/1132357323 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:32:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.478+0000 7f00bad9d640 1 -- 172.21.15.64:0/1132357323 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584230629d0 msgr2=0x558423089eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.478+0000 7f00bad9d640 1 --2- 172.21.15.64:0/1132357323 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584230629d0 0x558423089eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.478+0000 7f00bad9d640 1 -- 172.21.15.64:0/1132357323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584230632f0 msgr2=0x55842308a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.478+0000 7f00bad9d640 1 --2- 172.21.15.64:0/1132357323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584230632f0 0x55842308a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.478+0000 7f00bad9d640 1 -- 172.21.15.64:0/1132357323 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55842306df90 con 0x5584230720c0 2024-09-12T03:32:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.478+0000 7f00bad9d640 1 --2- 172.21.15.64:0/1132357323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584230720c0 0x55842306c240 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f00ac00b540 tx=0x7f00ac004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a2060a79ed34b9c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.479+0000 7f00a3fff640 1 -- 172.21.15.64:0/1132357323 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac025030 con 0x5584230720c0 2024-09-12T03:32:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.479+0000 7f00a3fff640 1 -- 172.21.15.64:0/1132357323 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00ac039070 con 0x5584230720c0 2024-09-12T03:32:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.479+0000 7f00a3fff640 1 -- 172.21.15.64:0/1132357323 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac0109b0 con 0x5584230720c0 2024-09-12T03:32:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.479+0000 7f00c38dc740 1 -- 172.21.15.64:0/1132357323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584230720c0 msgr2=0x55842306c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.479+0000 7f00c38dc740 1 --2- 172.21.15.64:0/1132357323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584230720c0 0x55842306c240 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f00ac00b540 tx=0x7f00ac004570 comp rx=0 tx=0).stop 2024-09-12T03:32:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.479+0000 7f00c38dc740 1 -- 172.21.15.64:0/1132357323 shutdown_connections 2024-09-12T03:32:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.479+0000 7f00c38dc740 1 --2- 172.21.15.64:0/1132357323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584230720c0 0x55842306c240 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.479+0000 7f00c38dc740 1 --2- 172.21.15.64:0/1132357323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584230632f0 0x55842308a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.479+0000 7f00c38dc740 1 --2- 172.21.15.64:0/1132357323 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584230629d0 0x558423089eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.479+0000 7f00c38dc740 1 -- 172.21.15.64:0/1132357323 >> 172.21.15.64:0/1132357323 conn(0x55842305f060 msgr2=0x558423071080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.479+0000 7f00c38dc740 1 -- 172.21.15.64:0/1132357323 shutdown_connections 2024-09-12T03:32:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.479+0000 7f00c38dc740 1 -- 172.21.15.64:0/1132357323 wait complete. 2024-09-12T03:32:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.479+0000 7f00c38dc740 1 -- start start 2024-09-12T03:32:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.480+0000 7f00c38dc740 1 Processor -- start 2024-09-12T03:32:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.480+0000 7f00c38dc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584230629d0 0x558423076940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.480+0000 7f00c38dc740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584230632f0 0x558423076e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.480+0000 7f00c38dc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584230720c0 0x558423079f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.480+0000 7f00c38dc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc1ab460 con 0x5584230720c0 2024-09-12T03:32:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.480+0000 7f00c38dc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc1ac8d0 con 0x5584230629d0 2024-09-12T03:32:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.480+0000 7f00c38dc740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc1abb30 con 0x5584230632f0 2024-09-12T03:32:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.480+0000 7f00c38dc740 10 client.0 _ll_register_callbacks cb 0x7f00ac00b600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:32:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.480+0000 7f00bad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584230720c0 0x558423079f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.480+0000 7f00bad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584230720c0 0x558423079f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38084/0 (socket says 172.21.15.64:38084) 2024-09-12T03:32:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.480+0000 7f00bad9d640 1 -- 172.21.15.64:0/3028976325 learned_addr learned my addr 172.21.15.64:0/3028976325 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:32:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.480+0000 7f00ba59c640 1 --2- 172.21.15.64:0/3028976325 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584230629d0 0x558423076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.480+0000 7f00b9d9b640 1 --2- 172.21.15.64:0/3028976325 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584230632f0 0x558423076e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.480+0000 7f00bad9d640 1 -- 172.21.15.64:0/3028976325 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584230632f0 msgr2=0x558423076e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.480+0000 7f00bad9d640 1 --2- 172.21.15.64:0/3028976325 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584230632f0 0x558423076e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.480+0000 7f00bad9d640 1 -- 172.21.15.64:0/3028976325 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584230629d0 msgr2=0x558423076940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.480+0000 7f00bad9d640 1 --2- 172.21.15.64:0/3028976325 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584230629d0 0x558423076940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.480+0000 7f00bad9d640 1 -- 172.21.15.64:0/3028976325 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55842307a5f0 con 0x5584230720c0 2024-09-12T03:32:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.480+0000 7f00bad9d640 1 --2- 172.21.15.64:0/3028976325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584230720c0 0x558423079f50 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f00ac003e50 tx=0x7f00ac004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b5431849badcd8b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.480+0000 7f00a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:32:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.481+0000 7f00a1ffb640 1 -- 172.21.15.64:0/3028976325 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac041020 con 0x5584230720c0 2024-09-12T03:32:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.481+0000 7f00a1ffb640 1 -- 172.21.15.64:0/3028976325 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f00ac010a40 con 0x5584230720c0 2024-09-12T03:32:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.481+0000 7f00c38dc740 1 -- 172.21.15.64:0/3028976325 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00ac039070 con 0x5584230720c0 2024-09-12T03:32:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.481+0000 7f00a1ffb640 1 -- 172.21.15.64:0/3028976325 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f00ac04c050 con 0x5584230720c0 2024-09-12T03:32:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.481+0000 7f00c38dc740 10 client.15636 fetch_fsmap learned FSMap version 25 2024-09-12T03:32:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.481+0000 7f00c38dc740 1 -- 172.21.15.64:0/3028976325 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5584230eda70 con 0x5584230720c0 2024-09-12T03:32:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.481+0000 7f00a1ffb640 1 -- 172.21.15.64:0/3028976325 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00ac039070 con 0x5584230720c0 2024-09-12T03:32:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.481+0000 7f00a1ffb640 1 -- 172.21.15.64:0/3028976325 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f00ac01e040 con 0x5584230720c0 2024-09-12T03:32:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.481+0000 7f00c38dc740 10 client.15636 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:32:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.481+0000 7f00c38dc740 10 client.15636 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:32:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.481+0000 7f00c38dc740 20 client.15636 populate_metadata read hostname 'smithi064' 2024-09-12T03:32:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.481+0000 7f00c38dc740 1 --2- 172.21.15.64:0/3028976325 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5584230ee0c0 0x5584230f04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.481+0000 7f00c38dc740 4 client.15636 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:32:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.481+0000 7f00c38dc740 1 -- 172.21.15.64:0/3028976325 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558422fced70 con 0x5584230ee0c0 2024-09-12T03:32:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.481+0000 7f00ba59c640 1 --2- 172.21.15.64:0/3028976325 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5584230ee0c0 0x5584230f04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.482+0000 7f00a1ffb640 10 client.15636 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:32:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.482+0000 7f00ba59c640 1 --2- 172.21.15.64:0/3028976325 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5584230ee0c0 0x5584230f04b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b3f037fff03dd60 server_cookie=5290b4706a2b154f in_seq=0 out_seq=0 2024-09-12T03:32:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.496+0000 7f00a1ffb640 1 -- 172.21.15.64:0/3028976325 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558422fced70 con 0x5584230ee0c0 2024-09-12T03:32:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.496+0000 7f00a1ffb640 10 client.15636 handle_command_reply: tid=0 2024-09-12T03:32:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.496+0000 7f00c38dc740 1 client.15636 shutdown 2024-09-12T03:32:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.497+0000 7f00c38dc740 1 -- 172.21.15.64:0/3028976325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584230720c0 msgr2=0x558423079f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.497+0000 7f00c38dc740 1 --2- 172.21.15.64:0/3028976325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584230720c0 0x558423079f50 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f00ac003e50 tx=0x7f00ac004310 comp rx=0 tx=0).stop 2024-09-12T03:32:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.497+0000 7f00c38dc740 1 -- 172.21.15.64:0/3028976325 shutdown_connections 2024-09-12T03:32:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.497+0000 7f00c38dc740 1 --2- 172.21.15.64:0/3028976325 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5584230ee0c0 0x5584230f04b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.497+0000 7f00c38dc740 1 --2- 172.21.15.64:0/3028976325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5584230720c0 0x558423079f50 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.497+0000 7f00c38dc740 1 --2- 172.21.15.64:0/3028976325 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584230632f0 0x558423076e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.497+0000 7f00c38dc740 1 --2- 172.21.15.64:0/3028976325 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584230629d0 0x558423076940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.497+0000 7f00a1ffb640 0 client.15636 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:32:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.497+0000 7f00c38dc740 1 -- 172.21.15.64:0/3028976325 >> 172.21.15.64:0/3028976325 conn(0x55842305f060 msgr2=0x55842306efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.497+0000 7f00c38dc740 1 -- 172.21.15.64:0/3028976325 shutdown_connections 2024-09-12T03:32:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.497+0000 7f00c38dc740 1 -- 172.21.15.64:0/3028976325 wait complete. 2024-09-12T03:32:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.497+0000 7f00c38dc740 20 client.15636 trim_cache size 0 max 16384 2024-09-12T03:32:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:32:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:32:48.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:32:48.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:32:48.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.497+0000 7f00c0b42640 1 -- 172.21.15.64:0/1604871899 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f00a80a2da0 msgr2=0x7f00a80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.497+0000 7f00c0b42640 1 --2- 172.21.15.64:0/1604871899 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f00a80a2da0 0x7f00a80a5260 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f00ac005d00 tx=0x7f00ac004730 comp rx=0 tx=0).stop 2024-09-12T03:32:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.497+0000 7f00c0b42640 1 -- 172.21.15.64:0/1604871899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00bc101350 msgr2=0x7f00bc198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.497+0000 7f00c0b42640 1 --2- 172.21.15.64:0/1604871899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00bc101350 0x7f00bc198810 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f00a4004770 tx=0x7f00a40042f0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.497+0000 7f00c0b42640 1 -- 172.21.15.64:0/1604871899 shutdown_connections 2024-09-12T03:32:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.497+0000 7f00c0b42640 1 --2- 172.21.15.64:0/1604871899 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f00a80a2da0 0x7f00a80a5260 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.497+0000 7f00c0b42640 1 --2- 172.21.15.64:0/1604871899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00bc1069c0 0x7f00bc19cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.497+0000 7f00c0b42640 1 --2- 172.21.15.64:0/1604871899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00bc101350 0x7f00bc198810 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.497+0000 7f00c0b42640 1 --2- 172.21.15.64:0/1604871899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00bc1009b0 0x7f00bc1982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.497+0000 7f00c0b42640 1 -- 172.21.15.64:0/1604871899 >> 172.21.15.64:0/1604871899 conn(0x7f00bc077990 msgr2=0x7f00bc103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.498+0000 7f00c0b42640 1 -- 172.21.15.64:0/1604871899 shutdown_connections 2024-09-12T03:32:48.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:32:48.498+0000 7f00c0b42640 1 -- 172.21.15.64:0/1604871899 wait complete. 2024-09-12T03:32:49.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:32:49 smithi064 ceph-mon[30952]: pgmap v109: 129 pgs: 129 active+clean; 4.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-09-12T03:32:49.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:32:49 smithi115 ceph-mon[34801]: pgmap v109: 129 pgs: 129 active+clean; 4.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-09-12T03:32:49.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:32:49 smithi195 ceph-mon[34786]: pgmap v109: 129 pgs: 129 active+clean; 4.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-09-12T03:32:51.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:32:51 smithi064 ceph-mon[30952]: pgmap v110: 129 pgs: 129 active+clean; 4.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-09-12T03:32:51.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:32:51 smithi115 ceph-mon[34801]: pgmap v110: 129 pgs: 129 active+clean; 4.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-09-12T03:32:51.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:32:51 smithi195 ceph-mon[34786]: pgmap v110: 129 pgs: 129 active+clean; 4.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-09-12T03:32:53.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:32:53 smithi064 ceph-mon[30952]: pgmap v111: 129 pgs: 129 active+clean; 4.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-09-12T03:32:53.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:32:53 smithi115 ceph-mon[34801]: pgmap v111: 129 pgs: 129 active+clean; 4.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-09-12T03:32:53.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:32:53 smithi195 ceph-mon[34786]: pgmap v111: 129 pgs: 129 active+clean; 4.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-09-12T03:32:54.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:32:54 smithi064 ceph-mon[30952]: pgmap v112: 129 pgs: 129 active+clean; 4.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-09-12T03:32:54.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:32:54 smithi115 ceph-mon[34801]: pgmap v112: 129 pgs: 129 active+clean; 4.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-09-12T03:32:54.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:32:54 smithi195 ceph-mon[34786]: pgmap v112: 129 pgs: 129 active+clean; 4.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-09-12T03:32:57.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:32:56 smithi064 ceph-mon[30952]: pgmap v113: 129 pgs: 129 active+clean; 4.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-09-12T03:32:57.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:32:56 smithi115 ceph-mon[34801]: pgmap v113: 129 pgs: 129 active+clean; 4.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-09-12T03:32:57.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:32:56 smithi195 ceph-mon[34786]: pgmap v113: 129 pgs: 129 active+clean; 4.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-09-12T03:32:59.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:32:58 smithi064 ceph-mon[30952]: pgmap v114: 129 pgs: 129 active+clean; 4.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-09-12T03:32:59.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:32:58 smithi115 ceph-mon[34801]: pgmap v114: 129 pgs: 129 active+clean; 4.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-09-12T03:32:59.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:32:58 smithi195 ceph-mon[34786]: pgmap v114: 129 pgs: 129 active+clean; 4.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-09-12T03:33:01.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:33:00 smithi064 ceph-mon[30952]: pgmap v115: 129 pgs: 129 active+clean; 4.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 12 op/s 2024-09-12T03:33:01.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:33:00 smithi115 ceph-mon[34801]: pgmap v115: 129 pgs: 129 active+clean; 4.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 12 op/s 2024-09-12T03:33:01.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:33:00 smithi195 ceph-mon[34786]: pgmap v115: 129 pgs: 129 active+clean; 4.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 12 op/s 2024-09-12T03:33:01.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.598+0000 7feb039c0640 1 -- 172.21.15.64:0/2555403071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feafc1008d0 msgr2=0x7feafc100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.598+0000 7feb039c0640 1 --2- 172.21.15.64:0/2555403071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feafc1008d0 0x7feafc100cb0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7feaf8009900 tx=0x7feaf802a410 comp rx=0 tx=0).stop 2024-09-12T03:33:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.598+0000 7feb039c0640 1 -- 172.21.15.64:0/2555403071 shutdown_connections 2024-09-12T03:33:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.598+0000 7feb039c0640 1 --2- 172.21.15.64:0/2555403071 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feafc10e480 0x7feafc110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.598+0000 7feb039c0640 1 --2- 172.21.15.64:0/2555403071 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feafc101280 0x7feafc10deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.598+0000 7feb039c0640 1 --2- 172.21.15.64:0/2555403071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feafc1008d0 0x7feafc100cb0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.598+0000 7feb039c0640 1 -- 172.21.15.64:0/2555403071 >> 172.21.15.64:0/2555403071 conn(0x7feafc0fc6b0 msgr2=0x7feafc0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.598+0000 7feb039c0640 1 -- 172.21.15.64:0/2555403071 shutdown_connections 2024-09-12T03:33:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.599+0000 7feb039c0640 1 -- 172.21.15.64:0/2555403071 wait complete. 2024-09-12T03:33:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.599+0000 7feb039c0640 1 Processor -- start 2024-09-12T03:33:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.599+0000 7feb039c0640 1 -- start start 2024-09-12T03:33:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.599+0000 7feb039c0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feafc1008d0 0x7feafc19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.599+0000 7feb039c0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feafc101280 0x7feafc19a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.599+0000 7feb039c0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feafc10e480 0x7feafc19ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.599+0000 7feb039c0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7feafc113f90 con 0x7feafc10e480 2024-09-12T03:33:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.599+0000 7feb039c0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7feafc113e10 con 0x7feafc101280 2024-09-12T03:33:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.599+0000 7feb039c0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7feafc114110 con 0x7feafc1008d0 2024-09-12T03:33:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.600+0000 7feb01f5d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feafc10e480 0x7feafc19ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.600+0000 7feb01f5d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feafc10e480 0x7feafc19ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52542/0 (socket says 172.21.15.64:52542) 2024-09-12T03:33:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.600+0000 7feb01f5d640 1 -- 172.21.15.64:0/1147435745 learned_addr learned my addr 172.21.15.64:0/1147435745 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:33:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.600+0000 7feb00f5b640 1 --2- 172.21.15.64:0/1147435745 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feafc101280 0x7feafc19a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.600+0000 7feb0175c640 1 --2- 172.21.15.64:0/1147435745 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feafc1008d0 0x7feafc19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.600+0000 7feb01f5d640 1 -- 172.21.15.64:0/1147435745 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feafc1008d0 msgr2=0x7feafc19a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.600+0000 7feb01f5d640 1 --2- 172.21.15.64:0/1147435745 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feafc1008d0 0x7feafc19a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.600+0000 7feb01f5d640 1 -- 172.21.15.64:0/1147435745 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feafc101280 msgr2=0x7feafc19a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.600+0000 7feb01f5d640 1 --2- 172.21.15.64:0/1147435745 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feafc101280 0x7feafc19a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.600+0000 7feb01f5d640 1 -- 172.21.15.64:0/1147435745 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feafc19f3f0 con 0x7feafc10e480 2024-09-12T03:33:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.600+0000 7feb01f5d640 1 --2- 172.21.15.64:0/1147435745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feafc10e480 0x7feafc19ecc0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7feaf400e9d0 tx=0x7feaf400edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.600+0000 7feaee7fc640 1 -- 172.21.15.64:0/1147435745 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf4004d20 con 0x7feafc10e480 2024-09-12T03:33:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.600+0000 7feaee7fc640 1 -- 172.21.15.64:0/1147435745 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feaf4005020 con 0x7feafc10e480 2024-09-12T03:33:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.600+0000 7feaee7fc640 1 -- 172.21.15.64:0/1147435745 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf40116b0 con 0x7feafc10e480 2024-09-12T03:33:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.600+0000 7feb039c0640 1 -- 172.21.15.64:0/1147435745 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feafc19f6e0 con 0x7feafc10e480 2024-09-12T03:33:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.600+0000 7feb039c0640 1 -- 172.21.15.64:0/1147435745 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feafc1a6f70 con 0x7feafc10e480 2024-09-12T03:33:01.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.603+0000 7feaee7fc640 1 -- 172.21.15.64:0/1147435745 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7feaf40051d0 con 0x7feafc10e480 2024-09-12T03:33:01.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.604+0000 7feb0682f740 1 Processor -- start 2024-09-12T03:33:01.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.604+0000 7feb0682f740 1 -- start start 2024-09-12T03:33:01.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.604+0000 7feb0682f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635b3a81af0 0x5635b3a81ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:01.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.604+0000 7feb0682f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635b3a82410 0x5635b3a8c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:01.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.604+0000 7feb0682f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635b3a8ccb0 0x5635b3a8f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:01.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.604+0000 7feb0682f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5635b39e62f0 con 0x5635b3a82410 2024-09-12T03:33:01.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.604+0000 7feb0682f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5635b398e270 con 0x5635b3a8ccb0 2024-09-12T03:33:01.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.604+0000 7feb0682f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5635b39e22b0 con 0x5635b3a81af0 2024-09-12T03:33:01.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.604+0000 7feb0175c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635b3a82410 0x5635b3a8c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:01.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.604+0000 7feaee7fc640 1 --2- 172.21.15.64:0/1147435745 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7feae80a2da0 0x7feae80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:01.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.604+0000 7feb0175c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635b3a82410 0x5635b3a8c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52558/0 (socket says 172.21.15.64:52558) 2024-09-12T03:33:01.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.604+0000 7feb0175c640 1 -- 172.21.15.64:0/3022210209 learned_addr learned my addr 172.21.15.64:0/3022210209 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:33:01.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.604+0000 7feb00f5b640 1 --2- 172.21.15.64:0/3022210209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635b3a81af0 0x5635b3a81ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:01.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.604+0000 7feb01f5d640 1 --2- 172.21.15.64:0/1147435745 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7feae80a2da0 0x7feae80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:01.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.604+0000 7feb00f5b640 1 --2- 172.21.15.64:0/3022210209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635b3a8ccb0 0x5635b3a8f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:01.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.605+0000 7feaee7fc640 1 -- 172.21.15.64:0/1147435745 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7feaf40cf9c0 con 0x7feafc10e480 2024-09-12T03:33:01.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.605+0000 7feb0175c640 1 -- 172.21.15.64:0/3022210209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635b3a81af0 msgr2=0x5635b3a81ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:01.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.605+0000 7feb0175c640 1 --2- 172.21.15.64:0/3022210209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635b3a81af0 0x5635b3a81ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:01.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.605+0000 7feb0175c640 1 -- 172.21.15.64:0/3022210209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635b3a8ccb0 msgr2=0x5635b3a8f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:01.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.605+0000 7feb0175c640 1 --2- 172.21.15.64:0/3022210209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635b3a8ccb0 0x5635b3a8f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:01.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.605+0000 7feb0175c640 1 -- 172.21.15.64:0/3022210209 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635b398d340 con 0x5635b3a82410 2024-09-12T03:33:01.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.605+0000 7feb0175c640 1 --2- 172.21.15.64:0/3022210209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635b3a82410 0x5635b3a8c770 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7feaf802a930 tx=0x7feaf8032cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8ef5435ed6fc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:01.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.605+0000 7feb01f5d640 1 --2- 172.21.15.64:0/1147435745 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7feae80a2da0 0x7feae80a5260 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7feaf4097f10 tx=0x7feaf4005850 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:01.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.605+0000 7feae77fe640 1 -- 172.21.15.64:0/3022210209 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf803f070 con 0x5635b3a82410 2024-09-12T03:33:01.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.605+0000 7feae77fe640 1 -- 172.21.15.64:0/3022210209 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feaf80316a0 con 0x5635b3a82410 2024-09-12T03:33:01.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.605+0000 7feae77fe640 1 -- 172.21.15.64:0/3022210209 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf8035070 con 0x5635b3a82410 2024-09-12T03:33:01.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.606+0000 7feb0682f740 1 -- 172.21.15.64:0/3022210209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635b3a82410 msgr2=0x5635b3a8c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:01.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.606+0000 7feb0682f740 1 --2- 172.21.15.64:0/3022210209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635b3a82410 0x5635b3a8c770 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7feaf802a930 tx=0x7feaf8032cc0 comp rx=0 tx=0).stop 2024-09-12T03:33:01.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.606+0000 7feb0682f740 1 -- 172.21.15.64:0/3022210209 shutdown_connections 2024-09-12T03:33:01.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.606+0000 7feb0682f740 1 --2- 172.21.15.64:0/3022210209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635b3a8ccb0 0x5635b3a8f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:01.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.606+0000 7feb0682f740 1 --2- 172.21.15.64:0/3022210209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635b3a82410 0x5635b3a8c770 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:01.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.606+0000 7feb0682f740 1 --2- 172.21.15.64:0/3022210209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635b3a81af0 0x5635b3a81ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:01.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.606+0000 7feb0682f740 1 -- 172.21.15.64:0/3022210209 >> 172.21.15.64:0/3022210209 conn(0x5635b3a7e060 msgr2=0x5635b3a7e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:01.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.606+0000 7feb0682f740 1 -- 172.21.15.64:0/3022210209 shutdown_connections 2024-09-12T03:33:01.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.606+0000 7feb0682f740 1 -- 172.21.15.64:0/3022210209 wait complete. 2024-09-12T03:33:01.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.606+0000 7feb0682f740 1 -- start start 2024-09-12T03:33:01.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.606+0000 7feb0682f740 1 Processor -- start 2024-09-12T03:33:01.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.607+0000 7feb0682f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635b3a81af0 0x5635b3a89ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:01.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.607+0000 7feb0682f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635b3a82410 0x5635b3a8a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:01.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.607+0000 7feb0682f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635b3a8ccb0 0x5635b3a8a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:01.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.607+0000 7feb0682f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5635b3a8f880 con 0x5635b3a82410 2024-09-12T03:33:01.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.607+0000 7feb0682f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5635b396f290 con 0x5635b3a8ccb0 2024-09-12T03:33:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.607+0000 7feb0682f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5635b3a8fa00 con 0x5635b3a81af0 2024-09-12T03:33:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.607+0000 7feb0682f740 10 client.0 _ll_register_callbacks cb 0x5635b39a0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.607+0000 7feb0175c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635b3a82410 0x5635b3a8a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.607+0000 7feb0175c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635b3a82410 0x5635b3a8a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52570/0 (socket says 172.21.15.64:52570) 2024-09-12T03:33:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.607+0000 7feb0175c640 1 -- 172.21.15.64:0/812787942 learned_addr learned my addr 172.21.15.64:0/812787942 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:33:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.607+0000 7feb00f5b640 1 --2- 172.21.15.64:0/812787942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635b3a81af0 0x5635b3a89ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.607+0000 7feb00f5b640 1 --2- 172.21.15.64:0/812787942 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635b3a8ccb0 0x5635b3a8a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.607+0000 7feb0175c640 1 -- 172.21.15.64:0/812787942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635b3a81af0 msgr2=0x5635b3a89ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.607+0000 7feb0175c640 1 --2- 172.21.15.64:0/812787942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635b3a81af0 0x5635b3a89ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.607+0000 7feb0175c640 1 -- 172.21.15.64:0/812787942 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635b3a8ccb0 msgr2=0x5635b3a8a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.607+0000 7feb0175c640 1 --2- 172.21.15.64:0/812787942 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635b3a8ccb0 0x5635b3a8a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.607+0000 7feb0175c640 1 -- 172.21.15.64:0/812787942 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635b36f9210 con 0x5635b3a82410 2024-09-12T03:33:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.608+0000 7feb0175c640 1 --2- 172.21.15.64:0/812787942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635b3a82410 0x5635b3a8a0e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7feaf802a450 tx=0x7feaf80050e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf149660c4e9ec3b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.608+0000 7feae4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:33:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.608+0000 7feae4ff9640 1 -- 172.21.15.64:0/812787942 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf804b060 con 0x5635b3a82410 2024-09-12T03:33:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.608+0000 7feae4ff9640 1 -- 172.21.15.64:0/812787942 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7feaf8004230 con 0x5635b3a82410 2024-09-12T03:33:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.608+0000 7feb0682f740 1 -- 172.21.15.64:0/812787942 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feaf80316a0 con 0x5635b3a82410 2024-09-12T03:33:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.608+0000 7feae4ff9640 1 -- 172.21.15.64:0/812787942 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7feaf8052050 con 0x5635b3a82410 2024-09-12T03:33:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.608+0000 7feb0682f740 10 client.15648 fetch_fsmap learned FSMap version 25 2024-09-12T03:33:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.608+0000 7feb0682f740 1 -- 172.21.15.64:0/812787942 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5635b37c5620 con 0x5635b3a82410 2024-09-12T03:33:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.608+0000 7feae4ff9640 1 -- 172.21.15.64:0/812787942 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feaf80316a0 con 0x5635b3a82410 2024-09-12T03:33:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.608+0000 7feae4ff9640 1 -- 172.21.15.64:0/812787942 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7feaf8003930 con 0x5635b3a82410 2024-09-12T03:33:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.608+0000 7feb0682f740 10 client.15648 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:33:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.608+0000 7feb0682f740 10 client.15648 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:33:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.608+0000 7feb0682f740 20 client.15648 populate_metadata read hostname 'smithi064' 2024-09-12T03:33:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.608+0000 7feb0682f740 1 --2- 172.21.15.64:0/812787942 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5635b3a9ca90 0x5635b3a9ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.608+0000 7feb0682f740 4 client.15648 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:33:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.608+0000 7feb0682f740 1 -- 172.21.15.64:0/812787942 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635b39f3a20 con 0x5635b3a9ca90 2024-09-12T03:33:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.609+0000 7feb00f5b640 1 --2- 172.21.15.64:0/812787942 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5635b3a9ca90 0x5635b3a9ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.609+0000 7feae4ff9640 10 client.15648 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:33:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.609+0000 7feb00f5b640 1 --2- 172.21.15.64:0/812787942 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5635b3a9ca90 0x5635b3a9ee80 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8ddea969f343188 server_cookie=622264c1f78212fc in_seq=0 out_seq=0 2024-09-12T03:33:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.610+0000 7feae4ff9640 1 -- 172.21.15.64:0/812787942 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5635b39f3a20 con 0x5635b3a9ca90 2024-09-12T03:33:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.610+0000 7feae4ff9640 10 client.15648 handle_command_reply: tid=0 2024-09-12T03:33:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.610+0000 7feb0682f740 1 client.15648 shutdown 2024-09-12T03:33:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.610+0000 7feb0682f740 1 -- 172.21.15.64:0/812787942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635b3a82410 msgr2=0x5635b3a8a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.610+0000 7feb0682f740 1 --2- 172.21.15.64:0/812787942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635b3a82410 0x5635b3a8a0e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7feaf802a450 tx=0x7feaf80050e0 comp rx=0 tx=0).stop 2024-09-12T03:33:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.610+0000 7feb0682f740 1 -- 172.21.15.64:0/812787942 shutdown_connections 2024-09-12T03:33:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.610+0000 7feb0682f740 1 --2- 172.21.15.64:0/812787942 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5635b3a9ca90 0x5635b3a9ee80 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.610+0000 7feb0682f740 1 --2- 172.21.15.64:0/812787942 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635b3a8ccb0 0x5635b3a8a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.610+0000 7feb0682f740 1 --2- 172.21.15.64:0/812787942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635b3a82410 0x5635b3a8a0e0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.610+0000 7feae4ff9640 0 client.15648 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:33:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.610+0000 7feb0682f740 1 --2- 172.21.15.64:0/812787942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635b3a81af0 0x5635b3a89ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.610+0000 7feb0682f740 1 -- 172.21.15.64:0/812787942 >> 172.21.15.64:0/812787942 conn(0x5635b3a7e060 msgr2=0x5635b3a83d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.610+0000 7feb0682f740 1 -- 172.21.15.64:0/812787942 shutdown_connections 2024-09-12T03:33:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.610+0000 7feb0682f740 1 -- 172.21.15.64:0/812787942 wait complete. 2024-09-12T03:33:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.610+0000 7feb0682f740 20 client.15648 trim_cache size 0 max 16384 2024-09-12T03:33:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.625+0000 7feb0682f740 1 Processor -- start 2024-09-12T03:33:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.625+0000 7feb0682f740 1 -- start start 2024-09-12T03:33:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.625+0000 7feb0682f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635b3a81af0 0x5635b3a8a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.625+0000 7feb0682f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635b3a82410 0x5635b3aa4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.625+0000 7feb0682f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635b3a8ccb0 0x5635b3aa4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.625+0000 7feb0682f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7feafc1b39d0 con 0x5635b3a81af0 2024-09-12T03:33:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.625+0000 7feb0682f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7feafc1b12a0 con 0x5635b3a8ccb0 2024-09-12T03:33:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.625+0000 7feb0682f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7feafc1af0f0 con 0x5635b3a82410 2024-09-12T03:33:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.626+0000 7feb00f5b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635b3a81af0 0x5635b3a8a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.626+0000 7feb00f5b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635b3a81af0 0x5635b3a8a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52576/0 (socket says 172.21.15.64:52576) 2024-09-12T03:33:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.626+0000 7feb00f5b640 1 -- 172.21.15.64:0/2127512226 learned_addr learned my addr 172.21.15.64:0/2127512226 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:33:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.626+0000 7feb0175c640 1 --2- 172.21.15.64:0/2127512226 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635b3a82410 0x5635b3aa4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.626+0000 7feb00f5b640 1 -- 172.21.15.64:0/2127512226 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635b3a82410 msgr2=0x5635b3aa4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.626+0000 7feb00f5b640 1 --2- 172.21.15.64:0/2127512226 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635b3a82410 0x5635b3aa4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.626+0000 7feb00f5b640 1 -- 172.21.15.64:0/2127512226 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635b3a8ccb0 msgr2=0x5635b3aa4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.626+0000 7feb00f5b640 1 --2- 172.21.15.64:0/2127512226 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635b3a8ccb0 0x5635b3aa4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.626+0000 7feb00f5b640 1 -- 172.21.15.64:0/2127512226 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635b3a8f3e0 con 0x5635b3a81af0 2024-09-12T03:33:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.626+0000 7feb0175c640 1 --2- 172.21.15.64:0/2127512226 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635b3a82410 0x5635b3aa4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:33:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.626+0000 7feb00f5b640 1 --2- 172.21.15.64:0/2127512226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635b3a81af0 0x5635b3a8a450 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7feaf0000c00 tx=0x7feaf000a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14c801885a99a1ed server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.626+0000 7feae6ffd640 1 -- 172.21.15.64:0/2127512226 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf0020410 con 0x5635b3a81af0 2024-09-12T03:33:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.626+0000 7feae6ffd640 1 -- 172.21.15.64:0/2127512226 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feaf0020a60 con 0x5635b3a81af0 2024-09-12T03:33:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.626+0000 7feae6ffd640 1 -- 172.21.15.64:0/2127512226 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf001ea00 con 0x5635b3a81af0 2024-09-12T03:33:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.627+0000 7feb0682f740 1 -- 172.21.15.64:0/2127512226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635b3a81af0 msgr2=0x5635b3a8a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.627+0000 7feb0682f740 1 --2- 172.21.15.64:0/2127512226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635b3a81af0 0x5635b3a8a450 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7feaf0000c00 tx=0x7feaf000a6c0 comp rx=0 tx=0).stop 2024-09-12T03:33:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.627+0000 7feb0682f740 1 -- 172.21.15.64:0/2127512226 shutdown_connections 2024-09-12T03:33:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.627+0000 7feb0682f740 1 --2- 172.21.15.64:0/2127512226 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635b3a8ccb0 0x5635b3aa4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.627+0000 7feb0682f740 1 --2- 172.21.15.64:0/2127512226 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635b3a82410 0x5635b3aa4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.627+0000 7feb0682f740 1 --2- 172.21.15.64:0/2127512226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635b3a81af0 0x5635b3a8a450 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.627+0000 7feb0682f740 1 -- 172.21.15.64:0/2127512226 >> 172.21.15.64:0/2127512226 conn(0x5635b3a7e060 msgr2=0x5635b3a8bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.627+0000 7feb0682f740 1 -- 172.21.15.64:0/2127512226 shutdown_connections 2024-09-12T03:33:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.627+0000 7feb0682f740 1 -- 172.21.15.64:0/2127512226 wait complete. 2024-09-12T03:33:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.627+0000 7feb0682f740 1 -- start start 2024-09-12T03:33:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.627+0000 7feb0682f740 1 Processor -- start 2024-09-12T03:33:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.628+0000 7feb0682f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635b3a81af0 0x5635b3a95230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.628+0000 7feb0682f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635b3a82410 0x5635b3a90550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.628+0000 7feb0682f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635b3a8ccb0 0x5635b3a90a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.628+0000 7feb0682f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7feafc1af760 con 0x5635b3a81af0 2024-09-12T03:33:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.628+0000 7feb0682f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7feafc1b0bd0 con 0x5635b3a8ccb0 2024-09-12T03:33:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.628+0000 7feb0682f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7feafc1afe30 con 0x5635b3a82410 2024-09-12T03:33:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.628+0000 7feb0682f740 10 client.0 _ll_register_callbacks cb 0x7feaf000aa10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.628+0000 7feb00f5b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635b3a81af0 0x5635b3a95230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.628+0000 7feb00f5b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635b3a81af0 0x5635b3a95230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52582/0 (socket says 172.21.15.64:52582) 2024-09-12T03:33:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.628+0000 7feb00f5b640 1 -- 172.21.15.64:0/1002811338 learned_addr learned my addr 172.21.15.64:0/1002811338 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:33:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.628+0000 7feb0175c640 1 --2- 172.21.15.64:0/1002811338 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635b3a82410 0x5635b3a90550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.628+0000 7feb00f5b640 1 --2- 172.21.15.64:0/1002811338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635b3a8ccb0 0x5635b3a90a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.628+0000 7feb00f5b640 1 -- 172.21.15.64:0/1002811338 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635b3a82410 msgr2=0x5635b3a90550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.628+0000 7feb00f5b640 1 --2- 172.21.15.64:0/1002811338 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635b3a82410 0x5635b3a90550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.628+0000 7feb00f5b640 1 -- 172.21.15.64:0/1002811338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635b3a8ccb0 msgr2=0x5635b3a90a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.628+0000 7feb00f5b640 1 --2- 172.21.15.64:0/1002811338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635b3a8ccb0 0x5635b3a90a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.628+0000 7feb00f5b640 1 -- 172.21.15.64:0/1002811338 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635b3a910d0 con 0x5635b3a81af0 2024-09-12T03:33:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.628+0000 7feb0175c640 1 --2- 172.21.15.64:0/1002811338 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635b3a82410 0x5635b3a90550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:33:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.629+0000 7feb00f5b640 1 --2- 172.21.15.64:0/1002811338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635b3a81af0 0x5635b3a95230 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7feaf0005890 tx=0x7feaf0021c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=878636815ab21ad8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.629+0000 7feae4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:33:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.629+0000 7feae4ff9640 1 -- 172.21.15.64:0/1002811338 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf0005650 con 0x5635b3a81af0 2024-09-12T03:33:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.629+0000 7feae4ff9640 1 -- 172.21.15.64:0/1002811338 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7feaf00046c0 con 0x5635b3a81af0 2024-09-12T03:33:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.629+0000 7feb0682f740 1 -- 172.21.15.64:0/1002811338 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feaf0020a60 con 0x5635b3a81af0 2024-09-12T03:33:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.629+0000 7feae4ff9640 1 -- 172.21.15.64:0/1002811338 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7feaf00048c0 con 0x5635b3a81af0 2024-09-12T03:33:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.629+0000 7feb0682f740 10 client.15654 fetch_fsmap learned FSMap version 25 2024-09-12T03:33:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.629+0000 7feb0682f740 1 -- 172.21.15.64:0/1002811338 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5635b3b07250 con 0x5635b3a81af0 2024-09-12T03:33:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.629+0000 7feae4ff9640 1 -- 172.21.15.64:0/1002811338 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feaf0020a60 con 0x5635b3a81af0 2024-09-12T03:33:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.629+0000 7feae4ff9640 1 -- 172.21.15.64:0/1002811338 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7feaf0029ab0 con 0x5635b3a81af0 2024-09-12T03:33:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.629+0000 7feb0682f740 10 client.15654 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:33:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.629+0000 7feb0682f740 10 client.15654 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:33:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.629+0000 7feb0682f740 20 client.15654 populate_metadata read hostname 'smithi064' 2024-09-12T03:33:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.630+0000 7feb0682f740 1 --2- 172.21.15.64:0/1002811338 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5635b3b07980 0x5635b3b09da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.630+0000 7feb0682f740 4 client.15654 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:33:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.630+0000 7feb0682f740 1 -- 172.21.15.64:0/1002811338 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5635b39edd70 con 0x5635b3b07980 2024-09-12T03:33:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.630+0000 7feb0175c640 1 --2- 172.21.15.64:0/1002811338 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5635b3b07980 0x5635b3b09da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.630+0000 7feb0175c640 1 --2- 172.21.15.64:0/1002811338 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5635b3b07980 0x5635b3b09da0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5c3d5ec6116725 server_cookie=db6486a65bf99129 in_seq=0 out_seq=0 2024-09-12T03:33:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.630+0000 7feae4ff9640 10 client.15654 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:33:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.644+0000 7feae4ff9640 1 -- 172.21.15.64:0/1002811338 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5635b39edd70 con 0x5635b3b07980 2024-09-12T03:33:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.644+0000 7feae4ff9640 10 client.15654 handle_command_reply: tid=0 2024-09-12T03:33:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.644+0000 7feb0682f740 1 client.15654 shutdown 2024-09-12T03:33:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.644+0000 7feb0682f740 1 -- 172.21.15.64:0/1002811338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635b3a81af0 msgr2=0x5635b3a95230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.644+0000 7feb0682f740 1 --2- 172.21.15.64:0/1002811338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635b3a81af0 0x5635b3a95230 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7feaf0005890 tx=0x7feaf0021c90 comp rx=0 tx=0).stop 2024-09-12T03:33:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.644+0000 7feb0682f740 1 -- 172.21.15.64:0/1002811338 shutdown_connections 2024-09-12T03:33:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.644+0000 7feb0682f740 1 --2- 172.21.15.64:0/1002811338 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5635b3b07980 0x5635b3b09da0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.644+0000 7feb0682f740 1 --2- 172.21.15.64:0/1002811338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635b3a8ccb0 0x5635b3a90a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.644+0000 7feb0682f740 1 --2- 172.21.15.64:0/1002811338 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635b3a82410 0x5635b3a90550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.644+0000 7feae4ff9640 0 client.15654 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:33:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.644+0000 7feb0682f740 1 --2- 172.21.15.64:0/1002811338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635b3a81af0 0x5635b3a95230 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.644+0000 7feb0682f740 1 -- 172.21.15.64:0/1002811338 >> 172.21.15.64:0/1002811338 conn(0x5635b3a7e060 msgr2=0x5635b3a7c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.644+0000 7feb0682f740 1 -- 172.21.15.64:0/1002811338 shutdown_connections 2024-09-12T03:33:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.644+0000 7feb0682f740 1 -- 172.21.15.64:0/1002811338 wait complete. 2024-09-12T03:33:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.644+0000 7feb0682f740 20 client.15654 trim_cache size 0 max 16384 2024-09-12T03:33:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:33:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:33:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:33:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:33:01.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.645+0000 7feb039c0640 1 -- 172.21.15.64:0/1147435745 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7feae80a2da0 msgr2=0x7feae80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:01.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.645+0000 7feb039c0640 1 --2- 172.21.15.64:0/1147435745 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7feae80a2da0 0x7feae80a5260 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7feaf4097f10 tx=0x7feaf4005850 comp rx=0 tx=0).stop 2024-09-12T03:33:01.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.645+0000 7feb039c0640 1 -- 172.21.15.64:0/1147435745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feafc10e480 msgr2=0x7feafc19ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:01.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.645+0000 7feb039c0640 1 --2- 172.21.15.64:0/1147435745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feafc10e480 0x7feafc19ecc0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7feaf400e9d0 tx=0x7feaf400edd0 comp rx=0 tx=0).stop 2024-09-12T03:33:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.645+0000 7feb039c0640 1 -- 172.21.15.64:0/1147435745 shutdown_connections 2024-09-12T03:33:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.645+0000 7feb039c0640 1 --2- 172.21.15.64:0/1147435745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feafc10e480 0x7feafc19ecc0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.645+0000 7feb039c0640 1 --2- 172.21.15.64:0/1147435745 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7feae80a2da0 0x7feae80a5260 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.645+0000 7feb039c0640 1 --2- 172.21.15.64:0/1147435745 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feafc101280 0x7feafc19a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.645+0000 7feb039c0640 1 --2- 172.21.15.64:0/1147435745 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feafc1008d0 0x7feafc19a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.645+0000 7feb039c0640 1 -- 172.21.15.64:0/1147435745 >> 172.21.15.64:0/1147435745 conn(0x7feafc0fc6b0 msgr2=0x7feafc0feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.645+0000 7feb039c0640 1 -- 172.21.15.64:0/1147435745 shutdown_connections 2024-09-12T03:33:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:01.645+0000 7feb039c0640 1 -- 172.21.15.64:0/1147435745 wait complete. 2024-09-12T03:33:03.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:33:02 smithi064 ceph-mon[30952]: pgmap v116: 129 pgs: 129 active+clean; 4.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 64 KiB/s wr, 35 op/s 2024-09-12T03:33:03.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:33:02 smithi115 ceph-mon[34801]: pgmap v116: 129 pgs: 129 active+clean; 4.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 64 KiB/s wr, 35 op/s 2024-09-12T03:33:03.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:33:02 smithi195 ceph-mon[34786]: pgmap v116: 129 pgs: 129 active+clean; 4.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 64 KiB/s wr, 35 op/s 2024-09-12T03:33:05.021 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:33:04 smithi064 ceph-mon[30952]: pgmap v117: 129 pgs: 129 active+clean; 4.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 50 KiB/s wr, 27 op/s 2024-09-12T03:33:05.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:33:04 smithi115 ceph-mon[34801]: pgmap v117: 129 pgs: 129 active+clean; 4.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 50 KiB/s wr, 27 op/s 2024-09-12T03:33:05.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:33:04 smithi195 ceph-mon[34786]: pgmap v117: 129 pgs: 129 active+clean; 4.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 50 KiB/s wr, 27 op/s 2024-09-12T03:33:07.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:33:06 smithi064 ceph-mon[30952]: pgmap v118: 129 pgs: 129 active+clean; 4.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 50 KiB/s wr, 27 op/s 2024-09-12T03:33:07.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:33:06 smithi115 ceph-mon[34801]: pgmap v118: 129 pgs: 129 active+clean; 4.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 50 KiB/s wr, 27 op/s 2024-09-12T03:33:07.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:33:06 smithi195 ceph-mon[34786]: pgmap v118: 129 pgs: 129 active+clean; 4.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 50 KiB/s wr, 27 op/s 2024-09-12T03:33:09.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:33:08 smithi064 ceph-mon[30952]: pgmap v119: 129 pgs: 129 active+clean; 4.8 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 62 KiB/s wr, 32 op/s 2024-09-12T03:33:09.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:33:08 smithi115 ceph-mon[34801]: pgmap v119: 129 pgs: 129 active+clean; 4.8 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 62 KiB/s wr, 32 op/s 2024-09-12T03:33:09.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:33:08 smithi195 ceph-mon[34786]: pgmap v119: 129 pgs: 129 active+clean; 4.8 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 62 KiB/s wr, 32 op/s 2024-09-12T03:33:11.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:33:10 smithi064 ceph-mon[30952]: pgmap v120: 129 pgs: 129 active+clean; 4.8 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 52 KiB/s wr, 27 op/s 2024-09-12T03:33:11.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:33:10 smithi115 ceph-mon[34801]: pgmap v120: 129 pgs: 129 active+clean; 4.8 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 52 KiB/s wr, 27 op/s 2024-09-12T03:33:11.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:33:10 smithi195 ceph-mon[34786]: pgmap v120: 129 pgs: 129 active+clean; 4.8 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 52 KiB/s wr, 27 op/s 2024-09-12T03:33:13.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:33:12 smithi064 ceph-mon[30952]: pgmap v121: 129 pgs: 129 active+clean; 4.9 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 65 KiB/s wr, 33 op/s 2024-09-12T03:33:13.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:33:12 smithi115 ceph-mon[34801]: pgmap v121: 129 pgs: 129 active+clean; 4.9 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 65 KiB/s wr, 33 op/s 2024-09-12T03:33:13.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:33:12 smithi195 ceph-mon[34786]: pgmap v121: 129 pgs: 129 active+clean; 4.9 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 65 KiB/s wr, 33 op/s 2024-09-12T03:33:14.746 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.746+0000 7f62bb1e8640 1 -- 172.21.15.64:0/1248896228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b4107b70 msgr2=0x7f62b4107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:14.746 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.746+0000 7f62bb1e8640 1 --2- 172.21.15.64:0/1248896228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b4107b70 0x7f62b4107f50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f62a4009930 tx=0x7f62a402a420 comp rx=0 tx=0).stop 2024-09-12T03:33:14.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.746+0000 7f62bb1e8640 1 -- 172.21.15.64:0/1248896228 shutdown_connections 2024-09-12T03:33:14.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.746+0000 7f62bb1e8640 1 --2- 172.21.15.64:0/1248896228 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62b4102470 0x7f62b410e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:14.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.746+0000 7f62bb1e8640 1 --2- 172.21.15.64:0/1248896228 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62b4101ad0 0x7f62b4101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:14.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.746+0000 7f62bb1e8640 1 --2- 172.21.15.64:0/1248896228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b4107b70 0x7f62b4107f50 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:14.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.746+0000 7f62bb1e8640 1 -- 172.21.15.64:0/1248896228 >> 172.21.15.64:0/1248896228 conn(0x7f62b40fd7d0 msgr2=0x7f62b40ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:14.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.746+0000 7f62bb1e8640 1 -- 172.21.15.64:0/1248896228 shutdown_connections 2024-09-12T03:33:14.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.746+0000 7f62bb1e8640 1 -- 172.21.15.64:0/1248896228 wait complete. 2024-09-12T03:33:14.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.747+0000 7f62bb1e8640 1 Processor -- start 2024-09-12T03:33:14.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.747+0000 7f62bb1e8640 1 -- start start 2024-09-12T03:33:14.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.747+0000 7f62bb1e8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62b4101ad0 0x7f62b4198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:14.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.747+0000 7f62bb1e8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62b4102470 0x7f62b41987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:14.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.747+0000 7f62bb1e8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b4107b70 0x7f62b419cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.747+0000 7f62bb1e8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f62b4111e20 con 0x7f62b4107b70 2024-09-12T03:33:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.747+0000 7f62bb1e8640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f62b4111ca0 con 0x7f62b4101ad0 2024-09-12T03:33:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.747+0000 7f62bb1e8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f62b4111fa0 con 0x7f62b4102470 2024-09-12T03:33:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.747+0000 7f62b9785640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b4107b70 0x7f62b419cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.747+0000 7f62b9785640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b4107b70 0x7f62b419cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51838/0 (socket says 172.21.15.64:51838) 2024-09-12T03:33:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.747+0000 7f62b9785640 1 -- 172.21.15.64:0/359976613 learned_addr learned my addr 172.21.15.64:0/359976613 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:33:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.747+0000 7f62b9785640 1 -- 172.21.15.64:0/359976613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62b4102470 msgr2=0x7f62b41987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.747+0000 7f62b9785640 1 --2- 172.21.15.64:0/359976613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62b4102470 0x7f62b41987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.747+0000 7f62b9785640 1 -- 172.21.15.64:0/359976613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62b4101ad0 msgr2=0x7f62b4198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.747+0000 7f62b8f84640 1 --2- 172.21.15.64:0/359976613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62b4101ad0 0x7f62b4198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.747+0000 7f62b9785640 1 --2- 172.21.15.64:0/359976613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62b4101ad0 0x7f62b4198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.747+0000 7f62b9785640 1 -- 172.21.15.64:0/359976613 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b419d250 con 0x7f62b4107b70 2024-09-12T03:33:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.747+0000 7f62b8f84640 1 --2- 172.21.15.64:0/359976613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62b4101ad0 0x7f62b4198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:33:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.748+0000 7f62b9785640 1 --2- 172.21.15.64:0/359976613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b4107b70 0x7f62b419cb20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f62ac00eb10 tx=0x7f62ac00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.748+0000 7f62b1ffb640 1 -- 172.21.15.64:0/359976613 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac004b20 con 0x7f62b4107b70 2024-09-12T03:33:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.748+0000 7f62bb1e8640 1 -- 172.21.15.64:0/359976613 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62b419d540 con 0x7f62b4107b70 2024-09-12T03:33:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.748+0000 7f62b1ffb640 1 -- 172.21.15.64:0/359976613 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62ac004cc0 con 0x7f62b4107b70 2024-09-12T03:33:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.748+0000 7f62b1ffb640 1 -- 172.21.15.64:0/359976613 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac010690 con 0x7f62b4107b70 2024-09-12T03:33:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.748+0000 7f62bb1e8640 1 -- 172.21.15.64:0/359976613 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62b41a4dd0 con 0x7f62b4107b70 2024-09-12T03:33:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.751+0000 7f62b1ffb640 1 -- 172.21.15.64:0/359976613 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f62ac005020 con 0x7f62b4107b70 2024-09-12T03:33:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.751+0000 7f62be05e740 1 Processor -- start 2024-09-12T03:33:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.751+0000 7f62be05e740 1 -- start start 2024-09-12T03:33:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.752+0000 7f62be05e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e61ccdaf0 0x557e61ccded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.752+0000 7f62be05e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e61cce410 0x557e61cd8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.752+0000 7f62be05e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e61cd8cb0 0x557e61cdb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.752+0000 7f62be05e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x557e61c322f0 con 0x557e61cd8cb0 2024-09-12T03:33:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.752+0000 7f62be05e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557e61bda270 con 0x557e61ccdaf0 2024-09-12T03:33:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.752+0000 7f62be05e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557e61c2e2b0 con 0x557e61cce410 2024-09-12T03:33:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.752+0000 7f62b3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e61cd8cb0 0x557e61cdb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.752+0000 7f62b3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e61cd8cb0 0x557e61cdb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51848/0 (socket says 172.21.15.64:51848) 2024-09-12T03:33:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.752+0000 7f62b3fff640 1 -- 172.21.15.64:0/3353724861 learned_addr learned my addr 172.21.15.64:0/3353724861 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:33:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.752+0000 7f62b8f84640 1 --2- 172.21.15.64:0/3353724861 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e61cce410 0x557e61cd8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.752+0000 7f62b3fff640 1 --2- 172.21.15.64:0/3353724861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e61ccdaf0 0x557e61ccded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.752+0000 7f62b1ffb640 1 --2- 172.21.15.64:0/359976613 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f629c0a2da0 0x7f629c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.752+0000 7f62b3fff640 1 -- 172.21.15.64:0/3353724861 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e61cce410 msgr2=0x557e61cd8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.752+0000 7f62b3fff640 1 --2- 172.21.15.64:0/3353724861 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e61cce410 0x557e61cd8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.752+0000 7f62b3fff640 1 -- 172.21.15.64:0/3353724861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e61ccdaf0 msgr2=0x557e61ccded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.752+0000 7f62b3fff640 1 --2- 172.21.15.64:0/3353724861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e61ccdaf0 0x557e61ccded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.752+0000 7f62b3fff640 1 -- 172.21.15.64:0/3353724861 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e61bd9340 con 0x557e61cd8cb0 2024-09-12T03:33:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.752+0000 7f62b9785640 1 --2- 172.21.15.64:0/359976613 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f629c0a2da0 0x7f629c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.752+0000 7f62b1ffb640 1 -- 172.21.15.64:0/359976613 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f62ac0cfff0 con 0x7f62b4107b70 2024-09-12T03:33:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.753+0000 7f62b3fff640 1 --2- 172.21.15.64:0/3353724861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e61cd8cb0 0x557e61cdb0a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f62a80127c0 tx=0x7f62a800d3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ada31f022a07bbe0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.753+0000 7f629affd640 1 -- 172.21.15.64:0/3353724861 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a8004040 con 0x557e61cd8cb0 2024-09-12T03:33:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.753+0000 7f629affd640 1 -- 172.21.15.64:0/3353724861 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62a80041e0 con 0x557e61cd8cb0 2024-09-12T03:33:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.753+0000 7f629affd640 1 -- 172.21.15.64:0/3353724861 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a8003970 con 0x557e61cd8cb0 2024-09-12T03:33:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.754+0000 7f62b9785640 1 --2- 172.21.15.64:0/359976613 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f629c0a2da0 0x7f629c0a5260 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f62ac098560 tx=0x7f62ac00efd0 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.754+0000 7f62be05e740 1 -- 172.21.15.64:0/3353724861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e61cd8cb0 msgr2=0x557e61cdb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.754+0000 7f62be05e740 1 --2- 172.21.15.64:0/3353724861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e61cd8cb0 0x557e61cdb0a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f62a80127c0 tx=0x7f62a800d3c0 comp rx=0 tx=0).stop 2024-09-12T03:33:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.754+0000 7f62be05e740 1 -- 172.21.15.64:0/3353724861 shutdown_connections 2024-09-12T03:33:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.754+0000 7f62be05e740 1 --2- 172.21.15.64:0/3353724861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e61cd8cb0 0x557e61cdb0a0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.754+0000 7f62be05e740 1 --2- 172.21.15.64:0/3353724861 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e61cce410 0x557e61cd8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.754+0000 7f62be05e740 1 --2- 172.21.15.64:0/3353724861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e61ccdaf0 0x557e61ccded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.754+0000 7f62be05e740 1 -- 172.21.15.64:0/3353724861 >> 172.21.15.64:0/3353724861 conn(0x557e61cca060 msgr2=0x557e61cca440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.754+0000 7f62be05e740 1 -- 172.21.15.64:0/3353724861 shutdown_connections 2024-09-12T03:33:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.754+0000 7f62be05e740 1 -- 172.21.15.64:0/3353724861 wait complete. 2024-09-12T03:33:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.754+0000 7f62be05e740 1 -- start start 2024-09-12T03:33:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.754+0000 7f62be05e740 1 Processor -- start 2024-09-12T03:33:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.755+0000 7f62be05e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e61ccdaf0 0x557e61cd5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.755+0000 7f62be05e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e61cce410 0x557e61cd6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.755+0000 7f62be05e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e61cd8cb0 0x557e61cd6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.755+0000 7f62be05e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x557e61cdb880 con 0x557e61cd8cb0 2024-09-12T03:33:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.755+0000 7f62be05e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557e61bbb290 con 0x557e61ccdaf0 2024-09-12T03:33:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.755+0000 7f62be05e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557e61cdba00 con 0x557e61cce410 2024-09-12T03:33:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.755+0000 7f62be05e740 10 client.0 _ll_register_callbacks cb 0x557e61bec140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.755+0000 7f62b3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e61cd8cb0 0x557e61cd6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.755+0000 7f62b3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e61cd8cb0 0x557e61cd6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51856/0 (socket says 172.21.15.64:51856) 2024-09-12T03:33:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.755+0000 7f62b3fff640 1 -- 172.21.15.64:0/4042814199 learned_addr learned my addr 172.21.15.64:0/4042814199 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:33:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.755+0000 7f62b8f84640 1 --2- 172.21.15.64:0/4042814199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e61cce410 0x557e61cd6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.755+0000 7f62b3fff640 1 --2- 172.21.15.64:0/4042814199 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e61ccdaf0 0x557e61cd5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.755+0000 7f62b3fff640 1 -- 172.21.15.64:0/4042814199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e61cce410 msgr2=0x557e61cd6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.755+0000 7f62b3fff640 1 --2- 172.21.15.64:0/4042814199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e61cce410 0x557e61cd6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.755+0000 7f62b3fff640 1 -- 172.21.15.64:0/4042814199 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e61ccdaf0 msgr2=0x557e61cd5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.755+0000 7f62b3fff640 1 --2- 172.21.15.64:0/4042814199 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e61ccdaf0 0x557e61cd5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.755+0000 7f62b3fff640 1 -- 172.21.15.64:0/4042814199 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e61945210 con 0x557e61cd8cb0 2024-09-12T03:33:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.755+0000 7f62b3fff640 1 --2- 172.21.15.64:0/4042814199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e61cd8cb0 0x557e61cd6680 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f62a8003650 tx=0x7f62a800af00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46611477858a5fd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.755+0000 7f6293fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:33:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.756+0000 7f6293fff640 1 -- 172.21.15.64:0/4042814199 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a8012e00 con 0x557e61cd8cb0 2024-09-12T03:33:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.756+0000 7f62be05e740 1 -- 172.21.15.64:0/4042814199 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62a80041e0 con 0x557e61cd8cb0 2024-09-12T03:33:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.756+0000 7f6293fff640 1 -- 172.21.15.64:0/4042814199 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f62a8012070 con 0x557e61cd8cb0 2024-09-12T03:33:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.756+0000 7f6293fff640 1 -- 172.21.15.64:0/4042814199 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f62a8012270 con 0x557e61cd8cb0 2024-09-12T03:33:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.756+0000 7f62be05e740 10 client.15666 fetch_fsmap learned FSMap version 25 2024-09-12T03:33:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.756+0000 7f62be05e740 1 -- 172.21.15.64:0/4042814199 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557e61a11620 con 0x557e61cd8cb0 2024-09-12T03:33:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.756+0000 7f6293fff640 1 -- 172.21.15.64:0/4042814199 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62a80041e0 con 0x557e61cd8cb0 2024-09-12T03:33:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.756+0000 7f6293fff640 1 -- 172.21.15.64:0/4042814199 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f62a8005440 con 0x557e61cd8cb0 2024-09-12T03:33:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.756+0000 7f62be05e740 10 client.15666 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:33:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.756+0000 7f62be05e740 10 client.15666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:33:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.756+0000 7f62be05e740 20 client.15666 populate_metadata read hostname 'smithi064' 2024-09-12T03:33:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.756+0000 7f62be05e740 1 --2- 172.21.15.64:0/4042814199 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x557e61ce8c80 0x557e61ceb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.756+0000 7f62be05e740 4 client.15666 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:33:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.756+0000 7f62be05e740 1 -- 172.21.15.64:0/4042814199 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e61c3fa20 con 0x557e61ce8c80 2024-09-12T03:33:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.757+0000 7f62b8f84640 1 --2- 172.21.15.64:0/4042814199 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x557e61ce8c80 0x557e61ceb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.757+0000 7f62b8f84640 1 --2- 172.21.15.64:0/4042814199 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x557e61ce8c80 0x557e61ceb070 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a57482b47292dc95 server_cookie=cf071a207a6f93be in_seq=0 out_seq=0 2024-09-12T03:33:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.757+0000 7f6293fff640 10 client.15666 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:33:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.758+0000 7f6293fff640 1 -- 172.21.15.64:0/4042814199 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557e61c3fa20 con 0x557e61ce8c80 2024-09-12T03:33:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.758+0000 7f6293fff640 10 client.15666 handle_command_reply: tid=0 2024-09-12T03:33:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.758+0000 7f62be05e740 1 client.15666 shutdown 2024-09-12T03:33:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.758+0000 7f62be05e740 1 -- 172.21.15.64:0/4042814199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e61cd8cb0 msgr2=0x557e61cd6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.758+0000 7f62be05e740 1 --2- 172.21.15.64:0/4042814199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e61cd8cb0 0x557e61cd6680 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f62a8003650 tx=0x7f62a800af00 comp rx=0 tx=0).stop 2024-09-12T03:33:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.758+0000 7f62be05e740 1 -- 172.21.15.64:0/4042814199 shutdown_connections 2024-09-12T03:33:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.758+0000 7f62be05e740 1 --2- 172.21.15.64:0/4042814199 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x557e61ce8c80 0x557e61ceb070 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.758+0000 7f62be05e740 1 --2- 172.21.15.64:0/4042814199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e61cd8cb0 0x557e61cd6680 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.758+0000 7f62be05e740 1 --2- 172.21.15.64:0/4042814199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e61cce410 0x557e61cd6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.758+0000 7f6293fff640 0 client.15666 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:33:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.758+0000 7f62be05e740 1 --2- 172.21.15.64:0/4042814199 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e61ccdaf0 0x557e61cd5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.758+0000 7f62be05e740 1 -- 172.21.15.64:0/4042814199 >> 172.21.15.64:0/4042814199 conn(0x557e61cca060 msgr2=0x557e61ccfd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.758+0000 7f62be05e740 1 -- 172.21.15.64:0/4042814199 shutdown_connections 2024-09-12T03:33:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.758+0000 7f62be05e740 1 -- 172.21.15.64:0/4042814199 wait complete. 2024-09-12T03:33:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.758+0000 7f62be05e740 20 client.15666 trim_cache size 0 max 16384 2024-09-12T03:33:14.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.773+0000 7f62be05e740 1 Processor -- start 2024-09-12T03:33:14.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.773+0000 7f62be05e740 1 -- start start 2024-09-12T03:33:14.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.773+0000 7f62be05e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e61ccdaf0 0x557e61cd44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:14.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.773+0000 7f62be05e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e61cce410 0x557e61cd64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:14.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.773+0000 7f62be05e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e61cd8cb0 0x557e61cf07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:14.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.773+0000 7f62be05e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f62b41b1950 con 0x557e61cce410 2024-09-12T03:33:14.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.773+0000 7f62be05e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f62b41af220 con 0x557e61ccdaf0 2024-09-12T03:33:14.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.773+0000 7f62be05e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f62b41ad070 con 0x557e61cd8cb0 2024-09-12T03:33:14.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.773+0000 7f62b8f84640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e61cce410 0x557e61cd64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:14.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.774+0000 7f62b8f84640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e61cce410 0x557e61cd64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51866/0 (socket says 172.21.15.64:51866) 2024-09-12T03:33:14.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.774+0000 7f62b8f84640 1 -- 172.21.15.64:0/163145087 learned_addr learned my addr 172.21.15.64:0/163145087 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:33:14.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.774+0000 7f62b3fff640 1 --2- 172.21.15.64:0/163145087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e61ccdaf0 0x557e61cd44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.774+0000 7f62b3fff640 1 --2- 172.21.15.64:0/163145087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e61cd8cb0 0x557e61cf07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.774+0000 7f62b8f84640 1 -- 172.21.15.64:0/163145087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e61cd8cb0 msgr2=0x557e61cf07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.774+0000 7f62b8f84640 1 --2- 172.21.15.64:0/163145087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e61cd8cb0 0x557e61cf07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.774+0000 7f62b8f84640 1 -- 172.21.15.64:0/163145087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e61ccdaf0 msgr2=0x557e61cd44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.774+0000 7f62b8f84640 1 --2- 172.21.15.64:0/163145087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e61ccdaf0 0x557e61cd44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.774+0000 7f62b8f84640 1 -- 172.21.15.64:0/163145087 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e61cc42f0 con 0x557e61cce410 2024-09-12T03:33:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.774+0000 7f62b3fff640 1 --2- 172.21.15.64:0/163145087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e61cd8cb0 0x557e61cf07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:33:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.774+0000 7f62b8f84640 1 --2- 172.21.15.64:0/163145087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e61cce410 0x557e61cd64b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f62a402a940 tx=0x7f62a4005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce495f8d2d5380c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.774+0000 7f629a7fc640 1 -- 172.21.15.64:0/163145087 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a4035040 con 0x557e61cce410 2024-09-12T03:33:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.774+0000 7f629a7fc640 1 -- 172.21.15.64:0/163145087 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62a4049070 con 0x557e61cce410 2024-09-12T03:33:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.774+0000 7f629a7fc640 1 -- 172.21.15.64:0/163145087 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a4004da0 con 0x557e61cce410 2024-09-12T03:33:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.775+0000 7f62be05e740 1 -- 172.21.15.64:0/163145087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e61cce410 msgr2=0x557e61cd64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.775+0000 7f62be05e740 1 --2- 172.21.15.64:0/163145087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e61cce410 0x557e61cd64b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f62a402a940 tx=0x7f62a4005b50 comp rx=0 tx=0).stop 2024-09-12T03:33:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.775+0000 7f62be05e740 1 -- 172.21.15.64:0/163145087 shutdown_connections 2024-09-12T03:33:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.775+0000 7f62be05e740 1 --2- 172.21.15.64:0/163145087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e61cd8cb0 0x557e61cf07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.775+0000 7f62be05e740 1 --2- 172.21.15.64:0/163145087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e61cce410 0x557e61cd64b0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.775+0000 7f62be05e740 1 --2- 172.21.15.64:0/163145087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e61ccdaf0 0x557e61cd44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.775+0000 7f62be05e740 1 -- 172.21.15.64:0/163145087 >> 172.21.15.64:0/163145087 conn(0x557e61cca060 msgr2=0x557e61cd7c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.775+0000 7f62be05e740 1 -- 172.21.15.64:0/163145087 shutdown_connections 2024-09-12T03:33:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.775+0000 7f62be05e740 1 -- 172.21.15.64:0/163145087 wait complete. 2024-09-12T03:33:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.775+0000 7f62be05e740 1 -- start start 2024-09-12T03:33:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.775+0000 7f62be05e740 1 Processor -- start 2024-09-12T03:33:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.776+0000 7f62be05e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e61ccdaf0 0x557e61d4baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.776+0000 7f62be05e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e61cce410 0x557e61d4c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.776+0000 7f62be05e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e61cd8cb0 0x557e61cdc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.776+0000 7f62be05e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f62b41ad6e0 con 0x557e61cce410 2024-09-12T03:33:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.776+0000 7f62be05e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f62b41aeb50 con 0x557e61cd8cb0 2024-09-12T03:33:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.776+0000 7f62be05e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f62b41addb0 con 0x557e61ccdaf0 2024-09-12T03:33:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.776+0000 7f62be05e740 10 client.0 _ll_register_callbacks cb 0x7f62a800dd80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.776+0000 7f62b8f84640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e61cce410 0x557e61d4c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.776+0000 7f62b8f84640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e61cce410 0x557e61d4c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51876/0 (socket says 172.21.15.64:51876) 2024-09-12T03:33:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.776+0000 7f62b8f84640 1 -- 172.21.15.64:0/2586220015 learned_addr learned my addr 172.21.15.64:0/2586220015 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:33:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.776+0000 7f62b3fff640 1 --2- 172.21.15.64:0/2586220015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e61cd8cb0 0x557e61cdc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.776+0000 7f62b3fff640 1 --2- 172.21.15.64:0/2586220015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e61ccdaf0 0x557e61d4baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.776+0000 7f62b8f84640 1 -- 172.21.15.64:0/2586220015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e61ccdaf0 msgr2=0x557e61d4baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.776+0000 7f62b8f84640 1 --2- 172.21.15.64:0/2586220015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e61ccdaf0 0x557e61d4baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.776+0000 7f62b8f84640 1 -- 172.21.15.64:0/2586220015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e61cd8cb0 msgr2=0x557e61cdc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.776+0000 7f62b8f84640 1 --2- 172.21.15.64:0/2586220015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e61cd8cb0 0x557e61cdc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.776+0000 7f62b8f84640 1 -- 172.21.15.64:0/2586220015 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e61cdca10 con 0x557e61cce410 2024-09-12T03:33:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.776+0000 7f62b3fff640 1 --2- 172.21.15.64:0/2586220015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e61ccdaf0 0x557e61d4baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:33:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.777+0000 7f62b8f84640 1 --2- 172.21.15.64:0/2586220015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e61cce410 0x557e61d4c030 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f62a4005ab0 tx=0x7f62a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a3d96fbfa11eb98 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.777+0000 7f6293fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:33:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.777+0000 7f6293fff640 1 -- 172.21.15.64:0/2586220015 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a4035040 con 0x557e61cce410 2024-09-12T03:33:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.777+0000 7f6293fff640 1 -- 172.21.15.64:0/2586220015 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f62a4044040 con 0x557e61cce410 2024-09-12T03:33:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.777+0000 7f62be05e740 1 -- 172.21.15.64:0/2586220015 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62a4049070 con 0x557e61cce410 2024-09-12T03:33:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.777+0000 7f6293fff640 1 -- 172.21.15.64:0/2586220015 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f62a40316b0 con 0x557e61cce410 2024-09-12T03:33:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.777+0000 7f62be05e740 10 client.15672 fetch_fsmap learned FSMap version 25 2024-09-12T03:33:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.777+0000 7f62be05e740 1 -- 172.21.15.64:0/2586220015 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557e61d532e0 con 0x557e61cce410 2024-09-12T03:33:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.777+0000 7f6293fff640 1 -- 172.21.15.64:0/2586220015 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62a4049070 con 0x557e61cce410 2024-09-12T03:33:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.777+0000 7f6293fff640 1 -- 172.21.15.64:0/2586220015 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f62a404f9c0 con 0x557e61cce410 2024-09-12T03:33:14.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.777+0000 7f62be05e740 10 client.15672 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:33:14.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.777+0000 7f62be05e740 10 client.15672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:33:14.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.777+0000 7f62be05e740 20 client.15672 populate_metadata read hostname 'smithi064' 2024-09-12T03:33:14.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.777+0000 7f62be05e740 1 --2- 172.21.15.64:0/2586220015 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x557e61d539f0 0x557e61d55e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:14.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.777+0000 7f62be05e740 4 client.15672 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:33:14.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.777+0000 7f62be05e740 1 -- 172.21.15.64:0/2586220015 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557e61c39d70 con 0x557e61d539f0 2024-09-12T03:33:14.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.778+0000 7f62b3fff640 1 --2- 172.21.15.64:0/2586220015 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x557e61d539f0 0x557e61d55e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:14.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.778+0000 7f6293fff640 10 client.15672 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:33:14.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.778+0000 7f62b3fff640 1 --2- 172.21.15.64:0/2586220015 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x557e61d539f0 0x557e61d55e10 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=170a41e0e8b0dcd server_cookie=5fd3431bce6a22fc in_seq=0 out_seq=0 2024-09-12T03:33:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.792+0000 7f6293fff640 1 -- 172.21.15.64:0/2586220015 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557e61c39d70 con 0x557e61d539f0 2024-09-12T03:33:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.792+0000 7f6293fff640 10 client.15672 handle_command_reply: tid=0 2024-09-12T03:33:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.792+0000 7f62be05e740 1 client.15672 shutdown 2024-09-12T03:33:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.792+0000 7f62be05e740 1 -- 172.21.15.64:0/2586220015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e61cce410 msgr2=0x557e61d4c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.792+0000 7f62be05e740 1 --2- 172.21.15.64:0/2586220015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e61cce410 0x557e61d4c030 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f62a4005ab0 tx=0x7f62a40046c0 comp rx=0 tx=0).stop 2024-09-12T03:33:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.792+0000 7f62be05e740 1 -- 172.21.15.64:0/2586220015 shutdown_connections 2024-09-12T03:33:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.792+0000 7f62be05e740 1 --2- 172.21.15.64:0/2586220015 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x557e61d539f0 0x557e61d55e10 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.792+0000 7f62be05e740 1 --2- 172.21.15.64:0/2586220015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557e61cd8cb0 0x557e61cdc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.792+0000 7f62be05e740 1 --2- 172.21.15.64:0/2586220015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e61cce410 0x557e61d4c030 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.792+0000 7f6293fff640 0 client.15672 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:33:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.792+0000 7f62be05e740 1 --2- 172.21.15.64:0/2586220015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557e61ccdaf0 0x557e61d4baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.792+0000 7f62be05e740 1 -- 172.21.15.64:0/2586220015 >> 172.21.15.64:0/2586220015 conn(0x557e61cca060 msgr2=0x557e61cd7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.793+0000 7f62be05e740 1 -- 172.21.15.64:0/2586220015 shutdown_connections 2024-09-12T03:33:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.793+0000 7f62be05e740 1 -- 172.21.15.64:0/2586220015 wait complete. 2024-09-12T03:33:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.793+0000 7f62be05e740 20 client.15672 trim_cache size 0 max 16384 2024-09-12T03:33:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:33:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:33:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:33:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:33:14.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.793+0000 7f62bb1e8640 1 -- 172.21.15.64:0/359976613 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f629c0a2da0 msgr2=0x7f629c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:14.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.793+0000 7f62bb1e8640 1 --2- 172.21.15.64:0/359976613 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f629c0a2da0 0x7f629c0a5260 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f62ac098560 tx=0x7f62ac00efd0 comp rx=0 tx=0).stop 2024-09-12T03:33:14.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.793+0000 7f62bb1e8640 1 -- 172.21.15.64:0/359976613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b4107b70 msgr2=0x7f62b419cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:14.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.793+0000 7f62bb1e8640 1 --2- 172.21.15.64:0/359976613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b4107b70 0x7f62b419cb20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f62ac00eb10 tx=0x7f62ac00c2d0 comp rx=0 tx=0).stop 2024-09-12T03:33:14.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.793+0000 7f62bb1e8640 1 -- 172.21.15.64:0/359976613 shutdown_connections 2024-09-12T03:33:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.793+0000 7f62bb1e8640 1 --2- 172.21.15.64:0/359976613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b4107b70 0x7f62b419cb20 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.793+0000 7f62bb1e8640 1 --2- 172.21.15.64:0/359976613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62b4102470 0x7f62b41987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.793+0000 7f62bb1e8640 1 --2- 172.21.15.64:0/359976613 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f629c0a2da0 0x7f629c0a5260 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.793+0000 7f62bb1e8640 1 --2- 172.21.15.64:0/359976613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62b4101ad0 0x7f62b4198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.793+0000 7f62bb1e8640 1 -- 172.21.15.64:0/359976613 >> 172.21.15.64:0/359976613 conn(0x7f62b40fd7d0 msgr2=0x7f62b40fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.793+0000 7f62bb1e8640 1 -- 172.21.15.64:0/359976613 shutdown_connections 2024-09-12T03:33:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:14.794+0000 7f62bb1e8640 1 -- 172.21.15.64:0/359976613 wait complete. 2024-09-12T03:33:15.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:33:14 smithi064 ceph-mon[30952]: pgmap v122: 129 pgs: 129 active+clean; 4.9 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-09-12T03:33:15.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:33:14 smithi115 ceph-mon[34801]: pgmap v122: 129 pgs: 129 active+clean; 4.9 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-09-12T03:33:15.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:33:14 smithi195 ceph-mon[34786]: pgmap v122: 129 pgs: 129 active+clean; 4.9 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-09-12T03:33:17.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:33:16 smithi064 ceph-mon[30952]: pgmap v123: 129 pgs: 129 active+clean; 4.9 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-09-12T03:33:17.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:33:16 smithi115 ceph-mon[34801]: pgmap v123: 129 pgs: 129 active+clean; 4.9 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-09-12T03:33:17.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:33:16 smithi195 ceph-mon[34786]: pgmap v123: 129 pgs: 129 active+clean; 4.9 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-09-12T03:33:19.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:33:18 smithi064 ceph-mon[30952]: pgmap v124: 129 pgs: 129 active+clean; 5.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 43 KiB/s wr, 18 op/s 2024-09-12T03:33:19.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:33:18 smithi115 ceph-mon[34801]: pgmap v124: 129 pgs: 129 active+clean; 5.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 43 KiB/s wr, 18 op/s 2024-09-12T03:33:19.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:33:18 smithi195 ceph-mon[34786]: pgmap v124: 129 pgs: 129 active+clean; 5.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 43 KiB/s wr, 18 op/s 2024-09-12T03:33:21.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:33:20 smithi064 ceph-mon[30952]: pgmap v125: 129 pgs: 129 active+clean; 5.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-12T03:33:21.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:33:20 smithi115 ceph-mon[34801]: pgmap v125: 129 pgs: 129 active+clean; 5.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-12T03:33:21.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:33:20 smithi195 ceph-mon[34786]: pgmap v125: 129 pgs: 129 active+clean; 5.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-12T03:33:23.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:33:22 smithi064 ceph-mon[30952]: pgmap v126: 129 pgs: 129 active+clean; 5.2 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 18 op/s 2024-09-12T03:33:23.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:33:22 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:33:23.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:33:22 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:33:23.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:33:22 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:33:23.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:33:22 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:33:23.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:33:22 smithi115 ceph-mon[34801]: pgmap v126: 129 pgs: 129 active+clean; 5.2 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 18 op/s 2024-09-12T03:33:23.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:33:22 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:33:23.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:33:22 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:33:23.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:33:22 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:33:23.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:33:22 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:33:23.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:33:22 smithi195 ceph-mon[34786]: pgmap v126: 129 pgs: 129 active+clean; 5.2 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 18 op/s 2024-09-12T03:33:23.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:33:22 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:33:23.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:33:22 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:33:23.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:33:22 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:33:23.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:33:22 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:33:25.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:33:24 smithi115 ceph-mon[34801]: pgmap v127: 129 pgs: 129 active+clean; 5.2 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 KiB/s wr, 12 op/s 2024-09-12T03:33:25.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:33:24 smithi195 ceph-mon[34786]: pgmap v127: 129 pgs: 129 active+clean; 5.2 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 KiB/s wr, 12 op/s 2024-09-12T03:33:25.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:33:24 smithi064 ceph-mon[30952]: pgmap v127: 129 pgs: 129 active+clean; 5.2 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 KiB/s wr, 12 op/s 2024-09-12T03:33:27.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:33:26 smithi115 ceph-mon[34801]: pgmap v128: 129 pgs: 129 active+clean; 5.2 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 KiB/s wr, 12 op/s 2024-09-12T03:33:27.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:33:26 smithi195 ceph-mon[34786]: pgmap v128: 129 pgs: 129 active+clean; 5.2 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 KiB/s wr, 12 op/s 2024-09-12T03:33:27.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:33:26 smithi064 ceph-mon[30952]: pgmap v128: 129 pgs: 129 active+clean; 5.2 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 KiB/s wr, 12 op/s 2024-09-12T03:33:27.894 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.893+0000 7fc70df9f640 1 -- 172.21.15.64:0/1444862314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc708103680 msgr2=0x7fc70810bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.893+0000 7fc70df9f640 1 --2- 172.21.15.64:0/1444862314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc708103680 0x7fc70810bb60 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc009900 tx=0x7fc6fc02a480 comp rx=0 tx=0).stop 2024-09-12T03:33:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.893+0000 7fc70df9f640 1 -- 172.21.15.64:0/1444862314 shutdown_connections 2024-09-12T03:33:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.893+0000 7fc70df9f640 1 --2- 172.21.15.64:0/1444862314 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc70810c130 0x7fc70810e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.893+0000 7fc70df9f640 1 --2- 172.21.15.64:0/1444862314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc708103680 0x7fc70810bb60 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.893+0000 7fc70df9f640 1 --2- 172.21.15.64:0/1444862314 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc708102d60 0x7fc708103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.893+0000 7fc70df9f640 1 -- 172.21.15.64:0/1444862314 >> 172.21.15.64:0/1444862314 conn(0x7fc7080fc6b0 msgr2=0x7fc7080fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.893+0000 7fc70df9f640 1 -- 172.21.15.64:0/1444862314 shutdown_connections 2024-09-12T03:33:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.893+0000 7fc70df9f640 1 -- 172.21.15.64:0/1444862314 wait complete. 2024-09-12T03:33:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.894+0000 7fc70df9f640 1 Processor -- start 2024-09-12T03:33:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.894+0000 7fc70df9f640 1 -- start start 2024-09-12T03:33:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.894+0000 7fc70df9f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc708102d60 0x7fc7081009b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.894+0000 7fc70df9f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc708103680 0x7fc708100ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.894+0000 7fc70df9f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc70810c130 0x7fc708101430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.894+0000 7fc70df9f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc708111db0 con 0x7fc708103680 2024-09-12T03:33:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.894+0000 7fc70df9f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc708111c30 con 0x7fc708102d60 2024-09-12T03:33:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.894+0000 7fc70df9f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc708111f30 con 0x7fc70810c130 2024-09-12T03:33:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.894+0000 7fc706ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc708103680 0x7fc708100ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.894+0000 7fc706ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc708103680 0x7fc708100ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41528/0 (socket says 172.21.15.64:41528) 2024-09-12T03:33:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.894+0000 7fc706ffd640 1 -- 172.21.15.64:0/3835879590 learned_addr learned my addr 172.21.15.64:0/3835879590 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:33:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.895+0000 7fc7077fe640 1 --2- 172.21.15.64:0/3835879590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc708102d60 0x7fc7081009b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.895+0000 7fc707fff640 1 --2- 172.21.15.64:0/3835879590 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc70810c130 0x7fc708101430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.895+0000 7fc706ffd640 1 -- 172.21.15.64:0/3835879590 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc70810c130 msgr2=0x7fc708101430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.895+0000 7fc706ffd640 1 --2- 172.21.15.64:0/3835879590 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc70810c130 0x7fc708101430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.895+0000 7fc706ffd640 1 -- 172.21.15.64:0/3835879590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc708102d60 msgr2=0x7fc7081009b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.895+0000 7fc706ffd640 1 --2- 172.21.15.64:0/3835879590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc708102d60 0x7fc7081009b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.895+0000 7fc706ffd640 1 -- 172.21.15.64:0/3835879590 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7081a67c0 con 0x7fc708103680 2024-09-12T03:33:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.895+0000 7fc7077fe640 1 --2- 172.21.15.64:0/3835879590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc708102d60 0x7fc7081009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:33:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.895+0000 7fc707fff640 1 --2- 172.21.15.64:0/3835879590 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc70810c130 0x7fc708101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:33:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.895+0000 7fc706ffd640 1 --2- 172.21.15.64:0/3835879590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc708103680 0x7fc708100ef0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc005940 tx=0x7fc6fc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.895+0000 7fc704ff9640 1 -- 172.21.15.64:0/3835879590 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc03f030 con 0x7fc708103680 2024-09-12T03:33:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.895+0000 7fc704ff9640 1 -- 172.21.15.64:0/3835879590 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6fc003cb0 con 0x7fc708103680 2024-09-12T03:33:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.895+0000 7fc70df9f640 1 -- 172.21.15.64:0/3835879590 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7081a6a50 con 0x7fc708103680 2024-09-12T03:33:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.895+0000 7fc704ff9640 1 -- 172.21.15.64:0/3835879590 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc035070 con 0x7fc708103680 2024-09-12T03:33:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.895+0000 7fc70df9f640 1 -- 172.21.15.64:0/3835879590 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7081a6f90 con 0x7fc708103680 2024-09-12T03:33:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.897+0000 7fc704ff9640 1 -- 172.21.15.64:0/3835879590 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc6fc042030 con 0x7fc708103680 2024-09-12T03:33:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.898+0000 7fc710e1b740 1 Processor -- start 2024-09-12T03:33:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.898+0000 7fc710e1b740 1 -- start start 2024-09-12T03:33:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.898+0000 7fc710e1b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a9bfa3af0 0x556a9bfa3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.898+0000 7fc704ff9640 1 --2- 172.21.15.64:0/3835879590 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fc6f00a2e20 0x7fc6f00a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.898+0000 7fc710e1b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a9bfa4410 0x556a9bfae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.898+0000 7fc710e1b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a9bfaecb0 0x556a9bfb10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.898+0000 7fc710e1b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556a9bf082f0 con 0x556a9bfaecb0 2024-09-12T03:33:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.898+0000 7fc710e1b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556a9beb0270 con 0x556a9bfa4410 2024-09-12T03:33:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.898+0000 7fc710e1b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556a9bf042b0 con 0x556a9bfa3af0 2024-09-12T03:33:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.898+0000 7fc7077fe640 1 --2- 172.21.15.64:0/3835879590 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fc6f00a2e20 0x7fc6f00a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.898+0000 7fc707fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a9bfaecb0 0x556a9bfb10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.898+0000 7fc704ff9640 1 -- 172.21.15.64:0/3835879590 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7fc6fc0f00d0 con 0x7fc708103680 2024-09-12T03:33:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.898+0000 7fc707fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a9bfaecb0 0x556a9bfb10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41544/0 (socket says 172.21.15.64:41544) 2024-09-12T03:33:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.898+0000 7fc707fff640 1 -- 172.21.15.64:0/2528357928 learned_addr learned my addr 172.21.15.64:0/2528357928 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:33:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.899+0000 7fc706ffd640 1 --2- 172.21.15.64:0/2528357928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a9bfa4410 0x556a9bfae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.899+0000 7fc707fff640 1 --2- 172.21.15.64:0/2528357928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a9bfa3af0 0x556a9bfa3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.899+0000 7fc707fff640 1 -- 172.21.15.64:0/2528357928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a9bfa3af0 msgr2=0x556a9bfa3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.899+0000 7fc707fff640 1 --2- 172.21.15.64:0/2528357928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a9bfa3af0 0x556a9bfa3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.899+0000 7fc707fff640 1 -- 172.21.15.64:0/2528357928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a9bfa4410 msgr2=0x556a9bfae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.899+0000 7fc707fff640 1 --2- 172.21.15.64:0/2528357928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a9bfa4410 0x556a9bfae770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.899+0000 7fc707fff640 1 -- 172.21.15.64:0/2528357928 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a9beaf340 con 0x556a9bfaecb0 2024-09-12T03:33:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.899+0000 7fc7077fe640 1 --2- 172.21.15.64:0/3835879590 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fc6f00a2e20 0x7fc6f00a52e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8009b40 tx=0x7fc6f8005e50 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.899+0000 7fc707fff640 1 --2- 172.21.15.64:0/2528357928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a9bfaecb0 0x556a9bfb10a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc700009f40 tx=0x7fc700009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=285f97b61644a952 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.899+0000 7fc6edffb640 1 -- 172.21.15.64:0/2528357928 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc700019030 con 0x556a9bfaecb0 2024-09-12T03:33:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.899+0000 7fc6edffb640 1 -- 172.21.15.64:0/2528357928 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc700011db0 con 0x556a9bfaecb0 2024-09-12T03:33:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.899+0000 7fc6edffb640 1 -- 172.21.15.64:0/2528357928 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70000fa50 con 0x556a9bfaecb0 2024-09-12T03:33:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.900+0000 7fc710e1b740 1 -- 172.21.15.64:0/2528357928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a9bfaecb0 msgr2=0x556a9bfb10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.900+0000 7fc710e1b740 1 --2- 172.21.15.64:0/2528357928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a9bfaecb0 0x556a9bfb10a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc700009f40 tx=0x7fc700009820 comp rx=0 tx=0).stop 2024-09-12T03:33:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.900+0000 7fc710e1b740 1 -- 172.21.15.64:0/2528357928 shutdown_connections 2024-09-12T03:33:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.900+0000 7fc710e1b740 1 --2- 172.21.15.64:0/2528357928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a9bfaecb0 0x556a9bfb10a0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.900+0000 7fc710e1b740 1 --2- 172.21.15.64:0/2528357928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a9bfa4410 0x556a9bfae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.900+0000 7fc710e1b740 1 --2- 172.21.15.64:0/2528357928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a9bfa3af0 0x556a9bfa3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.900+0000 7fc710e1b740 1 -- 172.21.15.64:0/2528357928 >> 172.21.15.64:0/2528357928 conn(0x556a9bfa0060 msgr2=0x556a9bfa0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.900+0000 7fc710e1b740 1 -- 172.21.15.64:0/2528357928 shutdown_connections 2024-09-12T03:33:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.900+0000 7fc710e1b740 1 -- 172.21.15.64:0/2528357928 wait complete. 2024-09-12T03:33:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.900+0000 7fc710e1b740 1 -- start start 2024-09-12T03:33:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.901+0000 7fc710e1b740 1 Processor -- start 2024-09-12T03:33:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.901+0000 7fc710e1b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a9bfa3af0 0x556a9bfabbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.901+0000 7fc710e1b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a9bfa4410 0x556a9bfac110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.901+0000 7fc710e1b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a9bfaecb0 0x556a9bfac650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.901+0000 7fc710e1b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556a9bfb1880 con 0x556a9bfaecb0 2024-09-12T03:33:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.901+0000 7fc710e1b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556a9be91290 con 0x556a9bfa3af0 2024-09-12T03:33:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.901+0000 7fc710e1b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556a9bfb1a00 con 0x556a9bfa4410 2024-09-12T03:33:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.901+0000 7fc710e1b740 10 client.0 _ll_register_callbacks cb 0x556a9bec2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.901+0000 7fc707fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a9bfaecb0 0x556a9bfac650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.901+0000 7fc707fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a9bfaecb0 0x556a9bfac650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41546/0 (socket says 172.21.15.64:41546) 2024-09-12T03:33:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.901+0000 7fc707fff640 1 -- 172.21.15.64:0/2694562423 learned_addr learned my addr 172.21.15.64:0/2694562423 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:33:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.901+0000 7fc7077fe640 1 --2- 172.21.15.64:0/2694562423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a9bfa3af0 0x556a9bfabbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.901+0000 7fc706ffd640 1 --2- 172.21.15.64:0/2694562423 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a9bfa4410 0x556a9bfac110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.901+0000 7fc707fff640 1 -- 172.21.15.64:0/2694562423 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a9bfa4410 msgr2=0x556a9bfac110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.901+0000 7fc707fff640 1 --2- 172.21.15.64:0/2694562423 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a9bfa4410 0x556a9bfac110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.901+0000 7fc707fff640 1 -- 172.21.15.64:0/2694562423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a9bfa3af0 msgr2=0x556a9bfabbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.901+0000 7fc707fff640 1 --2- 172.21.15.64:0/2694562423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a9bfa3af0 0x556a9bfabbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.901+0000 7fc707fff640 1 -- 172.21.15.64:0/2694562423 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a9bc1b210 con 0x556a9bfaecb0 2024-09-12T03:33:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.901+0000 7fc706ffd640 1 --2- 172.21.15.64:0/2694562423 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a9bfa4410 0x556a9bfac110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:33:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.902+0000 7fc707fff640 1 --2- 172.21.15.64:0/2694562423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a9bfaecb0 0x556a9bfac650 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc700011750 tx=0x7fc700011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d016fa27e8bc20f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.902+0000 7fc6e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:33:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.902+0000 7fc6e6ffd640 1 -- 172.21.15.64:0/2694562423 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc700019040 con 0x556a9bfaecb0 2024-09-12T03:33:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.902+0000 7fc6e6ffd640 1 -- 172.21.15.64:0/2694562423 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7fc700013270 con 0x556a9bfaecb0 2024-09-12T03:33:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.902+0000 7fc710e1b740 1 -- 172.21.15.64:0/2694562423 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc700011db0 con 0x556a9bfaecb0 2024-09-12T03:33:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.902+0000 7fc6e6ffd640 1 -- 172.21.15.64:0/2694562423 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc70002f050 con 0x556a9bfaecb0 2024-09-12T03:33:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.902+0000 7fc710e1b740 10 client.15684 fetch_fsmap learned FSMap version 25 2024-09-12T03:33:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.902+0000 7fc710e1b740 1 -- 172.21.15.64:0/2694562423 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556a9bce7620 con 0x556a9bfaecb0 2024-09-12T03:33:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.902+0000 7fc6e6ffd640 1 -- 172.21.15.64:0/2694562423 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc700011db0 con 0x556a9bfaecb0 2024-09-12T03:33:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.902+0000 7fc6e6ffd640 1 -- 172.21.15.64:0/2694562423 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc70000fad0 con 0x556a9bfaecb0 2024-09-12T03:33:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.902+0000 7fc710e1b740 10 client.15684 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:33:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.902+0000 7fc710e1b740 10 client.15684 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:33:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.902+0000 7fc710e1b740 20 client.15684 populate_metadata read hostname 'smithi064' 2024-09-12T03:33:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.902+0000 7fc710e1b740 1 --2- 172.21.15.64:0/2694562423 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x556a9bfbec40 0x556a9bfc1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.902+0000 7fc710e1b740 4 client.15684 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:33:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.902+0000 7fc710e1b740 1 -- 172.21.15.64:0/2694562423 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a9bf15a20 con 0x556a9bfbec40 2024-09-12T03:33:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.903+0000 7fc7077fe640 1 --2- 172.21.15.64:0/2694562423 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x556a9bfbec40 0x556a9bfc1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.903+0000 7fc7077fe640 1 --2- 172.21.15.64:0/2694562423 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x556a9bfbec40 0x556a9bfc1030 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27100abd00227d00 server_cookie=ae06002fdfd75d92 in_seq=0 out_seq=0 2024-09-12T03:33:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.903+0000 7fc6e6ffd640 10 client.15684 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:33:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.904+0000 7fc6e6ffd640 1 -- 172.21.15.64:0/2694562423 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556a9bf15a20 con 0x556a9bfbec40 2024-09-12T03:33:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.904+0000 7fc6e6ffd640 10 client.15684 handle_command_reply: tid=0 2024-09-12T03:33:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.904+0000 7fc710e1b740 1 client.15684 shutdown 2024-09-12T03:33:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.904+0000 7fc710e1b740 1 -- 172.21.15.64:0/2694562423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a9bfaecb0 msgr2=0x556a9bfac650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.904+0000 7fc710e1b740 1 --2- 172.21.15.64:0/2694562423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a9bfaecb0 0x556a9bfac650 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc700011750 tx=0x7fc700011bb0 comp rx=0 tx=0).stop 2024-09-12T03:33:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.904+0000 7fc710e1b740 1 -- 172.21.15.64:0/2694562423 shutdown_connections 2024-09-12T03:33:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.904+0000 7fc710e1b740 1 --2- 172.21.15.64:0/2694562423 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x556a9bfbec40 0x556a9bfc1030 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.904+0000 7fc710e1b740 1 --2- 172.21.15.64:0/2694562423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a9bfaecb0 0x556a9bfac650 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.904+0000 7fc710e1b740 1 --2- 172.21.15.64:0/2694562423 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a9bfa4410 0x556a9bfac110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.904+0000 7fc6e6ffd640 0 client.15684 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:33:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.904+0000 7fc710e1b740 1 --2- 172.21.15.64:0/2694562423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a9bfa3af0 0x556a9bfabbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.904+0000 7fc710e1b740 1 -- 172.21.15.64:0/2694562423 >> 172.21.15.64:0/2694562423 conn(0x556a9bfa0060 msgr2=0x556a9bfa5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.904+0000 7fc710e1b740 1 -- 172.21.15.64:0/2694562423 shutdown_connections 2024-09-12T03:33:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.904+0000 7fc710e1b740 1 -- 172.21.15.64:0/2694562423 wait complete. 2024-09-12T03:33:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.905+0000 7fc710e1b740 20 client.15684 trim_cache size 0 max 16384 2024-09-12T03:33:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.919+0000 7fc710e1b740 1 Processor -- start 2024-09-12T03:33:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.919+0000 7fc710e1b740 1 -- start start 2024-09-12T03:33:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.919+0000 7fc710e1b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a9bfa3af0 0x556a9bfad430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.920+0000 7fc710e1b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a9bfa4410 0x556a9bfc65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.920+0000 7fc710e1b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a9bfaecb0 0x556a9bfc6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.920+0000 7fc710e1b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc7081b3aa0 con 0x556a9bfa3af0 2024-09-12T03:33:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.920+0000 7fc710e1b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc7081b1370 con 0x556a9bfaecb0 2024-09-12T03:33:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.920+0000 7fc710e1b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc7081af1c0 con 0x556a9bfa4410 2024-09-12T03:33:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.920+0000 7fc7077fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a9bfa3af0 0x556a9bfad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.920+0000 7fc7077fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a9bfa3af0 0x556a9bfad430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41548/0 (socket says 172.21.15.64:41548) 2024-09-12T03:33:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.920+0000 7fc7077fe640 1 -- 172.21.15.64:0/311046656 learned_addr learned my addr 172.21.15.64:0/311046656 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:33:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.920+0000 7fc707fff640 1 --2- 172.21.15.64:0/311046656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a9bfaecb0 0x556a9bfc6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.920+0000 7fc706ffd640 1 --2- 172.21.15.64:0/311046656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a9bfa4410 0x556a9bfc65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.920+0000 7fc7077fe640 1 -- 172.21.15.64:0/311046656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a9bfa4410 msgr2=0x556a9bfc65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.920+0000 7fc7077fe640 1 --2- 172.21.15.64:0/311046656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a9bfa4410 0x556a9bfc65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.920+0000 7fc7077fe640 1 -- 172.21.15.64:0/311046656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a9bfaecb0 msgr2=0x556a9bfc6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.920+0000 7fc7077fe640 1 --2- 172.21.15.64:0/311046656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a9bfaecb0 0x556a9bfc6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.920+0000 7fc7077fe640 1 -- 172.21.15.64:0/311046656 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a9bf9fe90 con 0x556a9bfa3af0 2024-09-12T03:33:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.920+0000 7fc7077fe640 1 --2- 172.21.15.64:0/311046656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a9bfa3af0 0x556a9bfad430 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc6f800ed40 tx=0x7fc6f8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feafca469356a4e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.920+0000 7fc6ed7fa640 1 -- 172.21.15.64:0/311046656 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f8004020 con 0x556a9bfa3af0 2024-09-12T03:33:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.920+0000 7fc6ed7fa640 1 -- 172.21.15.64:0/311046656 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6f8017040 con 0x556a9bfa3af0 2024-09-12T03:33:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.920+0000 7fc6ed7fa640 1 -- 172.21.15.64:0/311046656 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f8003e30 con 0x556a9bfa3af0 2024-09-12T03:33:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.921+0000 7fc710e1b740 1 -- 172.21.15.64:0/311046656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a9bfa3af0 msgr2=0x556a9bfad430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.921+0000 7fc710e1b740 1 --2- 172.21.15.64:0/311046656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a9bfa3af0 0x556a9bfad430 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc6f800ed40 tx=0x7fc6f8014000 comp rx=0 tx=0).stop 2024-09-12T03:33:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.921+0000 7fc710e1b740 1 -- 172.21.15.64:0/311046656 shutdown_connections 2024-09-12T03:33:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.921+0000 7fc710e1b740 1 --2- 172.21.15.64:0/311046656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a9bfaecb0 0x556a9bfc6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.921+0000 7fc710e1b740 1 --2- 172.21.15.64:0/311046656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a9bfa4410 0x556a9bfc65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.921+0000 7fc710e1b740 1 --2- 172.21.15.64:0/311046656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a9bfa3af0 0x556a9bfad430 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.921+0000 7fc710e1b740 1 -- 172.21.15.64:0/311046656 >> 172.21.15.64:0/311046656 conn(0x556a9bfa0060 msgr2=0x556a9bf9f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.921+0000 7fc710e1b740 1 -- 172.21.15.64:0/311046656 shutdown_connections 2024-09-12T03:33:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.921+0000 7fc710e1b740 1 -- 172.21.15.64:0/311046656 wait complete. 2024-09-12T03:33:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.921+0000 7fc710e1b740 1 -- start start 2024-09-12T03:33:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.922+0000 7fc710e1b740 1 Processor -- start 2024-09-12T03:33:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.922+0000 7fc710e1b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a9bfa3af0 0x556a9bfb71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.922+0000 7fc710e1b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a9bfa4410 0x556a9bfb4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.922+0000 7fc710e1b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a9bfaecb0 0x556a9bfb4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.922+0000 7fc710e1b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc7081af830 con 0x556a9bfa4410 2024-09-12T03:33:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.922+0000 7fc710e1b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc7081b0ca0 con 0x556a9bfa3af0 2024-09-12T03:33:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.922+0000 7fc710e1b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc7081aff00 con 0x556a9bfaecb0 2024-09-12T03:33:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.922+0000 7fc710e1b740 10 client.0 _ll_register_callbacks cb 0x7fc6fc005760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.922+0000 7fc706ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a9bfa4410 0x556a9bfb4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.922+0000 7fc706ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a9bfa4410 0x556a9bfb4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41558/0 (socket says 172.21.15.64:41558) 2024-09-12T03:33:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.922+0000 7fc706ffd640 1 -- 172.21.15.64:0/410031364 learned_addr learned my addr 172.21.15.64:0/410031364 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:33:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.922+0000 7fc707fff640 1 --2- 172.21.15.64:0/410031364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a9bfaecb0 0x556a9bfb4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.922+0000 7fc7077fe640 1 --2- 172.21.15.64:0/410031364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a9bfa3af0 0x556a9bfb71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.922+0000 7fc706ffd640 1 -- 172.21.15.64:0/410031364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a9bfaecb0 msgr2=0x556a9bfb4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.922+0000 7fc706ffd640 1 --2- 172.21.15.64:0/410031364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a9bfaecb0 0x556a9bfb4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.922+0000 7fc706ffd640 1 -- 172.21.15.64:0/410031364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a9bfa3af0 msgr2=0x556a9bfb71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.922+0000 7fc706ffd640 1 --2- 172.21.15.64:0/410031364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a9bfa3af0 0x556a9bfb71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.922+0000 7fc706ffd640 1 -- 172.21.15.64:0/410031364 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a9bfb5030 con 0x556a9bfa4410 2024-09-12T03:33:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.922+0000 7fc707fff640 1 --2- 172.21.15.64:0/410031364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a9bfaecb0 0x556a9bfb4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:33:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.922+0000 7fc7077fe640 1 --2- 172.21.15.64:0/410031364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a9bfa3af0 0x556a9bfb71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:33:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.923+0000 7fc706ffd640 1 --2- 172.21.15.64:0/410031364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a9bfa4410 0x556a9bfb4500 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc0b8680 tx=0x7fc6fc0c1d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cd101cc4858971 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.923+0000 7fc6e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:33:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.923+0000 7fc6e6ffd640 1 -- 172.21.15.64:0/410031364 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc049040 con 0x556a9bfa4410 2024-09-12T03:33:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.923+0000 7fc6e6ffd640 1 -- 172.21.15.64:0/410031364 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7fc6fc0ed6d0 con 0x556a9bfa4410 2024-09-12T03:33:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.923+0000 7fc710e1b740 1 -- 172.21.15.64:0/410031364 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6f8017040 con 0x556a9bfa4410 2024-09-12T03:33:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.923+0000 7fc6e6ffd640 1 -- 172.21.15.64:0/410031364 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc6fc0dae60 con 0x556a9bfa4410 2024-09-12T03:33:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.923+0000 7fc710e1b740 10 client.15690 fetch_fsmap learned FSMap version 25 2024-09-12T03:33:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.923+0000 7fc710e1b740 1 -- 172.21.15.64:0/410031364 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556a9c0293a0 con 0x556a9bfa4410 2024-09-12T03:33:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.923+0000 7fc6e6ffd640 1 -- 172.21.15.64:0/410031364 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6f8017040 con 0x556a9bfa4410 2024-09-12T03:33:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.923+0000 7fc6e6ffd640 1 -- 172.21.15.64:0/410031364 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc6fc0b8890 con 0x556a9bfa4410 2024-09-12T03:33:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.923+0000 7fc710e1b740 10 client.15690 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:33:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.923+0000 7fc710e1b740 10 client.15690 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:33:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.923+0000 7fc710e1b740 20 client.15690 populate_metadata read hostname 'smithi064' 2024-09-12T03:33:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.923+0000 7fc710e1b740 1 --2- 172.21.15.64:0/410031364 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x556a9bfbec40 0x556a9c02bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.923+0000 7fc710e1b740 4 client.15690 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:33:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.923+0000 7fc710e1b740 1 -- 172.21.15.64:0/410031364 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556a9bf0fd70 con 0x556a9bfbec40 2024-09-12T03:33:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.924+0000 7fc7077fe640 1 --2- 172.21.15.64:0/410031364 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x556a9bfbec40 0x556a9c02bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.924+0000 7fc6e6ffd640 10 client.15690 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:33:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.924+0000 7fc7077fe640 1 --2- 172.21.15.64:0/410031364 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x556a9bfbec40 0x556a9c02bb90 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c0eafce003796c1 server_cookie=f3e353c7d3995c1e in_seq=0 out_seq=0 2024-09-12T03:33:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.942+0000 7fc6e6ffd640 1 -- 172.21.15.64:0/410031364 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556a9bf0fd70 con 0x556a9bfbec40 2024-09-12T03:33:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.942+0000 7fc6e6ffd640 10 client.15690 handle_command_reply: tid=0 2024-09-12T03:33:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.942+0000 7fc710e1b740 1 client.15690 shutdown 2024-09-12T03:33:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.942+0000 7fc710e1b740 1 -- 172.21.15.64:0/410031364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a9bfa4410 msgr2=0x556a9bfb4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.942+0000 7fc710e1b740 1 --2- 172.21.15.64:0/410031364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a9bfa4410 0x556a9bfb4500 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc0b8680 tx=0x7fc6fc0c1d60 comp rx=0 tx=0).stop 2024-09-12T03:33:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.942+0000 7fc710e1b740 1 -- 172.21.15.64:0/410031364 shutdown_connections 2024-09-12T03:33:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.942+0000 7fc710e1b740 1 --2- 172.21.15.64:0/410031364 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x556a9bfbec40 0x556a9c02bb90 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.942+0000 7fc710e1b740 1 --2- 172.21.15.64:0/410031364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a9bfaecb0 0x556a9bfb4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.942+0000 7fc710e1b740 1 --2- 172.21.15.64:0/410031364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a9bfa4410 0x556a9bfb4500 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.942+0000 7fc710e1b740 1 --2- 172.21.15.64:0/410031364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a9bfa3af0 0x556a9bfb71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.942+0000 7fc6e6ffd640 0 client.15690 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:33:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.942+0000 7fc710e1b740 1 -- 172.21.15.64:0/410031364 >> 172.21.15.64:0/410031364 conn(0x556a9bfa0060 msgr2=0x556a9bfb9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.942+0000 7fc710e1b740 1 -- 172.21.15.64:0/410031364 shutdown_connections 2024-09-12T03:33:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.942+0000 7fc710e1b740 1 -- 172.21.15.64:0/410031364 wait complete. 2024-09-12T03:33:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.942+0000 7fc710e1b740 20 client.15690 trim_cache size 0 max 16384 2024-09-12T03:33:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:33:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:33:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:33:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:33:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.942+0000 7fc70df9f640 1 -- 172.21.15.64:0/3835879590 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fc6f00a2e20 msgr2=0x7fc6f00a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.942+0000 7fc70df9f640 1 --2- 172.21.15.64:0/3835879590 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fc6f00a2e20 0x7fc6f00a52e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8009b40 tx=0x7fc6f8005e50 comp rx=0 tx=0).stop 2024-09-12T03:33:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.942+0000 7fc70df9f640 1 -- 172.21.15.64:0/3835879590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc708103680 msgr2=0x7fc708100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.942+0000 7fc70df9f640 1 --2- 172.21.15.64:0/3835879590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc708103680 0x7fc708100ef0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc005940 tx=0x7fc6fc0042f0 comp rx=0 tx=0).stop 2024-09-12T03:33:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.942+0000 7fc70df9f640 1 -- 172.21.15.64:0/3835879590 shutdown_connections 2024-09-12T03:33:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.942+0000 7fc70df9f640 1 --2- 172.21.15.64:0/3835879590 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc70810c130 0x7fc708101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.942+0000 7fc70df9f640 1 --2- 172.21.15.64:0/3835879590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc708103680 0x7fc708100ef0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.942+0000 7fc70df9f640 1 --2- 172.21.15.64:0/3835879590 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fc6f00a2e20 0x7fc6f00a52e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.942+0000 7fc70df9f640 1 --2- 172.21.15.64:0/3835879590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc708102d60 0x7fc7081009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.942+0000 7fc70df9f640 1 -- 172.21.15.64:0/3835879590 >> 172.21.15.64:0/3835879590 conn(0x7fc7080fc6b0 msgr2=0x7fc7080fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.942+0000 7fc70df9f640 1 -- 172.21.15.64:0/3835879590 shutdown_connections 2024-09-12T03:33:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:27.942+0000 7fc70df9f640 1 -- 172.21.15.64:0/3835879590 wait complete. 2024-09-12T03:33:28.537 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.537 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.537 INFO:tasks.workunit.client.0.smithi064.stdout:127/600: open fd = 3 2024-09-12T03:33:28.537 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.537 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.537 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.537 INFO:tasks.workunit.client.0.smithi064.stdout:128/600: open fd = 3 2024-09-12T03:33:28.538 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.538 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.538 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.538 INFO:tasks.workunit.client.0.smithi064.stdout:129/600: open fd = 3 2024-09-12T03:33:28.538 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.538 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.538 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.538 INFO:tasks.workunit.client.0.smithi064.stdout:130/600: open fd = 3 2024-09-12T03:33:28.538 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.538 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.538 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.539 INFO:tasks.workunit.client.0.smithi064.stdout:131/600: open fd = 3 2024-09-12T03:33:28.539 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.539 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.539 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.539 INFO:tasks.workunit.client.0.smithi064.stdout:132/600: open fd = 3 2024-09-12T03:33:28.539 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.539 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.539 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.539 INFO:tasks.workunit.client.0.smithi064.stdout:133/600: open fd = 3 2024-09-12T03:33:28.539 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.539 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.539 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.540 INFO:tasks.workunit.client.0.smithi064.stdout:134/600: open fd = 3 2024-09-12T03:33:28.540 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.540 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.540 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.540 INFO:tasks.workunit.client.0.smithi064.stdout:135/600: open fd = 3 2024-09-12T03:33:28.540 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.540 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.540 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.540 INFO:tasks.workunit.client.0.smithi064.stdout:136/600: open fd = 3 2024-09-12T03:33:28.540 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.540 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.540 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.541 INFO:tasks.workunit.client.0.smithi064.stdout:137/600: open fd = 3 2024-09-12T03:33:28.541 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.541 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.541 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.541 INFO:tasks.workunit.client.0.smithi064.stdout:138/600: open fd = 3 2024-09-12T03:33:28.541 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.541 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.541 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.541 INFO:tasks.workunit.client.0.smithi064.stdout:139/600: open fd = 3 2024-09-12T03:33:28.541 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.541 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.541 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.541 INFO:tasks.workunit.client.0.smithi064.stdout:140/600: open fd = 3 2024-09-12T03:33:28.545 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.545 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.545 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.545 INFO:tasks.workunit.client.0.smithi064.stdout:141/600: open fd = 3 2024-09-12T03:33:28.545 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.546 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.546 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.546 INFO:tasks.workunit.client.0.smithi064.stdout:142/600: open fd = 3 2024-09-12T03:33:28.546 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.546 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.546 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.546 INFO:tasks.workunit.client.0.smithi064.stdout:143/600: open fd = 3 2024-09-12T03:33:28.546 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.546 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.546 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.546 INFO:tasks.workunit.client.0.smithi064.stdout:144/600: open fd = 3 2024-09-12T03:33:28.546 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.547 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.547 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.547 INFO:tasks.workunit.client.0.smithi064.stdout:145/600: open fd = 3 2024-09-12T03:33:28.547 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.547 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.547 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.547 INFO:tasks.workunit.client.0.smithi064.stdout:146/600: open fd = 3 2024-09-12T03:33:28.547 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.547 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.547 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.547 INFO:tasks.workunit.client.0.smithi064.stdout:147/600: open fd = 3 2024-09-12T03:33:28.547 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.547 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.548 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.548 INFO:tasks.workunit.client.0.smithi064.stdout:148/600: open fd = 3 2024-09-12T03:33:28.548 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.548 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.548 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.548 INFO:tasks.workunit.client.0.smithi064.stdout:149/600: open fd = 3 2024-09-12T03:33:28.548 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.548 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.548 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.548 INFO:tasks.workunit.client.0.smithi064.stdout:150/600: open fd = 3 2024-09-12T03:33:28.549 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.549 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.549 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.549 INFO:tasks.workunit.client.0.smithi064.stdout:151/600: open fd = 3 2024-09-12T03:33:28.549 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.549 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.549 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.549 INFO:tasks.workunit.client.0.smithi064.stdout:152/600: open fd = 3 2024-09-12T03:33:28.549 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.549 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.549 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.549 INFO:tasks.workunit.client.0.smithi064.stdout:153/600: open fd = 3 2024-09-12T03:33:28.549 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.549 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.550 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.550 INFO:tasks.workunit.client.0.smithi064.stdout:154/600: open fd = 3 2024-09-12T03:33:28.550 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.550 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.550 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.550 INFO:tasks.workunit.client.0.smithi064.stdout:155/600: open fd = 3 2024-09-12T03:33:28.550 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.550 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.550 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.550 INFO:tasks.workunit.client.0.smithi064.stdout:156/600: open fd = 3 2024-09-12T03:33:28.550 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.550 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.550 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.551 INFO:tasks.workunit.client.0.smithi064.stdout:157/600: open fd = 3 2024-09-12T03:33:28.551 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.551 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.551 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.551 INFO:tasks.workunit.client.0.smithi064.stdout:158/600: open fd = 3 2024-09-12T03:33:28.551 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.551 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.551 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.551 INFO:tasks.workunit.client.0.smithi064.stdout:159/600: open fd = 3 2024-09-12T03:33:28.551 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.551 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.552 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.552 INFO:tasks.workunit.client.0.smithi064.stdout:160/600: open fd = 3 2024-09-12T03:33:28.552 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.552 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.552 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.552 INFO:tasks.workunit.client.0.smithi064.stdout:161/600: open fd = 3 2024-09-12T03:33:28.552 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.552 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.552 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.552 INFO:tasks.workunit.client.0.smithi064.stdout:162/600: open fd = 3 2024-09-12T03:33:28.552 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.552 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.552 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.553 INFO:tasks.workunit.client.0.smithi064.stdout:163/600: open fd = 3 2024-09-12T03:33:28.553 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.553 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.553 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.553 INFO:tasks.workunit.client.0.smithi064.stdout:164/600: open fd = 3 2024-09-12T03:33:28.553 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.553 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.553 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.553 INFO:tasks.workunit.client.0.smithi064.stdout:165/600: open fd = 3 2024-09-12T03:33:28.553 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.553 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.553 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.553 INFO:tasks.workunit.client.0.smithi064.stdout:166/600: open fd = 3 2024-09-12T03:33:28.554 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.554 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.554 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.554 INFO:tasks.workunit.client.0.smithi064.stdout:167/600: open fd = 3 2024-09-12T03:33:28.554 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.554 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.554 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.554 INFO:tasks.workunit.client.0.smithi064.stdout:168/600: open fd = 3 2024-09-12T03:33:28.554 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.554 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.554 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.554 INFO:tasks.workunit.client.0.smithi064.stdout:169/600: open fd = 3 2024-09-12T03:33:28.555 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.555 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.555 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.555 INFO:tasks.workunit.client.0.smithi064.stdout:170/600: open fd = 3 2024-09-12T03:33:28.555 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.555 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.555 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.555 INFO:tasks.workunit.client.0.smithi064.stdout:171/600: open fd = 3 2024-09-12T03:33:28.555 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.555 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.555 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.555 INFO:tasks.workunit.client.0.smithi064.stdout:172/600: open fd = 3 2024-09-12T03:33:28.556 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.556 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.556 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.556 INFO:tasks.workunit.client.0.smithi064.stdout:173/600: open fd = 3 2024-09-12T03:33:28.556 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.556 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.556 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.556 INFO:tasks.workunit.client.0.smithi064.stdout:174/600: open fd = 3 2024-09-12T03:33:28.556 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.556 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.556 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.556 INFO:tasks.workunit.client.0.smithi064.stdout:175/600: open fd = 3 2024-09-12T03:33:28.556 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.556 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.557 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.557 INFO:tasks.workunit.client.0.smithi064.stdout:176/600: open fd = 3 2024-09-12T03:33:28.557 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.557 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.557 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.557 INFO:tasks.workunit.client.0.smithi064.stdout:177/600: open fd = 3 2024-09-12T03:33:28.557 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.557 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.557 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.557 INFO:tasks.workunit.client.0.smithi064.stdout:178/600: open fd = 3 2024-09-12T03:33:28.557 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.558 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.558 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.558 INFO:tasks.workunit.client.0.smithi064.stdout:179/600: open fd = 3 2024-09-12T03:33:28.558 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.558 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.558 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.558 INFO:tasks.workunit.client.0.smithi064.stdout:180/600: open fd = 3 2024-09-12T03:33:28.558 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.558 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.558 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.558 INFO:tasks.workunit.client.0.smithi064.stdout:181/600: open fd = 3 2024-09-12T03:33:28.558 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.558 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.559 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.559 INFO:tasks.workunit.client.0.smithi064.stdout:182/600: open fd = 3 2024-09-12T03:33:28.559 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.559 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.559 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.559 INFO:tasks.workunit.client.0.smithi064.stdout:183/600: open fd = 3 2024-09-12T03:33:28.559 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.559 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.559 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.559 INFO:tasks.workunit.client.0.smithi064.stdout:184/600: open fd = 3 2024-09-12T03:33:28.559 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.559 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.559 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.559 INFO:tasks.workunit.client.0.smithi064.stdout:185/600: open fd = 3 2024-09-12T03:33:28.560 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.560 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.560 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.560 INFO:tasks.workunit.client.0.smithi064.stdout:186/600: open fd = 3 2024-09-12T03:33:28.560 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.560 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.560 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.560 INFO:tasks.workunit.client.0.smithi064.stdout:187/600: open fd = 3 2024-09-12T03:33:28.560 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.560 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.560 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:33:28.560 INFO:tasks.workunit.client.0.smithi064.stdout:188/600: open fd = 3 2024-09-12T03:33:28.561 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:28.561 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:33:29.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:33:28 smithi115 ceph-mon[34801]: pgmap v129: 129 pgs: 129 active+clean; 5.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 18 op/s 2024-09-12T03:33:29.301 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:33:28 smithi064 ceph-mon[30952]: pgmap v129: 129 pgs: 129 active+clean; 5.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 18 op/s 2024-09-12T03:33:29.301 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:33:28 smithi195 ceph-mon[34786]: pgmap v129: 129 pgs: 129 active+clean; 5.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 18 op/s 2024-09-12T03:33:31.145 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:33:30 smithi115 ceph-mon[34801]: pgmap v130: 129 pgs: 129 active+clean; 5.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-09-12T03:33:31.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:33:30 smithi195 ceph-mon[34786]: pgmap v130: 129 pgs: 129 active+clean; 5.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-09-12T03:33:31.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:33:30 smithi064 ceph-mon[30952]: pgmap v130: 129 pgs: 129 active+clean; 5.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-09-12T03:33:33.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:33:32 smithi115 ceph-mon[34801]: pgmap v131: 129 pgs: 129 active+clean; 1.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 18 op/s 2024-09-12T03:33:33.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:33:32 smithi195 ceph-mon[34786]: pgmap v131: 129 pgs: 129 active+clean; 1.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 18 op/s 2024-09-12T03:33:33.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:33:32 smithi064 ceph-mon[30952]: pgmap v131: 129 pgs: 129 active+clean; 1.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 18 op/s 2024-09-12T03:33:35.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:33:34 smithi115 ceph-mon[34801]: pgmap v132: 129 pgs: 129 active+clean; 1.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-09-12T03:33:35.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:33:34 smithi195 ceph-mon[34786]: pgmap v132: 129 pgs: 129 active+clean; 1.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-09-12T03:33:35.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:33:34 smithi064 ceph-mon[30952]: pgmap v132: 129 pgs: 129 active+clean; 1.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-09-12T03:33:37.074 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:33:36 smithi064 ceph-mon[30952]: pgmap v133: 129 pgs: 129 active+clean; 1.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-09-12T03:33:37.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:33:36 smithi115 ceph-mon[34801]: pgmap v133: 129 pgs: 129 active+clean; 1.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-09-12T03:33:37.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:33:36 smithi195 ceph-mon[34786]: pgmap v133: 129 pgs: 129 active+clean; 1.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-09-12T03:33:39.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:33:38 smithi115 ceph-mon[34801]: pgmap v134: 129 pgs: 129 active+clean; 1.6 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 19 op/s 2024-09-12T03:33:39.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:33:38 smithi195 ceph-mon[34786]: pgmap v134: 129 pgs: 129 active+clean; 1.6 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 19 op/s 2024-09-12T03:33:39.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:33:38 smithi064 ceph-mon[30952]: pgmap v134: 129 pgs: 129 active+clean; 1.6 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 19 op/s 2024-09-12T03:33:41.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.043+0000 7f53b963c640 1 -- 172.21.15.64:0/3226992915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53b410c0d0 msgr2=0x7f53b410e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:41.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.043+0000 7f53b963c640 1 --2- 172.21.15.64:0/3226992915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53b410c0d0 0x7f53b410e550 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f53ac009ca0 tx=0x7f53ac02a950 comp rx=0 tx=0).stop 2024-09-12T03:33:41.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.043+0000 7f53b963c640 1 -- 172.21.15.64:0/3226992915 shutdown_connections 2024-09-12T03:33:41.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.043+0000 7f53b963c640 1 --2- 172.21.15.64:0/3226992915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53b410c0d0 0x7f53b410e550 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.043+0000 7f53b963c640 1 --2- 172.21.15.64:0/3226992915 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53b41035a0 0x7f53b410ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.043+0000 7f53b963c640 1 --2- 172.21.15.64:0/3226992915 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53b4102c80 0x7f53b4103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.043+0000 7f53b963c640 1 -- 172.21.15.64:0/3226992915 >> 172.21.15.64:0/3226992915 conn(0x7f53b4077990 msgr2=0x7f53b40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:41.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.043+0000 7f53b963c640 1 -- 172.21.15.64:0/3226992915 shutdown_connections 2024-09-12T03:33:41.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.043+0000 7f53b963c640 1 -- 172.21.15.64:0/3226992915 wait complete. 2024-09-12T03:33:41.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.044+0000 7f53b963c640 1 Processor -- start 2024-09-12T03:33:41.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.044+0000 7f53b963c640 1 -- start start 2024-09-12T03:33:41.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.044+0000 7f53b963c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53b4102c80 0x7f53b419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:41.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.044+0000 7f53b963c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53b41035a0 0x7f53b419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:41.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.044+0000 7f53b963c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53b410c0d0 0x7f53b419ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:41.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.044+0000 7f53b963c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f53b4111d60 con 0x7f53b4102c80 2024-09-12T03:33:41.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.044+0000 7f53b963c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f53b4111be0 con 0x7f53b410c0d0 2024-09-12T03:33:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.044+0000 7f53b963c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f53b4111ee0 con 0x7f53b41035a0 2024-09-12T03:33:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.044+0000 7f53b2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53b4102c80 0x7f53b419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.045+0000 7f53b2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53b4102c80 0x7f53b419a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53548/0 (socket says 172.21.15.64:53548) 2024-09-12T03:33:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.045+0000 7f53b2ffd640 1 -- 172.21.15.64:0/1315877909 learned_addr learned my addr 172.21.15.64:0/1315877909 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:33:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.045+0000 7f53b27fc640 1 --2- 172.21.15.64:0/1315877909 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53b41035a0 0x7f53b419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.045+0000 7f53b37fe640 1 --2- 172.21.15.64:0/1315877909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53b410c0d0 0x7f53b419ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.045+0000 7f53b2ffd640 1 -- 172.21.15.64:0/1315877909 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53b41035a0 msgr2=0x7f53b419a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.045+0000 7f53b2ffd640 1 --2- 172.21.15.64:0/1315877909 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53b41035a0 0x7f53b419a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.045+0000 7f53b2ffd640 1 -- 172.21.15.64:0/1315877909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53b410c0d0 msgr2=0x7f53b419ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.045+0000 7f53b2ffd640 1 --2- 172.21.15.64:0/1315877909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53b410c0d0 0x7f53b419ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.045+0000 7f53b2ffd640 1 -- 172.21.15.64:0/1315877909 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53b419f440 con 0x7f53b4102c80 2024-09-12T03:33:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.045+0000 7f53b37fe640 1 --2- 172.21.15.64:0/1315877909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53b410c0d0 0x7f53b419ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:33:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.045+0000 7f53b27fc640 1 --2- 172.21.15.64:0/1315877909 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53b41035a0 0x7f53b419a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:33:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.045+0000 7f53b2ffd640 1 --2- 172.21.15.64:0/1315877909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53b4102c80 0x7f53b419a450 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f53a400e9f0 tx=0x7f53a400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.045+0000 7f539bfff640 1 -- 172.21.15.64:0/1315877909 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a400cd80 con 0x7f53b4102c80 2024-09-12T03:33:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.045+0000 7f539bfff640 1 -- 172.21.15.64:0/1315877909 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53a40054d0 con 0x7f53b4102c80 2024-09-12T03:33:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.045+0000 7f53b963c640 1 -- 172.21.15.64:0/1315877909 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53b419f730 con 0x7f53b4102c80 2024-09-12T03:33:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.045+0000 7f539bfff640 1 -- 172.21.15.64:0/1315877909 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a4010490 con 0x7f53b4102c80 2024-09-12T03:33:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.045+0000 7f53b963c640 1 -- 172.21.15.64:0/1315877909 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53b41a6fc0 con 0x7f53b4102c80 2024-09-12T03:33:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.048+0000 7f539bfff640 1 -- 172.21.15.64:0/1315877909 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f53a4005020 con 0x7f53b4102c80 2024-09-12T03:33:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.048+0000 7f53bc3d4740 1 Processor -- start 2024-09-12T03:33:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.048+0000 7f53bc3d4740 1 -- start start 2024-09-12T03:33:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.048+0000 7f53bc3d4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa9a6b9af0 0x55fa9a6b9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.048+0000 7f539bfff640 1 --2- 172.21.15.64:0/1315877909 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f539c0a2e20 0x7f539c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.048+0000 7f53bc3d4740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9a6ba410 0x55fa9a6c4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.048+0000 7f53bc3d4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fa9a6c4cb0 0x55fa9a6c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.048+0000 7f53bc3d4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55fa9a61e2f0 con 0x55fa9a6b9af0 2024-09-12T03:33:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.048+0000 7f53bc3d4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55fa9a5c6270 con 0x55fa9a6c4cb0 2024-09-12T03:33:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.048+0000 7f53bc3d4740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fa9a61a2b0 con 0x55fa9a6ba410 2024-09-12T03:33:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.048+0000 7f53b37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa9a6b9af0 0x55fa9a6b9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.049+0000 7f53b37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa9a6b9af0 0x55fa9a6b9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53562/0 (socket says 172.21.15.64:53562) 2024-09-12T03:33:41.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.049+0000 7f53b37fe640 1 -- 172.21.15.64:0/1235004990 learned_addr learned my addr 172.21.15.64:0/1235004990 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:33:41.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.049+0000 7f53b2ffd640 1 --2- 172.21.15.64:0/1315877909 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f539c0a2e20 0x7f539c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:41.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.049+0000 7f539bfff640 1 -- 172.21.15.64:0/1315877909 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f53a40cfc50 con 0x7f53b4102c80 2024-09-12T03:33:41.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.049+0000 7f53b37fe640 1 --2- 172.21.15.64:0/1235004990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fa9a6c4cb0 0x55fa9a6c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:41.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.049+0000 7f53b27fc640 1 --2- 172.21.15.64:0/1235004990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9a6ba410 0x55fa9a6c4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:41.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.049+0000 7f53b37fe640 1 -- 172.21.15.64:0/1235004990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9a6ba410 msgr2=0x55fa9a6c4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:41.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.049+0000 7f53b37fe640 1 --2- 172.21.15.64:0/1235004990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9a6ba410 0x55fa9a6c4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.049+0000 7f53b37fe640 1 -- 172.21.15.64:0/1235004990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fa9a6c4cb0 msgr2=0x55fa9a6c70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:41.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.049+0000 7f53b37fe640 1 --2- 172.21.15.64:0/1235004990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fa9a6c4cb0 0x55fa9a6c70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.049+0000 7f53b37fe640 1 -- 172.21.15.64:0/1235004990 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa9a5c5340 con 0x55fa9a6b9af0 2024-09-12T03:33:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.049+0000 7f53b2ffd640 1 --2- 172.21.15.64:0/1315877909 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f539c0a2e20 0x7f539c0a52e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f53a409bd30 tx=0x7f53a401bb30 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.049+0000 7f53b37fe640 1 --2- 172.21.15.64:0/1235004990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa9a6b9af0 0x55fa9a6b9ed0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f53ac030030 tx=0x7f53ac009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ceef0c5425daa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.049+0000 7f53997fa640 1 -- 172.21.15.64:0/1235004990 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ac034070 con 0x55fa9a6b9af0 2024-09-12T03:33:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.049+0000 7f53997fa640 1 -- 172.21.15.64:0/1235004990 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53ac003e20 con 0x55fa9a6b9af0 2024-09-12T03:33:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.049+0000 7f53997fa640 1 -- 172.21.15.64:0/1235004990 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ac004030 con 0x55fa9a6b9af0 2024-09-12T03:33:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.050+0000 7f53bc3d4740 1 -- 172.21.15.64:0/1235004990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa9a6b9af0 msgr2=0x55fa9a6b9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.050+0000 7f53bc3d4740 1 --2- 172.21.15.64:0/1235004990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa9a6b9af0 0x55fa9a6b9ed0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f53ac030030 tx=0x7f53ac009360 comp rx=0 tx=0).stop 2024-09-12T03:33:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.050+0000 7f53bc3d4740 1 -- 172.21.15.64:0/1235004990 shutdown_connections 2024-09-12T03:33:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.050+0000 7f53bc3d4740 1 --2- 172.21.15.64:0/1235004990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fa9a6c4cb0 0x55fa9a6c70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.050+0000 7f53bc3d4740 1 --2- 172.21.15.64:0/1235004990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9a6ba410 0x55fa9a6c4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.050+0000 7f53bc3d4740 1 --2- 172.21.15.64:0/1235004990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa9a6b9af0 0x55fa9a6b9ed0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.050+0000 7f53bc3d4740 1 -- 172.21.15.64:0/1235004990 >> 172.21.15.64:0/1235004990 conn(0x55fa9a6b6060 msgr2=0x55fa9a6b6440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:41.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.050+0000 7f53bc3d4740 1 -- 172.21.15.64:0/1235004990 shutdown_connections 2024-09-12T03:33:41.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.050+0000 7f53bc3d4740 1 -- 172.21.15.64:0/1235004990 wait complete. 2024-09-12T03:33:41.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.051+0000 7f53bc3d4740 1 -- start start 2024-09-12T03:33:41.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.051+0000 7f53bc3d4740 1 Processor -- start 2024-09-12T03:33:41.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.051+0000 7f53bc3d4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fa9a6b9af0 0x55fa9a6c1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:41.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.051+0000 7f53bc3d4740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9a6ba410 0x55fa9a6c20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:41.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.051+0000 7f53bc3d4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa9a6c4cb0 0x55fa9a6c2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:41.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.051+0000 7f53bc3d4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55fa9a6c7880 con 0x55fa9a6c4cb0 2024-09-12T03:33:41.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.051+0000 7f53bc3d4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55fa9a5a7290 con 0x55fa9a6b9af0 2024-09-12T03:33:41.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.051+0000 7f53bc3d4740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fa9a6c7a00 con 0x55fa9a6ba410 2024-09-12T03:33:41.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.051+0000 7f53bc3d4740 10 client.0 _ll_register_callbacks cb 0x55fa9a5d8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:41.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.051+0000 7f53b37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa9a6c4cb0 0x55fa9a6c2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:41.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.051+0000 7f53b37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa9a6c4cb0 0x55fa9a6c2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53570/0 (socket says 172.21.15.64:53570) 2024-09-12T03:33:41.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.051+0000 7f53b37fe640 1 -- 172.21.15.64:0/197969075 learned_addr learned my addr 172.21.15.64:0/197969075 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:33:41.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.051+0000 7f53b37fe640 1 --2- 172.21.15.64:0/197969075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fa9a6b9af0 0x55fa9a6c1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:41.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.051+0000 7f53b27fc640 1 --2- 172.21.15.64:0/197969075 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9a6ba410 0x55fa9a6c20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:41.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.051+0000 7f53b37fe640 1 -- 172.21.15.64:0/197969075 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9a6ba410 msgr2=0x55fa9a6c20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:41.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.051+0000 7f53b37fe640 1 --2- 172.21.15.64:0/197969075 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9a6ba410 0x55fa9a6c20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.052+0000 7f53b37fe640 1 -- 172.21.15.64:0/197969075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fa9a6b9af0 msgr2=0x55fa9a6c1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:41.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.052+0000 7f53b37fe640 1 --2- 172.21.15.64:0/197969075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fa9a6b9af0 0x55fa9a6c1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.052+0000 7f53b37fe640 1 -- 172.21.15.64:0/197969075 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa9a331210 con 0x55fa9a6c4cb0 2024-09-12T03:33:41.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.052+0000 7f53b27fc640 1 --2- 172.21.15.64:0/197969075 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9a6ba410 0x55fa9a6c20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:33:41.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.052+0000 7f53b37fe640 1 --2- 172.21.15.64:0/197969075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa9a6c4cb0 0x55fa9a6c2610 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f53ac02fca0 tx=0x7f53ac02a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bb5cfdb2bebd670 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:41.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.052+0000 7f5396ff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:33:41.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.052+0000 7f5396ff5640 1 -- 172.21.15.64:0/197969075 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ac034040 con 0x55fa9a6c4cb0 2024-09-12T03:33:41.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.052+0000 7f5396ff5640 1 -- 172.21.15.64:0/197969075 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f53ac003b80 con 0x55fa9a6c4cb0 2024-09-12T03:33:41.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.052+0000 7f53bc3d4740 1 -- 172.21.15.64:0/197969075 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53ac003e20 con 0x55fa9a6c4cb0 2024-09-12T03:33:41.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.052+0000 7f5396ff5640 1 -- 172.21.15.64:0/197969075 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f53ac02f670 con 0x55fa9a6c4cb0 2024-09-12T03:33:41.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.052+0000 7f53bc3d4740 10 client.15702 fetch_fsmap learned FSMap version 25 2024-09-12T03:33:41.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.052+0000 7f53bc3d4740 1 -- 172.21.15.64:0/197969075 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fa9a3fd620 con 0x55fa9a6c4cb0 2024-09-12T03:33:41.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.052+0000 7f5396ff5640 1 -- 172.21.15.64:0/197969075 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53ac003e20 con 0x55fa9a6c4cb0 2024-09-12T03:33:41.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.052+0000 7f5396ff5640 1 -- 172.21.15.64:0/197969075 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f53ac0398c0 con 0x55fa9a6c4cb0 2024-09-12T03:33:41.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.052+0000 7f53bc3d4740 10 client.15702 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:33:41.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.052+0000 7f53bc3d4740 10 client.15702 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:33:41.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.052+0000 7f53bc3d4740 20 client.15702 populate_metadata read hostname 'smithi064' 2024-09-12T03:33:41.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.053+0000 7f53bc3d4740 1 --2- 172.21.15.64:0/197969075 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55fa9a6d4ab0 0x55fa9a6d6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:41.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.053+0000 7f53bc3d4740 4 client.15702 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:33:41.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.053+0000 7f53bc3d4740 1 -- 172.21.15.64:0/197969075 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa9a62ba20 con 0x55fa9a6d4ab0 2024-09-12T03:33:41.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.053+0000 7f53b27fc640 1 --2- 172.21.15.64:0/197969075 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55fa9a6d4ab0 0x55fa9a6d6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:41.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.053+0000 7f53b27fc640 1 --2- 172.21.15.64:0/197969075 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55fa9a6d4ab0 0x55fa9a6d6ea0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d3cf5c6a56546c3 server_cookie=a82f1260b4056b19 in_seq=0 out_seq=0 2024-09-12T03:33:41.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.053+0000 7f5396ff5640 10 client.15702 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:33:41.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.054+0000 7f5396ff5640 1 -- 172.21.15.64:0/197969075 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fa9a62ba20 con 0x55fa9a6d4ab0 2024-09-12T03:33:41.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.054+0000 7f5396ff5640 10 client.15702 handle_command_reply: tid=0 2024-09-12T03:33:41.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.054+0000 7f53bc3d4740 1 client.15702 shutdown 2024-09-12T03:33:41.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.054+0000 7f53bc3d4740 1 -- 172.21.15.64:0/197969075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa9a6c4cb0 msgr2=0x55fa9a6c2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:41.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.054+0000 7f53bc3d4740 1 --2- 172.21.15.64:0/197969075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa9a6c4cb0 0x55fa9a6c2610 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f53ac02fca0 tx=0x7f53ac02a950 comp rx=0 tx=0).stop 2024-09-12T03:33:41.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.054+0000 7f53bc3d4740 1 -- 172.21.15.64:0/197969075 shutdown_connections 2024-09-12T03:33:41.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.054+0000 7f53bc3d4740 1 --2- 172.21.15.64:0/197969075 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55fa9a6d4ab0 0x55fa9a6d6ea0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.054+0000 7f53bc3d4740 1 --2- 172.21.15.64:0/197969075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa9a6c4cb0 0x55fa9a6c2610 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.054+0000 7f53bc3d4740 1 --2- 172.21.15.64:0/197969075 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9a6ba410 0x55fa9a6c20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.054+0000 7f53bc3d4740 1 --2- 172.21.15.64:0/197969075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fa9a6b9af0 0x55fa9a6c1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.054+0000 7f5396ff5640 0 client.15702 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:33:41.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.054+0000 7f53bc3d4740 1 -- 172.21.15.64:0/197969075 >> 172.21.15.64:0/197969075 conn(0x55fa9a6b6060 msgr2=0x55fa9a6bbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:41.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.054+0000 7f53bc3d4740 1 -- 172.21.15.64:0/197969075 shutdown_connections 2024-09-12T03:33:41.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.054+0000 7f53bc3d4740 1 -- 172.21.15.64:0/197969075 wait complete. 2024-09-12T03:33:41.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.054+0000 7f53bc3d4740 20 client.15702 trim_cache size 0 max 16384 2024-09-12T03:33:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.069+0000 7f53bc3d4740 1 Processor -- start 2024-09-12T03:33:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.069+0000 7f53bc3d4740 1 -- start start 2024-09-12T03:33:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.069+0000 7f53bc3d4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fa9a6b9af0 0x55fa9a6bc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.069+0000 7f53bc3d4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa9a6ba410 0x55fa9a6c2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.069+0000 7f53bc3d4740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9a6c4cb0 0x55fa9a6d9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.069+0000 7f53bc3d4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f53b41b3a80 con 0x55fa9a6ba410 2024-09-12T03:33:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.069+0000 7f53bc3d4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f53b41b1350 con 0x55fa9a6b9af0 2024-09-12T03:33:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.069+0000 7f53bc3d4740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f53b41af1a0 con 0x55fa9a6c4cb0 2024-09-12T03:33:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.069+0000 7f53b27fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa9a6ba410 0x55fa9a6c2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.069+0000 7f53b27fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa9a6ba410 0x55fa9a6c2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53572/0 (socket says 172.21.15.64:53572) 2024-09-12T03:33:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.069+0000 7f53b27fc640 1 -- 172.21.15.64:0/247514812 learned_addr learned my addr 172.21.15.64:0/247514812 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:33:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.069+0000 7f53b37fe640 1 --2- 172.21.15.64:0/247514812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fa9a6b9af0 0x55fa9a6bc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.069+0000 7f53b37fe640 1 --2- 172.21.15.64:0/247514812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9a6c4cb0 0x55fa9a6d9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.069+0000 7f53b27fc640 1 -- 172.21.15.64:0/247514812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9a6c4cb0 msgr2=0x55fa9a6d9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.069+0000 7f53b27fc640 1 --2- 172.21.15.64:0/247514812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9a6c4cb0 0x55fa9a6d9ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.069+0000 7f53b27fc640 1 -- 172.21.15.64:0/247514812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fa9a6b9af0 msgr2=0x55fa9a6bc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.069+0000 7f53b27fc640 1 --2- 172.21.15.64:0/247514812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fa9a6b9af0 0x55fa9a6bc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.069+0000 7f53b27fc640 1 -- 172.21.15.64:0/247514812 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa9a6ba240 con 0x55fa9a6ba410 2024-09-12T03:33:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.070+0000 7f53b27fc640 1 --2- 172.21.15.64:0/247514812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa9a6ba410 0x55fa9a6c2440 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x55fa9a6c1400 tx=0x7f53a8014840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540d830235f3830a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.070+0000 7f5398ff9640 1 -- 172.21.15.64:0/247514812 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a80142d0 con 0x55fa9a6ba410 2024-09-12T03:33:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.070+0000 7f5398ff9640 1 -- 172.21.15.64:0/247514812 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53a801d910 con 0x55fa9a6ba410 2024-09-12T03:33:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.070+0000 7f5398ff9640 1 -- 172.21.15.64:0/247514812 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a80129e0 con 0x55fa9a6ba410 2024-09-12T03:33:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.070+0000 7f53bc3d4740 1 -- 172.21.15.64:0/247514812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa9a6ba410 msgr2=0x55fa9a6c2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.070+0000 7f53bc3d4740 1 --2- 172.21.15.64:0/247514812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa9a6ba410 0x55fa9a6c2440 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x55fa9a6c1400 tx=0x7f53a8014840 comp rx=0 tx=0).stop 2024-09-12T03:33:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.071+0000 7f53bc3d4740 1 -- 172.21.15.64:0/247514812 shutdown_connections 2024-09-12T03:33:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.071+0000 7f53bc3d4740 1 --2- 172.21.15.64:0/247514812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9a6c4cb0 0x55fa9a6d9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.071+0000 7f53bc3d4740 1 --2- 172.21.15.64:0/247514812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa9a6ba410 0x55fa9a6c2440 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.071+0000 7f53bc3d4740 1 --2- 172.21.15.64:0/247514812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fa9a6b9af0 0x55fa9a6bc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.071+0000 7f53bc3d4740 1 -- 172.21.15.64:0/247514812 >> 172.21.15.64:0/247514812 conn(0x55fa9a6b6060 msgr2=0x55fa9a6c3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.071+0000 7f53bc3d4740 1 -- 172.21.15.64:0/247514812 shutdown_connections 2024-09-12T03:33:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.071+0000 7f53bc3d4740 1 -- 172.21.15.64:0/247514812 wait complete. 2024-09-12T03:33:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.071+0000 7f53bc3d4740 1 -- start start 2024-09-12T03:33:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.071+0000 7f53bc3d4740 1 Processor -- start 2024-09-12T03:33:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.071+0000 7f53bc3d4740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fa9a6b9af0 0x55fa9a6da4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.071+0000 7f53bc3d4740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9a6ba410 0x55fa9a6d7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.071+0000 7f53bc3d4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa9a6c4cb0 0x55fa9a6d7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.071+0000 7f53bc3d4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f53b41af810 con 0x55fa9a6c4cb0 2024-09-12T03:33:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.071+0000 7f53bc3d4740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f53b41b0c80 con 0x55fa9a6b9af0 2024-09-12T03:33:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.071+0000 7f53bc3d4740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f53b41afee0 con 0x55fa9a6ba410 2024-09-12T03:33:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.071+0000 7f53bc3d4740 10 client.0 _ll_register_callbacks cb 0x7f53a4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.071+0000 7f53b37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa9a6c4cb0 0x55fa9a6d7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.071+0000 7f53b37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa9a6c4cb0 0x55fa9a6d7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53574/0 (socket says 172.21.15.64:53574) 2024-09-12T03:33:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.071+0000 7f53b37fe640 1 -- 172.21.15.64:0/1483543062 learned_addr learned my addr 172.21.15.64:0/1483543062 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:33:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.071+0000 7f53b37fe640 1 --2- 172.21.15.64:0/1483543062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fa9a6b9af0 0x55fa9a6da4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.072+0000 7f53b27fc640 1 --2- 172.21.15.64:0/1483543062 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9a6ba410 0x55fa9a6d7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.072+0000 7f53b37fe640 1 -- 172.21.15.64:0/1483543062 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9a6ba410 msgr2=0x55fa9a6d7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.072+0000 7f53b37fe640 1 --2- 172.21.15.64:0/1483543062 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9a6ba410 0x55fa9a6d7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.072+0000 7f53b37fe640 1 -- 172.21.15.64:0/1483543062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fa9a6b9af0 msgr2=0x55fa9a6da4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.072+0000 7f53b37fe640 1 --2- 172.21.15.64:0/1483543062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fa9a6b9af0 0x55fa9a6da4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.072+0000 7f53b37fe640 1 -- 172.21.15.64:0/1483543062 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa9a6d8420 con 0x55fa9a6c4cb0 2024-09-12T03:33:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.072+0000 7f53b27fc640 1 --2- 172.21.15.64:0/1483543062 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9a6ba410 0x55fa9a6d7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:33:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.072+0000 7f53b37fe640 1 --2- 172.21.15.64:0/1483543062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa9a6c4cb0 0x55fa9a6d7d80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f53ac039c60 tx=0x7f53ac02b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d6828dc2d06c02f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.072+0000 7f5396ff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:33:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.072+0000 7f5396ff5640 1 -- 172.21.15.64:0/1483543062 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ac034040 con 0x55fa9a6c4cb0 2024-09-12T03:33:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.072+0000 7f5396ff5640 1 -- 172.21.15.64:0/1483543062 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f53ac002030 con 0x55fa9a6c4cb0 2024-09-12T03:33:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.072+0000 7f53bc3d4740 1 -- 172.21.15.64:0/1483543062 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53a801d910 con 0x55fa9a6c4cb0 2024-09-12T03:33:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.072+0000 7f5396ff5640 1 -- 172.21.15.64:0/1483543062 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f53ac0098c0 con 0x55fa9a6c4cb0 2024-09-12T03:33:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.072+0000 7f53bc3d4740 10 client.15708 fetch_fsmap learned FSMap version 25 2024-09-12T03:33:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.072+0000 7f53bc3d4740 1 -- 172.21.15.64:0/1483543062 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fa9a73f290 con 0x55fa9a6c4cb0 2024-09-12T03:33:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.072+0000 7f5396ff5640 1 -- 172.21.15.64:0/1483543062 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53a801d910 con 0x55fa9a6c4cb0 2024-09-12T03:33:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.073+0000 7f5396ff5640 1 -- 172.21.15.64:0/1483543062 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f53ac030760 con 0x55fa9a6c4cb0 2024-09-12T03:33:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.073+0000 7f53bc3d4740 10 client.15708 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:33:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.073+0000 7f53bc3d4740 10 client.15708 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:33:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.073+0000 7f53bc3d4740 20 client.15708 populate_metadata read hostname 'smithi064' 2024-09-12T03:33:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.073+0000 7f53bc3d4740 1 --2- 172.21.15.64:0/1483543062 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55fa9a73f7f0 0x55fa9a741c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.073+0000 7f53bc3d4740 4 client.15708 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:33:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.073+0000 7f53bc3d4740 1 -- 172.21.15.64:0/1483543062 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fa9a625d70 con 0x55fa9a73f7f0 2024-09-12T03:33:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.073+0000 7f53b27fc640 1 --2- 172.21.15.64:0/1483543062 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55fa9a73f7f0 0x55fa9a741c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.073+0000 7f5396ff5640 10 client.15708 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:33:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.073+0000 7f53b27fc640 1 --2- 172.21.15.64:0/1483543062 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55fa9a73f7f0 0x55fa9a741c10 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b3553e547591ed0 server_cookie=9ee4036d1bac74ae in_seq=0 out_seq=0 2024-09-12T03:33:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.086+0000 7f5396ff5640 1 -- 172.21.15.64:0/1483543062 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fa9a625d70 con 0x55fa9a73f7f0 2024-09-12T03:33:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.086+0000 7f5396ff5640 10 client.15708 handle_command_reply: tid=0 2024-09-12T03:33:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.086+0000 7f53bc3d4740 1 client.15708 shutdown 2024-09-12T03:33:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.087+0000 7f53bc3d4740 1 -- 172.21.15.64:0/1483543062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa9a6c4cb0 msgr2=0x55fa9a6d7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.087+0000 7f53bc3d4740 1 --2- 172.21.15.64:0/1483543062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa9a6c4cb0 0x55fa9a6d7d80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f53ac039c60 tx=0x7f53ac02b0d0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.087+0000 7f53bc3d4740 1 -- 172.21.15.64:0/1483543062 shutdown_connections 2024-09-12T03:33:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.087+0000 7f53bc3d4740 1 --2- 172.21.15.64:0/1483543062 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55fa9a73f7f0 0x55fa9a741c10 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.087+0000 7f53bc3d4740 1 --2- 172.21.15.64:0/1483543062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa9a6c4cb0 0x55fa9a6d7d80 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.087+0000 7f53bc3d4740 1 --2- 172.21.15.64:0/1483543062 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa9a6ba410 0x55fa9a6d7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.087+0000 7f53bc3d4740 1 --2- 172.21.15.64:0/1483543062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fa9a6b9af0 0x55fa9a6da4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.087+0000 7f53bc3d4740 1 -- 172.21.15.64:0/1483543062 >> 172.21.15.64:0/1483543062 conn(0x55fa9a6b6060 msgr2=0x55fa9a6cfcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.087+0000 7f5396ff5640 0 client.15708 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:33:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.087+0000 7f53bc3d4740 1 -- 172.21.15.64:0/1483543062 shutdown_connections 2024-09-12T03:33:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.087+0000 7f53bc3d4740 1 -- 172.21.15.64:0/1483543062 wait complete. 2024-09-12T03:33:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.087+0000 7f53bc3d4740 20 client.15708 trim_cache size 0 max 16384 2024-09-12T03:33:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:33:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:33:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:33:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:33:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.087+0000 7f53b963c640 1 -- 172.21.15.64:0/1315877909 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f539c0a2e20 msgr2=0x7f539c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.087+0000 7f53b963c640 1 --2- 172.21.15.64:0/1315877909 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f539c0a2e20 0x7f539c0a52e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f53a409bd30 tx=0x7f53a401bb30 comp rx=0 tx=0).stop 2024-09-12T03:33:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.087+0000 7f53b963c640 1 -- 172.21.15.64:0/1315877909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53b4102c80 msgr2=0x7f53b419a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.087+0000 7f53b963c640 1 --2- 172.21.15.64:0/1315877909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53b4102c80 0x7f53b419a450 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f53a400e9f0 tx=0x7f53a400edf0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.087+0000 7f53b963c640 1 -- 172.21.15.64:0/1315877909 shutdown_connections 2024-09-12T03:33:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.087+0000 7f53b963c640 1 --2- 172.21.15.64:0/1315877909 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f539c0a2e20 0x7f539c0a52e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.087+0000 7f53b963c640 1 --2- 172.21.15.64:0/1315877909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53b410c0d0 0x7f53b419ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.087+0000 7f53b963c640 1 --2- 172.21.15.64:0/1315877909 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53b41035a0 0x7f53b419a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.087+0000 7f53b963c640 1 --2- 172.21.15.64:0/1315877909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53b4102c80 0x7f53b419a450 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.087+0000 7f53b963c640 1 -- 172.21.15.64:0/1315877909 >> 172.21.15.64:0/1315877909 conn(0x7f53b4077990 msgr2=0x7f53b4100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.088+0000 7f53b963c640 1 -- 172.21.15.64:0/1315877909 shutdown_connections 2024-09-12T03:33:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:41.088+0000 7f53b963c640 1 -- 172.21.15.64:0/1315877909 wait complete. 2024-09-12T03:33:41.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:33:40 smithi115 ceph-mon[34801]: pgmap v135: 129 pgs: 129 active+clean; 1.6 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-12T03:33:41.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:33:40 smithi195 ceph-mon[34786]: pgmap v135: 129 pgs: 129 active+clean; 1.6 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-12T03:33:41.271 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:33:40 smithi064 ceph-mon[30952]: pgmap v135: 129 pgs: 129 active+clean; 1.6 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-12T03:33:43.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:33:42 smithi115 ceph-mon[34801]: pgmap v136: 129 pgs: 129 active+clean; 1.8 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 45 KiB/s wr, 20 op/s 2024-09-12T03:33:43.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:33:42 smithi195 ceph-mon[34786]: pgmap v136: 129 pgs: 129 active+clean; 1.8 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 45 KiB/s wr, 20 op/s 2024-09-12T03:33:43.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:33:42 smithi064 ceph-mon[30952]: pgmap v136: 129 pgs: 129 active+clean; 1.8 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 45 KiB/s wr, 20 op/s 2024-09-12T03:33:45.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:33:44 smithi115 ceph-mon[34801]: pgmap v137: 129 pgs: 129 active+clean; 1.8 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-09-12T03:33:45.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:33:44 smithi195 ceph-mon[34786]: pgmap v137: 129 pgs: 129 active+clean; 1.8 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-09-12T03:33:45.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:33:44 smithi064 ceph-mon[30952]: pgmap v137: 129 pgs: 129 active+clean; 1.8 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-09-12T03:33:47.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:33:46 smithi115 ceph-mon[34801]: pgmap v138: 129 pgs: 129 active+clean; 1.8 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-09-12T03:33:47.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:33:46 smithi195 ceph-mon[34786]: pgmap v138: 129 pgs: 129 active+clean; 1.8 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-09-12T03:33:47.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:33:46 smithi064 ceph-mon[30952]: pgmap v138: 129 pgs: 129 active+clean; 1.8 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-09-12T03:33:49.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:33:48 smithi115 ceph-mon[34801]: pgmap v139: 129 pgs: 129 active+clean; 1.9 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 17 op/s 2024-09-12T03:33:49.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:33:48 smithi195 ceph-mon[34786]: pgmap v139: 129 pgs: 129 active+clean; 1.9 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 17 op/s 2024-09-12T03:33:49.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:33:48 smithi064 ceph-mon[30952]: pgmap v139: 129 pgs: 129 active+clean; 1.9 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 17 op/s 2024-09-12T03:33:51.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:33:50 smithi115 ceph-mon[34801]: pgmap v140: 129 pgs: 129 active+clean; 1.9 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 12 op/s 2024-09-12T03:33:51.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:33:50 smithi195 ceph-mon[34786]: pgmap v140: 129 pgs: 129 active+clean; 1.9 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 12 op/s 2024-09-12T03:33:51.271 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:33:50 smithi064 ceph-mon[30952]: pgmap v140: 129 pgs: 129 active+clean; 1.9 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 12 op/s 2024-09-12T03:33:53.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:33:52 smithi115 ceph-mon[34801]: pgmap v141: 129 pgs: 129 active+clean; 2.0 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 17 op/s 2024-09-12T03:33:53.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:33:52 smithi195 ceph-mon[34786]: pgmap v141: 129 pgs: 129 active+clean; 2.0 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 17 op/s 2024-09-12T03:33:53.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:33:52 smithi064 ceph-mon[30952]: pgmap v141: 129 pgs: 129 active+clean; 2.0 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 17 op/s 2024-09-12T03:33:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.186+0000 7fbbca18d640 1 -- 172.21.15.64:0/996484671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbc4102480 msgr2=0x7fbbc410e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.186+0000 7fbbca18d640 1 --2- 172.21.15.64:0/996484671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbc4102480 0x7fbbc410e9b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc009cd0 tx=0x7fbbbc02a8e0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.186+0000 7fbbca18d640 1 -- 172.21.15.64:0/996484671 shutdown_connections 2024-09-12T03:33:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.186+0000 7fbbca18d640 1 --2- 172.21.15.64:0/996484671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbc4102480 0x7fbbc410e9b0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.186+0000 7fbbca18d640 1 --2- 172.21.15.64:0/996484671 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbbc4101ae0 0x7fbbc4101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.186+0000 7fbbca18d640 1 --2- 172.21.15.64:0/996484671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbc4107b80 0x7fbbc4107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.186+0000 7fbbca18d640 1 -- 172.21.15.64:0/996484671 >> 172.21.15.64:0/996484671 conn(0x7fbbc40fd7e0 msgr2=0x7fbbc40ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.186+0000 7fbbca18d640 1 -- 172.21.15.64:0/996484671 shutdown_connections 2024-09-12T03:33:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.186+0000 7fbbca18d640 1 -- 172.21.15.64:0/996484671 wait complete. 2024-09-12T03:33:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.186+0000 7fbbca18d640 1 Processor -- start 2024-09-12T03:33:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.187+0000 7fbbca18d640 1 -- start start 2024-09-12T03:33:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.187+0000 7fbbca18d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbbc4101ae0 0x7fbbc4198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.187+0000 7fbbca18d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbc4102480 0x7fbbc4198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.187+0000 7fbbca18d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbc4107b80 0x7fbbc419cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.187+0000 7fbbca18d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc4111e30 con 0x7fbbc4107b80 2024-09-12T03:33:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.187+0000 7fbbca18d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc4111cb0 con 0x7fbbc4102480 2024-09-12T03:33:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.187+0000 7fbbca18d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc4111fb0 con 0x7fbbc4101ae0 2024-09-12T03:33:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.187+0000 7fbbc3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbc4107b80 0x7fbbc419cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.187+0000 7fbbc3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbc4107b80 0x7fbbc419cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56524/0 (socket says 172.21.15.64:56524) 2024-09-12T03:33:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.187+0000 7fbbc3fff640 1 -- 172.21.15.64:0/1297358072 learned_addr learned my addr 172.21.15.64:0/1297358072 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:33:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.187+0000 7fbbc37fe640 1 --2- 172.21.15.64:0/1297358072 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbbc4101ae0 0x7fbbc4198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.187+0000 7fbbbaffd640 1 --2- 172.21.15.64:0/1297358072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbc4102480 0x7fbbc4198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.187+0000 7fbbc3fff640 1 -- 172.21.15.64:0/1297358072 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbbc4101ae0 msgr2=0x7fbbc4198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.188+0000 7fbbc3fff640 1 --2- 172.21.15.64:0/1297358072 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbbc4101ae0 0x7fbbc4198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.188+0000 7fbbc3fff640 1 -- 172.21.15.64:0/1297358072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbc4102480 msgr2=0x7fbbc4198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.188+0000 7fbbc3fff640 1 --2- 172.21.15.64:0/1297358072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbc4102480 0x7fbbc4198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.188+0000 7fbbc3fff640 1 -- 172.21.15.64:0/1297358072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbc419d240 con 0x7fbbc4107b80 2024-09-12T03:33:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.188+0000 7fbbc3fff640 1 --2- 172.21.15.64:0/1297358072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbc4107b80 0x7fbbc419cb10 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc005e30 tx=0x7fbbbc003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.188+0000 7fbbc17fa640 1 -- 172.21.15.64:0/1297358072 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc004240 con 0x7fbbc4107b80 2024-09-12T03:33:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.188+0000 7fbbc17fa640 1 -- 172.21.15.64:0/1297358072 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbbc031ce0 con 0x7fbbc4107b80 2024-09-12T03:33:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.188+0000 7fbbc17fa640 1 -- 172.21.15.64:0/1297358072 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc046480 con 0x7fbbc4107b80 2024-09-12T03:33:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.188+0000 7fbbca18d640 1 -- 172.21.15.64:0/1297358072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbc419d4d0 con 0x7fbbc4107b80 2024-09-12T03:33:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.188+0000 7fbbca18d640 1 -- 172.21.15.64:0/1297358072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbc41a4dc0 con 0x7fbbc4107b80 2024-09-12T03:33:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.191+0000 7fbbc17fa640 1 -- 172.21.15.64:0/1297358072 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fbbbc0043e0 con 0x7fbbc4107b80 2024-09-12T03:33:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.191+0000 7fbbcd007740 1 Processor -- start 2024-09-12T03:33:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.191+0000 7fbbcd007740 1 -- start start 2024-09-12T03:33:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.191+0000 7fbbcd007740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56013ebc59b0 0x56013ebc5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.191+0000 7fbbcd007740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56013ebc62d0 0x56013ebd4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.191+0000 7fbbcd007740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56013ebd50c0 0x56013ebd74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.191+0000 7fbbcd007740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56013eb2a2f0 con 0x56013ebc62d0 2024-09-12T03:33:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.191+0000 7fbbcd007740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56013ead2270 con 0x56013ebd50c0 2024-09-12T03:33:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.191+0000 7fbbcd007740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56013eb262b0 con 0x56013ebc59b0 2024-09-12T03:33:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.191+0000 7fbbc37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56013ebc62d0 0x56013ebd4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.191+0000 7fbbc37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56013ebc62d0 0x56013ebd4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56540/0 (socket says 172.21.15.64:56540) 2024-09-12T03:33:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.191+0000 7fbbc37fe640 1 -- 172.21.15.64:0/1809285159 learned_addr learned my addr 172.21.15.64:0/1809285159 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:33:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.191+0000 7fbbbaffd640 1 --2- 172.21.15.64:0/1809285159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56013ebd50c0 0x56013ebd74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.191+0000 7fbbc17fa640 1 --2- 172.21.15.64:0/1297358072 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fbbb00a2e20 0x7fbbb00a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.192+0000 7fbbbaffd640 1 --2- 172.21.15.64:0/1809285159 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56013ebc59b0 0x56013ebc5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.192+0000 7fbbc37fe640 1 -- 172.21.15.64:0/1809285159 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56013ebc59b0 msgr2=0x56013ebc5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.192+0000 7fbbc37fe640 1 --2- 172.21.15.64:0/1809285159 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56013ebc59b0 0x56013ebc5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.192+0000 7fbbc37fe640 1 -- 172.21.15.64:0/1809285159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56013ebd50c0 msgr2=0x56013ebd74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.192+0000 7fbbc37fe640 1 --2- 172.21.15.64:0/1809285159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56013ebd50c0 0x56013ebd74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.192+0000 7fbbc37fe640 1 -- 172.21.15.64:0/1809285159 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56013ead1340 con 0x56013ebc62d0 2024-09-12T03:33:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.192+0000 7fbbbaffd640 1 --2- 172.21.15.64:0/1809285159 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56013ebc59b0 0x56013ebc5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:33:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.192+0000 7fbbc3fff640 1 --2- 172.21.15.64:0/1297358072 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fbbb00a2e20 0x7fbbb00a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.192+0000 7fbbc37fe640 1 --2- 172.21.15.64:0/1809285159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56013ebc62d0 0x56013ebd4b80 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fbbb400bc90 tx=0x7fbbb40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c584a82bfa69f89 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.192+0000 7fbbc17fa640 1 -- 172.21.15.64:0/1297358072 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7fbbbc02e030 con 0x7fbbc4107b80 2024-09-12T03:33:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.192+0000 7fbbb9ffb640 1 -- 172.21.15.64:0/1809285159 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb401c480 con 0x56013ebc62d0 2024-09-12T03:33:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.192+0000 7fbbb9ffb640 1 -- 172.21.15.64:0/1809285159 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbb401cad0 con 0x56013ebc62d0 2024-09-12T03:33:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.192+0000 7fbbb9ffb640 1 -- 172.21.15.64:0/1809285159 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb4011af0 con 0x56013ebc62d0 2024-09-12T03:33:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.193+0000 7fbbcd007740 1 -- 172.21.15.64:0/1809285159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56013ebc62d0 msgr2=0x56013ebd4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.193+0000 7fbbc3fff640 1 --2- 172.21.15.64:0/1297358072 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fbbb00a2e20 0x7fbbb00a52e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc0bc970 tx=0x7fbbbc0095c0 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.193+0000 7fbbcd007740 1 --2- 172.21.15.64:0/1809285159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56013ebc62d0 0x56013ebd4b80 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fbbb400bc90 tx=0x7fbbb40107a0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.193+0000 7fbbcd007740 1 -- 172.21.15.64:0/1809285159 shutdown_connections 2024-09-12T03:33:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.193+0000 7fbbcd007740 1 --2- 172.21.15.64:0/1809285159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56013ebd50c0 0x56013ebd74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.193+0000 7fbbcd007740 1 --2- 172.21.15.64:0/1809285159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56013ebc62d0 0x56013ebd4b80 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.193+0000 7fbbcd007740 1 --2- 172.21.15.64:0/1809285159 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56013ebc59b0 0x56013ebc5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.193+0000 7fbbcd007740 1 -- 172.21.15.64:0/1809285159 >> 172.21.15.64:0/1809285159 conn(0x56013ebc2060 msgr2=0x56013ebc2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.193+0000 7fbbcd007740 1 -- 172.21.15.64:0/1809285159 shutdown_connections 2024-09-12T03:33:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.193+0000 7fbbcd007740 1 -- 172.21.15.64:0/1809285159 wait complete. 2024-09-12T03:33:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.193+0000 7fbbcd007740 1 -- start start 2024-09-12T03:33:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.194+0000 7fbbcd007740 1 Processor -- start 2024-09-12T03:33:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.194+0000 7fbbcd007740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56013ebc59b0 0x56013ebd2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.194+0000 7fbbcd007740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56013ebc62d0 0x56013ebd2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.194+0000 7fbbcd007740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56013ebd50c0 0x56013ebd2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.194+0000 7fbbcd007740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56013ebd7c90 con 0x56013ebd50c0 2024-09-12T03:33:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.194+0000 7fbbcd007740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56013eab3290 con 0x56013ebc62d0 2024-09-12T03:33:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.194+0000 7fbbcd007740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56013ebd7e10 con 0x56013ebc59b0 2024-09-12T03:33:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.194+0000 7fbbcd007740 10 client.0 _ll_register_callbacks cb 0x56013eae4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.194+0000 7fbbbaffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56013ebd50c0 0x56013ebd2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.194+0000 7fbbbaffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56013ebd50c0 0x56013ebd2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56556/0 (socket says 172.21.15.64:56556) 2024-09-12T03:33:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.194+0000 7fbbbaffd640 1 -- 172.21.15.64:0/2619177720 learned_addr learned my addr 172.21.15.64:0/2619177720 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:33:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.194+0000 7fbbc37fe640 1 --2- 172.21.15.64:0/2619177720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56013ebc62d0 0x56013ebd2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.194+0000 7fbbbaffd640 1 --2- 172.21.15.64:0/2619177720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56013ebc59b0 0x56013ebd2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.194+0000 7fbbbaffd640 1 -- 172.21.15.64:0/2619177720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56013ebc59b0 msgr2=0x56013ebd2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.194+0000 7fbbbaffd640 1 --2- 172.21.15.64:0/2619177720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56013ebc59b0 0x56013ebd2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.194+0000 7fbbbaffd640 1 -- 172.21.15.64:0/2619177720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56013ebc62d0 msgr2=0x56013ebd2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.194+0000 7fbbbaffd640 1 --2- 172.21.15.64:0/2619177720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56013ebc62d0 0x56013ebd2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.194+0000 7fbbbaffd640 1 -- 172.21.15.64:0/2619177720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56013e83d210 con 0x56013ebd50c0 2024-09-12T03:33:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.195+0000 7fbbbaffd640 1 --2- 172.21.15.64:0/2619177720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56013ebd50c0 0x56013ebd2ac0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fbbac00c010 tx=0x7fbbac014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c65a2f897caae2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.195+0000 7fbba2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:33:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.195+0000 7fbba2ffd640 1 -- 172.21.15.64:0/2619177720 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac011b70 con 0x56013ebd50c0 2024-09-12T03:33:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.195+0000 7fbbcd007740 1 -- 172.21.15.64:0/2619177720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbb401cad0 con 0x56013ebd50c0 2024-09-12T03:33:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.195+0000 7fbba2ffd640 1 -- 172.21.15.64:0/2619177720 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7fbbac011d10 con 0x56013ebd50c0 2024-09-12T03:33:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.195+0000 7fbba2ffd640 1 -- 172.21.15.64:0/2619177720 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbbac026050 con 0x56013ebd50c0 2024-09-12T03:33:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.195+0000 7fbbcd007740 10 client.15720 fetch_fsmap learned FSMap version 25 2024-09-12T03:33:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.195+0000 7fbbcd007740 1 -- 172.21.15.64:0/2619177720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56013e909620 con 0x56013ebd50c0 2024-09-12T03:33:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.195+0000 7fbba2ffd640 1 -- 172.21.15.64:0/2619177720 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbb401cad0 con 0x56013ebd50c0 2024-09-12T03:33:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.195+0000 7fbba2ffd640 1 -- 172.21.15.64:0/2619177720 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fbbac00f3f0 con 0x56013ebd50c0 2024-09-12T03:33:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.195+0000 7fbbcd007740 10 client.15720 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:33:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.195+0000 7fbbcd007740 10 client.15720 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:33:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.195+0000 7fbbcd007740 20 client.15720 populate_metadata read hostname 'smithi064' 2024-09-12T03:33:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.195+0000 7fbbcd007740 1 --2- 172.21.15.64:0/2619177720 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x56013ebe5050 0x56013ebe7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.195+0000 7fbbcd007740 4 client.15720 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:33:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.195+0000 7fbbcd007740 1 -- 172.21.15.64:0/2619177720 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56013eb37a20 con 0x56013ebe5050 2024-09-12T03:33:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.196+0000 7fbbc37fe640 1 --2- 172.21.15.64:0/2619177720 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x56013ebe5050 0x56013ebe7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.196+0000 7fbbc37fe640 1 --2- 172.21.15.64:0/2619177720 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x56013ebe5050 0x56013ebe7440 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbc8f1efbbe6df40 server_cookie=2f6cd9e9e8fa6f96 in_seq=0 out_seq=0 2024-09-12T03:33:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.196+0000 7fbba2ffd640 10 client.15720 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:33:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.197+0000 7fbba2ffd640 1 -- 172.21.15.64:0/2619177720 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56013eb37a20 con 0x56013ebe5050 2024-09-12T03:33:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.197+0000 7fbba2ffd640 10 client.15720 handle_command_reply: tid=0 2024-09-12T03:33:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.197+0000 7fbbcd007740 1 client.15720 shutdown 2024-09-12T03:33:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.197+0000 7fbbcd007740 1 -- 172.21.15.64:0/2619177720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56013ebd50c0 msgr2=0x56013ebd2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.197+0000 7fbbcd007740 1 --2- 172.21.15.64:0/2619177720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56013ebd50c0 0x56013ebd2ac0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fbbac00c010 tx=0x7fbbac014040 comp rx=0 tx=0).stop 2024-09-12T03:33:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.197+0000 7fbbcd007740 1 -- 172.21.15.64:0/2619177720 shutdown_connections 2024-09-12T03:33:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.197+0000 7fbbcd007740 1 --2- 172.21.15.64:0/2619177720 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x56013ebe5050 0x56013ebe7440 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.197+0000 7fbbcd007740 1 --2- 172.21.15.64:0/2619177720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56013ebd50c0 0x56013ebd2ac0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.197+0000 7fbbcd007740 1 --2- 172.21.15.64:0/2619177720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56013ebc62d0 0x56013ebd2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.197+0000 7fbba2ffd640 0 client.15720 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:33:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.197+0000 7fbbcd007740 1 --2- 172.21.15.64:0/2619177720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56013ebc59b0 0x56013ebd2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.197+0000 7fbbcd007740 1 -- 172.21.15.64:0/2619177720 >> 172.21.15.64:0/2619177720 conn(0x56013ebc2060 msgr2=0x56013ebc7d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.197+0000 7fbbcd007740 1 -- 172.21.15.64:0/2619177720 shutdown_connections 2024-09-12T03:33:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.197+0000 7fbbcd007740 1 -- 172.21.15.64:0/2619177720 wait complete. 2024-09-12T03:33:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.197+0000 7fbbcd007740 20 client.15720 trim_cache size 0 max 16384 2024-09-12T03:33:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.212+0000 7fbbcd007740 1 Processor -- start 2024-09-12T03:33:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.212+0000 7fbbcd007740 1 -- start start 2024-09-12T03:33:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.212+0000 7fbbcd007740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56013ebc59b0 0x56013ebecd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.212+0000 7fbbcd007740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56013ebc62d0 0x56013ebed250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.212+0000 7fbbcd007740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56013ebd50c0 0x56013ebcf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.212+0000 7fbbcd007740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc41af5f0 con 0x56013ebd50c0 2024-09-12T03:33:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.212+0000 7fbbcd007740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc41acec0 con 0x56013ebc59b0 2024-09-12T03:33:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.212+0000 7fbbcd007740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc41aad10 con 0x56013ebc62d0 2024-09-12T03:33:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.212+0000 7fbbbaffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56013ebd50c0 0x56013ebcf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.212+0000 7fbbbaffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56013ebd50c0 0x56013ebcf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56566/0 (socket says 172.21.15.64:56566) 2024-09-12T03:33:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.212+0000 7fbbbaffd640 1 -- 172.21.15.64:0/3688121205 learned_addr learned my addr 172.21.15.64:0/3688121205 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:33:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.213+0000 7fbbbaffd640 1 --2- 172.21.15.64:0/3688121205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56013ebc59b0 0x56013ebecd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.213+0000 7fbbbaffd640 1 -- 172.21.15.64:0/3688121205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56013ebc62d0 msgr2=0x56013ebed250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.213+0000 7fbbc37fe640 1 --2- 172.21.15.64:0/3688121205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56013ebc62d0 0x56013ebed250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.213+0000 7fbbbaffd640 1 --2- 172.21.15.64:0/3688121205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56013ebc62d0 0x56013ebed250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.213+0000 7fbbbaffd640 1 -- 172.21.15.64:0/3688121205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56013ebc59b0 msgr2=0x56013ebecd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.213+0000 7fbbbaffd640 1 --2- 172.21.15.64:0/3688121205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56013ebc59b0 0x56013ebecd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.213+0000 7fbbbaffd640 1 -- 172.21.15.64:0/3688121205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56013ebd0ff0 con 0x56013ebd50c0 2024-09-12T03:33:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.213+0000 7fbbbaffd640 1 --2- 172.21.15.64:0/3688121205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56013ebd50c0 0x56013ebcf2c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x56013ebc58e0 tx=0x7fbbac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22cf3185163c0673 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.213+0000 7fbbb97fa640 1 -- 172.21.15.64:0/3688121205 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac030070 con 0x56013ebd50c0 2024-09-12T03:33:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.213+0000 7fbbb97fa640 1 -- 172.21.15.64:0/3688121205 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbac00fe50 con 0x56013ebd50c0 2024-09-12T03:33:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.213+0000 7fbbb97fa640 1 -- 172.21.15.64:0/3688121205 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac00fc10 con 0x56013ebd50c0 2024-09-12T03:33:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.214+0000 7fbbcd007740 1 -- 172.21.15.64:0/3688121205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56013ebd50c0 msgr2=0x56013ebcf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.214+0000 7fbbcd007740 1 --2- 172.21.15.64:0/3688121205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56013ebd50c0 0x56013ebcf2c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x56013ebc58e0 tx=0x7fbbac0046c0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.214+0000 7fbbcd007740 1 -- 172.21.15.64:0/3688121205 shutdown_connections 2024-09-12T03:33:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.214+0000 7fbbcd007740 1 --2- 172.21.15.64:0/3688121205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56013ebd50c0 0x56013ebcf2c0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.214+0000 7fbbcd007740 1 --2- 172.21.15.64:0/3688121205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56013ebc62d0 0x56013ebed250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.214+0000 7fbbcd007740 1 --2- 172.21.15.64:0/3688121205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56013ebc59b0 0x56013ebecd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.214+0000 7fbbcd007740 1 -- 172.21.15.64:0/3688121205 >> 172.21.15.64:0/3688121205 conn(0x56013ebc2060 msgr2=0x56013ebd40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.214+0000 7fbbcd007740 1 -- 172.21.15.64:0/3688121205 shutdown_connections 2024-09-12T03:33:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.214+0000 7fbbcd007740 1 -- 172.21.15.64:0/3688121205 wait complete. 2024-09-12T03:33:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.214+0000 7fbbcd007740 1 -- start start 2024-09-12T03:33:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.214+0000 7fbbcd007740 1 Processor -- start 2024-09-12T03:33:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.215+0000 7fbbcd007740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56013ebc59b0 0x56013ebdd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.215+0000 7fbbcd007740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56013ebc62d0 0x56013ebdd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.215+0000 7fbbcd007740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56013ebd50c0 0x56013ebda0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.215+0000 7fbbcd007740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc41ab380 con 0x56013ebd50c0 2024-09-12T03:33:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.215+0000 7fbbcd007740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc41ac7f0 con 0x56013ebc59b0 2024-09-12T03:33:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.215+0000 7fbbcd007740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc41aba50 con 0x56013ebc62d0 2024-09-12T03:33:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.215+0000 7fbbcd007740 10 client.0 _ll_register_callbacks cb 0x7fbbac00b8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.215+0000 7fbbbaffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56013ebd50c0 0x56013ebda0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.215+0000 7fbbbaffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56013ebd50c0 0x56013ebda0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56574/0 (socket says 172.21.15.64:56574) 2024-09-12T03:33:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.215+0000 7fbbbaffd640 1 -- 172.21.15.64:0/1905137007 learned_addr learned my addr 172.21.15.64:0/1905137007 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:33:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.215+0000 7fbbc37fe640 1 --2- 172.21.15.64:0/1905137007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56013ebc62d0 0x56013ebdd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.215+0000 7fbbbaffd640 1 --2- 172.21.15.64:0/1905137007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56013ebc59b0 0x56013ebdd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.215+0000 7fbbbaffd640 1 -- 172.21.15.64:0/1905137007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56013ebc62d0 msgr2=0x56013ebdd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.215+0000 7fbbbaffd640 1 --2- 172.21.15.64:0/1905137007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56013ebc62d0 0x56013ebdd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.215+0000 7fbbbaffd640 1 -- 172.21.15.64:0/1905137007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56013ebc59b0 msgr2=0x56013ebdd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.215+0000 7fbbbaffd640 1 --2- 172.21.15.64:0/1905137007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56013ebc59b0 0x56013ebdd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.215+0000 7fbbbaffd640 1 -- 172.21.15.64:0/1905137007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56013ebe77b0 con 0x56013ebd50c0 2024-09-12T03:33:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.215+0000 7fbbbaffd640 1 --2- 172.21.15.64:0/1905137007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56013ebd50c0 0x56013ebda0a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fbbac02e040 tx=0x7fbbac005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=facdfcb367deb9c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.215+0000 7fbba2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:33:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.215+0000 7fbba2ffd640 1 -- 172.21.15.64:0/1905137007 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac030040 con 0x56013ebd50c0 2024-09-12T03:33:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.215+0000 7fbba2ffd640 1 -- 172.21.15.64:0/1905137007 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7fbbac00f9b0 con 0x56013ebd50c0 2024-09-12T03:33:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.215+0000 7fbbcd007740 1 -- 172.21.15.64:0/1905137007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbac00fe50 con 0x56013ebd50c0 2024-09-12T03:33:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.216+0000 7fbba2ffd640 1 -- 172.21.15.64:0/1905137007 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbbac023040 con 0x56013ebd50c0 2024-09-12T03:33:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.216+0000 7fbbcd007740 10 client.15726 fetch_fsmap learned FSMap version 25 2024-09-12T03:33:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.216+0000 7fbbcd007740 1 -- 172.21.15.64:0/1905137007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56013ec503a0 con 0x56013ebd50c0 2024-09-12T03:33:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.216+0000 7fbba2ffd640 1 -- 172.21.15.64:0/1905137007 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbac00fe50 con 0x56013ebd50c0 2024-09-12T03:33:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.216+0000 7fbba2ffd640 1 -- 172.21.15.64:0/1905137007 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fbbac036040 con 0x56013ebd50c0 2024-09-12T03:33:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.216+0000 7fbbcd007740 10 client.15726 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:33:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.216+0000 7fbbcd007740 10 client.15726 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:33:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.216+0000 7fbbcd007740 20 client.15726 populate_metadata read hostname 'smithi064' 2024-09-12T03:33:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.216+0000 7fbbcd007740 1 --2- 172.21.15.64:0/1905137007 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x56013ec50a30 0x56013ec52e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.216+0000 7fbbcd007740 4 client.15726 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:33:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.216+0000 7fbbcd007740 1 -- 172.21.15.64:0/1905137007 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56013eb31d70 con 0x56013ec50a30 2024-09-12T03:33:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.216+0000 7fbbc37fe640 1 --2- 172.21.15.64:0/1905137007 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x56013ec50a30 0x56013ec52e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.217+0000 7fbbc37fe640 1 --2- 172.21.15.64:0/1905137007 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x56013ec50a30 0x56013ec52e50 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69611aaaa6b8a91f server_cookie=e0814d468fef224d in_seq=0 out_seq=0 2024-09-12T03:33:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.217+0000 7fbba2ffd640 10 client.15726 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:33:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.229+0000 7fbba2ffd640 1 -- 172.21.15.64:0/1905137007 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56013eb31d70 con 0x56013ec50a30 2024-09-12T03:33:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.229+0000 7fbba2ffd640 10 client.15726 handle_command_reply: tid=0 2024-09-12T03:33:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.229+0000 7fbbcd007740 1 client.15726 shutdown 2024-09-12T03:33:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.229+0000 7fbbcd007740 1 -- 172.21.15.64:0/1905137007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56013ebd50c0 msgr2=0x56013ebda0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.229+0000 7fbbcd007740 1 --2- 172.21.15.64:0/1905137007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56013ebd50c0 0x56013ebda0a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fbbac02e040 tx=0x7fbbac005740 comp rx=0 tx=0).stop 2024-09-12T03:33:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.230+0000 7fbbcd007740 1 -- 172.21.15.64:0/1905137007 shutdown_connections 2024-09-12T03:33:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.230+0000 7fbbcd007740 1 --2- 172.21.15.64:0/1905137007 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x56013ec50a30 0x56013ec52e50 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.230+0000 7fbbcd007740 1 --2- 172.21.15.64:0/1905137007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56013ebd50c0 0x56013ebda0a0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.230+0000 7fbbcd007740 1 --2- 172.21.15.64:0/1905137007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56013ebc62d0 0x56013ebdd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.230+0000 7fbba2ffd640 0 client.15726 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:33:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.230+0000 7fbbcd007740 1 --2- 172.21.15.64:0/1905137007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56013ebc59b0 0x56013ebdd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.230+0000 7fbbcd007740 1 -- 172.21.15.64:0/1905137007 >> 172.21.15.64:0/1905137007 conn(0x56013ebc2060 msgr2=0x56013ebc7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.230+0000 7fbbcd007740 1 -- 172.21.15.64:0/1905137007 shutdown_connections 2024-09-12T03:33:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.230+0000 7fbbcd007740 1 -- 172.21.15.64:0/1905137007 wait complete. 2024-09-12T03:33:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.230+0000 7fbbcd007740 20 client.15726 trim_cache size 0 max 16384 2024-09-12T03:33:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:33:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:33:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:33:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:33:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.230+0000 7fbbca18d640 1 -- 172.21.15.64:0/1297358072 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fbbb00a2e20 msgr2=0x7fbbb00a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.230+0000 7fbbca18d640 1 --2- 172.21.15.64:0/1297358072 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fbbb00a2e20 0x7fbbb00a52e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc0bc970 tx=0x7fbbbc0095c0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.230+0000 7fbbca18d640 1 -- 172.21.15.64:0/1297358072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbc4107b80 msgr2=0x7fbbc419cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.230+0000 7fbbca18d640 1 --2- 172.21.15.64:0/1297358072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbc4107b80 0x7fbbc419cb10 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc005e30 tx=0x7fbbbc003ba0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.230+0000 7fbbca18d640 1 -- 172.21.15.64:0/1297358072 shutdown_connections 2024-09-12T03:33:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.230+0000 7fbbca18d640 1 --2- 172.21.15.64:0/1297358072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbc4107b80 0x7fbbc419cb10 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.230+0000 7fbbca18d640 1 --2- 172.21.15.64:0/1297358072 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fbbb00a2e20 0x7fbbb00a52e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.230+0000 7fbbca18d640 1 --2- 172.21.15.64:0/1297358072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbc4102480 0x7fbbc4198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.230+0000 7fbbca18d640 1 --2- 172.21.15.64:0/1297358072 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbbc4101ae0 0x7fbbc4198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.230+0000 7fbbca18d640 1 -- 172.21.15.64:0/1297358072 >> 172.21.15.64:0/1297358072 conn(0x7fbbc40fd7e0 msgr2=0x7fbbc40fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:54.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.230+0000 7fbbca18d640 1 -- 172.21.15.64:0/1297358072 shutdown_connections 2024-09-12T03:33:54.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:33:54.230+0000 7fbbca18d640 1 -- 172.21.15.64:0/1297358072 wait complete. 2024-09-12T03:33:55.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:33:54 smithi115 ceph-mon[34801]: pgmap v142: 129 pgs: 129 active+clean; 2.0 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-09-12T03:33:55.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:33:54 smithi195 ceph-mon[34786]: pgmap v142: 129 pgs: 129 active+clean; 2.0 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-09-12T03:33:55.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:33:54 smithi064 ceph-mon[30952]: pgmap v142: 129 pgs: 129 active+clean; 2.0 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-09-12T03:33:57.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:33:56 smithi115 ceph-mon[34801]: pgmap v143: 129 pgs: 129 active+clean; 2.0 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-09-12T03:33:57.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:33:56 smithi195 ceph-mon[34786]: pgmap v143: 129 pgs: 129 active+clean; 2.0 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-09-12T03:33:57.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:33:56 smithi064 ceph-mon[30952]: pgmap v143: 129 pgs: 129 active+clean; 2.0 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-09-12T03:33:59.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:33:58 smithi115 ceph-mon[34801]: pgmap v144: 129 pgs: 129 active+clean; 2.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 KiB/s wr, 18 op/s 2024-09-12T03:33:59.209 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:33:58 smithi195 ceph-mon[34786]: pgmap v144: 129 pgs: 129 active+clean; 2.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 KiB/s wr, 18 op/s 2024-09-12T03:33:59.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:33:58 smithi064 ceph-mon[30952]: pgmap v144: 129 pgs: 129 active+clean; 2.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 KiB/s wr, 18 op/s 2024-09-12T03:34:01.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:34:00 smithi115 ceph-mon[34801]: pgmap v145: 129 pgs: 129 active+clean; 2.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-09-12T03:34:01.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:34:00 smithi195 ceph-mon[34786]: pgmap v145: 129 pgs: 129 active+clean; 2.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-09-12T03:34:01.271 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:34:00 smithi064 ceph-mon[30952]: pgmap v145: 129 pgs: 129 active+clean; 2.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-09-12T03:34:03.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:34:02 smithi115 ceph-mon[34801]: pgmap v146: 129 pgs: 129 active+clean; 2.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 66 KiB/s wr, 33 op/s 2024-09-12T03:34:03.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:34:02 smithi195 ceph-mon[34786]: pgmap v146: 129 pgs: 129 active+clean; 2.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 66 KiB/s wr, 33 op/s 2024-09-12T03:34:03.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:34:02 smithi064 ceph-mon[30952]: pgmap v146: 129 pgs: 129 active+clean; 2.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 66 KiB/s wr, 33 op/s 2024-09-12T03:34:05.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:34:04 smithi115 ceph-mon[34801]: pgmap v147: 129 pgs: 129 active+clean; 2.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-09-12T03:34:05.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:34:04 smithi195 ceph-mon[34786]: pgmap v147: 129 pgs: 129 active+clean; 2.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-09-12T03:34:05.271 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:34:04 smithi064 ceph-mon[30952]: pgmap v147: 129 pgs: 129 active+clean; 2.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-09-12T03:34:07.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:34:06 smithi115 ceph-mon[34801]: pgmap v148: 129 pgs: 129 active+clean; 2.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-09-12T03:34:07.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:34:06 smithi195 ceph-mon[34786]: pgmap v148: 129 pgs: 129 active+clean; 2.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-09-12T03:34:07.271 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:34:06 smithi064 ceph-mon[30952]: pgmap v148: 129 pgs: 129 active+clean; 2.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-09-12T03:34:07.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.331+0000 7f2effcd9640 1 -- 172.21.15.64:0/946041121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ef81009f0 msgr2=0x7f2ef8100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:07.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.331+0000 7f2effcd9640 1 --2- 172.21.15.64:0/946041121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ef81009f0 0x7f2ef8100e50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4009930 tx=0x7f2ef402a420 comp rx=0 tx=0).stop 2024-09-12T03:34:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.331+0000 7f2effcd9640 1 -- 172.21.15.64:0/946041121 shutdown_connections 2024-09-12T03:34:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.331+0000 7f2effcd9640 1 --2- 172.21.15.64:0/946041121 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ef8101390 0x7f2ef810d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.331+0000 7f2effcd9640 1 --2- 172.21.15.64:0/946041121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ef81009f0 0x7f2ef8100e50 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.331+0000 7f2effcd9640 1 --2- 172.21.15.64:0/946041121 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ef8106a00 0x7f2ef8106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.331+0000 7f2effcd9640 1 -- 172.21.15.64:0/946041121 >> 172.21.15.64:0/946041121 conn(0x7f2ef80fc6b0 msgr2=0x7f2ef80fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.331+0000 7f2effcd9640 1 -- 172.21.15.64:0/946041121 shutdown_connections 2024-09-12T03:34:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.331+0000 7f2effcd9640 1 -- 172.21.15.64:0/946041121 wait complete. 2024-09-12T03:34:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.332+0000 7f2effcd9640 1 Processor -- start 2024-09-12T03:34:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.332+0000 7f2effcd9640 1 -- start start 2024-09-12T03:34:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.332+0000 7f2effcd9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ef81009f0 0x7f2ef81990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.332+0000 7f2effcd9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ef8101390 0x7f2ef8199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.332+0000 7f2effcd9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ef8106a00 0x7f2ef81a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.332+0000 7f2effcd9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef8110cf0 con 0x7f2ef8106a00 2024-09-12T03:34:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.332+0000 7f2effcd9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef8110b70 con 0x7f2ef8101390 2024-09-12T03:34:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.332+0000 7f2effcd9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef8110e70 con 0x7f2ef81009f0 2024-09-12T03:34:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.332+0000 7f2efe276640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ef8106a00 0x7f2ef81a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.332+0000 7f2efe276640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ef8106a00 0x7f2ef81a4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56098/0 (socket says 172.21.15.64:56098) 2024-09-12T03:34:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.332+0000 7f2efe276640 1 -- 172.21.15.64:0/3669782374 learned_addr learned my addr 172.21.15.64:0/3669782374 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:34:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.332+0000 7f2efd274640 1 --2- 172.21.15.64:0/3669782374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ef8101390 0x7f2ef8199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.332+0000 7f2efda75640 1 --2- 172.21.15.64:0/3669782374 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ef81009f0 0x7f2ef81990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.333+0000 7f2efe276640 1 -- 172.21.15.64:0/3669782374 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ef81009f0 msgr2=0x7f2ef81990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.333+0000 7f2efe276640 1 --2- 172.21.15.64:0/3669782374 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ef81009f0 0x7f2ef81990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.333+0000 7f2efe276640 1 -- 172.21.15.64:0/3669782374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ef8101390 msgr2=0x7f2ef8199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.333+0000 7f2efe276640 1 --2- 172.21.15.64:0/3669782374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ef8101390 0x7f2ef8199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.333+0000 7f2efe276640 1 -- 172.21.15.64:0/3669782374 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef81a4680 con 0x7f2ef8106a00 2024-09-12T03:34:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.333+0000 7f2efe276640 1 --2- 172.21.15.64:0/3669782374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ef8106a00 0x7f2ef81a4140 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000cce0 tx=0x7f2ef000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.333+0000 7f2eeaffd640 1 -- 172.21.15.64:0/3669782374 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef000f450 con 0x7f2ef8106a00 2024-09-12T03:34:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.333+0000 7f2eeaffd640 1 -- 172.21.15.64:0/3669782374 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ef000faa0 con 0x7f2ef8106a00 2024-09-12T03:34:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.333+0000 7f2effcd9640 1 -- 172.21.15.64:0/3669782374 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ef81a4850 con 0x7f2ef8106a00 2024-09-12T03:34:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.333+0000 7f2eeaffd640 1 -- 172.21.15.64:0/3669782374 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef0018b70 con 0x7f2ef8106a00 2024-09-12T03:34:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.333+0000 7f2effcd9640 1 -- 172.21.15.64:0/3669782374 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ef81a4d70 con 0x7f2ef8106a00 2024-09-12T03:34:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.336+0000 7f2eeaffd640 1 -- 172.21.15.64:0/3669782374 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f2ef0018d10 con 0x7f2ef8106a00 2024-09-12T03:34:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.336+0000 7f2f02a6f740 1 Processor -- start 2024-09-12T03:34:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.336+0000 7f2f02a6f740 1 -- start start 2024-09-12T03:34:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.336+0000 7f2f02a6f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb6873baf0 0x55fb6873bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.336+0000 7f2f02a6f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb6873c410 0x55fb68746770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.336+0000 7f2f02a6f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb68746cb0 0x55fb687490a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.336+0000 7f2f02a6f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55fb686a02f0 con 0x55fb6873c410 2024-09-12T03:34:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.337+0000 7f2f02a6f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55fb68648270 con 0x55fb68746cb0 2024-09-12T03:34:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.337+0000 7f2f02a6f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fb6869c2b0 con 0x55fb6873baf0 2024-09-12T03:34:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.337+0000 7f2efda75640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb6873c410 0x55fb68746770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.337+0000 7f2efda75640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb6873c410 0x55fb68746770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56108/0 (socket says 172.21.15.64:56108) 2024-09-12T03:34:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.337+0000 7f2efda75640 1 -- 172.21.15.64:0/60083574 learned_addr learned my addr 172.21.15.64:0/60083574 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:34:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.337+0000 7f2efd274640 1 --2- 172.21.15.64:0/60083574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb68746cb0 0x55fb687490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.337+0000 7f2efd274640 1 --2- 172.21.15.64:0/60083574 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb6873baf0 0x55fb6873bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.337+0000 7f2eeaffd640 1 --2- 172.21.15.64:0/3669782374 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f2ee40a2e20 0x7f2ee40a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.337+0000 7f2efda75640 1 -- 172.21.15.64:0/60083574 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb6873baf0 msgr2=0x55fb6873bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.337+0000 7f2efda75640 1 --2- 172.21.15.64:0/60083574 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb6873baf0 0x55fb6873bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.337+0000 7f2efda75640 1 -- 172.21.15.64:0/60083574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb68746cb0 msgr2=0x55fb687490a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.337+0000 7f2efda75640 1 --2- 172.21.15.64:0/60083574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb68746cb0 0x55fb687490a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.337+0000 7f2efda75640 1 -- 172.21.15.64:0/60083574 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb68647340 con 0x55fb6873c410 2024-09-12T03:34:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.337+0000 7f2efe276640 1 --2- 172.21.15.64:0/3669782374 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f2ee40a2e20 0x7f2ee40a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.337+0000 7f2eeaffd640 1 -- 172.21.15.64:0/3669782374 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f2ef00cfb60 con 0x7f2ef8106a00 2024-09-12T03:34:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.337+0000 7f2efda75640 1 --2- 172.21.15.64:0/60083574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb6873c410 0x55fb68746770 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f2eec009ca0 tx=0x7f2eec011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67cd58fb95e61deb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.337+0000 7f2ee3fff640 1 -- 172.21.15.64:0/60083574 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eec01e070 con 0x55fb6873c410 2024-09-12T03:34:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.338+0000 7f2ee3fff640 1 -- 172.21.15.64:0/60083574 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2eec019ae0 con 0x55fb6873c410 2024-09-12T03:34:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.338+0000 7f2efe276640 1 --2- 172.21.15.64:0/3669782374 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f2ee40a2e20 0x7f2ee40a52e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2ef00ccf90 tx=0x7f2ef001baf0 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.338+0000 7f2ee3fff640 1 -- 172.21.15.64:0/60083574 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eec00e990 con 0x55fb6873c410 2024-09-12T03:34:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.338+0000 7f2f02a6f740 1 -- 172.21.15.64:0/60083574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb6873c410 msgr2=0x55fb68746770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.338+0000 7f2f02a6f740 1 --2- 172.21.15.64:0/60083574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb6873c410 0x55fb68746770 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f2eec009ca0 tx=0x7f2eec011810 comp rx=0 tx=0).stop 2024-09-12T03:34:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.338+0000 7f2f02a6f740 1 -- 172.21.15.64:0/60083574 shutdown_connections 2024-09-12T03:34:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.338+0000 7f2f02a6f740 1 --2- 172.21.15.64:0/60083574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb68746cb0 0x55fb687490a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.338+0000 7f2f02a6f740 1 --2- 172.21.15.64:0/60083574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb6873c410 0x55fb68746770 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.338+0000 7f2f02a6f740 1 --2- 172.21.15.64:0/60083574 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb6873baf0 0x55fb6873bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.338+0000 7f2f02a6f740 1 -- 172.21.15.64:0/60083574 >> 172.21.15.64:0/60083574 conn(0x55fb68738060 msgr2=0x55fb68738440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.338+0000 7f2f02a6f740 1 -- 172.21.15.64:0/60083574 shutdown_connections 2024-09-12T03:34:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.339+0000 7f2f02a6f740 1 -- 172.21.15.64:0/60083574 wait complete. 2024-09-12T03:34:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.339+0000 7f2f02a6f740 1 -- start start 2024-09-12T03:34:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.339+0000 7f2f02a6f740 1 Processor -- start 2024-09-12T03:34:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.339+0000 7f2f02a6f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb6873baf0 0x55fb68743bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.339+0000 7f2f02a6f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb6873c410 0x55fb68744130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.339+0000 7f2f02a6f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb68746cb0 0x55fb68744670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.339+0000 7f2f02a6f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55fb68749880 con 0x55fb6873c410 2024-09-12T03:34:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.339+0000 7f2f02a6f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55fb68629290 con 0x55fb68746cb0 2024-09-12T03:34:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.339+0000 7f2f02a6f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fb68749a00 con 0x55fb6873baf0 2024-09-12T03:34:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.339+0000 7f2f02a6f740 10 client.0 _ll_register_callbacks cb 0x55fb6865a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:34:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.340+0000 7f2efda75640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb6873c410 0x55fb68744130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.340+0000 7f2efda75640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb6873c410 0x55fb68744130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56118/0 (socket says 172.21.15.64:56118) 2024-09-12T03:34:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.340+0000 7f2efda75640 1 -- 172.21.15.64:0/3297018343 learned_addr learned my addr 172.21.15.64:0/3297018343 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:34:07.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.340+0000 7f2efd274640 1 --2- 172.21.15.64:0/3297018343 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb6873baf0 0x55fb68743bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:07.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.340+0000 7f2efd274640 1 --2- 172.21.15.64:0/3297018343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb68746cb0 0x55fb68744670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:07.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.340+0000 7f2efda75640 1 -- 172.21.15.64:0/3297018343 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb6873baf0 msgr2=0x55fb68743bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:07.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.340+0000 7f2efda75640 1 --2- 172.21.15.64:0/3297018343 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb6873baf0 0x55fb68743bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:07.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.340+0000 7f2efda75640 1 -- 172.21.15.64:0/3297018343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb68746cb0 msgr2=0x55fb68744670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:07.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.340+0000 7f2efda75640 1 --2- 172.21.15.64:0/3297018343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb68746cb0 0x55fb68744670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:07.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.340+0000 7f2efda75640 1 -- 172.21.15.64:0/3297018343 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb683b3210 con 0x55fb6873c410 2024-09-12T03:34:07.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.340+0000 7f2efd274640 1 --2- 172.21.15.64:0/3297018343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb68746cb0 0x55fb68744670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:34:07.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.340+0000 7f2efda75640 1 --2- 172.21.15.64:0/3297018343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb6873c410 0x55fb68744130 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f2eec009c70 tx=0x7f2eec019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb8cef43b53077c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.340+0000 7f2ee17fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:34:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.340+0000 7f2ee17fa640 1 -- 172.21.15.64:0/3297018343 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eec01e050 con 0x55fb6873c410 2024-09-12T03:34:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.340+0000 7f2ee17fa640 1 -- 172.21.15.64:0/3297018343 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f2eec004020 con 0x55fb6873c410 2024-09-12T03:34:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.340+0000 7f2f02a6f740 1 -- 172.21.15.64:0/3297018343 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2eec019ae0 con 0x55fb6873c410 2024-09-12T03:34:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.340+0000 7f2ee17fa640 1 -- 172.21.15.64:0/3297018343 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2eec032050 con 0x55fb6873c410 2024-09-12T03:34:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.340+0000 7f2f02a6f740 10 client.15738 fetch_fsmap learned FSMap version 25 2024-09-12T03:34:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.340+0000 7f2f02a6f740 1 -- 172.21.15.64:0/3297018343 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fb6847f620 con 0x55fb6873c410 2024-09-12T03:34:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.341+0000 7f2ee17fa640 1 -- 172.21.15.64:0/3297018343 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2eec019ae0 con 0x55fb6873c410 2024-09-12T03:34:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.341+0000 7f2ee17fa640 1 -- 172.21.15.64:0/3297018343 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2eec004750 con 0x55fb6873c410 2024-09-12T03:34:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.341+0000 7f2f02a6f740 10 client.15738 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:34:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.341+0000 7f2f02a6f740 10 client.15738 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:34:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.341+0000 7f2f02a6f740 20 client.15738 populate_metadata read hostname 'smithi064' 2024-09-12T03:34:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.341+0000 7f2f02a6f740 1 --2- 172.21.15.64:0/3297018343 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55fb68756bf0 0x55fb68758fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.341+0000 7f2f02a6f740 4 client.15738 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:34:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.341+0000 7f2f02a6f740 1 -- 172.21.15.64:0/3297018343 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb686ada20 con 0x55fb68756bf0 2024-09-12T03:34:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.341+0000 7f2efd274640 1 --2- 172.21.15.64:0/3297018343 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55fb68756bf0 0x55fb68758fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.342+0000 7f2efd274640 1 --2- 172.21.15.64:0/3297018343 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55fb68756bf0 0x55fb68758fe0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59939972ec3196b9 server_cookie=9880b7687311e5ae in_seq=0 out_seq=0 2024-09-12T03:34:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.342+0000 7f2ee17fa640 10 client.15738 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:34:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.342+0000 7f2ee17fa640 1 -- 172.21.15.64:0/3297018343 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fb686ada20 con 0x55fb68756bf0 2024-09-12T03:34:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.342+0000 7f2ee17fa640 10 client.15738 handle_command_reply: tid=0 2024-09-12T03:34:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.342+0000 7f2f02a6f740 1 client.15738 shutdown 2024-09-12T03:34:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.342+0000 7f2f02a6f740 1 -- 172.21.15.64:0/3297018343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb6873c410 msgr2=0x55fb68744130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.342+0000 7f2f02a6f740 1 --2- 172.21.15.64:0/3297018343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb6873c410 0x55fb68744130 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f2eec009c70 tx=0x7f2eec019660 comp rx=0 tx=0).stop 2024-09-12T03:34:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.343+0000 7f2f02a6f740 1 -- 172.21.15.64:0/3297018343 shutdown_connections 2024-09-12T03:34:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.343+0000 7f2f02a6f740 1 --2- 172.21.15.64:0/3297018343 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55fb68756bf0 0x55fb68758fe0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.343+0000 7f2f02a6f740 1 --2- 172.21.15.64:0/3297018343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb68746cb0 0x55fb68744670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.343+0000 7f2f02a6f740 1 --2- 172.21.15.64:0/3297018343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb6873c410 0x55fb68744130 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.343+0000 7f2f02a6f740 1 --2- 172.21.15.64:0/3297018343 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb6873baf0 0x55fb68743bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.343+0000 7f2ee17fa640 0 client.15738 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:34:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.343+0000 7f2f02a6f740 1 -- 172.21.15.64:0/3297018343 >> 172.21.15.64:0/3297018343 conn(0x55fb68738060 msgr2=0x55fb687371c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.343+0000 7f2f02a6f740 1 -- 172.21.15.64:0/3297018343 shutdown_connections 2024-09-12T03:34:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.343+0000 7f2f02a6f740 1 -- 172.21.15.64:0/3297018343 wait complete. 2024-09-12T03:34:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.343+0000 7f2f02a6f740 20 client.15738 trim_cache size 0 max 16384 2024-09-12T03:34:07.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.357+0000 7f2f02a6f740 1 Processor -- start 2024-09-12T03:34:07.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.357+0000 7f2f02a6f740 1 -- start start 2024-09-12T03:34:07.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.358+0000 7f2f02a6f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb6873baf0 0x55fb687444a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:07.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.358+0000 7f2f02a6f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb6873c410 0x55fb6873dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:07.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.358+0000 7f2f02a6f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb68746cb0 0x55fb6874f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:07.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.358+0000 7f2f02a6f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef81b17a0 con 0x55fb6873c410 2024-09-12T03:34:07.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.358+0000 7f2f02a6f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef81af070 con 0x55fb68746cb0 2024-09-12T03:34:07.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.358+0000 7f2f02a6f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef81acec0 con 0x55fb6873baf0 2024-09-12T03:34:07.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.358+0000 7f2efda75640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb6873c410 0x55fb6873dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:07.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.358+0000 7f2efda75640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb6873c410 0x55fb6873dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56126/0 (socket says 172.21.15.64:56126) 2024-09-12T03:34:07.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.358+0000 7f2efda75640 1 -- 172.21.15.64:0/2696733560 learned_addr learned my addr 172.21.15.64:0/2696733560 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:34:07.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.358+0000 7f2efd274640 1 --2- 172.21.15.64:0/2696733560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb6873baf0 0x55fb687444a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:07.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.358+0000 7f2efda75640 1 -- 172.21.15.64:0/2696733560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb6873baf0 msgr2=0x55fb687444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:07.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.358+0000 7f2efda75640 1 --2- 172.21.15.64:0/2696733560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb6873baf0 0x55fb687444a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:07.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.358+0000 7f2efda75640 1 -- 172.21.15.64:0/2696733560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb68746cb0 msgr2=0x55fb6874f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:07.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.358+0000 7f2efda75640 1 --2- 172.21.15.64:0/2696733560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb68746cb0 0x55fb6874f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:07.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.358+0000 7f2efda75640 1 -- 172.21.15.64:0/2696733560 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb6873f7a0 con 0x55fb6873c410 2024-09-12T03:34:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.358+0000 7f2efda75640 1 --2- 172.21.15.64:0/2696733560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb6873c410 0x55fb6873dc60 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f2eec009ca0 tx=0x7f2eec00e3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15288c96ddaa4001 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.358+0000 7f2ee37fe640 1 -- 172.21.15.64:0/2696733560 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eec015070 con 0x55fb6873c410 2024-09-12T03:34:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.358+0000 7f2ee37fe640 1 -- 172.21.15.64:0/2696733560 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2eec01f040 con 0x55fb6873c410 2024-09-12T03:34:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.358+0000 7f2ee37fe640 1 -- 172.21.15.64:0/2696733560 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eec01e070 con 0x55fb6873c410 2024-09-12T03:34:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.359+0000 7f2f02a6f740 1 -- 172.21.15.64:0/2696733560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb6873c410 msgr2=0x55fb6873dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.359+0000 7f2f02a6f740 1 --2- 172.21.15.64:0/2696733560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb6873c410 0x55fb6873dc60 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f2eec009ca0 tx=0x7f2eec00e3b0 comp rx=0 tx=0).stop 2024-09-12T03:34:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.359+0000 7f2f02a6f740 1 -- 172.21.15.64:0/2696733560 shutdown_connections 2024-09-12T03:34:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.359+0000 7f2f02a6f740 1 --2- 172.21.15.64:0/2696733560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb68746cb0 0x55fb6874f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.359+0000 7f2f02a6f740 1 --2- 172.21.15.64:0/2696733560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb6873c410 0x55fb6873dc60 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.359+0000 7f2f02a6f740 1 --2- 172.21.15.64:0/2696733560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb6873baf0 0x55fb687444a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.359+0000 7f2f02a6f740 1 -- 172.21.15.64:0/2696733560 >> 172.21.15.64:0/2696733560 conn(0x55fb68738060 msgr2=0x55fb6873d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.359+0000 7f2f02a6f740 1 -- 172.21.15.64:0/2696733560 shutdown_connections 2024-09-12T03:34:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.359+0000 7f2f02a6f740 1 -- 172.21.15.64:0/2696733560 wait complete. 2024-09-12T03:34:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.359+0000 7f2f02a6f740 1 -- start start 2024-09-12T03:34:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.359+0000 7f2f02a6f740 1 Processor -- start 2024-09-12T03:34:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.360+0000 7f2f02a6f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb6873baf0 0x55fb6875e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.360+0000 7f2f02a6f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb6873c410 0x55fb6875e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.360+0000 7f2f02a6f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb68746cb0 0x55fb687bc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.360+0000 7f2f02a6f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef81ad530 con 0x55fb68746cb0 2024-09-12T03:34:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.360+0000 7f2f02a6f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef81ae9a0 con 0x55fb6873baf0 2024-09-12T03:34:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.360+0000 7f2f02a6f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef81adc00 con 0x55fb6873c410 2024-09-12T03:34:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.360+0000 7f2f02a6f740 10 client.0 _ll_register_callbacks cb 0x7f2eec01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:34:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.360+0000 7f2efd274640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb68746cb0 0x55fb687bc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.360+0000 7f2efd274640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb68746cb0 0x55fb687bc970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56140/0 (socket says 172.21.15.64:56140) 2024-09-12T03:34:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.360+0000 7f2efd274640 1 -- 172.21.15.64:0/2552760457 learned_addr learned my addr 172.21.15.64:0/2552760457 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:34:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.360+0000 7f2efd274640 1 --2- 172.21.15.64:0/2552760457 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb6873baf0 0x55fb6875e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.360+0000 7f2efda75640 1 --2- 172.21.15.64:0/2552760457 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb6873c410 0x55fb6875e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.360+0000 7f2efd274640 1 -- 172.21.15.64:0/2552760457 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb6873c410 msgr2=0x55fb6875e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.360+0000 7f2efd274640 1 --2- 172.21.15.64:0/2552760457 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb6873c410 0x55fb6875e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.360+0000 7f2efd274640 1 -- 172.21.15.64:0/2552760457 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb6873baf0 msgr2=0x55fb6875e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.360+0000 7f2efd274640 1 --2- 172.21.15.64:0/2552760457 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb6873baf0 0x55fb6875e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.360+0000 7f2efd274640 1 -- 172.21.15.64:0/2552760457 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb687bd010 con 0x55fb68746cb0 2024-09-12T03:34:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.360+0000 7f2efda75640 1 --2- 172.21.15.64:0/2552760457 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb6873c410 0x55fb6875e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:34:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.360+0000 7f2efd274640 1 --2- 172.21.15.64:0/2552760457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb68746cb0 0x55fb687bc970 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4000c00 tx=0x7f2ef4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4589afb58edf352 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.360+0000 7f2ee17fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:34:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.361+0000 7f2ee17fa640 1 -- 172.21.15.64:0/2552760457 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4035040 con 0x55fb68746cb0 2024-09-12T03:34:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.361+0000 7f2ee17fa640 1 -- 172.21.15.64:0/2552760457 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f2ef404bb10 con 0x55fb68746cb0 2024-09-12T03:34:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.361+0000 7f2f02a6f740 1 -- 172.21.15.64:0/2552760457 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2eec01f040 con 0x55fb68746cb0 2024-09-12T03:34:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.361+0000 7f2ee17fa640 1 -- 172.21.15.64:0/2552760457 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2ef405b050 con 0x55fb68746cb0 2024-09-12T03:34:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.361+0000 7f2f02a6f740 10 client.15744 fetch_fsmap learned FSMap version 25 2024-09-12T03:34:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.361+0000 7f2f02a6f740 1 -- 172.21.15.64:0/2552760457 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fb687c55d0 con 0x55fb68746cb0 2024-09-12T03:34:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.361+0000 7f2ee17fa640 1 -- 172.21.15.64:0/2552760457 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2eec01f040 con 0x55fb68746cb0 2024-09-12T03:34:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.361+0000 7f2ee17fa640 1 -- 172.21.15.64:0/2552760457 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2ef4052070 con 0x55fb68746cb0 2024-09-12T03:34:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.361+0000 7f2f02a6f740 10 client.15744 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:34:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.361+0000 7f2f02a6f740 10 client.15744 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:34:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.361+0000 7f2f02a6f740 20 client.15744 populate_metadata read hostname 'smithi064' 2024-09-12T03:34:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.361+0000 7f2f02a6f740 1 --2- 172.21.15.64:0/2552760457 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55fb687c5d00 0x55fb687c8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:07.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.361+0000 7f2f02a6f740 4 client.15744 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:34:07.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.361+0000 7f2f02a6f740 1 -- 172.21.15.64:0/2552760457 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fb686a7d70 con 0x55fb687c5d00 2024-09-12T03:34:07.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.362+0000 7f2efda75640 1 --2- 172.21.15.64:0/2552760457 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55fb687c5d00 0x55fb687c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:07.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.362+0000 7f2ee17fa640 10 client.15744 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:34:07.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.362+0000 7f2efda75640 1 --2- 172.21.15.64:0/2552760457 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55fb687c5d00 0x55fb687c8120 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e1e3c640da6fe3a server_cookie=91c76a0f2eecb669 in_seq=0 out_seq=0 2024-09-12T03:34:07.377 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.377+0000 7f2ee17fa640 1 -- 172.21.15.64:0/2552760457 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fb686a7d70 con 0x55fb687c5d00 2024-09-12T03:34:07.377 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.377+0000 7f2ee17fa640 10 client.15744 handle_command_reply: tid=0 2024-09-12T03:34:07.378 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.377+0000 7f2f02a6f740 1 client.15744 shutdown 2024-09-12T03:34:07.378 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.377+0000 7f2f02a6f740 1 -- 172.21.15.64:0/2552760457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb68746cb0 msgr2=0x55fb687bc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:07.378 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.377+0000 7f2f02a6f740 1 --2- 172.21.15.64:0/2552760457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb68746cb0 0x55fb687bc970 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4000c00 tx=0x7f2ef4009260 comp rx=0 tx=0).stop 2024-09-12T03:34:07.378 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.377+0000 7f2f02a6f740 1 -- 172.21.15.64:0/2552760457 shutdown_connections 2024-09-12T03:34:07.378 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.377+0000 7f2f02a6f740 1 --2- 172.21.15.64:0/2552760457 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55fb687c5d00 0x55fb687c8120 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:07.378 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.377+0000 7f2f02a6f740 1 --2- 172.21.15.64:0/2552760457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb68746cb0 0x55fb687bc970 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:07.378 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.377+0000 7f2f02a6f740 1 --2- 172.21.15.64:0/2552760457 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb6873c410 0x55fb6875e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:07.378 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.377+0000 7f2ee17fa640 0 client.15744 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:34:07.378 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.377+0000 7f2f02a6f740 1 --2- 172.21.15.64:0/2552760457 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb6873baf0 0x55fb6875e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:07.378 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.377+0000 7f2f02a6f740 1 -- 172.21.15.64:0/2552760457 >> 172.21.15.64:0/2552760457 conn(0x55fb68738060 msgr2=0x55fb687371c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:07.378 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.377+0000 7f2f02a6f740 1 -- 172.21.15.64:0/2552760457 shutdown_connections 2024-09-12T03:34:07.378 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.377+0000 7f2f02a6f740 1 -- 172.21.15.64:0/2552760457 wait complete. 2024-09-12T03:34:07.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.377+0000 7f2f02a6f740 20 client.15744 trim_cache size 0 max 16384 2024-09-12T03:34:07.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:34:07.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:34:07.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:34:07.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:34:07.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.378+0000 7f2effcd9640 1 -- 172.21.15.64:0/3669782374 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f2ee40a2e20 msgr2=0x7f2ee40a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:07.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.378+0000 7f2effcd9640 1 --2- 172.21.15.64:0/3669782374 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f2ee40a2e20 0x7f2ee40a52e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2ef00ccf90 tx=0x7f2ef001baf0 comp rx=0 tx=0).stop 2024-09-12T03:34:07.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.378+0000 7f2effcd9640 1 -- 172.21.15.64:0/3669782374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ef8106a00 msgr2=0x7f2ef81a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:07.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.378+0000 7f2effcd9640 1 --2- 172.21.15.64:0/3669782374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ef8106a00 0x7f2ef81a4140 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000cce0 tx=0x7f2ef000a620 comp rx=0 tx=0).stop 2024-09-12T03:34:07.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.378+0000 7f2effcd9640 1 -- 172.21.15.64:0/3669782374 shutdown_connections 2024-09-12T03:34:07.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.378+0000 7f2effcd9640 1 --2- 172.21.15.64:0/3669782374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ef8106a00 0x7f2ef81a4140 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:07.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.378+0000 7f2effcd9640 1 --2- 172.21.15.64:0/3669782374 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f2ee40a2e20 0x7f2ee40a52e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:07.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.378+0000 7f2effcd9640 1 --2- 172.21.15.64:0/3669782374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ef8101390 0x7f2ef8199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:07.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.378+0000 7f2effcd9640 1 --2- 172.21.15.64:0/3669782374 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ef81009f0 0x7f2ef81990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:07.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.378+0000 7f2effcd9640 1 -- 172.21.15.64:0/3669782374 >> 172.21.15.64:0/3669782374 conn(0x7f2ef80fc6b0 msgr2=0x7f2ef80fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:07.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.378+0000 7f2effcd9640 1 -- 172.21.15.64:0/3669782374 shutdown_connections 2024-09-12T03:34:07.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:07.378+0000 7f2effcd9640 1 -- 172.21.15.64:0/3669782374 wait complete. 2024-09-12T03:34:09.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:34:08 smithi115 ceph-mon[34801]: pgmap v149: 129 pgs: 129 active+clean; 2.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 67 KiB/s wr, 33 op/s 2024-09-12T03:34:09.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:34:08 smithi195 ceph-mon[34786]: pgmap v149: 129 pgs: 129 active+clean; 2.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 67 KiB/s wr, 33 op/s 2024-09-12T03:34:09.271 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:34:08 smithi064 ceph-mon[30952]: pgmap v149: 129 pgs: 129 active+clean; 2.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 67 KiB/s wr, 33 op/s 2024-09-12T03:34:11.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:34:10 smithi115 ceph-mon[34801]: pgmap v150: 129 pgs: 129 active+clean; 2.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 49 KiB/s wr, 25 op/s 2024-09-12T03:34:11.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:34:10 smithi195 ceph-mon[34786]: pgmap v150: 129 pgs: 129 active+clean; 2.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 49 KiB/s wr, 25 op/s 2024-09-12T03:34:11.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:34:10 smithi064 ceph-mon[30952]: pgmap v150: 129 pgs: 129 active+clean; 2.5 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 49 KiB/s wr, 25 op/s 2024-09-12T03:34:13.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:34:12 smithi115 ceph-mon[34801]: pgmap v151: 129 pgs: 129 active+clean; 2.7 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 69 KiB/s wr, 33 op/s 2024-09-12T03:34:13.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:34:12 smithi195 ceph-mon[34786]: pgmap v151: 129 pgs: 129 active+clean; 2.7 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 69 KiB/s wr, 33 op/s 2024-09-12T03:34:13.271 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:34:12 smithi064 ceph-mon[30952]: pgmap v151: 129 pgs: 129 active+clean; 2.7 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 69 KiB/s wr, 33 op/s 2024-09-12T03:34:15.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:34:14 smithi115 ceph-mon[34801]: pgmap v152: 129 pgs: 129 active+clean; 2.7 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-12T03:34:15.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:34:14 smithi195 ceph-mon[34786]: pgmap v152: 129 pgs: 129 active+clean; 2.7 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-12T03:34:15.271 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:34:14 smithi064 ceph-mon[30952]: pgmap v152: 129 pgs: 129 active+clean; 2.7 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-12T03:34:17.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:34:16 smithi115 ceph-mon[34801]: pgmap v153: 129 pgs: 129 active+clean; 2.7 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-12T03:34:17.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:34:16 smithi195 ceph-mon[34786]: pgmap v153: 129 pgs: 129 active+clean; 2.7 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-12T03:34:17.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:34:16 smithi064 ceph-mon[30952]: pgmap v153: 129 pgs: 129 active+clean; 2.7 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-12T03:34:19.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:34:18 smithi115 ceph-mon[34801]: pgmap v154: 129 pgs: 129 active+clean; 2.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-09-12T03:34:19.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:34:18 smithi195 ceph-mon[34786]: pgmap v154: 129 pgs: 129 active+clean; 2.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-09-12T03:34:19.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:34:18 smithi064 ceph-mon[30952]: pgmap v154: 129 pgs: 129 active+clean; 2.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-09-12T03:34:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.477+0000 7fdc793cf640 1 -- 172.21.15.64:0/1852063537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc74103620 msgr2=0x7fdc7410bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.477+0000 7fdc793cf640 1 --2- 172.21.15.64:0/1852063537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc74103620 0x7fdc7410bb00 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fdc68009900 tx=0x7fdc6802a480 comp rx=0 tx=0).stop 2024-09-12T03:34:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.477+0000 7fdc793cf640 1 -- 172.21.15.64:0/1852063537 shutdown_connections 2024-09-12T03:34:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.477+0000 7fdc793cf640 1 --2- 172.21.15.64:0/1852063537 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc7410c0d0 0x7fdc7410e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.477+0000 7fdc793cf640 1 --2- 172.21.15.64:0/1852063537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc74103620 0x7fdc7410bb00 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.477+0000 7fdc793cf640 1 --2- 172.21.15.64:0/1852063537 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc74102d00 0x7fdc741030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.477+0000 7fdc793cf640 1 -- 172.21.15.64:0/1852063537 >> 172.21.15.64:0/1852063537 conn(0x7fdc74077990 msgr2=0x7fdc740fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.477+0000 7fdc793cf640 1 -- 172.21.15.64:0/1852063537 shutdown_connections 2024-09-12T03:34:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.477+0000 7fdc793cf640 1 -- 172.21.15.64:0/1852063537 wait complete. 2024-09-12T03:34:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.477+0000 7fdc793cf640 1 Processor -- start 2024-09-12T03:34:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.478+0000 7fdc793cf640 1 -- start start 2024-09-12T03:34:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.478+0000 7fdc793cf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc74102d00 0x7fdc74100950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.478+0000 7fdc793cf640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc74103620 0x7fdc74100e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.478+0000 7fdc793cf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc7410c0d0 0x7fdc741013d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.478+0000 7fdc793cf640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdc74111d60 con 0x7fdc74102d00 2024-09-12T03:34:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.478+0000 7fdc793cf640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdc74111be0 con 0x7fdc7410c0d0 2024-09-12T03:34:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.478+0000 7fdc793cf640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdc74111ee0 con 0x7fdc74103620 2024-09-12T03:34:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.478+0000 7fdc72ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc74102d00 0x7fdc74100950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.478+0000 7fdc72ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc74102d00 0x7fdc74100950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35404/0 (socket says 172.21.15.64:35404) 2024-09-12T03:34:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.478+0000 7fdc72ffd640 1 -- 172.21.15.64:0/2291205950 learned_addr learned my addr 172.21.15.64:0/2291205950 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:34:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.478+0000 7fdc727fc640 1 --2- 172.21.15.64:0/2291205950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc74103620 0x7fdc74100e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.478+0000 7fdc737fe640 1 --2- 172.21.15.64:0/2291205950 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc7410c0d0 0x7fdc741013d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.479+0000 7fdc72ffd640 1 -- 172.21.15.64:0/2291205950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc74103620 msgr2=0x7fdc74100e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.479+0000 7fdc72ffd640 1 --2- 172.21.15.64:0/2291205950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc74103620 0x7fdc74100e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.479+0000 7fdc72ffd640 1 -- 172.21.15.64:0/2291205950 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc7410c0d0 msgr2=0x7fdc741013d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.479+0000 7fdc72ffd640 1 --2- 172.21.15.64:0/2291205950 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc7410c0d0 0x7fdc741013d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.479+0000 7fdc72ffd640 1 -- 172.21.15.64:0/2291205950 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc741a6700 con 0x7fdc74102d00 2024-09-12T03:34:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.479+0000 7fdc72ffd640 1 --2- 172.21.15.64:0/2291205950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc74102d00 0x7fdc74100950 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fdc6400c920 tx=0x7fdc6400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.479+0000 7fdc5bfff640 1 -- 172.21.15.64:0/2291205950 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc64007c70 con 0x7fdc74102d00 2024-09-12T03:34:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.479+0000 7fdc5bfff640 1 -- 172.21.15.64:0/2291205950 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc64007e10 con 0x7fdc74102d00 2024-09-12T03:34:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.479+0000 7fdc793cf640 1 -- 172.21.15.64:0/2291205950 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc741a69f0 con 0x7fdc74102d00 2024-09-12T03:34:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.479+0000 7fdc5bfff640 1 -- 172.21.15.64:0/2291205950 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc64018820 con 0x7fdc74102d00 2024-09-12T03:34:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.479+0000 7fdc793cf640 1 -- 172.21.15.64:0/2291205950 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc741a6f10 con 0x7fdc74102d00 2024-09-12T03:34:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.482+0000 7fdc5bfff640 1 -- 172.21.15.64:0/2291205950 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fdc640189c0 con 0x7fdc74102d00 2024-09-12T03:34:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.482+0000 7fdc7c245740 1 Processor -- start 2024-09-12T03:34:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.482+0000 7fdc7c245740 1 -- start start 2024-09-12T03:34:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.482+0000 7fdc7c245740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d354a8baf0 0x55d354a8bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.482+0000 7fdc7c245740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d354a8c410 0x55d354a96770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.482+0000 7fdc7c245740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d354a96cb0 0x55d354a990a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.482+0000 7fdc7c245740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d3549f02f0 con 0x55d354a96cb0 2024-09-12T03:34:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.482+0000 7fdc7c245740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d354998270 con 0x55d354a8baf0 2024-09-12T03:34:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.482+0000 7fdc7c245740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d3549ec2b0 con 0x55d354a8c410 2024-09-12T03:34:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.482+0000 7fdc5bfff640 1 --2- 172.21.15.64:0/2291205950 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fdc5c0a2da0 0x7fdc5c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.482+0000 7fdc727fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d354a96cb0 0x55d354a990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.482+0000 7fdc727fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d354a96cb0 0x55d354a990a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35410/0 (socket says 172.21.15.64:35410) 2024-09-12T03:34:20.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.482+0000 7fdc727fc640 1 -- 172.21.15.64:0/2278264053 learned_addr learned my addr 172.21.15.64:0/2278264053 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:34:20.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.483+0000 7fdc737fe640 1 --2- 172.21.15.64:0/2291205950 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fdc5c0a2da0 0x7fdc5c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:20.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.483+0000 7fdc5bfff640 1 -- 172.21.15.64:0/2291205950 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7fdc640cf8a0 con 0x7fdc74102d00 2024-09-12T03:34:20.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.483+0000 7fdc72ffd640 1 --2- 172.21.15.64:0/2278264053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d354a8c410 0x55d354a96770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:20.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.483+0000 7fdc737fe640 1 --2- 172.21.15.64:0/2278264053 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d354a8baf0 0x55d354a8bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:20.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.483+0000 7fdc727fc640 1 -- 172.21.15.64:0/2278264053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d354a8c410 msgr2=0x55d354a96770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:20.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.483+0000 7fdc727fc640 1 --2- 172.21.15.64:0/2278264053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d354a8c410 0x55d354a96770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:20.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.483+0000 7fdc727fc640 1 -- 172.21.15.64:0/2278264053 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d354a8baf0 msgr2=0x55d354a8bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:20.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.483+0000 7fdc727fc640 1 --2- 172.21.15.64:0/2278264053 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d354a8baf0 0x55d354a8bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:20.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.483+0000 7fdc727fc640 1 -- 172.21.15.64:0/2278264053 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d354997340 con 0x55d354a96cb0 2024-09-12T03:34:20.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.483+0000 7fdc737fe640 1 --2- 172.21.15.64:0/2291205950 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fdc5c0a2da0 0x7fdc5c0a5260 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c005fd0 tx=0x7fdc6c009b60 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:20.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.483+0000 7fdc727fc640 1 --2- 172.21.15.64:0/2278264053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d354a96cb0 0x55d354a990a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fdc68009900 tx=0x7fdc68032c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95764138379128b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:20.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.483+0000 7fdc597fa640 1 -- 172.21.15.64:0/2278264053 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc68040070 con 0x55d354a96cb0 2024-09-12T03:34:20.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.483+0000 7fdc597fa640 1 -- 172.21.15.64:0/2278264053 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc68004030 con 0x55d354a96cb0 2024-09-12T03:34:20.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.483+0000 7fdc597fa640 1 -- 172.21.15.64:0/2278264053 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc68035040 con 0x55d354a96cb0 2024-09-12T03:34:20.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.484+0000 7fdc7c245740 1 -- 172.21.15.64:0/2278264053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d354a96cb0 msgr2=0x55d354a990a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:20.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.484+0000 7fdc7c245740 1 --2- 172.21.15.64:0/2278264053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d354a96cb0 0x55d354a990a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fdc68009900 tx=0x7fdc68032c70 comp rx=0 tx=0).stop 2024-09-12T03:34:20.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.484+0000 7fdc7c245740 1 -- 172.21.15.64:0/2278264053 shutdown_connections 2024-09-12T03:34:20.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.484+0000 7fdc7c245740 1 --2- 172.21.15.64:0/2278264053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d354a96cb0 0x55d354a990a0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:20.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.484+0000 7fdc7c245740 1 --2- 172.21.15.64:0/2278264053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d354a8c410 0x55d354a96770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:20.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.484+0000 7fdc7c245740 1 --2- 172.21.15.64:0/2278264053 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d354a8baf0 0x55d354a8bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:20.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.484+0000 7fdc7c245740 1 -- 172.21.15.64:0/2278264053 >> 172.21.15.64:0/2278264053 conn(0x55d354a88060 msgr2=0x55d354a88440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:20.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.484+0000 7fdc7c245740 1 -- 172.21.15.64:0/2278264053 shutdown_connections 2024-09-12T03:34:20.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.484+0000 7fdc7c245740 1 -- 172.21.15.64:0/2278264053 wait complete. 2024-09-12T03:34:20.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.484+0000 7fdc7c245740 1 -- start start 2024-09-12T03:34:20.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.484+0000 7fdc7c245740 1 Processor -- start 2024-09-12T03:34:20.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.485+0000 7fdc7c245740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d354a8baf0 0x55d354a93be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:20.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.485+0000 7fdc7c245740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d354a8c410 0x55d354a94120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:20.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.485+0000 7fdc7c245740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d354a96cb0 0x55d354a94660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:20.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.485+0000 7fdc7c245740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d354a99880 con 0x55d354a8c410 2024-09-12T03:34:20.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.485+0000 7fdc7c245740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d354979290 con 0x55d354a8baf0 2024-09-12T03:34:20.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.485+0000 7fdc7c245740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d354a99a00 con 0x55d354a96cb0 2024-09-12T03:34:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.485+0000 7fdc7c245740 10 client.0 _ll_register_callbacks cb 0x55d3549aa140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:34:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.485+0000 7fdc727fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d354a8c410 0x55d354a94120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.485+0000 7fdc727fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d354a8c410 0x55d354a94120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35412/0 (socket says 172.21.15.64:35412) 2024-09-12T03:34:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.485+0000 7fdc727fc640 1 -- 172.21.15.64:0/3271025622 learned_addr learned my addr 172.21.15.64:0/3271025622 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:34:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.485+0000 7fdc72ffd640 1 --2- 172.21.15.64:0/3271025622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d354a8baf0 0x55d354a93be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.485+0000 7fdc737fe640 1 --2- 172.21.15.64:0/3271025622 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d354a96cb0 0x55d354a94660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.485+0000 7fdc727fc640 1 -- 172.21.15.64:0/3271025622 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d354a96cb0 msgr2=0x55d354a94660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.485+0000 7fdc727fc640 1 --2- 172.21.15.64:0/3271025622 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d354a96cb0 0x55d354a94660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.485+0000 7fdc727fc640 1 -- 172.21.15.64:0/3271025622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d354a8baf0 msgr2=0x55d354a93be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.485+0000 7fdc727fc640 1 --2- 172.21.15.64:0/3271025622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d354a8baf0 0x55d354a93be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.485+0000 7fdc727fc640 1 -- 172.21.15.64:0/3271025622 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d354703210 con 0x55d354a8c410 2024-09-12T03:34:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.485+0000 7fdc727fc640 1 --2- 172.21.15.64:0/3271025622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d354a8c410 0x55d354a94120 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdc6802a9a0 tx=0x7fdc68031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f011ee20120cfda3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.485+0000 7fdc56ff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:34:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.486+0000 7fdc56ff5640 1 -- 172.21.15.64:0/3271025622 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc68040070 con 0x55d354a8c410 2024-09-12T03:34:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.486+0000 7fdc56ff5640 1 -- 172.21.15.64:0/3271025622 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7fdc680043e0 con 0x55d354a8c410 2024-09-12T03:34:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.486+0000 7fdc7c245740 1 -- 172.21.15.64:0/3271025622 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc68004030 con 0x55d354a8c410 2024-09-12T03:34:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.486+0000 7fdc56ff5640 1 -- 172.21.15.64:0/3271025622 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdc68056050 con 0x55d354a8c410 2024-09-12T03:34:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.486+0000 7fdc7c245740 10 client.15756 fetch_fsmap learned FSMap version 25 2024-09-12T03:34:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.486+0000 7fdc7c245740 1 -- 172.21.15.64:0/3271025622 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d3547cf620 con 0x55d354a8c410 2024-09-12T03:34:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.486+0000 7fdc56ff5640 1 -- 172.21.15.64:0/3271025622 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc68004030 con 0x55d354a8c410 2024-09-12T03:34:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.486+0000 7fdc56ff5640 1 -- 172.21.15.64:0/3271025622 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fdc68003cc0 con 0x55d354a8c410 2024-09-12T03:34:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.486+0000 7fdc7c245740 10 client.15756 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:34:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.486+0000 7fdc7c245740 10 client.15756 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:34:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.486+0000 7fdc7c245740 20 client.15756 populate_metadata read hostname 'smithi064' 2024-09-12T03:34:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.486+0000 7fdc7c245740 1 --2- 172.21.15.64:0/3271025622 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55d354aa6ad0 0x55d354aa8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.486+0000 7fdc7c245740 4 client.15756 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:34:20.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.486+0000 7fdc7c245740 1 -- 172.21.15.64:0/3271025622 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3549fda20 con 0x55d354aa6ad0 2024-09-12T03:34:20.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.487+0000 7fdc72ffd640 1 --2- 172.21.15.64:0/3271025622 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55d354aa6ad0 0x55d354aa8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:20.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.487+0000 7fdc72ffd640 1 --2- 172.21.15.64:0/3271025622 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55d354aa6ad0 0x55d354aa8ec0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea54954e84de16b1 server_cookie=7ead972e728c4920 in_seq=0 out_seq=0 2024-09-12T03:34:20.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.487+0000 7fdc56ff5640 10 client.15756 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:34:20.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.488+0000 7fdc56ff5640 1 -- 172.21.15.64:0/3271025622 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d3549fda20 con 0x55d354aa6ad0 2024-09-12T03:34:20.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.488+0000 7fdc56ff5640 10 client.15756 handle_command_reply: tid=0 2024-09-12T03:34:20.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.488+0000 7fdc7c245740 1 client.15756 shutdown 2024-09-12T03:34:20.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.488+0000 7fdc7c245740 1 -- 172.21.15.64:0/3271025622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d354a8c410 msgr2=0x55d354a94120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:20.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.488+0000 7fdc7c245740 1 --2- 172.21.15.64:0/3271025622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d354a8c410 0x55d354a94120 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdc6802a9a0 tx=0x7fdc68031400 comp rx=0 tx=0).stop 2024-09-12T03:34:20.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.488+0000 7fdc7c245740 1 -- 172.21.15.64:0/3271025622 shutdown_connections 2024-09-12T03:34:20.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.488+0000 7fdc7c245740 1 --2- 172.21.15.64:0/3271025622 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55d354aa6ad0 0x55d354aa8ec0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:20.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.488+0000 7fdc7c245740 1 --2- 172.21.15.64:0/3271025622 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d354a96cb0 0x55d354a94660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:20.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.488+0000 7fdc7c245740 1 --2- 172.21.15.64:0/3271025622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d354a8c410 0x55d354a94120 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:20.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.488+0000 7fdc7c245740 1 --2- 172.21.15.64:0/3271025622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d354a8baf0 0x55d354a93be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:20.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.488+0000 7fdc56ff5640 0 client.15756 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:34:20.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.488+0000 7fdc7c245740 1 -- 172.21.15.64:0/3271025622 >> 172.21.15.64:0/3271025622 conn(0x55d354a88060 msgr2=0x55d354a8dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:20.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.488+0000 7fdc7c245740 1 -- 172.21.15.64:0/3271025622 shutdown_connections 2024-09-12T03:34:20.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.488+0000 7fdc7c245740 1 -- 172.21.15.64:0/3271025622 wait complete. 2024-09-12T03:34:20.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.489+0000 7fdc7c245740 20 client.15756 trim_cache size 0 max 16384 2024-09-12T03:34:20.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.503+0000 7fdc7c245740 1 Processor -- start 2024-09-12T03:34:20.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.503+0000 7fdc7c245740 1 -- start start 2024-09-12T03:34:20.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.503+0000 7fdc7c245740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d354a8baf0 0x55d354a94490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:20.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.503+0000 7fdc7c245740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d354a8c410 0x55d354a95b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:20.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.503+0000 7fdc7c245740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d354a96cb0 0x55d354aabee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:20.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.503+0000 7fdc7c245740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdc741b3910 con 0x55d354a8baf0 2024-09-12T03:34:20.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.503+0000 7fdc7c245740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdc741b11e0 con 0x55d354a96cb0 2024-09-12T03:34:20.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.503+0000 7fdc7c245740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdc741af030 con 0x55d354a8c410 2024-09-12T03:34:20.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.503+0000 7fdc72ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d354a8baf0 0x55d354a94490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:20.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.503+0000 7fdc72ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d354a8baf0 0x55d354a94490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35418/0 (socket says 172.21.15.64:35418) 2024-09-12T03:34:20.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.503+0000 7fdc72ffd640 1 -- 172.21.15.64:0/3763901034 learned_addr learned my addr 172.21.15.64:0/3763901034 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:34:20.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.503+0000 7fdc737fe640 1 --2- 172.21.15.64:0/3763901034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d354a96cb0 0x55d354aabee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:20.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.504+0000 7fdc727fc640 1 --2- 172.21.15.64:0/3763901034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d354a8c410 0x55d354a95b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:20.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.504+0000 7fdc72ffd640 1 -- 172.21.15.64:0/3763901034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d354a8c410 msgr2=0x55d354a95b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:20.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.504+0000 7fdc72ffd640 1 --2- 172.21.15.64:0/3763901034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d354a8c410 0x55d354a95b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.504+0000 7fdc72ffd640 1 -- 172.21.15.64:0/3763901034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d354a96cb0 msgr2=0x55d354aabee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.504+0000 7fdc72ffd640 1 --2- 172.21.15.64:0/3763901034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d354a96cb0 0x55d354aabee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.504+0000 7fdc72ffd640 1 -- 172.21.15.64:0/3763901034 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d354a8c240 con 0x55d354a8baf0 2024-09-12T03:34:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.504+0000 7fdc727fc640 1 --2- 172.21.15.64:0/3763901034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d354a8c410 0x55d354a95b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:34:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.504+0000 7fdc72ffd640 1 --2- 172.21.15.64:0/3763901034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d354a8baf0 0x55d354a94490 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fdc6409b6f0 tx=0x7fdc64086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64643f48d15e13ad server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.504+0000 7fdc58ff9640 1 -- 172.21.15.64:0/3763901034 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc640a4660 con 0x55d354a8baf0 2024-09-12T03:34:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.504+0000 7fdc58ff9640 1 -- 172.21.15.64:0/3763901034 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc64099cf0 con 0x55d354a8baf0 2024-09-12T03:34:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.504+0000 7fdc58ff9640 1 -- 172.21.15.64:0/3763901034 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc64098200 con 0x55d354a8baf0 2024-09-12T03:34:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.504+0000 7fdc7c245740 1 -- 172.21.15.64:0/3763901034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d354a8baf0 msgr2=0x55d354a94490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.504+0000 7fdc7c245740 1 --2- 172.21.15.64:0/3763901034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d354a8baf0 0x55d354a94490 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fdc6409b6f0 tx=0x7fdc64086040 comp rx=0 tx=0).stop 2024-09-12T03:34:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.504+0000 7fdc7c245740 1 -- 172.21.15.64:0/3763901034 shutdown_connections 2024-09-12T03:34:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.504+0000 7fdc7c245740 1 --2- 172.21.15.64:0/3763901034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d354a96cb0 0x55d354aabee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.504+0000 7fdc7c245740 1 --2- 172.21.15.64:0/3763901034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d354a8c410 0x55d354a95b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.505+0000 7fdc7c245740 1 --2- 172.21.15.64:0/3763901034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d354a8baf0 0x55d354a94490 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.505+0000 7fdc7c245740 1 -- 172.21.15.64:0/3763901034 >> 172.21.15.64:0/3763901034 conn(0x55d354a88060 msgr2=0x55d354a95440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.505+0000 7fdc7c245740 1 -- 172.21.15.64:0/3763901034 shutdown_connections 2024-09-12T03:34:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.505+0000 7fdc7c245740 1 -- 172.21.15.64:0/3763901034 wait complete. 2024-09-12T03:34:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.505+0000 7fdc7c245740 1 -- start start 2024-09-12T03:34:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.505+0000 7fdc7c245740 1 Processor -- start 2024-09-12T03:34:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.505+0000 7fdc7c245740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d354a8baf0 0x55d354aa9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.505+0000 7fdc7c245740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d354a8c410 0x55d354aa99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.505+0000 7fdc7c245740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d354a96cb0 0x55d354b0f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.505+0000 7fdc7c245740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdc741af6a0 con 0x55d354a8c410 2024-09-12T03:34:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.505+0000 7fdc7c245740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdc741b0b10 con 0x55d354a96cb0 2024-09-12T03:34:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.505+0000 7fdc7c245740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdc741afd70 con 0x55d354a8baf0 2024-09-12T03:34:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.505+0000 7fdc7c245740 10 client.0 _ll_register_callbacks cb 0x7fdc68032bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:34:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.505+0000 7fdc727fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d354a8c410 0x55d354aa99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.505+0000 7fdc727fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d354a8c410 0x55d354aa99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35422/0 (socket says 172.21.15.64:35422) 2024-09-12T03:34:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.505+0000 7fdc727fc640 1 -- 172.21.15.64:0/4283323793 learned_addr learned my addr 172.21.15.64:0/4283323793 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:34:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.506+0000 7fdc737fe640 1 --2- 172.21.15.64:0/4283323793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d354a96cb0 0x55d354b0f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.506+0000 7fdc72ffd640 1 --2- 172.21.15.64:0/4283323793 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d354a8baf0 0x55d354aa9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.506+0000 7fdc727fc640 1 -- 172.21.15.64:0/4283323793 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d354a8baf0 msgr2=0x55d354aa9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.506+0000 7fdc727fc640 1 --2- 172.21.15.64:0/4283323793 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d354a8baf0 0x55d354aa9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.506+0000 7fdc727fc640 1 -- 172.21.15.64:0/4283323793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d354a96cb0 msgr2=0x55d354b0f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.506+0000 7fdc727fc640 1 --2- 172.21.15.64:0/4283323793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d354a96cb0 0x55d354b0f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.506+0000 7fdc727fc640 1 -- 172.21.15.64:0/4283323793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d354b0f940 con 0x55d354a8c410 2024-09-12T03:34:20.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.506+0000 7fdc737fe640 1 --2- 172.21.15.64:0/4283323793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d354a96cb0 0x55d354b0f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:34:20.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.506+0000 7fdc727fc640 1 --2- 172.21.15.64:0/4283323793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d354a8c410 0x55d354aa99c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fdc68004720 tx=0x7fdc68004c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=363fc38fcf81f292 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:20.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.506+0000 7fdc56ff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:34:20.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.506+0000 7fdc56ff5640 1 -- 172.21.15.64:0/4283323793 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc68040070 con 0x55d354a8c410 2024-09-12T03:34:20.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.506+0000 7fdc56ff5640 1 -- 172.21.15.64:0/4283323793 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7fdc680051d0 con 0x55d354a8c410 2024-09-12T03:34:20.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.506+0000 7fdc7c245740 1 -- 172.21.15.64:0/4283323793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc64099cf0 con 0x55d354a8c410 2024-09-12T03:34:20.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.506+0000 7fdc56ff5640 1 -- 172.21.15.64:0/4283323793 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdc68035040 con 0x55d354a8c410 2024-09-12T03:34:20.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.506+0000 7fdc7c245740 10 client.15762 fetch_fsmap learned FSMap version 25 2024-09-12T03:34:20.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.506+0000 7fdc7c245740 1 -- 172.21.15.64:0/4283323793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d354b10b90 con 0x55d354a8c410 2024-09-12T03:34:20.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.506+0000 7fdc56ff5640 1 -- 172.21.15.64:0/4283323793 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc64099cf0 con 0x55d354a8c410 2024-09-12T03:34:20.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.507+0000 7fdc56ff5640 1 -- 172.21.15.64:0/4283323793 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fdc680315c0 con 0x55d354a8c410 2024-09-12T03:34:20.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.507+0000 7fdc7c245740 10 client.15762 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:34:20.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.507+0000 7fdc7c245740 10 client.15762 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:34:20.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.507+0000 7fdc7c245740 20 client.15762 populate_metadata read hostname 'smithi064' 2024-09-12T03:34:20.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.507+0000 7fdc7c245740 1 --2- 172.21.15.64:0/4283323793 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55d354b11310 0x55d354b13730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:20.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.507+0000 7fdc7c245740 4 client.15762 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:34:20.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.507+0000 7fdc7c245740 1 -- 172.21.15.64:0/4283323793 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d3549f7d70 con 0x55d354b11310 2024-09-12T03:34:20.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.507+0000 7fdc72ffd640 1 --2- 172.21.15.64:0/4283323793 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55d354b11310 0x55d354b13730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:20.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.507+0000 7fdc56ff5640 10 client.15762 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:34:20.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.507+0000 7fdc72ffd640 1 --2- 172.21.15.64:0/4283323793 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55d354b11310 0x55d354b13730 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9df848ea371780de server_cookie=56790d29f0bbab70 in_seq=0 out_seq=0 2024-09-12T03:34:20.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.521+0000 7fdc56ff5640 1 -- 172.21.15.64:0/4283323793 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d3549f7d70 con 0x55d354b11310 2024-09-12T03:34:20.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.521+0000 7fdc56ff5640 10 client.15762 handle_command_reply: tid=0 2024-09-12T03:34:20.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.521+0000 7fdc7c245740 1 client.15762 shutdown 2024-09-12T03:34:20.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.521+0000 7fdc7c245740 1 -- 172.21.15.64:0/4283323793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d354a8c410 msgr2=0x55d354aa99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:20.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.521+0000 7fdc7c245740 1 --2- 172.21.15.64:0/4283323793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d354a8c410 0x55d354aa99c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fdc68004720 tx=0x7fdc68004c40 comp rx=0 tx=0).stop 2024-09-12T03:34:20.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.521+0000 7fdc7c245740 1 -- 172.21.15.64:0/4283323793 shutdown_connections 2024-09-12T03:34:20.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.521+0000 7fdc7c245740 1 --2- 172.21.15.64:0/4283323793 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55d354b11310 0x55d354b13730 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:20.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.521+0000 7fdc7c245740 1 --2- 172.21.15.64:0/4283323793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d354a96cb0 0x55d354b0f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:20.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.521+0000 7fdc56ff5640 0 client.15762 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:34:20.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.521+0000 7fdc7c245740 1 --2- 172.21.15.64:0/4283323793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d354a8c410 0x55d354aa99c0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:20.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.521+0000 7fdc7c245740 1 --2- 172.21.15.64:0/4283323793 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d354a8baf0 0x55d354aa9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:20.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.521+0000 7fdc7c245740 1 -- 172.21.15.64:0/4283323793 >> 172.21.15.64:0/4283323793 conn(0x55d354a88060 msgr2=0x55d354a95c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:20.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.521+0000 7fdc7c245740 1 -- 172.21.15.64:0/4283323793 shutdown_connections 2024-09-12T03:34:20.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.521+0000 7fdc7c245740 1 -- 172.21.15.64:0/4283323793 wait complete. 2024-09-12T03:34:20.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.521+0000 7fdc7c245740 20 client.15762 trim_cache size 0 max 16384 2024-09-12T03:34:20.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:34:20.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:34:20.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:34:20.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:34:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.522+0000 7fdc793cf640 1 -- 172.21.15.64:0/2291205950 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fdc5c0a2da0 msgr2=0x7fdc5c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.522+0000 7fdc793cf640 1 --2- 172.21.15.64:0/2291205950 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fdc5c0a2da0 0x7fdc5c0a5260 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c005fd0 tx=0x7fdc6c009b60 comp rx=0 tx=0).stop 2024-09-12T03:34:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.522+0000 7fdc793cf640 1 -- 172.21.15.64:0/2291205950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc74102d00 msgr2=0x7fdc74100950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.522+0000 7fdc793cf640 1 --2- 172.21.15.64:0/2291205950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc74102d00 0x7fdc74100950 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fdc6400c920 tx=0x7fdc6400cde0 comp rx=0 tx=0).stop 2024-09-12T03:34:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.522+0000 7fdc793cf640 1 -- 172.21.15.64:0/2291205950 shutdown_connections 2024-09-12T03:34:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.522+0000 7fdc793cf640 1 --2- 172.21.15.64:0/2291205950 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fdc5c0a2da0 0x7fdc5c0a5260 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.522+0000 7fdc793cf640 1 --2- 172.21.15.64:0/2291205950 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc7410c0d0 0x7fdc741013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.522+0000 7fdc793cf640 1 --2- 172.21.15.64:0/2291205950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc74103620 0x7fdc74100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.522+0000 7fdc793cf640 1 --2- 172.21.15.64:0/2291205950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc74102d00 0x7fdc74100950 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:20.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.522+0000 7fdc793cf640 1 -- 172.21.15.64:0/2291205950 >> 172.21.15.64:0/2291205950 conn(0x7fdc74077990 msgr2=0x7fdc740fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:20.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.522+0000 7fdc793cf640 1 -- 172.21.15.64:0/2291205950 shutdown_connections 2024-09-12T03:34:20.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:20.522+0000 7fdc793cf640 1 -- 172.21.15.64:0/2291205950 wait complete. 2024-09-12T03:34:21.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:34:20 smithi115 ceph-mon[34801]: pgmap v155: 129 pgs: 129 active+clean; 2.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-12T03:34:21.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:34:20 smithi195 ceph-mon[34786]: pgmap v155: 129 pgs: 129 active+clean; 2.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-12T03:34:21.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:34:20 smithi064 ceph-mon[30952]: pgmap v155: 129 pgs: 129 active+clean; 2.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-12T03:34:23.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:34:22 smithi115 ceph-mon[34801]: pgmap v156: 129 pgs: 129 active+clean; 3.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 54 KiB/s wr, 21 op/s 2024-09-12T03:34:23.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:34:22 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:34:23.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:34:22 smithi195 ceph-mon[34786]: pgmap v156: 129 pgs: 129 active+clean; 3.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 54 KiB/s wr, 21 op/s 2024-09-12T03:34:23.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:34:22 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:34:23.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:34:22 smithi064 ceph-mon[30952]: pgmap v156: 129 pgs: 129 active+clean; 3.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 54 KiB/s wr, 21 op/s 2024-09-12T03:34:23.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:34:22 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:34:24.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:34:23 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:34:24.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:34:23 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:34:24.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:34:23 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:34:24.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:34:23 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:34:24.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:34:23 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:34:24.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:34:23 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:34:24.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:34:23 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:34:24.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:34:23 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:34:24.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:34:23 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:34:25.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:34:25 smithi195 ceph-mon[34786]: pgmap v157: 129 pgs: 129 active+clean; 3.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-12T03:34:25.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:34:25 smithi064 ceph-mon[30952]: pgmap v157: 129 pgs: 129 active+clean; 3.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-12T03:34:25.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:34:25 smithi115 ceph-mon[34801]: pgmap v157: 129 pgs: 129 active+clean; 3.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-12T03:34:27.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:34:27 smithi195 ceph-mon[34786]: pgmap v158: 129 pgs: 129 active+clean; 3.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-12T03:34:27.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:34:27 smithi064 ceph-mon[30952]: pgmap v158: 129 pgs: 129 active+clean; 3.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-12T03:34:27.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:34:27 smithi115 ceph-mon[34801]: pgmap v158: 129 pgs: 129 active+clean; 3.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-12T03:34:29.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:34:29 smithi195 ceph-mon[34786]: pgmap v159: 129 pgs: 129 active+clean; 3.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-09-12T03:34:29.525 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:34:29 smithi064 ceph-mon[30952]: pgmap v159: 129 pgs: 129 active+clean; 3.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-09-12T03:34:29.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:34:29 smithi115 ceph-mon[34801]: pgmap v159: 129 pgs: 129 active+clean; 3.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-09-12T03:34:31.069 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.070 INFO:tasks.workunit.client.0.smithi064.stdout:189/600: open fd = 3 2024-09-12T03:34:31.070 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.070 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.070 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.070 INFO:tasks.workunit.client.0.smithi064.stdout:190/600: open fd = 3 2024-09-12T03:34:31.070 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.070 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.070 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.070 INFO:tasks.workunit.client.0.smithi064.stdout:191/600: open fd = 3 2024-09-12T03:34:31.070 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.070 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.070 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.071 INFO:tasks.workunit.client.0.smithi064.stdout:192/600: open fd = 3 2024-09-12T03:34:31.071 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.071 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.071 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.071 INFO:tasks.workunit.client.0.smithi064.stdout:193/600: open fd = 3 2024-09-12T03:34:31.071 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.071 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.071 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.071 INFO:tasks.workunit.client.0.smithi064.stdout:194/600: open fd = 3 2024-09-12T03:34:31.071 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.071 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.071 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.071 INFO:tasks.workunit.client.0.smithi064.stdout:195/600: open fd = 3 2024-09-12T03:34:31.072 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.072 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.072 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.072 INFO:tasks.workunit.client.0.smithi064.stdout:196/600: open fd = 3 2024-09-12T03:34:31.072 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.072 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.072 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.072 INFO:tasks.workunit.client.0.smithi064.stdout:197/600: open fd = 3 2024-09-12T03:34:31.072 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.072 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.072 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.072 INFO:tasks.workunit.client.0.smithi064.stdout:198/600: open fd = 3 2024-09-12T03:34:31.072 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.073 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.073 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.073 INFO:tasks.workunit.client.0.smithi064.stdout:199/600: open fd = 3 2024-09-12T03:34:31.073 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.073 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.073 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.073 INFO:tasks.workunit.client.0.smithi064.stdout:200/600: open fd = 3 2024-09-12T03:34:31.073 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.073 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.073 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.073 INFO:tasks.workunit.client.0.smithi064.stdout:201/600: open fd = 3 2024-09-12T03:34:31.073 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.074 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.074 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.074 INFO:tasks.workunit.client.0.smithi064.stdout:202/600: open fd = 3 2024-09-12T03:34:31.074 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.074 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.074 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.074 INFO:tasks.workunit.client.0.smithi064.stdout:203/600: open fd = 3 2024-09-12T03:34:31.074 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.074 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.075 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.075 INFO:tasks.workunit.client.0.smithi064.stdout:204/600: open fd = 3 2024-09-12T03:34:31.075 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.075 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.075 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.075 INFO:tasks.workunit.client.0.smithi064.stdout:205/600: open fd = 3 2024-09-12T03:34:31.075 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.075 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.075 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.075 INFO:tasks.workunit.client.0.smithi064.stdout:206/600: open fd = 3 2024-09-12T03:34:31.075 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.075 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.076 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.076 INFO:tasks.workunit.client.0.smithi064.stdout:207/600: open fd = 3 2024-09-12T03:34:31.076 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.076 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.076 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.076 INFO:tasks.workunit.client.0.smithi064.stdout:208/600: open fd = 3 2024-09-12T03:34:31.076 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.076 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.076 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.076 INFO:tasks.workunit.client.0.smithi064.stdout:209/600: open fd = 3 2024-09-12T03:34:31.076 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.076 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.077 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.077 INFO:tasks.workunit.client.0.smithi064.stdout:210/600: open fd = 3 2024-09-12T03:34:31.077 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.077 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.077 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.077 INFO:tasks.workunit.client.0.smithi064.stdout:211/600: open fd = 3 2024-09-12T03:34:31.077 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.077 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.077 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.077 INFO:tasks.workunit.client.0.smithi064.stdout:212/600: open fd = 3 2024-09-12T03:34:31.077 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.077 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.077 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.078 INFO:tasks.workunit.client.0.smithi064.stdout:213/600: open fd = 3 2024-09-12T03:34:31.078 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.078 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.078 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.078 INFO:tasks.workunit.client.0.smithi064.stdout:214/600: open fd = 3 2024-09-12T03:34:31.078 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.078 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.078 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.078 INFO:tasks.workunit.client.0.smithi064.stdout:215/600: open fd = 3 2024-09-12T03:34:31.078 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.078 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.078 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.078 INFO:tasks.workunit.client.0.smithi064.stdout:216/600: open fd = 3 2024-09-12T03:34:31.079 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.079 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.079 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.079 INFO:tasks.workunit.client.0.smithi064.stdout:217/600: open fd = 3 2024-09-12T03:34:31.079 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.079 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.079 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.079 INFO:tasks.workunit.client.0.smithi064.stdout:218/600: open fd = 3 2024-09-12T03:34:31.079 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.079 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.079 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.079 INFO:tasks.workunit.client.0.smithi064.stdout:219/600: open fd = 3 2024-09-12T03:34:31.079 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.079 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.080 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.080 INFO:tasks.workunit.client.0.smithi064.stdout:220/600: open fd = 3 2024-09-12T03:34:31.080 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.080 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.080 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.080 INFO:tasks.workunit.client.0.smithi064.stdout:221/600: open fd = 3 2024-09-12T03:34:31.080 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.080 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.080 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.080 INFO:tasks.workunit.client.0.smithi064.stdout:222/600: open fd = 3 2024-09-12T03:34:31.080 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.080 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.080 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.080 INFO:tasks.workunit.client.0.smithi064.stdout:223/600: open fd = 3 2024-09-12T03:34:31.081 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.081 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.081 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.081 INFO:tasks.workunit.client.0.smithi064.stdout:224/600: open fd = 3 2024-09-12T03:34:31.081 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.081 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.081 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.081 INFO:tasks.workunit.client.0.smithi064.stdout:225/600: open fd = 3 2024-09-12T03:34:31.081 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.082 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.082 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.082 INFO:tasks.workunit.client.0.smithi064.stdout:226/600: open fd = 3 2024-09-12T03:34:31.082 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.082 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.082 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.082 INFO:tasks.workunit.client.0.smithi064.stdout:227/600: open fd = 3 2024-09-12T03:34:31.082 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.082 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.082 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.082 INFO:tasks.workunit.client.0.smithi064.stdout:228/600: open fd = 3 2024-09-12T03:34:31.082 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.082 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.083 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.083 INFO:tasks.workunit.client.0.smithi064.stdout:229/600: open fd = 3 2024-09-12T03:34:31.083 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.083 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.083 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.083 INFO:tasks.workunit.client.0.smithi064.stdout:230/600: open fd = 3 2024-09-12T03:34:31.083 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.083 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.083 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.083 INFO:tasks.workunit.client.0.smithi064.stdout:231/600: open fd = 3 2024-09-12T03:34:31.083 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.083 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.083 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.083 INFO:tasks.workunit.client.0.smithi064.stdout:232/600: open fd = 3 2024-09-12T03:34:31.084 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.084 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.084 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.084 INFO:tasks.workunit.client.0.smithi064.stdout:233/600: open fd = 3 2024-09-12T03:34:31.084 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.084 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.084 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.084 INFO:tasks.workunit.client.0.smithi064.stdout:234/600: open fd = 3 2024-09-12T03:34:31.084 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.084 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.084 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.084 INFO:tasks.workunit.client.0.smithi064.stdout:235/600: open fd = 3 2024-09-12T03:34:31.084 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.085 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.085 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.085 INFO:tasks.workunit.client.0.smithi064.stdout:236/600: open fd = 3 2024-09-12T03:34:31.085 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.085 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.085 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.085 INFO:tasks.workunit.client.0.smithi064.stdout:237/600: open fd = 3 2024-09-12T03:34:31.085 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.085 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.085 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.085 INFO:tasks.workunit.client.0.smithi064.stdout:238/600: open fd = 3 2024-09-12T03:34:31.085 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.085 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.085 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.086 INFO:tasks.workunit.client.0.smithi064.stdout:239/600: open fd = 3 2024-09-12T03:34:31.086 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.086 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.086 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.086 INFO:tasks.workunit.client.0.smithi064.stdout:240/600: open fd = 3 2024-09-12T03:34:31.086 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.086 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.086 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.086 INFO:tasks.workunit.client.0.smithi064.stdout:241/600: open fd = 3 2024-09-12T03:34:31.086 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.086 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.086 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.086 INFO:tasks.workunit.client.0.smithi064.stdout:242/600: open fd = 3 2024-09-12T03:34:31.086 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.087 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.087 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.087 INFO:tasks.workunit.client.0.smithi064.stdout:243/600: open fd = 3 2024-09-12T03:34:31.087 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.087 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.087 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.087 INFO:tasks.workunit.client.0.smithi064.stdout:244/600: open fd = 3 2024-09-12T03:34:31.087 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.087 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.087 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.087 INFO:tasks.workunit.client.0.smithi064.stdout:245/600: open fd = 3 2024-09-12T03:34:31.087 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.087 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.087 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.088 INFO:tasks.workunit.client.0.smithi064.stdout:246/600: open fd = 3 2024-09-12T03:34:31.088 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.088 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.088 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.088 INFO:tasks.workunit.client.0.smithi064.stdout:247/600: open fd = 3 2024-09-12T03:34:31.088 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.088 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.088 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.088 INFO:tasks.workunit.client.0.smithi064.stdout:248/600: open fd = 3 2024-09-12T03:34:31.088 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.088 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.088 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.088 INFO:tasks.workunit.client.0.smithi064.stdout:249/600: open fd = 3 2024-09-12T03:34:31.089 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.089 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.089 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:34:31.089 INFO:tasks.workunit.client.0.smithi064.stdout:250/600: open fd = 3 2024-09-12T03:34:31.089 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.089 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:34:31.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:34:31 smithi195 ceph-mon[34786]: pgmap v160: 129 pgs: 129 active+clean; 3.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-09-12T03:34:31.521 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:34:31 smithi064 ceph-mon[30952]: pgmap v160: 129 pgs: 129 active+clean; 3.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-09-12T03:34:31.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:34:31 smithi115 ceph-mon[34801]: pgmap v160: 129 pgs: 129 active+clean; 3.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-09-12T03:34:33.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:34:33 smithi195 ceph-mon[34786]: pgmap v161: 129 pgs: 129 active+clean; 3.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-09-12T03:34:33.521 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:34:33 smithi064 ceph-mon[30952]: pgmap v161: 129 pgs: 129 active+clean; 3.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-09-12T03:34:33.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.623+0000 7fdbf8ce6640 1 -- 172.21.15.64:0/558945217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbf4101ad0 msgr2=0x7fdbf4101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:33.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.623+0000 7fdbf8ce6640 1 --2- 172.21.15.64:0/558945217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbf4101ad0 0x7fdbf4101f30 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8009930 tx=0x7fdbe802a420 comp rx=0 tx=0).stop 2024-09-12T03:34:33.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.623+0000 7fdbf8ce6640 1 -- 172.21.15.64:0/558945217 shutdown_connections 2024-09-12T03:34:33.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.623+0000 7fdbf8ce6640 1 --2- 172.21.15.64:0/558945217 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbf4102470 0x7fdbf410e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:33.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.624+0000 7fdbf8ce6640 1 --2- 172.21.15.64:0/558945217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbf4101ad0 0x7fdbf4101f30 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:33.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.624+0000 7fdbf8ce6640 1 --2- 172.21.15.64:0/558945217 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdbf4107b70 0x7fdbf4107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:33.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.624+0000 7fdbf8ce6640 1 -- 172.21.15.64:0/558945217 >> 172.21.15.64:0/558945217 conn(0x7fdbf40fd7d0 msgr2=0x7fdbf40ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:33.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.624+0000 7fdbf8ce6640 1 -- 172.21.15.64:0/558945217 shutdown_connections 2024-09-12T03:34:33.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.624+0000 7fdbf8ce6640 1 -- 172.21.15.64:0/558945217 wait complete. 2024-09-12T03:34:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.624+0000 7fdbf8ce6640 1 Processor -- start 2024-09-12T03:34:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.624+0000 7fdbf8ce6640 1 -- start start 2024-09-12T03:34:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.625+0000 7fdbf8ce6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbf4101ad0 0x7fdbf4198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.625+0000 7fdbf8ce6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbf4102470 0x7fdbf4198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.625+0000 7fdbf8ce6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdbf4107b70 0x7fdbf419cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.625+0000 7fdbf8ce6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf4111d90 con 0x7fdbf4101ad0 2024-09-12T03:34:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.625+0000 7fdbf8ce6640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf4111c10 con 0x7fdbf4107b70 2024-09-12T03:34:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.625+0000 7fdbf8ce6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf4111f10 con 0x7fdbf4102470 2024-09-12T03:34:33.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.625+0000 7fdbf259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbf4101ad0 0x7fdbf4198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:33.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.625+0000 7fdbf259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbf4101ad0 0x7fdbf4198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36998/0 (socket says 172.21.15.64:36998) 2024-09-12T03:34:33.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.625+0000 7fdbf259c640 1 -- 172.21.15.64:0/530483944 learned_addr learned my addr 172.21.15.64:0/530483944 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:34:33.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.625+0000 7fdbf1d9b640 1 --2- 172.21.15.64:0/530483944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbf4102470 0x7fdbf4198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:33.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.625+0000 7fdbf2d9d640 1 --2- 172.21.15.64:0/530483944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdbf4107b70 0x7fdbf419cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:33.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.625+0000 7fdbf259c640 1 -- 172.21.15.64:0/530483944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbf4102470 msgr2=0x7fdbf4198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:33.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.625+0000 7fdbf259c640 1 --2- 172.21.15.64:0/530483944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbf4102470 0x7fdbf4198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:33.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.625+0000 7fdbf259c640 1 -- 172.21.15.64:0/530483944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdbf4107b70 msgr2=0x7fdbf419cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:33.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.625+0000 7fdbf259c640 1 --2- 172.21.15.64:0/530483944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdbf4107b70 0x7fdbf419cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:33.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.625+0000 7fdbf259c640 1 -- 172.21.15.64:0/530483944 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf419d240 con 0x7fdbf4101ad0 2024-09-12T03:34:33.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.625+0000 7fdbf2d9d640 1 --2- 172.21.15.64:0/530483944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdbf4107b70 0x7fdbf419cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:34:33.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.625+0000 7fdbf259c640 1 --2- 172.21.15.64:0/530483944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbf4101ad0 0x7fdbf4198250 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fdbe400e9f0 tx=0x7fdbe400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:33.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.626+0000 7fdbe37fe640 1 -- 172.21.15.64:0/530483944 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe400cd80 con 0x7fdbf4101ad0 2024-09-12T03:34:33.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.626+0000 7fdbe37fe640 1 -- 172.21.15.64:0/530483944 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdbe40108e0 con 0x7fdbf4101ad0 2024-09-12T03:34:33.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.626+0000 7fdbf8ce6640 1 -- 172.21.15.64:0/530483944 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbf419d530 con 0x7fdbf4101ad0 2024-09-12T03:34:33.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.626+0000 7fdbe37fe640 1 -- 172.21.15.64:0/530483944 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4018920 con 0x7fdbf4101ad0 2024-09-12T03:34:33.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.626+0000 7fdbf8ce6640 1 -- 172.21.15.64:0/530483944 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbf41a4dc0 con 0x7fdbf4101ad0 2024-09-12T03:34:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.628+0000 7fdbe37fe640 1 -- 172.21.15.64:0/530483944 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fdbe4010a90 con 0x7fdbf4101ad0 2024-09-12T03:34:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.628+0000 7fdbfba77740 1 Processor -- start 2024-09-12T03:34:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.629+0000 7fdbfba77740 1 -- start start 2024-09-12T03:34:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.629+0000 7fdbfba77740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b8832e9d0 0x557b8832edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.629+0000 7fdbfba77740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b8832f2f0 0x557b8833db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.629+0000 7fdbfba77740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b8833e0c0 0x557b883404b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.629+0000 7fdbfba77740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x557b882932f0 con 0x557b8832f2f0 2024-09-12T03:34:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.629+0000 7fdbfba77740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557b8823b270 con 0x557b8833e0c0 2024-09-12T03:34:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.629+0000 7fdbfba77740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557b8828f2b0 con 0x557b8832e9d0 2024-09-12T03:34:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.629+0000 7fdbf259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b8832f2f0 0x557b8833db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.629+0000 7fdbf259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b8832f2f0 0x557b8833db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37000/0 (socket says 172.21.15.64:37000) 2024-09-12T03:34:33.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.629+0000 7fdbf259c640 1 -- 172.21.15.64:0/3521723155 learned_addr learned my addr 172.21.15.64:0/3521723155 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:34:33.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.629+0000 7fdbe37fe640 1 --2- 172.21.15.64:0/530483944 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fdbdc0a2e20 0x7fdbdc0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:33.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.629+0000 7fdbf2d9d640 1 --2- 172.21.15.64:0/3521723155 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b8832e9d0 0x557b8832edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:33.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.629+0000 7fdbf1d9b640 1 --2- 172.21.15.64:0/3521723155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b8833e0c0 0x557b883404b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:33.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.629+0000 7fdbf2d9d640 1 --2- 172.21.15.64:0/530483944 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fdbdc0a2e20 0x7fdbdc0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:33.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.629+0000 7fdbe37fe640 1 -- 172.21.15.64:0/530483944 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7fdbe40cfab0 con 0x7fdbf4101ad0 2024-09-12T03:34:33.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.630+0000 7fdbf2d9d640 1 -- 172.21.15.64:0/3521723155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b8833e0c0 msgr2=0x557b883404b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:33.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.630+0000 7fdbf2d9d640 1 --2- 172.21.15.64:0/3521723155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b8833e0c0 0x557b883404b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:33.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.630+0000 7fdbf2d9d640 1 -- 172.21.15.64:0/3521723155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b8832f2f0 msgr2=0x557b8833db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:33.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.630+0000 7fdbf2d9d640 1 --2- 172.21.15.64:0/3521723155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b8832f2f0 0x557b8833db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:33.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.630+0000 7fdbf2d9d640 1 -- 172.21.15.64:0/3521723155 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b8823a340 con 0x557b8832e9d0 2024-09-12T03:34:33.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.630+0000 7fdbf259c640 1 --2- 172.21.15.64:0/3521723155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b8832f2f0 0x557b8833db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:34:33.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.630+0000 7fdbf2d9d640 1 --2- 172.21.15.64:0/3521723155 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b8832e9d0 0x557b8832edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdbec0149c0 tx=0x7fdbec014e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dccbd8f6a2f68f5f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:33.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.630+0000 7fdbf2d9d640 1 --2- 172.21.15.64:0/530483944 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fdbdc0a2e20 0x7fdbdc0a52e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdbec01c000 tx=0x7fdbec01e000 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:33.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.630+0000 7fdbe0ff9640 1 -- 172.21.15.64:0/3521723155 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec014050 con 0x557b8832e9d0 2024-09-12T03:34:33.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.630+0000 7fdbe0ff9640 1 -- 172.21.15.64:0/3521723155 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdbec0141f0 con 0x557b8832e9d0 2024-09-12T03:34:33.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.630+0000 7fdbe0ff9640 1 -- 172.21.15.64:0/3521723155 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec0107e0 con 0x557b8832e9d0 2024-09-12T03:34:33.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.631+0000 7fdbfba77740 1 -- 172.21.15.64:0/3521723155 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b8832e9d0 msgr2=0x557b8832edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:33.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.631+0000 7fdbfba77740 1 --2- 172.21.15.64:0/3521723155 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b8832e9d0 0x557b8832edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdbec0149c0 tx=0x7fdbec014e80 comp rx=0 tx=0).stop 2024-09-12T03:34:33.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.631+0000 7fdbfba77740 1 -- 172.21.15.64:0/3521723155 shutdown_connections 2024-09-12T03:34:33.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.631+0000 7fdbfba77740 1 --2- 172.21.15.64:0/3521723155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b8833e0c0 0x557b883404b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:33.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.631+0000 7fdbfba77740 1 --2- 172.21.15.64:0/3521723155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b8832f2f0 0x557b8833db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:33.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.631+0000 7fdbfba77740 1 --2- 172.21.15.64:0/3521723155 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b8832e9d0 0x557b8832edb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:33.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.631+0000 7fdbfba77740 1 -- 172.21.15.64:0/3521723155 >> 172.21.15.64:0/3521723155 conn(0x557b8832b060 msgr2=0x557b8832b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:33.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.631+0000 7fdbfba77740 1 -- 172.21.15.64:0/3521723155 shutdown_connections 2024-09-12T03:34:33.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.631+0000 7fdbfba77740 1 -- 172.21.15.64:0/3521723155 wait complete. 2024-09-12T03:34:33.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.631+0000 7fdbfba77740 1 -- start start 2024-09-12T03:34:33.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.631+0000 7fdbfba77740 1 Processor -- start 2024-09-12T03:34:33.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.632+0000 7fdbfba77740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b8832e9d0 0x557b8833b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:33.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.632+0000 7fdbfba77740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b8832f2f0 0x557b8833b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:33.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.632+0000 7fdbfba77740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b8833e0c0 0x557b8833bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:33.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.632+0000 7fdbfba77740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x557b88340c90 con 0x557b8832f2f0 2024-09-12T03:34:33.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.632+0000 7fdbfba77740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557b8821c290 con 0x557b8832e9d0 2024-09-12T03:34:33.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.632+0000 7fdbfba77740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557b88340e10 con 0x557b8833e0c0 2024-09-12T03:34:33.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.632+0000 7fdbfba77740 10 client.0 _ll_register_callbacks cb 0x557b8824d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:34:33.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.632+0000 7fdbf1d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b8832f2f0 0x557b8833b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:33.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.632+0000 7fdbf1d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b8832f2f0 0x557b8833b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37004/0 (socket says 172.21.15.64:37004) 2024-09-12T03:34:33.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.632+0000 7fdbf1d9b640 1 -- 172.21.15.64:0/3296333699 learned_addr learned my addr 172.21.15.64:0/3296333699 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:34:33.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.632+0000 7fdbf259c640 1 --2- 172.21.15.64:0/3296333699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b8832e9d0 0x557b8833b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:33.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.632+0000 7fdbf2d9d640 1 --2- 172.21.15.64:0/3296333699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b8833e0c0 0x557b8833bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:33.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.632+0000 7fdbf1d9b640 1 -- 172.21.15.64:0/3296333699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b8833e0c0 msgr2=0x557b8833bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:33.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.632+0000 7fdbf1d9b640 1 --2- 172.21.15.64:0/3296333699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b8833e0c0 0x557b8833bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:33.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.632+0000 7fdbf1d9b640 1 -- 172.21.15.64:0/3296333699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b8832e9d0 msgr2=0x557b8833b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:33.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.632+0000 7fdbf1d9b640 1 --2- 172.21.15.64:0/3296333699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b8832e9d0 0x557b8833b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:33.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.632+0000 7fdbf1d9b640 1 -- 172.21.15.64:0/3296333699 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b87fa6210 con 0x557b8832f2f0 2024-09-12T03:34:33.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.632+0000 7fdbf1d9b640 1 --2- 172.21.15.64:0/3296333699 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b8832f2f0 0x557b8833b5f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdbe802a940 tx=0x7fdbe8009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d843807f0e479008 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:33.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.632+0000 7fdbd9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:34:33.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.633+0000 7fdbd9ffb640 1 -- 172.21.15.64:0/3296333699 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8035040 con 0x557b8832f2f0 2024-09-12T03:34:33.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.633+0000 7fdbd9ffb640 1 -- 172.21.15.64:0/3296333699 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7fdbe8031860 con 0x557b8832f2f0 2024-09-12T03:34:33.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.633+0000 7fdbfba77740 1 -- 172.21.15.64:0/3296333699 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbec0141f0 con 0x557b8832f2f0 2024-09-12T03:34:33.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.633+0000 7fdbd9ffb640 1 -- 172.21.15.64:0/3296333699 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdbe8004040 con 0x557b8832f2f0 2024-09-12T03:34:33.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.633+0000 7fdbfba77740 10 client.15774 fetch_fsmap learned FSMap version 25 2024-09-12T03:34:33.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.633+0000 7fdbfba77740 1 -- 172.21.15.64:0/3296333699 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557b88072620 con 0x557b8832f2f0 2024-09-12T03:34:33.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.633+0000 7fdbd9ffb640 1 -- 172.21.15.64:0/3296333699 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdbec0141f0 con 0x557b8832f2f0 2024-09-12T03:34:33.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.633+0000 7fdbd9ffb640 1 -- 172.21.15.64:0/3296333699 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fdbe803d450 con 0x557b8832f2f0 2024-09-12T03:34:33.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.633+0000 7fdbfba77740 10 client.15774 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:34:33.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.633+0000 7fdbfba77740 10 client.15774 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:34:33.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.633+0000 7fdbfba77740 20 client.15774 populate_metadata read hostname 'smithi064' 2024-09-12T03:34:33.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.633+0000 7fdbfba77740 1 --2- 172.21.15.64:0/3296333699 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x557b8834e0a0 0x557b883504b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:33.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.633+0000 7fdbfba77740 4 client.15774 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:34:33.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.633+0000 7fdbfba77740 1 -- 172.21.15.64:0/3296333699 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b882a0a20 con 0x557b8834e0a0 2024-09-12T03:34:33.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.634+0000 7fdbf259c640 1 --2- 172.21.15.64:0/3296333699 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x557b8834e0a0 0x557b883504b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:33.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.634+0000 7fdbd9ffb640 10 client.15774 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:34:33.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.634+0000 7fdbf259c640 1 --2- 172.21.15.64:0/3296333699 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x557b8834e0a0 0x557b883504b0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e9d9e6c41a31afc server_cookie=8d4099fee37c5d25 in_seq=0 out_seq=0 2024-09-12T03:34:33.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.635+0000 7fdbd9ffb640 1 -- 172.21.15.64:0/3296333699 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557b882a0a20 con 0x557b8834e0a0 2024-09-12T03:34:33.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.635+0000 7fdbd9ffb640 10 client.15774 handle_command_reply: tid=0 2024-09-12T03:34:33.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.635+0000 7fdbfba77740 1 client.15774 shutdown 2024-09-12T03:34:33.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.635+0000 7fdbfba77740 1 -- 172.21.15.64:0/3296333699 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b8832f2f0 msgr2=0x557b8833b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:33.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.635+0000 7fdbfba77740 1 --2- 172.21.15.64:0/3296333699 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b8832f2f0 0x557b8833b5f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdbe802a940 tx=0x7fdbe8009670 comp rx=0 tx=0).stop 2024-09-12T03:34:33.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.635+0000 7fdbfba77740 1 -- 172.21.15.64:0/3296333699 shutdown_connections 2024-09-12T03:34:33.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.635+0000 7fdbfba77740 1 --2- 172.21.15.64:0/3296333699 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x557b8834e0a0 0x557b883504b0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:33.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.635+0000 7fdbfba77740 1 --2- 172.21.15.64:0/3296333699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b8833e0c0 0x557b8833bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:33.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.635+0000 7fdbfba77740 1 --2- 172.21.15.64:0/3296333699 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b8832f2f0 0x557b8833b5f0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:33.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.635+0000 7fdbd9ffb640 0 client.15774 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:34:33.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.635+0000 7fdbfba77740 1 --2- 172.21.15.64:0/3296333699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b8832e9d0 0x557b8833b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:33.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.635+0000 7fdbfba77740 1 -- 172.21.15.64:0/3296333699 >> 172.21.15.64:0/3296333699 conn(0x557b8832b060 msgr2=0x557b88330d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:33.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.635+0000 7fdbfba77740 1 -- 172.21.15.64:0/3296333699 shutdown_connections 2024-09-12T03:34:33.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.635+0000 7fdbfba77740 1 -- 172.21.15.64:0/3296333699 wait complete. 2024-09-12T03:34:33.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.635+0000 7fdbfba77740 20 client.15774 trim_cache size 0 max 16384 2024-09-12T03:34:33.650 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:34:33 smithi115 ceph-mon[34801]: pgmap v161: 129 pgs: 129 active+clean; 3.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-09-12T03:34:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.650+0000 7fdbfba77740 1 Processor -- start 2024-09-12T03:34:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.650+0000 7fdbfba77740 1 -- start start 2024-09-12T03:34:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.650+0000 7fdbfba77740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b8832e9d0 0x557b88339960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.650+0000 7fdbfba77740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b8832f2f0 0x557b88355dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.650+0000 7fdbfba77740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b8833e0c0 0x557b88356300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.650+0000 7fdbfba77740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf41af5c0 con 0x557b8832f2f0 2024-09-12T03:34:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.650+0000 7fdbfba77740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf41ace90 con 0x557b8832e9d0 2024-09-12T03:34:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.650+0000 7fdbfba77740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf41aace0 con 0x557b8833e0c0 2024-09-12T03:34:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.650+0000 7fdbf1d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b8832f2f0 0x557b88355dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.650+0000 7fdbf1d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b8832f2f0 0x557b88355dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37016/0 (socket says 172.21.15.64:37016) 2024-09-12T03:34:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.650+0000 7fdbf1d9b640 1 -- 172.21.15.64:0/920035070 learned_addr learned my addr 172.21.15.64:0/920035070 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:34:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.650+0000 7fdbf259c640 1 --2- 172.21.15.64:0/920035070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b8832e9d0 0x557b88339960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.650+0000 7fdbf1d9b640 1 -- 172.21.15.64:0/920035070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b8833e0c0 msgr2=0x557b88356300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.650+0000 7fdbf1d9b640 1 --2- 172.21.15.64:0/920035070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b8833e0c0 0x557b88356300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.650+0000 7fdbf1d9b640 1 -- 172.21.15.64:0/920035070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b8832e9d0 msgr2=0x557b88339960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.650+0000 7fdbf1d9b640 1 --2- 172.21.15.64:0/920035070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b8832e9d0 0x557b88339960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.650+0000 7fdbf1d9b640 1 -- 172.21.15.64:0/920035070 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b8833a040 con 0x557b8832f2f0 2024-09-12T03:34:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.650+0000 7fdbf259c640 1 --2- 172.21.15.64:0/920035070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b8832e9d0 0x557b88339960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:34:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.651+0000 7fdbf1d9b640 1 --2- 172.21.15.64:0/920035070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b8832f2f0 0x557b88355dc0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fdbe803d820 tx=0x7fdbe8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d29f2805537fcaa0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.651+0000 7fdbdbfff640 1 -- 172.21.15.64:0/920035070 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8035070 con 0x557b8832f2f0 2024-09-12T03:34:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.651+0000 7fdbdbfff640 1 -- 172.21.15.64:0/920035070 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdbe803da90 con 0x557b8832f2f0 2024-09-12T03:34:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.651+0000 7fdbdbfff640 1 -- 172.21.15.64:0/920035070 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8005040 con 0x557b8832f2f0 2024-09-12T03:34:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.651+0000 7fdbfba77740 1 -- 172.21.15.64:0/920035070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b8832f2f0 msgr2=0x557b88355dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.651+0000 7fdbfba77740 1 --2- 172.21.15.64:0/920035070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b8832f2f0 0x557b88355dc0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fdbe803d820 tx=0x7fdbe8000bc0 comp rx=0 tx=0).stop 2024-09-12T03:34:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.651+0000 7fdbfba77740 1 -- 172.21.15.64:0/920035070 shutdown_connections 2024-09-12T03:34:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.651+0000 7fdbfba77740 1 --2- 172.21.15.64:0/920035070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b8833e0c0 0x557b88356300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.651+0000 7fdbfba77740 1 --2- 172.21.15.64:0/920035070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b8832f2f0 0x557b88355dc0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.651+0000 7fdbfba77740 1 --2- 172.21.15.64:0/920035070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b8832e9d0 0x557b88339960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.651+0000 7fdbfba77740 1 -- 172.21.15.64:0/920035070 >> 172.21.15.64:0/920035070 conn(0x557b8832b060 msgr2=0x557b8833b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.651+0000 7fdbfba77740 1 -- 172.21.15.64:0/920035070 shutdown_connections 2024-09-12T03:34:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.651+0000 7fdbfba77740 1 -- 172.21.15.64:0/920035070 wait complete. 2024-09-12T03:34:33.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.652+0000 7fdbfba77740 1 -- start start 2024-09-12T03:34:33.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.652+0000 7fdbfba77740 1 Processor -- start 2024-09-12T03:34:33.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.652+0000 7fdbfba77740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b8832e9d0 0x557b88341fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:33.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.652+0000 7fdbfba77740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b8832f2f0 0x557b88342500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:33.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.652+0000 7fdbfba77740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b8833e0c0 0x557b88342a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:33.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.652+0000 7fdbfba77740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf41ab350 con 0x557b8833e0c0 2024-09-12T03:34:33.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.652+0000 7fdbfba77740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf41ac7c0 con 0x557b8832e9d0 2024-09-12T03:34:33.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.652+0000 7fdbfba77740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf41aba20 con 0x557b8832f2f0 2024-09-12T03:34:33.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.652+0000 7fdbfba77740 10 client.0 _ll_register_callbacks cb 0x7fdbe8031c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:34:33.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.652+0000 7fdbf2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b8833e0c0 0x557b88342a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:33.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.652+0000 7fdbf2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b8833e0c0 0x557b88342a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37032/0 (socket says 172.21.15.64:37032) 2024-09-12T03:34:33.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.652+0000 7fdbf2d9d640 1 -- 172.21.15.64:0/4170470652 learned_addr learned my addr 172.21.15.64:0/4170470652 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:34:33.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.652+0000 7fdbf259c640 1 --2- 172.21.15.64:0/4170470652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b8832e9d0 0x557b88341fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:33.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.652+0000 7fdbf1d9b640 1 --2- 172.21.15.64:0/4170470652 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b8832f2f0 0x557b88342500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:33.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.652+0000 7fdbf2d9d640 1 -- 172.21.15.64:0/4170470652 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b8832f2f0 msgr2=0x557b88342500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:33.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.652+0000 7fdbf2d9d640 1 --2- 172.21.15.64:0/4170470652 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b8832f2f0 0x557b88342500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:33.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.652+0000 7fdbf2d9d640 1 -- 172.21.15.64:0/4170470652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b8832e9d0 msgr2=0x557b88341fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:33.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.652+0000 7fdbf2d9d640 1 --2- 172.21.15.64:0/4170470652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b8832e9d0 0x557b88341fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:33.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.653+0000 7fdbf2d9d640 1 -- 172.21.15.64:0/4170470652 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b883b7f90 con 0x557b8833e0c0 2024-09-12T03:34:33.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.653+0000 7fdbf2d9d640 1 --2- 172.21.15.64:0/4170470652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b8833e0c0 0x557b88342a40 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fdbec012000 tx=0x7fdbec005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4887486a31e52e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:33.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.653+0000 7fdbd9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:34:33.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.653+0000 7fdbd9ffb640 1 -- 172.21.15.64:0/4170470652 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec02d070 con 0x557b8833e0c0 2024-09-12T03:34:33.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.653+0000 7fdbd9ffb640 1 -- 172.21.15.64:0/4170470652 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7fdbec014050 con 0x557b8833e0c0 2024-09-12T03:34:33.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.653+0000 7fdbfba77740 1 -- 172.21.15.64:0/4170470652 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbe803da90 con 0x557b8833e0c0 2024-09-12T03:34:33.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.653+0000 7fdbd9ffb640 1 -- 172.21.15.64:0/4170470652 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdbec039050 con 0x557b8833e0c0 2024-09-12T03:34:33.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.653+0000 7fdbfba77740 10 client.15780 fetch_fsmap learned FSMap version 25 2024-09-12T03:34:33.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.653+0000 7fdbfba77740 1 -- 172.21.15.64:0/4170470652 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557b883b9280 con 0x557b8833e0c0 2024-09-12T03:34:33.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.653+0000 7fdbd9ffb640 1 -- 172.21.15.64:0/4170470652 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdbe803da90 con 0x557b8833e0c0 2024-09-12T03:34:33.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.653+0000 7fdbd9ffb640 1 -- 172.21.15.64:0/4170470652 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fdbec00f040 con 0x557b8833e0c0 2024-09-12T03:34:33.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.653+0000 7fdbfba77740 10 client.15780 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:34:33.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.653+0000 7fdbfba77740 10 client.15780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:34:33.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.653+0000 7fdbfba77740 20 client.15780 populate_metadata read hostname 'smithi064' 2024-09-12T03:34:33.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.654+0000 7fdbfba77740 1 --2- 172.21.15.64:0/4170470652 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x557b883b97a0 0x557b883bbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:33.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.654+0000 7fdbfba77740 4 client.15780 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:34:33.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.654+0000 7fdbfba77740 1 -- 172.21.15.64:0/4170470652 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557b8829ad70 con 0x557b883b97a0 2024-09-12T03:34:33.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.654+0000 7fdbf259c640 1 --2- 172.21.15.64:0/4170470652 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x557b883b97a0 0x557b883bbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:33.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.654+0000 7fdbd9ffb640 10 client.15780 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:34:33.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.654+0000 7fdbf259c640 1 --2- 172.21.15.64:0/4170470652 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x557b883b97a0 0x557b883bbbc0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a618f2d8c78085d server_cookie=55c0deb8fbba6b09 in_seq=0 out_seq=0 2024-09-12T03:34:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.668+0000 7fdbd9ffb640 1 -- 172.21.15.64:0/4170470652 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557b8829ad70 con 0x557b883b97a0 2024-09-12T03:34:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.668+0000 7fdbd9ffb640 10 client.15780 handle_command_reply: tid=0 2024-09-12T03:34:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.668+0000 7fdbfba77740 1 client.15780 shutdown 2024-09-12T03:34:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.668+0000 7fdbfba77740 1 -- 172.21.15.64:0/4170470652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b8833e0c0 msgr2=0x557b88342a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.668+0000 7fdbfba77740 1 --2- 172.21.15.64:0/4170470652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b8833e0c0 0x557b88342a40 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fdbec012000 tx=0x7fdbec005f90 comp rx=0 tx=0).stop 2024-09-12T03:34:33.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.668+0000 7fdbfba77740 1 -- 172.21.15.64:0/4170470652 shutdown_connections 2024-09-12T03:34:33.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.668+0000 7fdbfba77740 1 --2- 172.21.15.64:0/4170470652 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x557b883b97a0 0x557b883bbbc0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:33.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.668+0000 7fdbfba77740 1 --2- 172.21.15.64:0/4170470652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557b8833e0c0 0x557b88342a40 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:33.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.668+0000 7fdbfba77740 1 --2- 172.21.15.64:0/4170470652 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b8832f2f0 0x557b88342500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:33.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.668+0000 7fdbd9ffb640 0 client.15780 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:34:33.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.668+0000 7fdbfba77740 1 --2- 172.21.15.64:0/4170470652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b8832e9d0 0x557b88341fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:33.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.668+0000 7fdbfba77740 1 -- 172.21.15.64:0/4170470652 >> 172.21.15.64:0/4170470652 conn(0x557b8832b060 msgr2=0x557b88330d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:33.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.668+0000 7fdbfba77740 1 -- 172.21.15.64:0/4170470652 shutdown_connections 2024-09-12T03:34:33.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.668+0000 7fdbfba77740 1 -- 172.21.15.64:0/4170470652 wait complete. 2024-09-12T03:34:33.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.668+0000 7fdbfba77740 20 client.15780 trim_cache size 0 max 16384 2024-09-12T03:34:33.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:34:33.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:34:33.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:34:33.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:34:33.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.669+0000 7fdbf8ce6640 1 -- 172.21.15.64:0/530483944 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fdbdc0a2e20 msgr2=0x7fdbdc0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:33.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.669+0000 7fdbf8ce6640 1 --2- 172.21.15.64:0/530483944 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fdbdc0a2e20 0x7fdbdc0a52e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdbec01c000 tx=0x7fdbec01e000 comp rx=0 tx=0).stop 2024-09-12T03:34:33.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.669+0000 7fdbf8ce6640 1 -- 172.21.15.64:0/530483944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbf4101ad0 msgr2=0x7fdbf4198250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:33.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.669+0000 7fdbf8ce6640 1 --2- 172.21.15.64:0/530483944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbf4101ad0 0x7fdbf4198250 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fdbe400e9f0 tx=0x7fdbe400eeb0 comp rx=0 tx=0).stop 2024-09-12T03:34:33.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.669+0000 7fdbf8ce6640 1 -- 172.21.15.64:0/530483944 shutdown_connections 2024-09-12T03:34:33.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.669+0000 7fdbf8ce6640 1 --2- 172.21.15.64:0/530483944 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fdbdc0a2e20 0x7fdbdc0a52e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:33.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.669+0000 7fdbf8ce6640 1 --2- 172.21.15.64:0/530483944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdbf4107b70 0x7fdbf419cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:33.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.669+0000 7fdbf8ce6640 1 --2- 172.21.15.64:0/530483944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbf4102470 0x7fdbf4198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:33.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.669+0000 7fdbf8ce6640 1 --2- 172.21.15.64:0/530483944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbf4101ad0 0x7fdbf4198250 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:33.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.669+0000 7fdbf8ce6640 1 -- 172.21.15.64:0/530483944 >> 172.21.15.64:0/530483944 conn(0x7fdbf40fd7d0 msgr2=0x7fdbf40fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:33.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.669+0000 7fdbf8ce6640 1 -- 172.21.15.64:0/530483944 shutdown_connections 2024-09-12T03:34:33.673 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:33.669+0000 7fdbf8ce6640 1 -- 172.21.15.64:0/530483944 wait complete. 2024-09-12T03:34:35.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:34:35 smithi195 ceph-mon[34786]: pgmap v162: 129 pgs: 129 active+clean; 3.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 10 op/s 2024-09-12T03:34:35.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:34:35 smithi064 ceph-mon[30952]: pgmap v162: 129 pgs: 129 active+clean; 3.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 10 op/s 2024-09-12T03:34:35.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:34:35 smithi115 ceph-mon[34801]: pgmap v162: 129 pgs: 129 active+clean; 3.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 10 op/s 2024-09-12T03:34:37.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:34:37 smithi064 ceph-mon[30952]: pgmap v163: 129 pgs: 129 active+clean; 3.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 10 op/s 2024-09-12T03:34:37.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:34:37 smithi115 ceph-mon[34801]: pgmap v163: 129 pgs: 129 active+clean; 3.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 10 op/s 2024-09-12T03:34:37.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:34:37 smithi195 ceph-mon[34786]: pgmap v163: 129 pgs: 129 active+clean; 3.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 10 op/s 2024-09-12T03:34:39.521 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:34:39 smithi064 ceph-mon[30952]: pgmap v164: 129 pgs: 129 active+clean; 3.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-09-12T03:34:39.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:34:39 smithi115 ceph-mon[34801]: pgmap v164: 129 pgs: 129 active+clean; 3.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-09-12T03:34:39.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:34:39 smithi195 ceph-mon[34786]: pgmap v164: 129 pgs: 129 active+clean; 3.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-09-12T03:34:41.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:34:41 smithi064 ceph-mon[30952]: pgmap v165: 129 pgs: 129 active+clean; 3.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 13 op/s 2024-09-12T03:34:41.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:34:41 smithi115 ceph-mon[34801]: pgmap v165: 129 pgs: 129 active+clean; 3.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 13 op/s 2024-09-12T03:34:41.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:34:41 smithi195 ceph-mon[34786]: pgmap v165: 129 pgs: 129 active+clean; 3.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 13 op/s 2024-09-12T03:34:43.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:34:43 smithi064 ceph-mon[30952]: pgmap v166: 129 pgs: 129 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-09-12T03:34:43.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:34:43 smithi115 ceph-mon[34801]: pgmap v166: 129 pgs: 129 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-09-12T03:34:43.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:34:43 smithi195 ceph-mon[34786]: pgmap v166: 129 pgs: 129 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-09-12T03:34:45.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:34:45 smithi064 ceph-mon[30952]: pgmap v167: 129 pgs: 129 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-09-12T03:34:45.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:34:45 smithi115 ceph-mon[34801]: pgmap v167: 129 pgs: 129 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-09-12T03:34:45.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:34:45 smithi195 ceph-mon[34786]: pgmap v167: 129 pgs: 129 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-09-12T03:34:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.770+0000 7f0a82265640 1 -- 172.21.15.64:0/2759455200 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a7c1009b0 msgr2=0x7f0a7c100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.770+0000 7f0a82265640 1 --2- 172.21.15.64:0/2759455200 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a7c1009b0 0x7f0a7c100e10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0a70009930 tx=0x7f0a7002a420 comp rx=0 tx=0).stop 2024-09-12T03:34:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.770+0000 7f0a82265640 1 -- 172.21.15.64:0/2759455200 shutdown_connections 2024-09-12T03:34:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.770+0000 7f0a82265640 1 --2- 172.21.15.64:0/2759455200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a7c101350 0x7f0a7c10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.770+0000 7f0a82265640 1 --2- 172.21.15.64:0/2759455200 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a7c1009b0 0x7f0a7c100e10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.770+0000 7f0a82265640 1 --2- 172.21.15.64:0/2759455200 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a7c1069c0 0x7f0a7c106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.770+0000 7f0a82265640 1 -- 172.21.15.64:0/2759455200 >> 172.21.15.64:0/2759455200 conn(0x7f0a7c077990 msgr2=0x7f0a7c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.770+0000 7f0a82265640 1 -- 172.21.15.64:0/2759455200 shutdown_connections 2024-09-12T03:34:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.770+0000 7f0a82265640 1 -- 172.21.15.64:0/2759455200 wait complete. 2024-09-12T03:34:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.770+0000 7f0a82265640 1 Processor -- start 2024-09-12T03:34:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.770+0000 7f0a82265640 1 -- start start 2024-09-12T03:34:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.771+0000 7f0a82265640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a7c1009b0 0x7f0a7c0fcde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.771+0000 7f0a82265640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a7c101350 0x7f0a7c0fd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.771+0000 7f0a82265640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a7c1069c0 0x7f0a7c0fd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.771+0000 7f0a82265640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7c110cb0 con 0x7f0a7c1069c0 2024-09-12T03:34:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.771+0000 7f0a82265640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7c110b30 con 0x7f0a7c101350 2024-09-12T03:34:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.771+0000 7f0a82265640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7c110e30 con 0x7f0a7c1009b0 2024-09-12T03:34:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.771+0000 7f0a80802640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a7c1069c0 0x7f0a7c0fd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.771+0000 7f0a80802640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a7c1069c0 0x7f0a7c0fd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47638/0 (socket says 172.21.15.64:47638) 2024-09-12T03:34:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.771+0000 7f0a80802640 1 -- 172.21.15.64:0/2506330985 learned_addr learned my addr 172.21.15.64:0/2506330985 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:34:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.771+0000 7f0a7bfff640 1 --2- 172.21.15.64:0/2506330985 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a7c1009b0 0x7f0a7c0fcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.771+0000 7f0a7b7fe640 1 --2- 172.21.15.64:0/2506330985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a7c101350 0x7f0a7c0fd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.771+0000 7f0a80802640 1 -- 172.21.15.64:0/2506330985 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a7c1009b0 msgr2=0x7f0a7c0fcde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.771+0000 7f0a80802640 1 --2- 172.21.15.64:0/2506330985 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a7c1009b0 0x7f0a7c0fcde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.771+0000 7f0a80802640 1 -- 172.21.15.64:0/2506330985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a7c101350 msgr2=0x7f0a7c0fd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.771+0000 7f0a80802640 1 --2- 172.21.15.64:0/2506330985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a7c101350 0x7f0a7c0fd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.771+0000 7f0a80802640 1 -- 172.21.15.64:0/2506330985 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a7c1a6850 con 0x7f0a7c1069c0 2024-09-12T03:34:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.771+0000 7f0a7b7fe640 1 --2- 172.21.15.64:0/2506330985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a7c101350 0x7f0a7c0fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:34:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.772+0000 7f0a80802640 1 --2- 172.21.15.64:0/2506330985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a7c1069c0 0x7f0a7c0fd860 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f0a7400eb10 tx=0x7f0a7400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.772+0000 7f0a797fa640 1 -- 172.21.15.64:0/2506330985 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a74004b20 con 0x7f0a7c1069c0 2024-09-12T03:34:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.772+0000 7f0a797fa640 1 -- 172.21.15.64:0/2506330985 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a74004cc0 con 0x7f0a7c1069c0 2024-09-12T03:34:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.772+0000 7f0a82265640 1 -- 172.21.15.64:0/2506330985 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a7c1a6b40 con 0x7f0a7c1069c0 2024-09-12T03:34:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.772+0000 7f0a797fa640 1 -- 172.21.15.64:0/2506330985 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a74010690 con 0x7f0a7c1069c0 2024-09-12T03:34:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.772+0000 7f0a82265640 1 -- 172.21.15.64:0/2506330985 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a7c1a7040 con 0x7f0a7c1069c0 2024-09-12T03:34:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.774+0000 7f0a797fa640 1 -- 172.21.15.64:0/2506330985 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0a74005020 con 0x7f0a7c1069c0 2024-09-12T03:34:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.775+0000 7f0a85007740 1 Processor -- start 2024-09-12T03:34:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.775+0000 7f0a85007740 1 -- start start 2024-09-12T03:34:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.775+0000 7f0a85007740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562d76896af0 0x562d76896ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.775+0000 7f0a85007740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d76897410 0x562d768a1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.775+0000 7f0a85007740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d768a1cb0 0x562d768a40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.775+0000 7f0a85007740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562d767fb2f0 con 0x562d768a1cb0 2024-09-12T03:34:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.775+0000 7f0a85007740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562d767a3270 con 0x562d76896af0 2024-09-12T03:34:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.775+0000 7f0a85007740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562d767f72b0 con 0x562d76897410 2024-09-12T03:34:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.775+0000 7f0a7b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d768a1cb0 0x562d768a40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.775+0000 7f0a7b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d768a1cb0 0x562d768a40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47652/0 (socket says 172.21.15.64:47652) 2024-09-12T03:34:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.775+0000 7f0a7b7fe640 1 -- 172.21.15.64:0/3213051132 learned_addr learned my addr 172.21.15.64:0/3213051132 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:34:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.775+0000 7f0a7bfff640 1 --2- 172.21.15.64:0/3213051132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d76897410 0x562d768a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.775+0000 7f0a797fa640 1 --2- 172.21.15.64:0/2506330985 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f0a640a2da0 0x7f0a640a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.775+0000 7f0a7b7fe640 1 --2- 172.21.15.64:0/3213051132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562d76896af0 0x562d76896ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.775+0000 7f0a7b7fe640 1 -- 172.21.15.64:0/3213051132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d76897410 msgr2=0x562d768a1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.775+0000 7f0a7b7fe640 1 --2- 172.21.15.64:0/3213051132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d76897410 0x562d768a1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.775+0000 7f0a7b7fe640 1 -- 172.21.15.64:0/3213051132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562d76896af0 msgr2=0x562d76896ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.775+0000 7f0a7b7fe640 1 --2- 172.21.15.64:0/3213051132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562d76896af0 0x562d76896ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.775+0000 7f0a7b7fe640 1 -- 172.21.15.64:0/3213051132 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d767a2340 con 0x562d768a1cb0 2024-09-12T03:34:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.775+0000 7f0a80802640 1 --2- 172.21.15.64:0/2506330985 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f0a640a2da0 0x7f0a640a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.776+0000 7f0a797fa640 1 -- 172.21.15.64:0/2506330985 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f0a740cfe60 con 0x7f0a7c1069c0 2024-09-12T03:34:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.776+0000 7f0a7b7fe640 1 --2- 172.21.15.64:0/3213051132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d768a1cb0 0x562d768a40a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0a70033000 tx=0x7f0a70031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f60afa5c3d4d82c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.776+0000 7f0a627fc640 1 -- 172.21.15.64:0/3213051132 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a70035040 con 0x562d768a1cb0 2024-09-12T03:34:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.776+0000 7f0a627fc640 1 -- 172.21.15.64:0/3213051132 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a70045070 con 0x562d768a1cb0 2024-09-12T03:34:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.776+0000 7f0a627fc640 1 -- 172.21.15.64:0/3213051132 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a70003910 con 0x562d768a1cb0 2024-09-12T03:34:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.777+0000 7f0a85007740 1 -- 172.21.15.64:0/3213051132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d768a1cb0 msgr2=0x562d768a40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.777+0000 7f0a85007740 1 --2- 172.21.15.64:0/3213051132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d768a1cb0 0x562d768a40a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0a70033000 tx=0x7f0a70031400 comp rx=0 tx=0).stop 2024-09-12T03:34:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.777+0000 7f0a80802640 1 --2- 172.21.15.64:0/2506330985 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f0a640a2da0 0x7f0a640a5260 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0a7409bf40 tx=0x7f0a7406df70 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.777+0000 7f0a85007740 1 -- 172.21.15.64:0/3213051132 shutdown_connections 2024-09-12T03:34:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.777+0000 7f0a85007740 1 --2- 172.21.15.64:0/3213051132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d768a1cb0 0x562d768a40a0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.777+0000 7f0a85007740 1 --2- 172.21.15.64:0/3213051132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d76897410 0x562d768a1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.777+0000 7f0a85007740 1 --2- 172.21.15.64:0/3213051132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562d76896af0 0x562d76896ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.777+0000 7f0a85007740 1 -- 172.21.15.64:0/3213051132 >> 172.21.15.64:0/3213051132 conn(0x562d76893060 msgr2=0x562d76893440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.777+0000 7f0a85007740 1 -- 172.21.15.64:0/3213051132 shutdown_connections 2024-09-12T03:34:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.777+0000 7f0a85007740 1 -- 172.21.15.64:0/3213051132 wait complete. 2024-09-12T03:34:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.777+0000 7f0a85007740 1 -- start start 2024-09-12T03:34:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.777+0000 7f0a85007740 1 Processor -- start 2024-09-12T03:34:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.778+0000 7f0a85007740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d76896af0 0x562d7689ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.778+0000 7f0a85007740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562d76897410 0x562d7689f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.778+0000 7f0a85007740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d768a1cb0 0x562d7689f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.778+0000 7f0a85007740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562d768a4880 con 0x562d76896af0 2024-09-12T03:34:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.778+0000 7f0a85007740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562d76784290 con 0x562d76897410 2024-09-12T03:34:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.778+0000 7f0a85007740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562d768a4a00 con 0x562d768a1cb0 2024-09-12T03:34:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.778+0000 7f0a85007740 10 client.0 _ll_register_callbacks cb 0x562d767b5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:34:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.778+0000 7f0a7b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d76896af0 0x562d7689ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.778+0000 7f0a7b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d76896af0 0x562d7689ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47666/0 (socket says 172.21.15.64:47666) 2024-09-12T03:34:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.778+0000 7f0a7b7fe640 1 -- 172.21.15.64:0/1148343222 learned_addr learned my addr 172.21.15.64:0/1148343222 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:34:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.778+0000 7f0a7bfff640 1 --2- 172.21.15.64:0/1148343222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562d76897410 0x562d7689f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.778+0000 7f0a7b7fe640 1 --2- 172.21.15.64:0/1148343222 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d768a1cb0 0x562d7689f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.778+0000 7f0a7b7fe640 1 -- 172.21.15.64:0/1148343222 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d768a1cb0 msgr2=0x562d7689f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.778+0000 7f0a7b7fe640 1 --2- 172.21.15.64:0/1148343222 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d768a1cb0 0x562d7689f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.778+0000 7f0a7b7fe640 1 -- 172.21.15.64:0/1148343222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562d76897410 msgr2=0x562d7689f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.778+0000 7f0a7b7fe640 1 --2- 172.21.15.64:0/1148343222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562d76897410 0x562d7689f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.778+0000 7f0a7b7fe640 1 -- 172.21.15.64:0/1148343222 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d7650e210 con 0x562d76896af0 2024-09-12T03:34:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.778+0000 7f0a7bfff640 1 --2- 172.21.15.64:0/1148343222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562d76897410 0x562d7689f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:34:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.778+0000 7f0a7b7fe640 1 --2- 172.21.15.64:0/1148343222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d76896af0 0x562d7689ebb0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0a70005ea0 tx=0x7f0a70005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df6a463df6b114b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.778+0000 7f0a5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:34:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.779+0000 7f0a5b7fe640 1 -- 172.21.15.64:0/1148343222 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a70035040 con 0x562d76896af0 2024-09-12T03:34:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.779+0000 7f0a5b7fe640 1 -- 172.21.15.64:0/1148343222 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f0a7004d860 con 0x562d76896af0 2024-09-12T03:34:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.779+0000 7f0a85007740 1 -- 172.21.15.64:0/1148343222 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a70045070 con 0x562d76896af0 2024-09-12T03:34:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.779+0000 7f0a5b7fe640 1 -- 172.21.15.64:0/1148343222 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0a7005c050 con 0x562d76896af0 2024-09-12T03:34:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.779+0000 7f0a85007740 10 client.15792 fetch_fsmap learned FSMap version 25 2024-09-12T03:34:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.779+0000 7f0a85007740 1 -- 172.21.15.64:0/1148343222 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562d765da620 con 0x562d76896af0 2024-09-12T03:34:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.779+0000 7f0a5b7fe640 1 -- 172.21.15.64:0/1148343222 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a70045070 con 0x562d76896af0 2024-09-12T03:34:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.779+0000 7f0a5b7fe640 1 -- 172.21.15.64:0/1148343222 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f0a7003a050 con 0x562d76896af0 2024-09-12T03:34:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.779+0000 7f0a85007740 10 client.15792 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:34:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.779+0000 7f0a85007740 10 client.15792 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:34:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.779+0000 7f0a85007740 20 client.15792 populate_metadata read hostname 'smithi064' 2024-09-12T03:34:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.779+0000 7f0a85007740 1 --2- 172.21.15.64:0/1148343222 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x562d768b1ae0 0x562d768b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.779+0000 7f0a85007740 4 client.15792 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:34:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.779+0000 7f0a85007740 1 -- 172.21.15.64:0/1148343222 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d76808a20 con 0x562d768b1ae0 2024-09-12T03:34:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.780+0000 7f0a7bfff640 1 --2- 172.21.15.64:0/1148343222 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x562d768b1ae0 0x562d768b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.780+0000 7f0a7bfff640 1 --2- 172.21.15.64:0/1148343222 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x562d768b1ae0 0x562d768b3ed0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=870ee20b27fb4e42 server_cookie=5e9b2517bcd56894 in_seq=0 out_seq=0 2024-09-12T03:34:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.780+0000 7f0a5b7fe640 10 client.15792 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:34:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.781+0000 7f0a5b7fe640 1 -- 172.21.15.64:0/1148343222 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562d76808a20 con 0x562d768b1ae0 2024-09-12T03:34:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.781+0000 7f0a5b7fe640 10 client.15792 handle_command_reply: tid=0 2024-09-12T03:34:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.781+0000 7f0a85007740 1 client.15792 shutdown 2024-09-12T03:34:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.781+0000 7f0a85007740 1 -- 172.21.15.64:0/1148343222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d76896af0 msgr2=0x562d7689ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.781+0000 7f0a85007740 1 --2- 172.21.15.64:0/1148343222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d76896af0 0x562d7689ebb0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0a70005ea0 tx=0x7f0a70005750 comp rx=0 tx=0).stop 2024-09-12T03:34:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.782+0000 7f0a85007740 1 -- 172.21.15.64:0/1148343222 shutdown_connections 2024-09-12T03:34:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.782+0000 7f0a85007740 1 --2- 172.21.15.64:0/1148343222 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x562d768b1ae0 0x562d768b3ed0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.782+0000 7f0a85007740 1 --2- 172.21.15.64:0/1148343222 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d768a1cb0 0x562d7689f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.782+0000 7f0a85007740 1 --2- 172.21.15.64:0/1148343222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562d76897410 0x562d7689f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.782+0000 7f0a85007740 1 --2- 172.21.15.64:0/1148343222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d76896af0 0x562d7689ebb0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.782+0000 7f0a5b7fe640 0 client.15792 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:34:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.782+0000 7f0a85007740 1 -- 172.21.15.64:0/1148343222 >> 172.21.15.64:0/1148343222 conn(0x562d76893060 msgr2=0x562d76898d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.782+0000 7f0a85007740 1 -- 172.21.15.64:0/1148343222 shutdown_connections 2024-09-12T03:34:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.782+0000 7f0a85007740 1 -- 172.21.15.64:0/1148343222 wait complete. 2024-09-12T03:34:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.782+0000 7f0a85007740 20 client.15792 trim_cache size 0 max 16384 2024-09-12T03:34:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.796+0000 7f0a85007740 1 Processor -- start 2024-09-12T03:34:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.796+0000 7f0a85007740 1 -- start start 2024-09-12T03:34:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.796+0000 7f0a85007740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d76896af0 0x562d768a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.796+0000 7f0a85007740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562d76897410 0x562d7689f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.796+0000 7f0a85007740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d768a1cb0 0x562d7689be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.796+0000 7f0a85007740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7c1b3a60 con 0x562d76896af0 2024-09-12T03:34:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.796+0000 7f0a85007740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7c1b1330 con 0x562d76897410 2024-09-12T03:34:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.796+0000 7f0a85007740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7c1af180 con 0x562d768a1cb0 2024-09-12T03:34:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.797+0000 7f0a7b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d76896af0 0x562d768a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.797+0000 7f0a7b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d76896af0 0x562d768a0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47682/0 (socket says 172.21.15.64:47682) 2024-09-12T03:34:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.797+0000 7f0a7b7fe640 1 -- 172.21.15.64:0/550955577 learned_addr learned my addr 172.21.15.64:0/550955577 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:34:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.797+0000 7f0a7bfff640 1 --2- 172.21.15.64:0/550955577 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562d76897410 0x562d7689f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.797+0000 7f0a7b7fe640 1 -- 172.21.15.64:0/550955577 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d768a1cb0 msgr2=0x562d7689be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.797+0000 7f0a7b7fe640 1 --2- 172.21.15.64:0/550955577 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d768a1cb0 0x562d7689be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.797+0000 7f0a7b7fe640 1 -- 172.21.15.64:0/550955577 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562d76897410 msgr2=0x562d7689f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.797+0000 7f0a7b7fe640 1 --2- 172.21.15.64:0/550955577 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562d76897410 0x562d7689f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.797+0000 7f0a7b7fe640 1 -- 172.21.15.64:0/550955577 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d768a48d0 con 0x562d76896af0 2024-09-12T03:34:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.797+0000 7f0a7b7fe640 1 --2- 172.21.15.64:0/550955577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d76896af0 0x562d768a0ee0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f0a70006010 tx=0x7f0a700035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbbc783cd6055a1b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.797+0000 7f0a61ffb640 1 -- 172.21.15.64:0/550955577 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a70035040 con 0x562d76896af0 2024-09-12T03:34:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.797+0000 7f0a61ffb640 1 -- 172.21.15.64:0/550955577 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a70045040 con 0x562d76896af0 2024-09-12T03:34:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.797+0000 7f0a61ffb640 1 -- 172.21.15.64:0/550955577 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a70031a80 con 0x562d76896af0 2024-09-12T03:34:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.798+0000 7f0a85007740 1 -- 172.21.15.64:0/550955577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d76896af0 msgr2=0x562d768a0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.798+0000 7f0a85007740 1 --2- 172.21.15.64:0/550955577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d76896af0 0x562d768a0ee0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f0a70006010 tx=0x7f0a700035f0 comp rx=0 tx=0).stop 2024-09-12T03:34:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.798+0000 7f0a85007740 1 -- 172.21.15.64:0/550955577 shutdown_connections 2024-09-12T03:34:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.798+0000 7f0a85007740 1 --2- 172.21.15.64:0/550955577 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d768a1cb0 0x562d7689be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.798+0000 7f0a85007740 1 --2- 172.21.15.64:0/550955577 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562d76897410 0x562d7689f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.798+0000 7f0a85007740 1 --2- 172.21.15.64:0/550955577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d76896af0 0x562d768a0ee0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.798+0000 7f0a85007740 1 -- 172.21.15.64:0/550955577 >> 172.21.15.64:0/550955577 conn(0x562d76893060 msgr2=0x562d76898d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.798+0000 7f0a85007740 1 -- 172.21.15.64:0/550955577 shutdown_connections 2024-09-12T03:34:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.798+0000 7f0a85007740 1 -- 172.21.15.64:0/550955577 wait complete. 2024-09-12T03:34:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.798+0000 7f0a85007740 1 -- start start 2024-09-12T03:34:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.798+0000 7f0a85007740 1 Processor -- start 2024-09-12T03:34:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.799+0000 7f0a85007740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d76896af0 0x562d768b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.799+0000 7f0a85007740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d76897410 0x562d768b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.799+0000 7f0a85007740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562d768a1cb0 0x562d7691b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.799+0000 7f0a85007740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7c1af7f0 con 0x562d76897410 2024-09-12T03:34:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.799+0000 7f0a85007740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7c1b0c60 con 0x562d768a1cb0 2024-09-12T03:34:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.799+0000 7f0a85007740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7c1afec0 con 0x562d76896af0 2024-09-12T03:34:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.799+0000 7f0a85007740 10 client.0 _ll_register_callbacks cb 0x7f0a6c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:34:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.799+0000 7f0a7bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d76897410 0x562d768b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.799+0000 7f0a7bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d76897410 0x562d768b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47698/0 (socket says 172.21.15.64:47698) 2024-09-12T03:34:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.799+0000 7f0a7bfff640 1 -- 172.21.15.64:0/348644930 learned_addr learned my addr 172.21.15.64:0/348644930 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:34:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.799+0000 7f0a7b7fe640 1 --2- 172.21.15.64:0/348644930 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d76896af0 0x562d768b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.799+0000 7f0a7b7fe640 1 --2- 172.21.15.64:0/348644930 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562d768a1cb0 0x562d7691b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.799+0000 7f0a7bfff640 1 -- 172.21.15.64:0/348644930 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d76896af0 msgr2=0x562d768b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.799+0000 7f0a7bfff640 1 --2- 172.21.15.64:0/348644930 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d76896af0 0x562d768b4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.799+0000 7f0a7bfff640 1 -- 172.21.15.64:0/348644930 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562d768a1cb0 msgr2=0x562d7691b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.799+0000 7f0a7bfff640 1 --2- 172.21.15.64:0/348644930 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562d768a1cb0 0x562d7691b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.799+0000 7f0a7bfff640 1 -- 172.21.15.64:0/348644930 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d76899480 con 0x562d76897410 2024-09-12T03:34:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.799+0000 7f0a7bfff640 1 --2- 172.21.15.64:0/348644930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d76897410 0x562d768b5520 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x562d768a01f0 tx=0x7f0a6c010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8091e1d3d0e44ef server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.799+0000 7f0a5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:34:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.799+0000 7f0a5b7fe640 1 -- 172.21.15.64:0/348644930 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c004460 con 0x562d76897410 2024-09-12T03:34:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.799+0000 7f0a5b7fe640 1 -- 172.21.15.64:0/348644930 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f0a6c0035f0 con 0x562d76897410 2024-09-12T03:34:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.799+0000 7f0a85007740 1 -- 172.21.15.64:0/348644930 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a70045040 con 0x562d76897410 2024-09-12T03:34:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.800+0000 7f0a5b7fe640 1 -- 172.21.15.64:0/348644930 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0a6c0037f0 con 0x562d76897410 2024-09-12T03:34:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.800+0000 7f0a85007740 10 client.15798 fetch_fsmap learned FSMap version 25 2024-09-12T03:34:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.800+0000 7f0a85007740 1 -- 172.21.15.64:0/348644930 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562d7691c940 con 0x562d76897410 2024-09-12T03:34:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.800+0000 7f0a5b7fe640 1 -- 172.21.15.64:0/348644930 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a70045040 con 0x562d76897410 2024-09-12T03:34:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.800+0000 7f0a5b7fe640 1 -- 172.21.15.64:0/348644930 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f0a6c027b20 con 0x562d76897410 2024-09-12T03:34:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.800+0000 7f0a85007740 10 client.15798 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:34:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.800+0000 7f0a85007740 10 client.15798 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:34:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.800+0000 7f0a85007740 20 client.15798 populate_metadata read hostname 'smithi064' 2024-09-12T03:34:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.800+0000 7f0a85007740 1 --2- 172.21.15.64:0/348644930 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x562d7691d0f0 0x562d7691f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.800+0000 7f0a85007740 4 client.15798 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:34:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.800+0000 7f0a85007740 1 -- 172.21.15.64:0/348644930 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562d76802d70 con 0x562d7691d0f0 2024-09-12T03:34:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.800+0000 7f0a7b7fe640 1 --2- 172.21.15.64:0/348644930 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x562d7691d0f0 0x562d7691f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.801+0000 7f0a7b7fe640 1 --2- 172.21.15.64:0/348644930 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x562d7691d0f0 0x562d7691f510 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfce4ae8c1fa5869 server_cookie=fb8208663de6b540 in_seq=0 out_seq=0 2024-09-12T03:34:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.801+0000 7f0a5b7fe640 10 client.15798 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:34:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.814+0000 7f0a5b7fe640 1 -- 172.21.15.64:0/348644930 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562d76802d70 con 0x562d7691d0f0 2024-09-12T03:34:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.814+0000 7f0a5b7fe640 10 client.15798 handle_command_reply: tid=0 2024-09-12T03:34:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.814+0000 7f0a85007740 1 client.15798 shutdown 2024-09-12T03:34:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.814+0000 7f0a85007740 1 -- 172.21.15.64:0/348644930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d76897410 msgr2=0x562d768b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.814+0000 7f0a85007740 1 --2- 172.21.15.64:0/348644930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d76897410 0x562d768b5520 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x562d768a01f0 tx=0x7f0a6c010800 comp rx=0 tx=0).stop 2024-09-12T03:34:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.815+0000 7f0a85007740 1 -- 172.21.15.64:0/348644930 shutdown_connections 2024-09-12T03:34:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.815+0000 7f0a85007740 1 --2- 172.21.15.64:0/348644930 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x562d7691d0f0 0x562d7691f510 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.815+0000 7f0a85007740 1 --2- 172.21.15.64:0/348644930 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562d768a1cb0 0x562d7691b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.815+0000 7f0a85007740 1 --2- 172.21.15.64:0/348644930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d76897410 0x562d768b5520 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.815+0000 7f0a85007740 1 --2- 172.21.15.64:0/348644930 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d76896af0 0x562d768b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.815+0000 7f0a5b7fe640 0 client.15798 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:34:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.815+0000 7f0a85007740 1 -- 172.21.15.64:0/348644930 >> 172.21.15.64:0/348644930 conn(0x562d76893060 msgr2=0x562d768accc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.815+0000 7f0a85007740 1 -- 172.21.15.64:0/348644930 shutdown_connections 2024-09-12T03:34:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.815+0000 7f0a85007740 1 -- 172.21.15.64:0/348644930 wait complete. 2024-09-12T03:34:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.815+0000 7f0a85007740 20 client.15798 trim_cache size 0 max 16384 2024-09-12T03:34:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:34:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:34:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:34:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:34:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.815+0000 7f0a82265640 1 -- 172.21.15.64:0/2506330985 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f0a640a2da0 msgr2=0x7f0a640a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.815+0000 7f0a82265640 1 --2- 172.21.15.64:0/2506330985 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f0a640a2da0 0x7f0a640a5260 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0a7409bf40 tx=0x7f0a7406df70 comp rx=0 tx=0).stop 2024-09-12T03:34:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.815+0000 7f0a82265640 1 -- 172.21.15.64:0/2506330985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a7c1069c0 msgr2=0x7f0a7c0fd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.815+0000 7f0a82265640 1 --2- 172.21.15.64:0/2506330985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a7c1069c0 0x7f0a7c0fd860 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f0a7400eb10 tx=0x7f0a7400c2d0 comp rx=0 tx=0).stop 2024-09-12T03:34:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.815+0000 7f0a82265640 1 -- 172.21.15.64:0/2506330985 shutdown_connections 2024-09-12T03:34:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.815+0000 7f0a82265640 1 --2- 172.21.15.64:0/2506330985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a7c1069c0 0x7f0a7c0fd860 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.815+0000 7f0a82265640 1 --2- 172.21.15.64:0/2506330985 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f0a640a2da0 0x7f0a640a5260 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.815+0000 7f0a82265640 1 --2- 172.21.15.64:0/2506330985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a7c101350 0x7f0a7c0fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.815+0000 7f0a82265640 1 --2- 172.21.15.64:0/2506330985 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a7c1009b0 0x7f0a7c0fcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.815+0000 7f0a82265640 1 -- 172.21.15.64:0/2506330985 >> 172.21.15.64:0/2506330985 conn(0x7f0a7c077990 msgr2=0x7f0a7c069bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.815+0000 7f0a82265640 1 -- 172.21.15.64:0/2506330985 shutdown_connections 2024-09-12T03:34:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:46.816+0000 7f0a82265640 1 -- 172.21.15.64:0/2506330985 wait complete. 2024-09-12T03:34:47.521 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:34:47 smithi064 ceph-mon[30952]: pgmap v168: 129 pgs: 129 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-09-12T03:34:47.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:34:47 smithi115 ceph-mon[34801]: pgmap v168: 129 pgs: 129 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-09-12T03:34:47.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:34:47 smithi195 ceph-mon[34786]: pgmap v168: 129 pgs: 129 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-09-12T03:34:49.508 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:34:49 smithi195 ceph-mon[34786]: pgmap v169: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-09-12T03:34:49.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:34:49 smithi064 ceph-mon[30952]: pgmap v169: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-09-12T03:34:49.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:34:49 smithi115 ceph-mon[34801]: pgmap v169: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-09-12T03:34:51.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:34:51 smithi064 ceph-mon[30952]: pgmap v170: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-09-12T03:34:51.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:34:51 smithi115 ceph-mon[34801]: pgmap v170: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-09-12T03:34:51.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:34:51 smithi195 ceph-mon[34786]: pgmap v170: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-09-12T03:34:53.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:34:53 smithi064 ceph-mon[30952]: pgmap v171: 129 pgs: 129 active+clean; 4.0 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-09-12T03:34:53.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:34:53 smithi115 ceph-mon[34801]: pgmap v171: 129 pgs: 129 active+clean; 4.0 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-09-12T03:34:53.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:34:53 smithi195 ceph-mon[34786]: pgmap v171: 129 pgs: 129 active+clean; 4.0 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-09-12T03:34:55.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:34:55 smithi064 ceph-mon[30952]: pgmap v172: 129 pgs: 129 active+clean; 4.0 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-12T03:34:55.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:34:55 smithi115 ceph-mon[34801]: pgmap v172: 129 pgs: 129 active+clean; 4.0 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-12T03:34:55.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:34:55 smithi195 ceph-mon[34786]: pgmap v172: 129 pgs: 129 active+clean; 4.0 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-12T03:34:57.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:34:57 smithi064 ceph-mon[30952]: pgmap v173: 129 pgs: 129 active+clean; 4.0 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-12T03:34:57.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:34:57 smithi115 ceph-mon[34801]: pgmap v173: 129 pgs: 129 active+clean; 4.0 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-12T03:34:57.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:34:57 smithi195 ceph-mon[34786]: pgmap v173: 129 pgs: 129 active+clean; 4.0 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-12T03:34:59.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:34:59 smithi064 ceph-mon[30952]: pgmap v174: 129 pgs: 129 active+clean; 4.2 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-09-12T03:34:59.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:34:59 smithi115 ceph-mon[34801]: pgmap v174: 129 pgs: 129 active+clean; 4.2 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-09-12T03:34:59.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:34:59 smithi195 ceph-mon[34786]: pgmap v174: 129 pgs: 129 active+clean; 4.2 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-09-12T03:34:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.916+0000 7ffa03cd7640 1 -- 172.21.15.64:0/1310255160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9fc0745a0 msgr2=0x7ff9fc074980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.916+0000 7ffa03cd7640 1 --2- 172.21.15.64:0/1310255160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9fc0745a0 0x7ff9fc074980 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8009900 tx=0x7ff9f802a480 comp rx=0 tx=0).stop 2024-09-12T03:34:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.916+0000 7ffa03cd7640 1 -- 172.21.15.64:0/1310255160 shutdown_connections 2024-09-12T03:34:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.916+0000 7ffa03cd7640 1 --2- 172.21.15.64:0/1310255160 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9fc10e390 0x7ff9fc110780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.916+0000 7ffa03cd7640 1 --2- 172.21.15.64:0/1310255160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9fc074ec0 0x7ff9fc10de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.916+0000 7ffa03cd7640 1 --2- 172.21.15.64:0/1310255160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9fc0745a0 0x7ff9fc074980 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.916+0000 7ffa03cd7640 1 -- 172.21.15.64:0/1310255160 >> 172.21.15.64:0/1310255160 conn(0x7ff9fc0fd7d0 msgr2=0x7ff9fc0ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.916+0000 7ffa03cd7640 1 -- 172.21.15.64:0/1310255160 shutdown_connections 2024-09-12T03:34:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.916+0000 7ffa03cd7640 1 -- 172.21.15.64:0/1310255160 wait complete. 2024-09-12T03:34:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.917+0000 7ffa03cd7640 1 Processor -- start 2024-09-12T03:34:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.917+0000 7ffa03cd7640 1 -- start start 2024-09-12T03:34:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.917+0000 7ffa03cd7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9fc0745a0 0x7ff9fc19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.917+0000 7ffa03cd7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9fc074ec0 0x7ff9fc19a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.917+0000 7ffa03cd7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9fc10e390 0x7ff9fc19ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.917+0000 7ffa03cd7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc113fb0 con 0x7ff9fc0745a0 2024-09-12T03:34:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.917+0000 7ffa03cd7640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc113e30 con 0x7ff9fc10e390 2024-09-12T03:34:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.917+0000 7ffa03cd7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc114130 con 0x7ff9fc074ec0 2024-09-12T03:34:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.917+0000 7ffa01a73640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9fc0745a0 0x7ff9fc19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.917+0000 7ffa01a73640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9fc0745a0 0x7ff9fc19a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45242/0 (socket says 172.21.15.64:45242) 2024-09-12T03:34:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.917+0000 7ffa01a73640 1 -- 172.21.15.64:0/38324570 learned_addr learned my addr 172.21.15.64:0/38324570 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:34:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.917+0000 7ffa01272640 1 --2- 172.21.15.64:0/38324570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9fc074ec0 0x7ff9fc19a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.917+0000 7ffa02274640 1 --2- 172.21.15.64:0/38324570 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9fc10e390 0x7ff9fc19ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.917+0000 7ffa01a73640 1 -- 172.21.15.64:0/38324570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9fc074ec0 msgr2=0x7ff9fc19a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.917+0000 7ffa01a73640 1 --2- 172.21.15.64:0/38324570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9fc074ec0 0x7ff9fc19a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.917+0000 7ffa01a73640 1 -- 172.21.15.64:0/38324570 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9fc10e390 msgr2=0x7ff9fc19ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.917+0000 7ffa01a73640 1 --2- 172.21.15.64:0/38324570 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9fc10e390 0x7ff9fc19ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.917+0000 7ffa01a73640 1 -- 172.21.15.64:0/38324570 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9fc19f3f0 con 0x7ff9fc0745a0 2024-09-12T03:34:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.917+0000 7ffa01272640 1 --2- 172.21.15.64:0/38324570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9fc074ec0 0x7ff9fc19a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:34:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.917+0000 7ffa02274640 1 --2- 172.21.15.64:0/38324570 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9fc10e390 0x7ff9fc19ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:34:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.918+0000 7ffa01a73640 1 --2- 172.21.15.64:0/38324570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9fc0745a0 0x7ff9fc19a400 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8005ab0 tx=0x7ff9f8003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.918+0000 7ff9f2ffd640 1 -- 172.21.15.64:0/38324570 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f803f070 con 0x7ff9fc0745a0 2024-09-12T03:34:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.918+0000 7ff9f2ffd640 1 -- 172.21.15.64:0/38324570 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9f8031d60 con 0x7ff9fc0745a0 2024-09-12T03:34:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.918+0000 7ff9f2ffd640 1 -- 172.21.15.64:0/38324570 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8035030 con 0x7ff9fc0745a0 2024-09-12T03:34:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.918+0000 7ffa03cd7640 1 -- 172.21.15.64:0/38324570 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9fc19f680 con 0x7ff9fc0745a0 2024-09-12T03:34:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.918+0000 7ffa03cd7640 1 -- 172.21.15.64:0/38324570 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9fc1a6f70 con 0x7ff9fc0745a0 2024-09-12T03:34:59.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.920+0000 7ff9f2ffd640 1 -- 172.21.15.64:0/38324570 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ff9f8042070 con 0x7ff9fc0745a0 2024-09-12T03:34:59.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.921+0000 7ffa06a70740 1 Processor -- start 2024-09-12T03:34:59.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.921+0000 7ffa06a70740 1 -- start start 2024-09-12T03:34:59.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.921+0000 7ffa06a70740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4de07caf0 0x55e4de07ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:59.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.921+0000 7ffa06a70740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e4de07d410 0x55e4de087770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:59.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.921+0000 7ffa06a70740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4de087cb0 0x55e4de08a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:59.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.921+0000 7ffa06a70740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e4ddfe12f0 con 0x55e4de087cb0 2024-09-12T03:34:59.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.921+0000 7ffa06a70740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e4ddf89270 con 0x55e4de07caf0 2024-09-12T03:34:59.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.921+0000 7ffa06a70740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e4ddfdd2b0 con 0x55e4de07d410 2024-09-12T03:34:59.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.922+0000 7ffa01272640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4de087cb0 0x55e4de08a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:59.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.922+0000 7ff9f2ffd640 1 --2- 172.21.15.64:0/38324570 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7ff9e80a2da0 0x7ff9e80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:59.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.922+0000 7ffa02274640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4de07caf0 0x55e4de07ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:59.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.922+0000 7ffa01272640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4de087cb0 0x55e4de08a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45250/0 (socket says 172.21.15.64:45250) 2024-09-12T03:34:59.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.922+0000 7ffa01272640 1 -- 172.21.15.64:0/2074452846 learned_addr learned my addr 172.21.15.64:0/2074452846 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:34:59.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.922+0000 7ffa01a73640 1 --2- 172.21.15.64:0/2074452846 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e4de07d410 0x55e4de087770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:59.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.922+0000 7ffa02274640 1 --2- 172.21.15.64:0/38324570 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7ff9e80a2da0 0x7ff9e80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:59.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.922+0000 7ff9f2ffd640 1 -- 172.21.15.64:0/38324570 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7ff9f80efdd0 con 0x7ff9fc0745a0 2024-09-12T03:34:59.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.922+0000 7ffa02274640 1 -- 172.21.15.64:0/2074452846 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e4de07d410 msgr2=0x55e4de087770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:59.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.922+0000 7ffa02274640 1 --2- 172.21.15.64:0/2074452846 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e4de07d410 0x55e4de087770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:59.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.922+0000 7ffa02274640 1 -- 172.21.15.64:0/2074452846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4de087cb0 msgr2=0x55e4de08a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:59.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.922+0000 7ffa02274640 1 --2- 172.21.15.64:0/2074452846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4de087cb0 0x55e4de08a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:59.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.922+0000 7ffa02274640 1 -- 172.21.15.64:0/2074452846 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4ddf88340 con 0x55e4de07caf0 2024-09-12T03:34:59.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.922+0000 7ffa01272640 1 --2- 172.21.15.64:0/2074452846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4de087cb0 0x55e4de08a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:34:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.922+0000 7ffa01a73640 1 --2- 172.21.15.64:0/2074452846 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e4de07d410 0x55e4de087770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:34:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.922+0000 7ffa02274640 1 --2- 172.21.15.64:0/2074452846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4de07caf0 0x55e4de07ced0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4012860 tx=0x7ff9f4012d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=807565a234f39623 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.923+0000 7ffa02274640 1 --2- 172.21.15.64:0/38324570 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7ff9e80a2da0 0x7ff9e80a5260 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff9f400d000 tx=0x7ff9f401c000 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.923+0000 7ff9e7fff640 1 -- 172.21.15.64:0/2074452846 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f4004d20 con 0x55e4de07caf0 2024-09-12T03:34:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.923+0000 7ff9e7fff640 1 -- 172.21.15.64:0/2074452846 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9f4020030 con 0x55e4de07caf0 2024-09-12T03:34:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.923+0000 7ff9e7fff640 1 -- 172.21.15.64:0/2074452846 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f401c6a0 con 0x55e4de07caf0 2024-09-12T03:34:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.923+0000 7ffa06a70740 1 -- 172.21.15.64:0/2074452846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4de07caf0 msgr2=0x55e4de07ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.923+0000 7ffa06a70740 1 --2- 172.21.15.64:0/2074452846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4de07caf0 0x55e4de07ced0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4012860 tx=0x7ff9f4012d20 comp rx=0 tx=0).stop 2024-09-12T03:34:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.923+0000 7ffa06a70740 1 -- 172.21.15.64:0/2074452846 shutdown_connections 2024-09-12T03:34:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.923+0000 7ffa06a70740 1 --2- 172.21.15.64:0/2074452846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4de087cb0 0x55e4de08a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.923+0000 7ffa06a70740 1 --2- 172.21.15.64:0/2074452846 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e4de07d410 0x55e4de087770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.923+0000 7ffa06a70740 1 --2- 172.21.15.64:0/2074452846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4de07caf0 0x55e4de07ced0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.923+0000 7ffa06a70740 1 -- 172.21.15.64:0/2074452846 >> 172.21.15.64:0/2074452846 conn(0x55e4de079060 msgr2=0x55e4de079440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:59.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.924+0000 7ffa06a70740 1 -- 172.21.15.64:0/2074452846 shutdown_connections 2024-09-12T03:34:59.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.924+0000 7ffa06a70740 1 -- 172.21.15.64:0/2074452846 wait complete. 2024-09-12T03:34:59.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.924+0000 7ffa06a70740 1 -- start start 2024-09-12T03:34:59.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.924+0000 7ffa06a70740 1 Processor -- start 2024-09-12T03:34:59.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.924+0000 7ffa06a70740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e4de07caf0 0x55e4de084c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:59.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.924+0000 7ffa06a70740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4de07d410 0x55e4de085170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:59.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.924+0000 7ffa06a70740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4de087cb0 0x55e4de0856b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:59.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.924+0000 7ffa06a70740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e4de08a880 con 0x55e4de087cb0 2024-09-12T03:34:59.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.924+0000 7ffa06a70740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e4ddf6a290 con 0x55e4de07d410 2024-09-12T03:34:59.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.924+0000 7ffa06a70740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e4de08aa00 con 0x55e4de07caf0 2024-09-12T03:34:59.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.925+0000 7ffa06a70740 10 client.0 _ll_register_callbacks cb 0x55e4ddf9b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:34:59.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.925+0000 7ffa02274640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4de087cb0 0x55e4de0856b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:59.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.925+0000 7ffa02274640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4de087cb0 0x55e4de0856b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45266/0 (socket says 172.21.15.64:45266) 2024-09-12T03:34:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.925+0000 7ffa02274640 1 -- 172.21.15.64:0/3273152540 learned_addr learned my addr 172.21.15.64:0/3273152540 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:34:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.925+0000 7ffa01272640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4de07d410 0x55e4de085170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.925+0000 7ffa01a73640 1 --2- 172.21.15.64:0/3273152540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e4de07caf0 0x55e4de084c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.925+0000 7ffa02274640 1 -- 172.21.15.64:0/3273152540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e4de07caf0 msgr2=0x55e4de084c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.925+0000 7ffa02274640 1 --2- 172.21.15.64:0/3273152540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e4de07caf0 0x55e4de084c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.925+0000 7ffa02274640 1 -- 172.21.15.64:0/3273152540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4de07d410 msgr2=0x55e4de085170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.925+0000 7ffa02274640 1 --2- 172.21.15.64:0/3273152540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4de07d410 0x55e4de085170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.925+0000 7ffa02274640 1 -- 172.21.15.64:0/3273152540 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4ddcf4210 con 0x55e4de087cb0 2024-09-12T03:34:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.925+0000 7ffa01272640 1 --2- 172.21.15.64:0/3273152540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4de07d410 0x55e4de085170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:34:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.925+0000 7ffa02274640 1 --2- 172.21.15.64:0/3273152540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4de087cb0 0x55e4de0856b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4011fd0 tx=0x7ff9f4012540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c47e796984cac51 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.925+0000 7ff9e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:34:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.925+0000 7ff9e57fa640 1 -- 172.21.15.64:0/3273152540 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f4032060 con 0x55e4de087cb0 2024-09-12T03:34:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.925+0000 7ff9e57fa640 1 -- 172.21.15.64:0/3273152540 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7ff9f4004a00 con 0x55e4de087cb0 2024-09-12T03:34:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.926+0000 7ffa06a70740 1 -- 172.21.15.64:0/3273152540 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9f4020030 con 0x55e4de087cb0 2024-09-12T03:34:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.926+0000 7ff9e57fa640 1 -- 172.21.15.64:0/3273152540 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff9f4009700 con 0x55e4de087cb0 2024-09-12T03:34:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.926+0000 7ffa06a70740 10 client.15810 fetch_fsmap learned FSMap version 25 2024-09-12T03:34:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.926+0000 7ffa06a70740 1 -- 172.21.15.64:0/3273152540 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e4dddc0620 con 0x55e4de087cb0 2024-09-12T03:34:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.926+0000 7ff9e57fa640 1 -- 172.21.15.64:0/3273152540 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9f4020030 con 0x55e4de087cb0 2024-09-12T03:34:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.926+0000 7ff9e57fa640 1 -- 172.21.15.64:0/3273152540 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff9f401c8c0 con 0x55e4de087cb0 2024-09-12T03:34:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.926+0000 7ffa06a70740 10 client.15810 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:34:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.926+0000 7ffa06a70740 10 client.15810 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:34:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.926+0000 7ffa06a70740 20 client.15810 populate_metadata read hostname 'smithi064' 2024-09-12T03:34:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.926+0000 7ffa06a70740 1 --2- 172.21.15.64:0/3273152540 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55e4de097b70 0x55e4de099f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.926+0000 7ffa06a70740 4 client.15810 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:34:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.926+0000 7ffa06a70740 1 -- 172.21.15.64:0/3273152540 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4ddfeea20 con 0x55e4de097b70 2024-09-12T03:34:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.927+0000 7ffa01a73640 1 --2- 172.21.15.64:0/3273152540 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55e4de097b70 0x55e4de099f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.927+0000 7ffa01a73640 1 --2- 172.21.15.64:0/3273152540 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55e4de097b70 0x55e4de099f60 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2cd07919a266927 server_cookie=d22021612e61b110 in_seq=0 out_seq=0 2024-09-12T03:34:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.927+0000 7ff9e57fa640 10 client.15810 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:34:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.928+0000 7ff9e57fa640 1 -- 172.21.15.64:0/3273152540 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e4ddfeea20 con 0x55e4de097b70 2024-09-12T03:34:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.928+0000 7ff9e57fa640 10 client.15810 handle_command_reply: tid=0 2024-09-12T03:34:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.928+0000 7ffa06a70740 1 client.15810 shutdown 2024-09-12T03:34:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.928+0000 7ffa06a70740 1 -- 172.21.15.64:0/3273152540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4de087cb0 msgr2=0x55e4de0856b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.928+0000 7ffa06a70740 1 --2- 172.21.15.64:0/3273152540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4de087cb0 0x55e4de0856b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4011fd0 tx=0x7ff9f4012540 comp rx=0 tx=0).stop 2024-09-12T03:34:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.928+0000 7ffa06a70740 1 -- 172.21.15.64:0/3273152540 shutdown_connections 2024-09-12T03:34:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.928+0000 7ffa06a70740 1 --2- 172.21.15.64:0/3273152540 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55e4de097b70 0x55e4de099f60 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.928+0000 7ffa06a70740 1 --2- 172.21.15.64:0/3273152540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4de087cb0 0x55e4de0856b0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.928+0000 7ffa06a70740 1 --2- 172.21.15.64:0/3273152540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4de07d410 0x55e4de085170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.928+0000 7ff9e57fa640 0 client.15810 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:34:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.928+0000 7ffa06a70740 1 --2- 172.21.15.64:0/3273152540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e4de07caf0 0x55e4de084c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.928+0000 7ffa06a70740 1 -- 172.21.15.64:0/3273152540 >> 172.21.15.64:0/3273152540 conn(0x55e4de079060 msgr2=0x55e4de07edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.928+0000 7ffa06a70740 1 -- 172.21.15.64:0/3273152540 shutdown_connections 2024-09-12T03:34:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.928+0000 7ffa06a70740 1 -- 172.21.15.64:0/3273152540 wait complete. 2024-09-12T03:34:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.928+0000 7ffa06a70740 20 client.15810 trim_cache size 0 max 16384 2024-09-12T03:34:59.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.943+0000 7ffa06a70740 1 Processor -- start 2024-09-12T03:34:59.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.943+0000 7ffa06a70740 1 -- start start 2024-09-12T03:34:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.943+0000 7ffa06a70740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e4de07caf0 0x55e4de0854e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.943+0000 7ffa06a70740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4de07d410 0x55e4de086ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.943+0000 7ffa06a70740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4de087cb0 0x55e4de09f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.943+0000 7ffa06a70740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc1b3a10 con 0x55e4de07d410 2024-09-12T03:34:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.943+0000 7ffa06a70740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc1b12e0 con 0x55e4de087cb0 2024-09-12T03:34:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.943+0000 7ffa06a70740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc1af130 con 0x55e4de07caf0 2024-09-12T03:34:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.943+0000 7ffa01272640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4de07d410 0x55e4de086ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.943+0000 7ffa01272640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4de07d410 0x55e4de086ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45268/0 (socket says 172.21.15.64:45268) 2024-09-12T03:34:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.943+0000 7ffa01272640 1 -- 172.21.15.64:0/2247469319 learned_addr learned my addr 172.21.15.64:0/2247469319 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:34:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.943+0000 7ffa01a73640 1 --2- 172.21.15.64:0/2247469319 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e4de07caf0 0x55e4de0854e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.943+0000 7ffa01272640 1 -- 172.21.15.64:0/2247469319 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e4de07caf0 msgr2=0x55e4de0854e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.943+0000 7ffa01272640 1 --2- 172.21.15.64:0/2247469319 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e4de07caf0 0x55e4de0854e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.943+0000 7ffa02274640 1 --2- 172.21.15.64:0/2247469319 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4de087cb0 0x55e4de09f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.943+0000 7ffa01272640 1 -- 172.21.15.64:0/2247469319 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4de087cb0 msgr2=0x55e4de09f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.943+0000 7ffa01272640 1 --2- 172.21.15.64:0/2247469319 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4de087cb0 0x55e4de09f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.944+0000 7ffa01272640 1 -- 172.21.15.64:0/2247469319 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4de07e000 con 0x55e4de07d410 2024-09-12T03:34:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.944+0000 7ffa01272640 1 --2- 172.21.15.64:0/2247469319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4de07d410 0x55e4de086ba0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x55e4de07f690 tx=0x7ff9ec013940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=582e39132492db30 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.944+0000 7ff9e77fe640 1 -- 172.21.15.64:0/2247469319 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec021070 con 0x55e4de07d410 2024-09-12T03:34:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.944+0000 7ff9e77fe640 1 -- 172.21.15.64:0/2247469319 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9ec011950 con 0x55e4de07d410 2024-09-12T03:34:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.944+0000 7ff9e77fe640 1 -- 172.21.15.64:0/2247469319 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec01cb60 con 0x55e4de07d410 2024-09-12T03:34:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.944+0000 7ffa06a70740 1 -- 172.21.15.64:0/2247469319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4de07d410 msgr2=0x55e4de086ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.944+0000 7ffa06a70740 1 --2- 172.21.15.64:0/2247469319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4de07d410 0x55e4de086ba0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x55e4de07f690 tx=0x7ff9ec013940 comp rx=0 tx=0).stop 2024-09-12T03:34:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.945+0000 7ffa06a70740 1 -- 172.21.15.64:0/2247469319 shutdown_connections 2024-09-12T03:34:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.945+0000 7ffa06a70740 1 --2- 172.21.15.64:0/2247469319 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4de087cb0 0x55e4de09f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.945+0000 7ffa06a70740 1 --2- 172.21.15.64:0/2247469319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4de07d410 0x55e4de086ba0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.945+0000 7ffa06a70740 1 --2- 172.21.15.64:0/2247469319 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e4de07caf0 0x55e4de0854e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.945+0000 7ffa06a70740 1 -- 172.21.15.64:0/2247469319 >> 172.21.15.64:0/2247469319 conn(0x55e4de079060 msgr2=0x55e4de086490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.945+0000 7ffa06a70740 1 -- 172.21.15.64:0/2247469319 shutdown_connections 2024-09-12T03:34:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.945+0000 7ffa06a70740 1 -- 172.21.15.64:0/2247469319 wait complete. 2024-09-12T03:34:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.945+0000 7ffa06a70740 1 -- start start 2024-09-12T03:34:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.945+0000 7ffa06a70740 1 Processor -- start 2024-09-12T03:34:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.945+0000 7ffa06a70740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4de07caf0 0x55e4de08fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.945+0000 7ffa06a70740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e4de07d410 0x55e4de090200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.945+0000 7ffa06a70740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4de087cb0 0x55e4de081270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.945+0000 7ffa06a70740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc1af7a0 con 0x55e4de07caf0 2024-09-12T03:34:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.945+0000 7ffa06a70740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc1b0c10 con 0x55e4de087cb0 2024-09-12T03:34:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.945+0000 7ffa06a70740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc1afe70 con 0x55e4de07d410 2024-09-12T03:34:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.946+0000 7ffa06a70740 10 client.0 _ll_register_callbacks cb 0x7ff9ec0095b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:34:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.946+0000 7ffa01a73640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4de07caf0 0x55e4de08fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.946+0000 7ffa01a73640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4de07caf0 0x55e4de08fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45270/0 (socket says 172.21.15.64:45270) 2024-09-12T03:34:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.946+0000 7ffa01a73640 1 -- 172.21.15.64:0/3376632600 learned_addr learned my addr 172.21.15.64:0/3376632600 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:34:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.946+0000 7ffa01272640 1 --2- 172.21.15.64:0/3376632600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e4de07d410 0x55e4de090200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.946+0000 7ffa02274640 1 --2- 172.21.15.64:0/3376632600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4de087cb0 0x55e4de081270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.946+0000 7ffa01a73640 1 -- 172.21.15.64:0/3376632600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e4de07d410 msgr2=0x55e4de090200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.946+0000 7ffa01a73640 1 --2- 172.21.15.64:0/3376632600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e4de07d410 0x55e4de090200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.946+0000 7ffa01a73640 1 -- 172.21.15.64:0/3376632600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4de087cb0 msgr2=0x55e4de081270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:34:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.946+0000 7ffa01a73640 1 --2- 172.21.15.64:0/3376632600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4de087cb0 0x55e4de081270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.946+0000 7ffa01a73640 1 -- 172.21.15.64:0/3376632600 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4de0817b0 con 0x55e4de07caf0 2024-09-12T03:34:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.946+0000 7ffa01a73640 1 --2- 172.21.15.64:0/3376632600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4de07caf0 0x55e4de08fcc0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8005ec0 tx=0x7ff9f804af00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b74aa8a82fe5c369 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.946+0000 7ff9e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:34:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.946+0000 7ff9e57fa640 1 -- 172.21.15.64:0/3376632600 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8049040 con 0x55e4de07caf0 2024-09-12T03:34:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.946+0000 7ff9e57fa640 1 -- 172.21.15.64:0/3376632600 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7ff9f8037d70 con 0x55e4de07caf0 2024-09-12T03:34:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.946+0000 7ffa06a70740 1 -- 172.21.15.64:0/3376632600 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9ec011950 con 0x55e4de07caf0 2024-09-12T03:34:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.946+0000 7ff9e57fa640 1 -- 172.21.15.64:0/3376632600 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff9f80bbec0 con 0x55e4de07caf0 2024-09-12T03:34:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.946+0000 7ffa06a70740 10 client.15816 fetch_fsmap learned FSMap version 25 2024-09-12T03:34:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.946+0000 7ffa06a70740 1 -- 172.21.15.64:0/3376632600 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e4de101dd0 con 0x55e4de07caf0 2024-09-12T03:34:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.947+0000 7ff9e57fa640 1 -- 172.21.15.64:0/3376632600 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9ec011950 con 0x55e4de07caf0 2024-09-12T03:34:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.947+0000 7ff9e57fa640 1 -- 172.21.15.64:0/3376632600 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff9f803b580 con 0x55e4de07caf0 2024-09-12T03:34:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.947+0000 7ffa06a70740 10 client.15816 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:34:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.947+0000 7ffa06a70740 10 client.15816 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:34:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.947+0000 7ffa06a70740 20 client.15816 populate_metadata read hostname 'smithi064' 2024-09-12T03:34:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.947+0000 7ffa06a70740 1 --2- 172.21.15.64:0/3376632600 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55e4de102480 0x55e4de1048a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.947+0000 7ffa06a70740 4 client.15816 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:34:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.947+0000 7ffa06a70740 1 -- 172.21.15.64:0/3376632600 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e4ddfe8d70 con 0x55e4de102480 2024-09-12T03:34:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.947+0000 7ffa01272640 1 --2- 172.21.15.64:0/3376632600 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55e4de102480 0x55e4de1048a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.948+0000 7ffa01272640 1 --2- 172.21.15.64:0/3376632600 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55e4de102480 0x55e4de1048a0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e88892fd1acf180 server_cookie=58a98f14653c3af8 in_seq=0 out_seq=0 2024-09-12T03:34:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.948+0000 7ff9e57fa640 10 client.15816 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:34:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.961+0000 7ff9e57fa640 1 -- 172.21.15.64:0/3376632600 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e4ddfe8d70 con 0x55e4de102480 2024-09-12T03:34:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.961+0000 7ff9e57fa640 10 client.15816 handle_command_reply: tid=0 2024-09-12T03:34:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.961+0000 7ffa06a70740 1 client.15816 shutdown 2024-09-12T03:34:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.962+0000 7ffa06a70740 1 -- 172.21.15.64:0/3376632600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4de07caf0 msgr2=0x55e4de08fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.962+0000 7ffa06a70740 1 --2- 172.21.15.64:0/3376632600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4de07caf0 0x55e4de08fcc0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8005ec0 tx=0x7ff9f804af00 comp rx=0 tx=0).stop 2024-09-12T03:34:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.962+0000 7ffa06a70740 1 -- 172.21.15.64:0/3376632600 shutdown_connections 2024-09-12T03:34:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.962+0000 7ffa06a70740 1 --2- 172.21.15.64:0/3376632600 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55e4de102480 0x55e4de1048a0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.962+0000 7ffa06a70740 1 --2- 172.21.15.64:0/3376632600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e4de087cb0 0x55e4de081270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.962+0000 7ffa06a70740 1 --2- 172.21.15.64:0/3376632600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e4de07d410 0x55e4de090200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.962+0000 7ffa06a70740 1 --2- 172.21.15.64:0/3376632600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4de07caf0 0x55e4de08fcc0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.962+0000 7ff9e57fa640 0 client.15816 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:34:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.962+0000 7ffa06a70740 1 -- 172.21.15.64:0/3376632600 >> 172.21.15.64:0/3376632600 conn(0x55e4de079060 msgr2=0x55e4de07edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.962+0000 7ffa06a70740 1 -- 172.21.15.64:0/3376632600 shutdown_connections 2024-09-12T03:34:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.962+0000 7ffa06a70740 1 -- 172.21.15.64:0/3376632600 wait complete. 2024-09-12T03:34:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.962+0000 7ffa06a70740 20 client.15816 trim_cache size 0 max 16384 2024-09-12T03:34:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:34:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:34:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:34:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:34:59.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.962+0000 7ffa03cd7640 1 -- 172.21.15.64:0/38324570 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7ff9e80a2da0 msgr2=0x7ff9e80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:59.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.962+0000 7ffa03cd7640 1 --2- 172.21.15.64:0/38324570 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7ff9e80a2da0 0x7ff9e80a5260 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff9f400d000 tx=0x7ff9f401c000 comp rx=0 tx=0).stop 2024-09-12T03:34:59.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.962+0000 7ffa03cd7640 1 -- 172.21.15.64:0/38324570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9fc0745a0 msgr2=0x7ff9fc19a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:59.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.962+0000 7ffa03cd7640 1 --2- 172.21.15.64:0/38324570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9fc0745a0 0x7ff9fc19a400 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8005ab0 tx=0x7ff9f8003c80 comp rx=0 tx=0).stop 2024-09-12T03:34:59.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.962+0000 7ffa03cd7640 1 -- 172.21.15.64:0/38324570 shutdown_connections 2024-09-12T03:34:59.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.962+0000 7ffa03cd7640 1 --2- 172.21.15.64:0/38324570 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7ff9e80a2da0 0x7ff9e80a5260 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:59.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.962+0000 7ffa03cd7640 1 --2- 172.21.15.64:0/38324570 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9fc10e390 0x7ff9fc19ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:59.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.962+0000 7ffa03cd7640 1 --2- 172.21.15.64:0/38324570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9fc074ec0 0x7ff9fc19a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:59.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.962+0000 7ffa03cd7640 1 --2- 172.21.15.64:0/38324570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9fc0745a0 0x7ff9fc19a400 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:59.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.963+0000 7ffa03cd7640 1 -- 172.21.15.64:0/38324570 >> 172.21.15.64:0/38324570 conn(0x7ff9fc0fd7d0 msgr2=0x7ff9fc0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:59.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.963+0000 7ffa03cd7640 1 -- 172.21.15.64:0/38324570 shutdown_connections 2024-09-12T03:34:59.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:34:59.963+0000 7ffa03cd7640 1 -- 172.21.15.64:0/38324570 wait complete. 2024-09-12T03:35:01.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:35:01 smithi064 ceph-mon[30952]: pgmap v175: 129 pgs: 129 active+clean; 4.2 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-12T03:35:01.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:35:01 smithi115 ceph-mon[34801]: pgmap v175: 129 pgs: 129 active+clean; 4.2 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-12T03:35:01.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:35:01 smithi195 ceph-mon[34786]: pgmap v175: 129 pgs: 129 active+clean; 4.2 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-12T03:35:03.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:35:03 smithi064 ceph-mon[30952]: pgmap v176: 129 pgs: 129 active+clean; 4.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 77 KiB/s wr, 35 op/s 2024-09-12T03:35:03.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:35:03 smithi115 ceph-mon[34801]: pgmap v176: 129 pgs: 129 active+clean; 4.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 77 KiB/s wr, 35 op/s 2024-09-12T03:35:03.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:35:03 smithi195 ceph-mon[34786]: pgmap v176: 129 pgs: 129 active+clean; 4.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 77 KiB/s wr, 35 op/s 2024-09-12T03:35:05.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:35:05 smithi064 ceph-mon[30952]: pgmap v177: 129 pgs: 129 active+clean; 4.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 57 KiB/s wr, 27 op/s 2024-09-12T03:35:05.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:35:05 smithi115 ceph-mon[34801]: pgmap v177: 129 pgs: 129 active+clean; 4.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 57 KiB/s wr, 27 op/s 2024-09-12T03:35:05.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:35:05 smithi195 ceph-mon[34786]: pgmap v177: 129 pgs: 129 active+clean; 4.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 57 KiB/s wr, 27 op/s 2024-09-12T03:35:07.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:35:07 smithi064 ceph-mon[30952]: pgmap v178: 129 pgs: 129 active+clean; 4.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 57 KiB/s wr, 27 op/s 2024-09-12T03:35:07.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:35:07 smithi115 ceph-mon[34801]: pgmap v178: 129 pgs: 129 active+clean; 4.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 57 KiB/s wr, 27 op/s 2024-09-12T03:35:07.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:35:07 smithi195 ceph-mon[34786]: pgmap v178: 129 pgs: 129 active+clean; 4.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 57 KiB/s wr, 27 op/s 2024-09-12T03:35:09.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:35:09 smithi064 ceph-mon[30952]: pgmap v179: 129 pgs: 129 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 73 KiB/s wr, 32 op/s 2024-09-12T03:35:09.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:35:09 smithi115 ceph-mon[34801]: pgmap v179: 129 pgs: 129 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 73 KiB/s wr, 32 op/s 2024-09-12T03:35:09.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:35:09 smithi195 ceph-mon[34786]: pgmap v179: 129 pgs: 129 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 73 KiB/s wr, 32 op/s 2024-09-12T03:35:11.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:35:11 smithi064 ceph-mon[30952]: pgmap v180: 129 pgs: 129 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 59 KiB/s wr, 27 op/s 2024-09-12T03:35:11.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:35:11 smithi115 ceph-mon[34801]: pgmap v180: 129 pgs: 129 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 59 KiB/s wr, 27 op/s 2024-09-12T03:35:11.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:35:11 smithi195 ceph-mon[34786]: pgmap v180: 129 pgs: 129 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 59 KiB/s wr, 27 op/s 2024-09-12T03:35:13.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.063+0000 7f62ec445640 1 -- 172.21.15.64:0/3741543938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62e4102d80 msgr2=0x7f62e4103160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:35:13.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.063+0000 7f62ec445640 1 --2- 172.21.15.64:0/3741543938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62e4102d80 0x7f62e4103160 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f62d8009900 tx=0x7f62d802a480 comp rx=0 tx=0).stop 2024-09-12T03:35:13.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.063+0000 7f62ec445640 1 -- 172.21.15.64:0/3741543938 shutdown_connections 2024-09-12T03:35:13.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.063+0000 7f62ec445640 1 --2- 172.21.15.64:0/3741543938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62e410c150 0x7f62e410e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:13.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.063+0000 7f62ec445640 1 --2- 172.21.15.64:0/3741543938 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62e41036a0 0x7f62e410bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.063+0000 7f62ec445640 1 --2- 172.21.15.64:0/3741543938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62e4102d80 0x7f62e4103160 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.063+0000 7f62ec445640 1 -- 172.21.15.64:0/3741543938 >> 172.21.15.64:0/3741543938 conn(0x7f62e40fc6b0 msgr2=0x7f62e40fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:35:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.063+0000 7f62ec445640 1 -- 172.21.15.64:0/3741543938 shutdown_connections 2024-09-12T03:35:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.063+0000 7f62ec445640 1 -- 172.21.15.64:0/3741543938 wait complete. 2024-09-12T03:35:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.063+0000 7f62ec445640 1 Processor -- start 2024-09-12T03:35:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.063+0000 7f62ec445640 1 -- start start 2024-09-12T03:35:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.064+0000 7f62ec445640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62e4102d80 0x7f62e41009d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.064+0000 7f62ec445640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62e41036a0 0x7f62e4100f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.064+0000 7f62ec445640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62e410c150 0x7f62e4101450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.064+0000 7f62ec445640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f62e4111da0 con 0x7f62e410c150 2024-09-12T03:35:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.064+0000 7f62ec445640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f62e4111c20 con 0x7f62e4102d80 2024-09-12T03:35:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.064+0000 7f62ec445640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f62e4111f20 con 0x7f62e41036a0 2024-09-12T03:35:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.064+0000 7f62ea9e2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62e410c150 0x7f62e4101450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.064+0000 7f62ea9e2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62e410c150 0x7f62e4101450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44924/0 (socket says 172.21.15.64:44924) 2024-09-12T03:35:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.064+0000 7f62ea9e2640 1 -- 172.21.15.64:0/2667970673 learned_addr learned my addr 172.21.15.64:0/2667970673 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:35:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.064+0000 7f62e99e0640 1 --2- 172.21.15.64:0/2667970673 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62e41036a0 0x7f62e4100f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.064+0000 7f62ea1e1640 1 --2- 172.21.15.64:0/2667970673 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62e4102d80 0x7f62e41009d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.064+0000 7f62ea9e2640 1 -- 172.21.15.64:0/2667970673 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62e41036a0 msgr2=0x7f62e4100f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:35:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.064+0000 7f62ea9e2640 1 --2- 172.21.15.64:0/2667970673 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62e41036a0 0x7f62e4100f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.064+0000 7f62ea9e2640 1 -- 172.21.15.64:0/2667970673 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62e4102d80 msgr2=0x7f62e41009d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:35:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.064+0000 7f62ea9e2640 1 --2- 172.21.15.64:0/2667970673 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62e4102d80 0x7f62e41009d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.064+0000 7f62ea9e2640 1 -- 172.21.15.64:0/2667970673 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e41a6840 con 0x7f62e410c150 2024-09-12T03:35:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.064+0000 7f62ea9e2640 1 --2- 172.21.15.64:0/2667970673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62e410c150 0x7f62e4101450 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00eaa0 tx=0x7f62dc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:35:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.065+0000 7f62d77fe640 1 -- 172.21.15.64:0/2667970673 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc00cdb0 con 0x7f62e410c150 2024-09-12T03:35:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.065+0000 7f62d77fe640 1 -- 172.21.15.64:0/2667970673 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62dc005020 con 0x7f62e410c150 2024-09-12T03:35:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.065+0000 7f62ec445640 1 -- 172.21.15.64:0/2667970673 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62e41a6b30 con 0x7f62e410c150 2024-09-12T03:35:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.065+0000 7f62d77fe640 1 -- 172.21.15.64:0/2667970673 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc0116d0 con 0x7f62e410c150 2024-09-12T03:35:13.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.065+0000 7f62ec445640 1 -- 172.21.15.64:0/2667970673 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62e41a7080 con 0x7f62e410c150 2024-09-12T03:35:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.068+0000 7f62d77fe640 1 -- 172.21.15.64:0/2667970673 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f62dc0051d0 con 0x7f62e410c150 2024-09-12T03:35:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.068+0000 7f62ef1d9740 1 Processor -- start 2024-09-12T03:35:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.068+0000 7f62ef1d9740 1 -- start start 2024-09-12T03:35:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.069+0000 7f62ef1d9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db8f2d6af0 0x55db8f2d6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.069+0000 7f62ef1d9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db8f2d7410 0x55db8f2e1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.069+0000 7f62ef1d9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db8f2e1cb0 0x55db8f2e40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.069+0000 7f62ef1d9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55db8f23b2f0 con 0x55db8f2e1cb0 2024-09-12T03:35:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.069+0000 7f62ef1d9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55db8f1e3270 con 0x55db8f2d6af0 2024-09-12T03:35:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.069+0000 7f62ef1d9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55db8f2372b0 con 0x55db8f2d7410 2024-09-12T03:35:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.069+0000 7f62e99e0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db8f2e1cb0 0x55db8f2e40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.069+0000 7f62e99e0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db8f2e1cb0 0x55db8f2e40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44926/0 (socket says 172.21.15.64:44926) 2024-09-12T03:35:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.069+0000 7f62e99e0640 1 -- 172.21.15.64:0/3446553521 learned_addr learned my addr 172.21.15.64:0/3446553521 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:35:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.069+0000 7f62ea1e1640 1 --2- 172.21.15.64:0/3446553521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db8f2d7410 0x55db8f2e1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.069+0000 7f62e99e0640 1 --2- 172.21.15.64:0/3446553521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db8f2d6af0 0x55db8f2d6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.069+0000 7f62d77fe640 1 --2- 172.21.15.64:0/2667970673 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f62d00a2e20 0x7f62d00a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.069+0000 7f62e99e0640 1 -- 172.21.15.64:0/3446553521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db8f2d7410 msgr2=0x55db8f2e1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:35:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.069+0000 7f62e99e0640 1 --2- 172.21.15.64:0/3446553521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db8f2d7410 0x55db8f2e1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.069+0000 7f62e99e0640 1 -- 172.21.15.64:0/3446553521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db8f2d6af0 msgr2=0x55db8f2d6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:35:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.069+0000 7f62e99e0640 1 --2- 172.21.15.64:0/3446553521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db8f2d6af0 0x55db8f2d6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.069+0000 7f62e99e0640 1 -- 172.21.15.64:0/3446553521 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db8f1e2340 con 0x55db8f2e1cb0 2024-09-12T03:35:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.069+0000 7f62ea1e1640 1 --2- 172.21.15.64:0/3446553521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db8f2d7410 0x55db8f2e1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:35:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.069+0000 7f62ea9e2640 1 --2- 172.21.15.64:0/2667970673 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f62d00a2e20 0x7f62d00a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.069+0000 7f62d77fe640 1 -- 172.21.15.64:0/2667970673 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f62dc0cf910 con 0x7f62e410c150 2024-09-12T03:35:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.070+0000 7f62e99e0640 1 --2- 172.21.15.64:0/3446553521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db8f2e1cb0 0x55db8f2e40a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f62e00127a0 tx=0x7f62e000b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43daf1b8a6fc9508 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:35:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.070+0000 7f62d4bfe640 1 -- 172.21.15.64:0/3446553521 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e0012d20 con 0x55db8f2e1cb0 2024-09-12T03:35:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.070+0000 7f62d4bfe640 1 -- 172.21.15.64:0/3446553521 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62e0005020 con 0x55db8f2e1cb0 2024-09-12T03:35:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.070+0000 7f62d4bfe640 1 -- 172.21.15.64:0/3446553521 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e0004a00 con 0x55db8f2e1cb0 2024-09-12T03:35:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.070+0000 7f62ea9e2640 1 --2- 172.21.15.64:0/2667970673 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f62d00a2e20 0x7f62d00a52e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f62dc09b9f0 tx=0x7f62dc086040 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:35:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.071+0000 7f62ef1d9740 1 -- 172.21.15.64:0/3446553521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db8f2e1cb0 msgr2=0x55db8f2e40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:35:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.071+0000 7f62ef1d9740 1 --2- 172.21.15.64:0/3446553521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db8f2e1cb0 0x55db8f2e40a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f62e00127a0 tx=0x7f62e000b7c0 comp rx=0 tx=0).stop 2024-09-12T03:35:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.071+0000 7f62ef1d9740 1 -- 172.21.15.64:0/3446553521 shutdown_connections 2024-09-12T03:35:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.071+0000 7f62ef1d9740 1 --2- 172.21.15.64:0/3446553521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db8f2e1cb0 0x55db8f2e40a0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.071+0000 7f62ef1d9740 1 --2- 172.21.15.64:0/3446553521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db8f2d7410 0x55db8f2e1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.071+0000 7f62ef1d9740 1 --2- 172.21.15.64:0/3446553521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db8f2d6af0 0x55db8f2d6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.071+0000 7f62ef1d9740 1 -- 172.21.15.64:0/3446553521 >> 172.21.15.64:0/3446553521 conn(0x55db8f2d3060 msgr2=0x55db8f2d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:35:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.071+0000 7f62ef1d9740 1 -- 172.21.15.64:0/3446553521 shutdown_connections 2024-09-12T03:35:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.071+0000 7f62ef1d9740 1 -- 172.21.15.64:0/3446553521 wait complete. 2024-09-12T03:35:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.071+0000 7f62ef1d9740 1 -- start start 2024-09-12T03:35:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.071+0000 7f62ef1d9740 1 Processor -- start 2024-09-12T03:35:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.071+0000 7f62ef1d9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db8f2d6af0 0x55db8f2debd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.071+0000 7f62ef1d9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db8f2d7410 0x55db8f2df110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.071+0000 7f62ef1d9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db8f2e1cb0 0x55db8f2df650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.071+0000 7f62ef1d9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55db8f2e4880 con 0x55db8f2e1cb0 2024-09-12T03:35:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.072+0000 7f62ef1d9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55db8f1c4290 con 0x55db8f2d6af0 2024-09-12T03:35:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.072+0000 7f62ef1d9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55db8f2e4a00 con 0x55db8f2d7410 2024-09-12T03:35:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.072+0000 7f62ef1d9740 10 client.0 _ll_register_callbacks cb 0x55db8f1f5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:35:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.072+0000 7f62e99e0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db8f2e1cb0 0x55db8f2df650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.072+0000 7f62e99e0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db8f2e1cb0 0x55db8f2df650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44932/0 (socket says 172.21.15.64:44932) 2024-09-12T03:35:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.072+0000 7f62e99e0640 1 -- 172.21.15.64:0/3024503015 learned_addr learned my addr 172.21.15.64:0/3024503015 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:35:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.072+0000 7f62e99e0640 1 --2- 172.21.15.64:0/3024503015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db8f2d6af0 0x55db8f2debd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.072+0000 7f62ea1e1640 1 --2- 172.21.15.64:0/3024503015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db8f2d7410 0x55db8f2df110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.072+0000 7f62e99e0640 1 -- 172.21.15.64:0/3024503015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db8f2d7410 msgr2=0x55db8f2df110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:35:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.072+0000 7f62e99e0640 1 --2- 172.21.15.64:0/3024503015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db8f2d7410 0x55db8f2df110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.072+0000 7f62e99e0640 1 -- 172.21.15.64:0/3024503015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db8f2d6af0 msgr2=0x55db8f2debd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:35:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.072+0000 7f62e99e0640 1 --2- 172.21.15.64:0/3024503015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db8f2d6af0 0x55db8f2debd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.072+0000 7f62e99e0640 1 -- 172.21.15.64:0/3024503015 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db8ef4e210 con 0x55db8f2e1cb0 2024-09-12T03:35:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.072+0000 7f62e99e0640 1 --2- 172.21.15.64:0/3024503015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db8f2e1cb0 0x55db8f2df650 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f62e0005d00 tx=0x7f62e001b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c130dc6dd913c29 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:35:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.072+0000 7f62cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:35:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.072+0000 7f62cdffb640 1 -- 172.21.15.64:0/3024503015 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e0010770 con 0x55db8f2e1cb0 2024-09-12T03:35:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.072+0000 7f62cdffb640 1 -- 172.21.15.64:0/3024503015 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f62e0010910 con 0x55db8f2e1cb0 2024-09-12T03:35:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.072+0000 7f62ef1d9740 1 -- 172.21.15.64:0/3024503015 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62e0005020 con 0x55db8f2e1cb0 2024-09-12T03:35:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.073+0000 7f62cdffb640 1 -- 172.21.15.64:0/3024503015 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f62e0034050 con 0x55db8f2e1cb0 2024-09-12T03:35:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.073+0000 7f62ef1d9740 10 client.15828 fetch_fsmap learned FSMap version 25 2024-09-12T03:35:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.073+0000 7f62ef1d9740 1 -- 172.21.15.64:0/3024503015 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55db8f01a620 con 0x55db8f2e1cb0 2024-09-12T03:35:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.073+0000 7f62cdffb640 1 -- 172.21.15.64:0/3024503015 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62e0005020 con 0x55db8f2e1cb0 2024-09-12T03:35:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.073+0000 7f62cdffb640 1 -- 172.21.15.64:0/3024503015 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f62e002c020 con 0x55db8f2e1cb0 2024-09-12T03:35:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.073+0000 7f62ef1d9740 10 client.15828 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:35:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.073+0000 7f62ef1d9740 10 client.15828 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:35:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.073+0000 7f62ef1d9740 20 client.15828 populate_metadata read hostname 'smithi064' 2024-09-12T03:35:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.073+0000 7f62ef1d9740 1 --2- 172.21.15.64:0/3024503015 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55db8f2f1ad0 0x55db8f2f3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.073+0000 7f62ef1d9740 4 client.15828 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:35:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.073+0000 7f62ef1d9740 1 -- 172.21.15.64:0/3024503015 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db8f248a20 con 0x55db8f2f1ad0 2024-09-12T03:35:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.073+0000 7f62ea1e1640 1 --2- 172.21.15.64:0/3024503015 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55db8f2f1ad0 0x55db8f2f3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.074+0000 7f62ea1e1640 1 --2- 172.21.15.64:0/3024503015 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55db8f2f1ad0 0x55db8f2f3ec0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1811ed28a18ae332 server_cookie=cff14d82166d535d in_seq=0 out_seq=0 2024-09-12T03:35:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.074+0000 7f62cdffb640 10 client.15828 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:35:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.074+0000 7f62cdffb640 1 -- 172.21.15.64:0/3024503015 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55db8f248a20 con 0x55db8f2f1ad0 2024-09-12T03:35:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.074+0000 7f62cdffb640 10 client.15828 handle_command_reply: tid=0 2024-09-12T03:35:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.074+0000 7f62ef1d9740 1 client.15828 shutdown 2024-09-12T03:35:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.075+0000 7f62ef1d9740 1 -- 172.21.15.64:0/3024503015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db8f2e1cb0 msgr2=0x55db8f2df650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:35:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.075+0000 7f62ef1d9740 1 --2- 172.21.15.64:0/3024503015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db8f2e1cb0 0x55db8f2df650 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f62e0005d00 tx=0x7f62e001b040 comp rx=0 tx=0).stop 2024-09-12T03:35:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.075+0000 7f62ef1d9740 1 -- 172.21.15.64:0/3024503015 shutdown_connections 2024-09-12T03:35:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.075+0000 7f62ef1d9740 1 --2- 172.21.15.64:0/3024503015 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55db8f2f1ad0 0x55db8f2f3ec0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.075+0000 7f62ef1d9740 1 --2- 172.21.15.64:0/3024503015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db8f2e1cb0 0x55db8f2df650 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.075+0000 7f62ef1d9740 1 --2- 172.21.15.64:0/3024503015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db8f2d7410 0x55db8f2df110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.075+0000 7f62ef1d9740 1 --2- 172.21.15.64:0/3024503015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db8f2d6af0 0x55db8f2debd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.075+0000 7f62cdffb640 0 client.15828 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:35:13.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.075+0000 7f62ef1d9740 1 -- 172.21.15.64:0/3024503015 >> 172.21.15.64:0/3024503015 conn(0x55db8f2d3060 msgr2=0x55db8f2d8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:35:13.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.075+0000 7f62ef1d9740 1 -- 172.21.15.64:0/3024503015 shutdown_connections 2024-09-12T03:35:13.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.075+0000 7f62ef1d9740 1 -- 172.21.15.64:0/3024503015 wait complete. 2024-09-12T03:35:13.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.075+0000 7f62ef1d9740 20 client.15828 trim_cache size 0 max 16384 2024-09-12T03:35:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.089+0000 7f62ef1d9740 1 Processor -- start 2024-09-12T03:35:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.089+0000 7f62ef1d9740 1 -- start start 2024-09-12T03:35:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.090+0000 7f62ef1d9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db8f2d6af0 0x55db8f2d9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.090+0000 7f62ef1d9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db8f2d7410 0x55db8f2df100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.090+0000 7f62ef1d9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db8f2e1cb0 0x55db8f2df640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.090+0000 7f62ef1d9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f62e41b3af0 con 0x55db8f2e1cb0 2024-09-12T03:35:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.090+0000 7f62ef1d9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f62e41b13c0 con 0x55db8f2d6af0 2024-09-12T03:35:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.090+0000 7f62ef1d9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f62e41af210 con 0x55db8f2d7410 2024-09-12T03:35:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.090+0000 7f62e99e0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db8f2e1cb0 0x55db8f2df640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.090+0000 7f62e99e0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db8f2e1cb0 0x55db8f2df640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44936/0 (socket says 172.21.15.64:44936) 2024-09-12T03:35:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.090+0000 7f62e99e0640 1 -- 172.21.15.64:0/132415026 learned_addr learned my addr 172.21.15.64:0/132415026 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:35:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.090+0000 7f62ea1e1640 1 --2- 172.21.15.64:0/132415026 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db8f2d7410 0x55db8f2df100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.090+0000 7f62e99e0640 1 --2- 172.21.15.64:0/132415026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db8f2d6af0 0x55db8f2d9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.090+0000 7f62e99e0640 1 -- 172.21.15.64:0/132415026 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db8f2d7410 msgr2=0x55db8f2df100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:35:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.090+0000 7f62e99e0640 1 --2- 172.21.15.64:0/132415026 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db8f2d7410 0x55db8f2df100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.090+0000 7f62e99e0640 1 -- 172.21.15.64:0/132415026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db8f2d6af0 msgr2=0x55db8f2d9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:35:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.090+0000 7f62e99e0640 1 --2- 172.21.15.64:0/132415026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db8f2d6af0 0x55db8f2d9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.090+0000 7f62e99e0640 1 -- 172.21.15.64:0/132415026 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db8f2d2e90 con 0x55db8f2e1cb0 2024-09-12T03:35:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.090+0000 7f62e99e0640 1 --2- 172.21.15.64:0/132415026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db8f2e1cb0 0x55db8f2df640 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f62e00046c0 tx=0x7f62e0014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838cbddd0bd94b5d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:35:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.090+0000 7f62cffff640 1 -- 172.21.15.64:0/132415026 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e00056a0 con 0x55db8f2e1cb0 2024-09-12T03:35:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.090+0000 7f62cffff640 1 -- 172.21.15.64:0/132415026 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62e0012d20 con 0x55db8f2e1cb0 2024-09-12T03:35:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.090+0000 7f62cffff640 1 -- 172.21.15.64:0/132415026 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e0004730 con 0x55db8f2e1cb0 2024-09-12T03:35:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.091+0000 7f62ef1d9740 1 -- 172.21.15.64:0/132415026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db8f2e1cb0 msgr2=0x55db8f2df640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:35:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.091+0000 7f62ef1d9740 1 --2- 172.21.15.64:0/132415026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db8f2e1cb0 0x55db8f2df640 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f62e00046c0 tx=0x7f62e0014040 comp rx=0 tx=0).stop 2024-09-12T03:35:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.091+0000 7f62ef1d9740 1 -- 172.21.15.64:0/132415026 shutdown_connections 2024-09-12T03:35:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.091+0000 7f62ef1d9740 1 --2- 172.21.15.64:0/132415026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db8f2e1cb0 0x55db8f2df640 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.091+0000 7f62ef1d9740 1 --2- 172.21.15.64:0/132415026 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db8f2d7410 0x55db8f2df100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.091+0000 7f62ef1d9740 1 --2- 172.21.15.64:0/132415026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db8f2d6af0 0x55db8f2d9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.091+0000 7f62ef1d9740 1 -- 172.21.15.64:0/132415026 >> 172.21.15.64:0/132415026 conn(0x55db8f2d3060 msgr2=0x55db8f2e0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:35:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.091+0000 7f62ef1d9740 1 -- 172.21.15.64:0/132415026 shutdown_connections 2024-09-12T03:35:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.091+0000 7f62ef1d9740 1 -- 172.21.15.64:0/132415026 wait complete. 2024-09-12T03:35:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.091+0000 7f62ef1d9740 1 -- start start 2024-09-12T03:35:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.092+0000 7f62ef1d9740 1 Processor -- start 2024-09-12T03:35:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.092+0000 7f62ef1d9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db8f2d6af0 0x55db8f2e9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.092+0000 7f62ef1d9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db8f2d7410 0x55db8f2ea350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.092+0000 7f62ef1d9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db8f2e1cb0 0x55db8f2e5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.092+0000 7f62ef1d9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f62e41af880 con 0x55db8f2d6af0 2024-09-12T03:35:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.092+0000 7f62ef1d9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f62e41b0cf0 con 0x55db8f2d7410 2024-09-12T03:35:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.092+0000 7f62ef1d9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f62e41aff50 con 0x55db8f2e1cb0 2024-09-12T03:35:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.092+0000 7f62ef1d9740 10 client.0 _ll_register_callbacks cb 0x7f62e000be70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:35:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.092+0000 7f62e99e0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db8f2d6af0 0x55db8f2e9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.092+0000 7f62e99e0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db8f2d6af0 0x55db8f2e9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44940/0 (socket says 172.21.15.64:44940) 2024-09-12T03:35:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.092+0000 7f62e99e0640 1 -- 172.21.15.64:0/1253584571 learned_addr learned my addr 172.21.15.64:0/1253584571 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:35:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.092+0000 7f62e99e0640 1 --2- 172.21.15.64:0/1253584571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db8f2e1cb0 0x55db8f2e5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.092+0000 7f62ea1e1640 1 --2- 172.21.15.64:0/1253584571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db8f2d7410 0x55db8f2ea350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.092+0000 7f62e99e0640 1 -- 172.21.15.64:0/1253584571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db8f2e1cb0 msgr2=0x55db8f2e5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:35:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.092+0000 7f62e99e0640 1 --2- 172.21.15.64:0/1253584571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db8f2e1cb0 0x55db8f2e5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.092+0000 7f62e99e0640 1 -- 172.21.15.64:0/1253584571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db8f2d7410 msgr2=0x55db8f2ea350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:35:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.092+0000 7f62e99e0640 1 --2- 172.21.15.64:0/1253584571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db8f2d7410 0x55db8f2ea350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.092+0000 7f62e99e0640 1 -- 172.21.15.64:0/1253584571 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db8f2ce2d0 con 0x55db8f2d6af0 2024-09-12T03:35:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.092+0000 7f62ea1e1640 1 --2- 172.21.15.64:0/1253584571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db8f2d7410 0x55db8f2ea350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:35:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.093+0000 7f62e99e0640 1 --2- 172.21.15.64:0/1253584571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db8f2d6af0 0x55db8f2e9e10 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f62e0001e90 tx=0x7f62e0005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb65957a671b1d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:35:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.093+0000 7f62cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:35:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.093+0000 7f62cdffb640 1 -- 172.21.15.64:0/1253584571 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e0012270 con 0x55db8f2d6af0 2024-09-12T03:35:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.093+0000 7f62cdffb640 1 -- 172.21.15.64:0/1253584571 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f62e0010740 con 0x55db8f2d6af0 2024-09-12T03:35:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.093+0000 7f62ef1d9740 1 -- 172.21.15.64:0/1253584571 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62e0012d20 con 0x55db8f2d6af0 2024-09-12T03:35:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.093+0000 7f62cdffb640 1 -- 172.21.15.64:0/1253584571 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f62e0004940 con 0x55db8f2d6af0 2024-09-12T03:35:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.093+0000 7f62ef1d9740 10 client.15834 fetch_fsmap learned FSMap version 25 2024-09-12T03:35:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.093+0000 7f62ef1d9740 1 -- 172.21.15.64:0/1253584571 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55db8f35bf80 con 0x55db8f2d6af0 2024-09-12T03:35:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.093+0000 7f62cdffb640 1 -- 172.21.15.64:0/1253584571 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62e0012d20 con 0x55db8f2d6af0 2024-09-12T03:35:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.093+0000 7f62cdffb640 1 -- 172.21.15.64:0/1253584571 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f62e002c040 con 0x55db8f2d6af0 2024-09-12T03:35:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.093+0000 7f62ef1d9740 10 client.15834 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:35:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.093+0000 7f62ef1d9740 10 client.15834 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:35:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.093+0000 7f62ef1d9740 20 client.15834 populate_metadata read hostname 'smithi064' 2024-09-12T03:35:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.093+0000 7f62ef1d9740 1 --2- 172.21.15.64:0/1253584571 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55db8f35c590 0x55db8f35e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:13.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.093+0000 7f62ef1d9740 4 client.15834 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:35:13.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.093+0000 7f62ef1d9740 1 -- 172.21.15.64:0/1253584571 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55db8f242d70 con 0x55db8f35c590 2024-09-12T03:35:13.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.094+0000 7f62ea1e1640 1 --2- 172.21.15.64:0/1253584571 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55db8f35c590 0x55db8f35e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:13.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.094+0000 7f62ea1e1640 1 --2- 172.21.15.64:0/1253584571 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55db8f35c590 0x55db8f35e980 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3034d546f869b045 server_cookie=c41c95eb50fdf0a in_seq=0 out_seq=0 2024-09-12T03:35:13.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.094+0000 7f62cdffb640 10 client.15834 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:35:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.108+0000 7f62cdffb640 1 -- 172.21.15.64:0/1253584571 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55db8f242d70 con 0x55db8f35c590 2024-09-12T03:35:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.108+0000 7f62cdffb640 10 client.15834 handle_command_reply: tid=0 2024-09-12T03:35:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.109+0000 7f62ef1d9740 1 client.15834 shutdown 2024-09-12T03:35:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.109+0000 7f62ef1d9740 1 -- 172.21.15.64:0/1253584571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db8f2d6af0 msgr2=0x55db8f2e9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:35:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.109+0000 7f62ef1d9740 1 --2- 172.21.15.64:0/1253584571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db8f2d6af0 0x55db8f2e9e10 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f62e0001e90 tx=0x7f62e0005c50 comp rx=0 tx=0).stop 2024-09-12T03:35:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.109+0000 7f62ef1d9740 1 -- 172.21.15.64:0/1253584571 shutdown_connections 2024-09-12T03:35:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.109+0000 7f62ef1d9740 1 --2- 172.21.15.64:0/1253584571 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55db8f35c590 0x55db8f35e980 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.109+0000 7f62ef1d9740 1 --2- 172.21.15.64:0/1253584571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db8f2e1cb0 0x55db8f2e5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.109+0000 7f62ef1d9740 1 --2- 172.21.15.64:0/1253584571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55db8f2d7410 0x55db8f2ea350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.109+0000 7f62ef1d9740 1 --2- 172.21.15.64:0/1253584571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55db8f2d6af0 0x55db8f2e9e10 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.109+0000 7f62cdffb640 0 client.15834 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:35:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.109+0000 7f62ef1d9740 1 -- 172.21.15.64:0/1253584571 >> 172.21.15.64:0/1253584571 conn(0x55db8f2d3060 msgr2=0x55db8f2d19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:35:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.109+0000 7f62ef1d9740 1 -- 172.21.15.64:0/1253584571 shutdown_connections 2024-09-12T03:35:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.109+0000 7f62ef1d9740 1 -- 172.21.15.64:0/1253584571 wait complete. 2024-09-12T03:35:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.109+0000 7f62ef1d9740 20 client.15834 trim_cache size 0 max 16384 2024-09-12T03:35:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:35:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:35:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:35:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:35:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.109+0000 7f62ec445640 1 -- 172.21.15.64:0/2667970673 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f62d00a2e20 msgr2=0x7f62d00a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:35:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.109+0000 7f62ec445640 1 --2- 172.21.15.64:0/2667970673 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f62d00a2e20 0x7f62d00a52e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f62dc09b9f0 tx=0x7f62dc086040 comp rx=0 tx=0).stop 2024-09-12T03:35:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.109+0000 7f62ec445640 1 -- 172.21.15.64:0/2667970673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62e410c150 msgr2=0x7f62e4101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:35:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.110+0000 7f62ec445640 1 --2- 172.21.15.64:0/2667970673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62e410c150 0x7f62e4101450 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00eaa0 tx=0x7f62dc00ef60 comp rx=0 tx=0).stop 2024-09-12T03:35:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.110+0000 7f62ec445640 1 -- 172.21.15.64:0/2667970673 shutdown_connections 2024-09-12T03:35:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.110+0000 7f62ec445640 1 --2- 172.21.15.64:0/2667970673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62e410c150 0x7f62e4101450 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.110+0000 7f62ec445640 1 --2- 172.21.15.64:0/2667970673 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62e41036a0 0x7f62e4100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.110+0000 7f62ec445640 1 --2- 172.21.15.64:0/2667970673 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f62d00a2e20 0x7f62d00a52e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.110+0000 7f62ec445640 1 --2- 172.21.15.64:0/2667970673 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62e4102d80 0x7f62e41009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.110+0000 7f62ec445640 1 -- 172.21.15.64:0/2667970673 >> 172.21.15.64:0/2667970673 conn(0x7f62e40fc6b0 msgr2=0x7f62e40fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:35:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.110+0000 7f62ec445640 1 -- 172.21.15.64:0/2667970673 shutdown_connections 2024-09-12T03:35:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:13.110+0000 7f62ec445640 1 -- 172.21.15.64:0/2667970673 wait complete. 2024-09-12T03:35:13.521 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:35:13 smithi064 ceph-mon[30952]: pgmap v181: 129 pgs: 129 active+clean; 4.8 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 76 KiB/s wr, 33 op/s 2024-09-12T03:35:13.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:35:13 smithi115 ceph-mon[34801]: pgmap v181: 129 pgs: 129 active+clean; 4.8 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 76 KiB/s wr, 33 op/s 2024-09-12T03:35:13.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:35:13 smithi195 ceph-mon[34786]: pgmap v181: 129 pgs: 129 active+clean; 4.8 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 76 KiB/s wr, 33 op/s 2024-09-12T03:35:15.645 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:35:15 smithi115 ceph-mon[34801]: pgmap v182: 129 pgs: 129 active+clean; 4.8 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-12T03:35:15.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:35:15 smithi195 ceph-mon[34786]: pgmap v182: 129 pgs: 129 active+clean; 4.8 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-12T03:35:15.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:35:15 smithi064 ceph-mon[30952]: pgmap v182: 129 pgs: 129 active+clean; 4.8 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-12T03:35:17.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:35:17 smithi115 ceph-mon[34801]: pgmap v183: 129 pgs: 129 active+clean; 4.8 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-12T03:35:17.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:35:17 smithi195 ceph-mon[34786]: pgmap v183: 129 pgs: 129 active+clean; 4.8 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-12T03:35:17.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:35:17 smithi064 ceph-mon[30952]: pgmap v183: 129 pgs: 129 active+clean; 4.8 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-12T03:35:19.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:35:19 smithi115 ceph-mon[34801]: pgmap v184: 129 pgs: 129 active+clean; 5.0 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:35:19.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:35:19 smithi195 ceph-mon[34786]: pgmap v184: 129 pgs: 129 active+clean; 5.0 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:35:19.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:35:19 smithi064 ceph-mon[30952]: pgmap v184: 129 pgs: 129 active+clean; 5.0 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:35:21.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:35:21 smithi115 ceph-mon[34801]: pgmap v185: 129 pgs: 129 active+clean; 5.0 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:35:21.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:35:21 smithi195 ceph-mon[34786]: pgmap v185: 129 pgs: 129 active+clean; 5.0 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:35:21.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:35:21 smithi064 ceph-mon[30952]: pgmap v185: 129 pgs: 129 active+clean; 5.0 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:35:23.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:35:23 smithi115 ceph-mon[34801]: pgmap v186: 129 pgs: 129 active+clean; 5.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:35:23.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:35:23 smithi195 ceph-mon[34786]: pgmap v186: 129 pgs: 129 active+clean; 5.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:35:23.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:35:23 smithi064 ceph-mon[30952]: pgmap v186: 129 pgs: 129 active+clean; 5.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:35:24.645 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:35:24 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:35:24.645 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:35:24 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:35:24.645 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:35:24 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:35:24.645 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:35:24 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:35:24.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:35:24 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:35:24.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:35:24 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:35:24.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:35:24 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:35:24.707 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:35:24 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:35:24.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:35:24 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:35:24.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:35:24 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:35:24.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:35:24 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:35:24.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:35:24 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:35:25.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:35:25 smithi115 ceph-mon[34801]: pgmap v187: 129 pgs: 129 active+clean; 5.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-12T03:35:25.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:35:25 smithi195 ceph-mon[34786]: pgmap v187: 129 pgs: 129 active+clean; 5.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-12T03:35:25.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:35:25 smithi064 ceph-mon[30952]: pgmap v187: 129 pgs: 129 active+clean; 5.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-12T03:35:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.211+0000 7f777abcd640 1 -- 172.21.15.64:0/362443128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7774102c80 msgr2=0x7f7774103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:35:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.211+0000 7f777abcd640 1 --2- 172.21.15.64:0/362443128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7774102c80 0x7f7774103060 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f775c009900 tx=0x7f775c02a480 comp rx=0 tx=0).stop 2024-09-12T03:35:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.211+0000 7f777abcd640 1 -- 172.21.15.64:0/362443128 shutdown_connections 2024-09-12T03:35:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.211+0000 7f777abcd640 1 --2- 172.21.15.64:0/362443128 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f777410c0d0 0x7f777410e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.211+0000 7f777abcd640 1 --2- 172.21.15.64:0/362443128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77741035a0 0x7f777410ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.211+0000 7f777abcd640 1 --2- 172.21.15.64:0/362443128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7774102c80 0x7f7774103060 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.211+0000 7f777abcd640 1 -- 172.21.15.64:0/362443128 >> 172.21.15.64:0/362443128 conn(0x7f7774077990 msgr2=0x7f77740fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:35:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.211+0000 7f777abcd640 1 -- 172.21.15.64:0/362443128 shutdown_connections 2024-09-12T03:35:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.211+0000 7f777abcd640 1 -- 172.21.15.64:0/362443128 wait complete. 2024-09-12T03:35:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.211+0000 7f777abcd640 1 Processor -- start 2024-09-12T03:35:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.212+0000 7f777abcd640 1 -- start start 2024-09-12T03:35:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.212+0000 7f777abcd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7774102c80 0x7f7774198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.212+0000 7f777abcd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77741035a0 0x7f77741987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.212+0000 7f777abcd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f777410c0d0 0x7f777419cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.212+0000 7f777abcd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7774111d60 con 0x7f777410c0d0 2024-09-12T03:35:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.212+0000 7f777abcd640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7774111be0 con 0x7f77741035a0 2024-09-12T03:35:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.212+0000 7f777abcd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7774111ee0 con 0x7f7774102c80 2024-09-12T03:35:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.212+0000 7f777916a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f777410c0d0 0x7f777419cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.212+0000 7f777916a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f777410c0d0 0x7f777419cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51644/0 (socket says 172.21.15.64:51644) 2024-09-12T03:35:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.212+0000 7f777916a640 1 -- 172.21.15.64:0/2048189264 learned_addr learned my addr 172.21.15.64:0/2048189264 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:35:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.212+0000 7f7773fff640 1 --2- 172.21.15.64:0/2048189264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77741035a0 0x7f77741987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.212+0000 7f7778969640 1 --2- 172.21.15.64:0/2048189264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7774102c80 0x7f7774198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.212+0000 7f777916a640 1 -- 172.21.15.64:0/2048189264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7774102c80 msgr2=0x7f7774198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:35:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.212+0000 7f777916a640 1 --2- 172.21.15.64:0/2048189264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7774102c80 0x7f7774198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.212+0000 7f777916a640 1 -- 172.21.15.64:0/2048189264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77741035a0 msgr2=0x7f77741987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:35:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.212+0000 7f777916a640 1 --2- 172.21.15.64:0/2048189264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77741035a0 0x7f77741987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.212+0000 7f777916a640 1 -- 172.21.15.64:0/2048189264 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f777419d260 con 0x7f777410c0d0 2024-09-12T03:35:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.213+0000 7f777916a640 1 --2- 172.21.15.64:0/2048189264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f777410c0d0 0x7f777419cb30 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f776c009fd0 tx=0x7f776c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:35:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.213+0000 7f7771ffb640 1 -- 172.21.15.64:0/2048189264 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c017070 con 0x7f777410c0d0 2024-09-12T03:35:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.213+0000 7f7771ffb640 1 -- 172.21.15.64:0/2048189264 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f776c003e20 con 0x7f777410c0d0 2024-09-12T03:35:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.213+0000 7f7771ffb640 1 -- 172.21.15.64:0/2048189264 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c010460 con 0x7f777410c0d0 2024-09-12T03:35:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.213+0000 7f777abcd640 1 -- 172.21.15.64:0/2048189264 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f777419d550 con 0x7f777410c0d0 2024-09-12T03:35:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.213+0000 7f777abcd640 1 -- 172.21.15.64:0/2048189264 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77741a4de0 con 0x7f777410c0d0 2024-09-12T03:35:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.215+0000 7f7771ffb640 1 -- 172.21.15.64:0/2048189264 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f776c003970 con 0x7f777410c0d0 2024-09-12T03:35:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.216+0000 7f777da42740 1 Processor -- start 2024-09-12T03:35:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.216+0000 7f777da42740 1 -- start start 2024-09-12T03:35:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.216+0000 7f777da42740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565315609d0 0x556531560db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.216+0000 7f777da42740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565315612f0 0x55653156fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.216+0000 7f777da42740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565315700c0 0x5565315724b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.216+0000 7f777da42740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5565314c52f0 con 0x5565315612f0 2024-09-12T03:35:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.216+0000 7f777da42740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55653146d270 con 0x5565315700c0 2024-09-12T03:35:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.216+0000 7f777da42740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5565314c12b0 con 0x5565315609d0 2024-09-12T03:35:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.216+0000 7f7771ffb640 1 --2- 172.21.15.64:0/2048189264 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f77600a2e20 0x7f77600a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.216+0000 7f7778969640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565315612f0 0x55653156fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.216+0000 7f7778969640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565315612f0 0x55653156fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51650/0 (socket says 172.21.15.64:51650) 2024-09-12T03:35:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.216+0000 7f7778969640 1 -- 172.21.15.64:0/2258554243 learned_addr learned my addr 172.21.15.64:0/2258554243 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:35:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.216+0000 7f777916a640 1 --2- 172.21.15.64:0/2048189264 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f77600a2e20 0x7f77600a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.216+0000 7f7773fff640 1 --2- 172.21.15.64:0/2258554243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565315700c0 0x5565315724b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.216+0000 7f7771ffb640 1 -- 172.21.15.64:0/2048189264 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f776c0cf4f0 con 0x7f777410c0d0 2024-09-12T03:35:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.216+0000 7f7773fff640 1 --2- 172.21.15.64:0/2258554243 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565315609d0 0x556531560db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.217+0000 7f7778969640 1 -- 172.21.15.64:0/2258554243 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565315609d0 msgr2=0x556531560db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:35:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.217+0000 7f7778969640 1 --2- 172.21.15.64:0/2258554243 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565315609d0 0x556531560db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.217+0000 7f7778969640 1 -- 172.21.15.64:0/2258554243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565315700c0 msgr2=0x5565315724b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:35:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.217+0000 7f7778969640 1 --2- 172.21.15.64:0/2258554243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565315700c0 0x5565315724b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.217+0000 7f7778969640 1 -- 172.21.15.64:0/2258554243 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55653146c340 con 0x5565315612f0 2024-09-12T03:35:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.217+0000 7f7773fff640 1 --2- 172.21.15.64:0/2258554243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565315700c0 0x5565315724b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:35:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.217+0000 7f7778969640 1 --2- 172.21.15.64:0/2258554243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565315612f0 0x55653156fb80 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f775c033000 tx=0x7f775c032bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3963c9ed272da552 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:35:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.217+0000 7f777916a640 1 --2- 172.21.15.64:0/2048189264 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f77600a2e20 0x7f77600a52e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f776c0cce70 tx=0x7f776c06d6b0 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:35:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.217+0000 7f776affd640 1 -- 172.21.15.64:0/2258554243 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c035040 con 0x5565315612f0 2024-09-12T03:35:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.217+0000 7f776affd640 1 -- 172.21.15.64:0/2258554243 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f775c004030 con 0x5565315612f0 2024-09-12T03:35:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.217+0000 7f776affd640 1 -- 172.21.15.64:0/2258554243 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c0055a0 con 0x5565315612f0 2024-09-12T03:35:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.218+0000 7f777da42740 1 -- 172.21.15.64:0/2258554243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565315612f0 msgr2=0x55653156fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:35:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.218+0000 7f777da42740 1 --2- 172.21.15.64:0/2258554243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565315612f0 0x55653156fb80 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f775c033000 tx=0x7f775c032bb0 comp rx=0 tx=0).stop 2024-09-12T03:35:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.218+0000 7f777da42740 1 -- 172.21.15.64:0/2258554243 shutdown_connections 2024-09-12T03:35:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.218+0000 7f777da42740 1 --2- 172.21.15.64:0/2258554243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565315700c0 0x5565315724b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.218+0000 7f777da42740 1 --2- 172.21.15.64:0/2258554243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565315612f0 0x55653156fb80 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.218+0000 7f777da42740 1 --2- 172.21.15.64:0/2258554243 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565315609d0 0x556531560db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.218+0000 7f777da42740 1 -- 172.21.15.64:0/2258554243 >> 172.21.15.64:0/2258554243 conn(0x55653155d060 msgr2=0x55653155d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:35:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.218+0000 7f777da42740 1 -- 172.21.15.64:0/2258554243 shutdown_connections 2024-09-12T03:35:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.218+0000 7f777da42740 1 -- 172.21.15.64:0/2258554243 wait complete. 2024-09-12T03:35:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.218+0000 7f777da42740 1 -- start start 2024-09-12T03:35:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.218+0000 7f777da42740 1 Processor -- start 2024-09-12T03:35:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.219+0000 7f777da42740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565315609d0 0x55653156d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.219+0000 7f777da42740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565315612f0 0x55653156d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.219+0000 7f777da42740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565315700c0 0x55653156daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.219+0000 7f777da42740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556531572c90 con 0x5565315700c0 2024-09-12T03:35:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.219+0000 7f777da42740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55653144e290 con 0x5565315612f0 2024-09-12T03:35:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.219+0000 7f777da42740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556531572e10 con 0x5565315609d0 2024-09-12T03:35:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.219+0000 7f777da42740 10 client.0 _ll_register_callbacks cb 0x55653147f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:35:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.219+0000 7f7773fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565315700c0 0x55653156daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.219+0000 7f7773fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565315700c0 0x55653156daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51660/0 (socket says 172.21.15.64:51660) 2024-09-12T03:35:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.219+0000 7f7773fff640 1 -- 172.21.15.64:0/4125146766 learned_addr learned my addr 172.21.15.64:0/4125146766 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:35:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.219+0000 7f7773fff640 1 --2- 172.21.15.64:0/4125146766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565315609d0 0x55653156d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.219+0000 7f7778969640 1 --2- 172.21.15.64:0/4125146766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565315612f0 0x55653156d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.219+0000 7f7773fff640 1 -- 172.21.15.64:0/4125146766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565315609d0 msgr2=0x55653156d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:35:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.219+0000 7f7773fff640 1 --2- 172.21.15.64:0/4125146766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565315609d0 0x55653156d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.219+0000 7f7773fff640 1 -- 172.21.15.64:0/4125146766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565315612f0 msgr2=0x55653156d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:35:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.219+0000 7f7773fff640 1 --2- 172.21.15.64:0/4125146766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565315612f0 0x55653156d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.219+0000 7f7773fff640 1 -- 172.21.15.64:0/4125146766 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565311d8210 con 0x5565315700c0 2024-09-12T03:35:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.220+0000 7f7773fff640 1 --2- 172.21.15.64:0/4125146766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565315700c0 0x55653156daf0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f77640126c0 tx=0x7f7764014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d92e004a5175a44c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:35:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.220+0000 7f7753fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:35:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.220+0000 7f7753fff640 1 -- 172.21.15.64:0/4125146766 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7764004220 con 0x5565315700c0 2024-09-12T03:35:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.220+0000 7f7753fff640 1 -- 172.21.15.64:0/4125146766 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f77640043c0 con 0x5565315700c0 2024-09-12T03:35:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.220+0000 7f777da42740 1 -- 172.21.15.64:0/4125146766 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f775c004030 con 0x5565315700c0 2024-09-12T03:35:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.220+0000 7f7753fff640 1 -- 172.21.15.64:0/4125146766 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f776400d8c0 con 0x5565315700c0 2024-09-12T03:35:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.220+0000 7f777da42740 10 client.15846 fetch_fsmap learned FSMap version 25 2024-09-12T03:35:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.220+0000 7f777da42740 1 -- 172.21.15.64:0/4125146766 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5565312a4620 con 0x5565315700c0 2024-09-12T03:35:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.220+0000 7f7753fff640 1 -- 172.21.15.64:0/4125146766 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f775c004030 con 0x5565315700c0 2024-09-12T03:35:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.220+0000 7f7753fff640 1 -- 172.21.15.64:0/4125146766 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f776401e070 con 0x5565315700c0 2024-09-12T03:35:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.220+0000 7f777da42740 10 client.15846 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:35:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.220+0000 7f777da42740 10 client.15846 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:35:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.220+0000 7f777da42740 20 client.15846 populate_metadata read hostname 'smithi064' 2024-09-12T03:35:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.220+0000 7f777da42740 1 --2- 172.21.15.64:0/4125146766 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5565315801c0 0x5565315825d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.220+0000 7f777da42740 4 client.15846 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:35:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.220+0000 7f777da42740 1 -- 172.21.15.64:0/4125146766 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565314d2a20 con 0x5565315801c0 2024-09-12T03:35:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.221+0000 7f7778969640 1 --2- 172.21.15.64:0/4125146766 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5565315801c0 0x5565315825d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.221+0000 7f7778969640 1 --2- 172.21.15.64:0/4125146766 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5565315801c0 0x5565315825d0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a61d1ac2691dfb4e server_cookie=e11e0f9b74b04b4 in_seq=0 out_seq=0 2024-09-12T03:35:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.221+0000 7f7753fff640 10 client.15846 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:35:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.222+0000 7f7753fff640 1 -- 172.21.15.64:0/4125146766 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5565314d2a20 con 0x5565315801c0 2024-09-12T03:35:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.222+0000 7f7753fff640 10 client.15846 handle_command_reply: tid=0 2024-09-12T03:35:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.222+0000 7f777da42740 1 client.15846 shutdown 2024-09-12T03:35:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.222+0000 7f777da42740 1 -- 172.21.15.64:0/4125146766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565315700c0 msgr2=0x55653156daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:35:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.222+0000 7f777da42740 1 --2- 172.21.15.64:0/4125146766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565315700c0 0x55653156daf0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f77640126c0 tx=0x7f7764014040 comp rx=0 tx=0).stop 2024-09-12T03:35:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.222+0000 7f777da42740 1 -- 172.21.15.64:0/4125146766 shutdown_connections 2024-09-12T03:35:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.222+0000 7f777da42740 1 --2- 172.21.15.64:0/4125146766 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5565315801c0 0x5565315825d0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.222+0000 7f777da42740 1 --2- 172.21.15.64:0/4125146766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565315700c0 0x55653156daf0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.222+0000 7f7753fff640 0 client.15846 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:35:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.222+0000 7f777da42740 1 --2- 172.21.15.64:0/4125146766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565315612f0 0x55653156d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.222+0000 7f777da42740 1 --2- 172.21.15.64:0/4125146766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565315609d0 0x55653156d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.222+0000 7f777da42740 1 -- 172.21.15.64:0/4125146766 >> 172.21.15.64:0/4125146766 conn(0x55653155d060 msgr2=0x556531562dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:35:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.222+0000 7f777da42740 1 -- 172.21.15.64:0/4125146766 shutdown_connections 2024-09-12T03:35:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.223+0000 7f777da42740 1 -- 172.21.15.64:0/4125146766 wait complete. 2024-09-12T03:35:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.223+0000 7f777da42740 20 client.15846 trim_cache size 0 max 16384 2024-09-12T03:35:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.237+0000 7f777da42740 1 Processor -- start 2024-09-12T03:35:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.237+0000 7f777da42740 1 -- start start 2024-09-12T03:35:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.237+0000 7f777da42740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565315609d0 0x556531587ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.237+0000 7f777da42740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565315612f0 0x556531588420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.237+0000 7f777da42740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565315700c0 0x55653156a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.237+0000 7f777da42740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f77741af5c0 con 0x5565315700c0 2024-09-12T03:35:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.237+0000 7f777da42740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f77741ace90 con 0x5565315609d0 2024-09-12T03:35:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.237+0000 7f777da42740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f77741aace0 con 0x5565315612f0 2024-09-12T03:35:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.238+0000 7f7773fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565315700c0 0x55653156a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.238+0000 7f7773fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565315700c0 0x55653156a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51672/0 (socket says 172.21.15.64:51672) 2024-09-12T03:35:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.238+0000 7f7773fff640 1 -- 172.21.15.64:0/1969861575 learned_addr learned my addr 172.21.15.64:0/1969861575 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:35:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.238+0000 7f7778969640 1 --2- 172.21.15.64:0/1969861575 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565315612f0 0x556531588420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.238+0000 7f7773fff640 1 --2- 172.21.15.64:0/1969861575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565315609d0 0x556531587ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.238+0000 7f7773fff640 1 -- 172.21.15.64:0/1969861575 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565315612f0 msgr2=0x556531588420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:35:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.238+0000 7f7773fff640 1 --2- 172.21.15.64:0/1969861575 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565315612f0 0x556531588420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.238+0000 7f7773fff640 1 -- 172.21.15.64:0/1969861575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565315609d0 msgr2=0x556531587ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:35:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.238+0000 7f7773fff640 1 --2- 172.21.15.64:0/1969861575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565315609d0 0x556531587ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.238+0000 7f7773fff640 1 -- 172.21.15.64:0/1969861575 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55653156c000 con 0x5565315700c0 2024-09-12T03:35:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.238+0000 7f7778969640 1 --2- 172.21.15.64:0/1969861575 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565315612f0 0x556531588420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:35:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.238+0000 7f7773fff640 1 --2- 172.21.15.64:0/1969861575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565315700c0 0x55653156a2b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f7764010550 tx=0x7f776400b2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9341fd1155ce6c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:35:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.238+0000 7f776a7fc640 1 -- 172.21.15.64:0/1969861575 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7764033070 con 0x5565315700c0 2024-09-12T03:35:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.238+0000 7f776a7fc640 1 -- 172.21.15.64:0/1969861575 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77640122d0 con 0x5565315700c0 2024-09-12T03:35:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.238+0000 7f776a7fc640 1 -- 172.21.15.64:0/1969861575 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7764003f60 con 0x5565315700c0 2024-09-12T03:35:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.239+0000 7f777da42740 1 -- 172.21.15.64:0/1969861575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565315700c0 msgr2=0x55653156a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:35:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.239+0000 7f777da42740 1 --2- 172.21.15.64:0/1969861575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565315700c0 0x55653156a2b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f7764010550 tx=0x7f776400b2e0 comp rx=0 tx=0).stop 2024-09-12T03:35:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.239+0000 7f777da42740 1 -- 172.21.15.64:0/1969861575 shutdown_connections 2024-09-12T03:35:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.239+0000 7f777da42740 1 --2- 172.21.15.64:0/1969861575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565315700c0 0x55653156a2b0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.239+0000 7f777da42740 1 --2- 172.21.15.64:0/1969861575 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565315612f0 0x556531588420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.239+0000 7f777da42740 1 --2- 172.21.15.64:0/1969861575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565315609d0 0x556531587ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.239+0000 7f777da42740 1 -- 172.21.15.64:0/1969861575 >> 172.21.15.64:0/1969861575 conn(0x55653155d060 msgr2=0x55653156d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:35:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.239+0000 7f777da42740 1 -- 172.21.15.64:0/1969861575 shutdown_connections 2024-09-12T03:35:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.239+0000 7f777da42740 1 -- 172.21.15.64:0/1969861575 wait complete. 2024-09-12T03:35:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.239+0000 7f777da42740 1 -- start start 2024-09-12T03:35:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.239+0000 7f777da42740 1 Processor -- start 2024-09-12T03:35:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.240+0000 7f777da42740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565315609d0 0x556531574650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.240+0000 7f777da42740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565315612f0 0x556531574b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.240+0000 7f777da42740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565315700c0 0x5565315e3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.240+0000 7f777da42740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f77741ab350 con 0x5565315700c0 2024-09-12T03:35:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.240+0000 7f777da42740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f77741ac7c0 con 0x5565315612f0 2024-09-12T03:35:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.240+0000 7f777da42740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f77741aba20 con 0x5565315609d0 2024-09-12T03:35:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.240+0000 7f777da42740 10 client.0 _ll_register_callbacks cb 0x7f776400b9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:35:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.240+0000 7f7773fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565315700c0 0x5565315e3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.240+0000 7f7773fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565315700c0 0x5565315e3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51680/0 (socket says 172.21.15.64:51680) 2024-09-12T03:35:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.240+0000 7f7773fff640 1 -- 172.21.15.64:0/1028679243 learned_addr learned my addr 172.21.15.64:0/1028679243 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:35:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.240+0000 7f7773fff640 1 --2- 172.21.15.64:0/1028679243 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565315609d0 0x556531574650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.240+0000 7f7778969640 1 --2- 172.21.15.64:0/1028679243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565315612f0 0x556531574b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.240+0000 7f7773fff640 1 -- 172.21.15.64:0/1028679243 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565315609d0 msgr2=0x556531574650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:35:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.240+0000 7f7773fff640 1 --2- 172.21.15.64:0/1028679243 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565315609d0 0x556531574650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.240+0000 7f7773fff640 1 -- 172.21.15.64:0/1028679243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565315612f0 msgr2=0x556531574b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:35:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.240+0000 7f7773fff640 1 --2- 172.21.15.64:0/1028679243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565315612f0 0x556531574b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.240+0000 7f7773fff640 1 -- 172.21.15.64:0/1028679243 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565315e44c0 con 0x5565315700c0 2024-09-12T03:35:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.240+0000 7f7778969640 1 --2- 172.21.15.64:0/1028679243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565315612f0 0x556531574b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:35:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.240+0000 7f7773fff640 1 --2- 172.21.15.64:0/1028679243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565315700c0 0x5565315e3e80 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f7764003ee0 tx=0x7f77640037e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57cc110b4db7307b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:35:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.241+0000 7f7753fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:35:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.241+0000 7f7753fff640 1 -- 172.21.15.64:0/1028679243 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7764033040 con 0x5565315700c0 2024-09-12T03:35:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.241+0000 7f7753fff640 1 -- 172.21.15.64:0/1028679243 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f7764003f60 con 0x5565315700c0 2024-09-12T03:35:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.241+0000 7f777da42740 1 -- 172.21.15.64:0/1028679243 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77640122d0 con 0x5565315700c0 2024-09-12T03:35:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.241+0000 7f7753fff640 1 -- 172.21.15.64:0/1028679243 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7764042050 con 0x5565315700c0 2024-09-12T03:35:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.241+0000 7f777da42740 10 client.15852 fetch_fsmap learned FSMap version 25 2024-09-12T03:35:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.241+0000 7f777da42740 1 -- 172.21.15.64:0/1028679243 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5565315eb820 con 0x5565315700c0 2024-09-12T03:35:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.241+0000 7f7753fff640 1 -- 172.21.15.64:0/1028679243 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77640122d0 con 0x5565315700c0 2024-09-12T03:35:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.241+0000 7f7753fff640 1 -- 172.21.15.64:0/1028679243 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f7764039040 con 0x5565315700c0 2024-09-12T03:35:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.241+0000 7f777da42740 10 client.15852 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:35:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.241+0000 7f777da42740 10 client.15852 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:35:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.241+0000 7f777da42740 20 client.15852 populate_metadata read hostname 'smithi064' 2024-09-12T03:35:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.241+0000 7f777da42740 1 --2- 172.21.15.64:0/1028679243 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5565315ebe30 0x5565315ee220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.241+0000 7f777da42740 4 client.15852 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:35:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.241+0000 7f777da42740 1 -- 172.21.15.64:0/1028679243 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5565314ccd70 con 0x5565315ebe30 2024-09-12T03:35:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.242+0000 7f7778969640 1 --2- 172.21.15.64:0/1028679243 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5565315ebe30 0x5565315ee220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.242+0000 7f7778969640 1 --2- 172.21.15.64:0/1028679243 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5565315ebe30 0x5565315ee220 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a90373559f2afa4 server_cookie=ac3feb777ffaf4eb in_seq=0 out_seq=0 2024-09-12T03:35:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.242+0000 7f7753fff640 10 client.15852 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:35:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.256+0000 7f7753fff640 1 -- 172.21.15.64:0/1028679243 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5565314ccd70 con 0x5565315ebe30 2024-09-12T03:35:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.256+0000 7f7753fff640 10 client.15852 handle_command_reply: tid=0 2024-09-12T03:35:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.256+0000 7f777da42740 1 client.15852 shutdown 2024-09-12T03:35:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.256+0000 7f777da42740 1 -- 172.21.15.64:0/1028679243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565315700c0 msgr2=0x5565315e3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:35:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.256+0000 7f777da42740 1 --2- 172.21.15.64:0/1028679243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565315700c0 0x5565315e3e80 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f7764003ee0 tx=0x7f77640037e0 comp rx=0 tx=0).stop 2024-09-12T03:35:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.256+0000 7f777da42740 1 -- 172.21.15.64:0/1028679243 shutdown_connections 2024-09-12T03:35:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.256+0000 7f777da42740 1 --2- 172.21.15.64:0/1028679243 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5565315ebe30 0x5565315ee220 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.256+0000 7f777da42740 1 --2- 172.21.15.64:0/1028679243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565315700c0 0x5565315e3e80 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.256+0000 7f777da42740 1 --2- 172.21.15.64:0/1028679243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565315612f0 0x556531574b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.256+0000 7f777da42740 1 --2- 172.21.15.64:0/1028679243 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565315609d0 0x556531574650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.256+0000 7f7753fff640 0 client.15852 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:35:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.256+0000 7f777da42740 1 -- 172.21.15.64:0/1028679243 >> 172.21.15.64:0/1028679243 conn(0x55653155d060 msgr2=0x55653155b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:35:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.256+0000 7f777da42740 1 -- 172.21.15.64:0/1028679243 shutdown_connections 2024-09-12T03:35:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.256+0000 7f777da42740 1 -- 172.21.15.64:0/1028679243 wait complete. 2024-09-12T03:35:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.256+0000 7f777da42740 20 client.15852 trim_cache size 0 max 16384 2024-09-12T03:35:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:35:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:35:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:35:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:35:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.257+0000 7f777abcd640 1 -- 172.21.15.64:0/2048189264 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f77600a2e20 msgr2=0x7f77600a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:35:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.257+0000 7f777abcd640 1 --2- 172.21.15.64:0/2048189264 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f77600a2e20 0x7f77600a52e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f776c0cce70 tx=0x7f776c06d6b0 comp rx=0 tx=0).stop 2024-09-12T03:35:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.257+0000 7f777abcd640 1 -- 172.21.15.64:0/2048189264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f777410c0d0 msgr2=0x7f777419cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:35:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.257+0000 7f777abcd640 1 --2- 172.21.15.64:0/2048189264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f777410c0d0 0x7f777419cb30 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f776c009fd0 tx=0x7f776c00b260 comp rx=0 tx=0).stop 2024-09-12T03:35:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.257+0000 7f777abcd640 1 -- 172.21.15.64:0/2048189264 shutdown_connections 2024-09-12T03:35:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.257+0000 7f777abcd640 1 --2- 172.21.15.64:0/2048189264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f777410c0d0 0x7f777419cb30 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.257+0000 7f777abcd640 1 --2- 172.21.15.64:0/2048189264 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f77600a2e20 0x7f77600a52e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.257+0000 7f777abcd640 1 --2- 172.21.15.64:0/2048189264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77741035a0 0x7f77741987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.257+0000 7f777abcd640 1 --2- 172.21.15.64:0/2048189264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7774102c80 0x7f7774198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.257+0000 7f777abcd640 1 -- 172.21.15.64:0/2048189264 >> 172.21.15.64:0/2048189264 conn(0x7f7774077990 msgr2=0x7f77740fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:35:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.257+0000 7f777abcd640 1 -- 172.21.15.64:0/2048189264 shutdown_connections 2024-09-12T03:35:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:26.257+0000 7f777abcd640 1 -- 172.21.15.64:0/2048189264 wait complete. 2024-09-12T03:35:27.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:35:27 smithi115 ceph-mon[34801]: pgmap v188: 129 pgs: 129 active+clean; 5.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-12T03:35:27.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:35:27 smithi195 ceph-mon[34786]: pgmap v188: 129 pgs: 129 active+clean; 5.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-12T03:35:27.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:35:27 smithi064 ceph-mon[30952]: pgmap v188: 129 pgs: 129 active+clean; 5.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-12T03:35:29.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:35:29 smithi115 ceph-mon[34801]: pgmap v189: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 63 KiB/s wr, 21 op/s 2024-09-12T03:35:29.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:35:29 smithi195 ceph-mon[34786]: pgmap v189: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 63 KiB/s wr, 21 op/s 2024-09-12T03:35:29.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:35:29 smithi064 ceph-mon[30952]: pgmap v189: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 63 KiB/s wr, 21 op/s 2024-09-12T03:35:31.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:35:31 smithi115 ceph-mon[34801]: pgmap v190: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:35:31.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:35:31 smithi195 ceph-mon[34786]: pgmap v190: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:35:31.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:35:31 smithi064 ceph-mon[30952]: pgmap v190: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:35:33.616 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.616 INFO:tasks.workunit.client.0.smithi064.stdout:251/600: open fd = 3 2024-09-12T03:35:33.616 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.616 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.616 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.616 INFO:tasks.workunit.client.0.smithi064.stdout:252/600: open fd = 3 2024-09-12T03:35:33.617 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.617 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.617 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.617 INFO:tasks.workunit.client.0.smithi064.stdout:253/600: open fd = 3 2024-09-12T03:35:33.617 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.617 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.617 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.617 INFO:tasks.workunit.client.0.smithi064.stdout:254/600: open fd = 3 2024-09-12T03:35:33.617 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.617 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.617 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.617 INFO:tasks.workunit.client.0.smithi064.stdout:255/600: open fd = 3 2024-09-12T03:35:33.617 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.618 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.618 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.618 INFO:tasks.workunit.client.0.smithi064.stdout:256/600: open fd = 3 2024-09-12T03:35:33.618 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.618 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.618 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.618 INFO:tasks.workunit.client.0.smithi064.stdout:257/600: open fd = 3 2024-09-12T03:35:33.618 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.618 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.618 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.618 INFO:tasks.workunit.client.0.smithi064.stdout:258/600: open fd = 3 2024-09-12T03:35:33.618 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.618 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.619 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.619 INFO:tasks.workunit.client.0.smithi064.stdout:259/600: open fd = 3 2024-09-12T03:35:33.619 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.619 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.619 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.619 INFO:tasks.workunit.client.0.smithi064.stdout:260/600: open fd = 3 2024-09-12T03:35:33.619 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.619 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.619 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.619 INFO:tasks.workunit.client.0.smithi064.stdout:261/600: open fd = 3 2024-09-12T03:35:33.619 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.619 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.619 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.619 INFO:tasks.workunit.client.0.smithi064.stdout:262/600: open fd = 3 2024-09-12T03:35:33.619 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.620 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.620 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.620 INFO:tasks.workunit.client.0.smithi064.stdout:263/600: open fd = 3 2024-09-12T03:35:33.620 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.620 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.620 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.620 INFO:tasks.workunit.client.0.smithi064.stdout:264/600: open fd = 3 2024-09-12T03:35:33.620 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.620 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.620 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.620 INFO:tasks.workunit.client.0.smithi064.stdout:265/600: open fd = 3 2024-09-12T03:35:33.620 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.620 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.621 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.621 INFO:tasks.workunit.client.0.smithi064.stdout:266/600: open fd = 3 2024-09-12T03:35:33.621 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.621 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.621 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.621 INFO:tasks.workunit.client.0.smithi064.stdout:267/600: open fd = 3 2024-09-12T03:35:33.621 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.621 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.621 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.621 INFO:tasks.workunit.client.0.smithi064.stdout:268/600: open fd = 3 2024-09-12T03:35:33.621 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.621 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.621 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.621 INFO:tasks.workunit.client.0.smithi064.stdout:269/600: open fd = 3 2024-09-12T03:35:33.622 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.622 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.622 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.622 INFO:tasks.workunit.client.0.smithi064.stdout:270/600: open fd = 3 2024-09-12T03:35:33.622 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.622 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.622 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.622 INFO:tasks.workunit.client.0.smithi064.stdout:271/600: open fd = 3 2024-09-12T03:35:33.622 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.622 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.622 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.622 INFO:tasks.workunit.client.0.smithi064.stdout:272/600: open fd = 3 2024-09-12T03:35:33.622 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.622 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.623 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.623 INFO:tasks.workunit.client.0.smithi064.stdout:273/600: open fd = 3 2024-09-12T03:35:33.623 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.623 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.623 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.623 INFO:tasks.workunit.client.0.smithi064.stdout:274/600: open fd = 3 2024-09-12T03:35:33.623 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.623 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.623 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.623 INFO:tasks.workunit.client.0.smithi064.stdout:275/600: open fd = 3 2024-09-12T03:35:33.623 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.623 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.623 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.623 INFO:tasks.workunit.client.0.smithi064.stdout:276/600: open fd = 3 2024-09-12T03:35:33.623 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.624 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.624 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.624 INFO:tasks.workunit.client.0.smithi064.stdout:277/600: open fd = 3 2024-09-12T03:35:33.624 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.624 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.624 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.624 INFO:tasks.workunit.client.0.smithi064.stdout:278/600: open fd = 3 2024-09-12T03:35:33.624 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.624 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.624 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.624 INFO:tasks.workunit.client.0.smithi064.stdout:279/600: open fd = 3 2024-09-12T03:35:33.624 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.624 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.625 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.625 INFO:tasks.workunit.client.0.smithi064.stdout:280/600: open fd = 3 2024-09-12T03:35:33.625 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.625 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.625 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.625 INFO:tasks.workunit.client.0.smithi064.stdout:281/600: open fd = 3 2024-09-12T03:35:33.625 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.625 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.625 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.625 INFO:tasks.workunit.client.0.smithi064.stdout:282/600: open fd = 3 2024-09-12T03:35:33.625 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.625 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.625 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.626 INFO:tasks.workunit.client.0.smithi064.stdout:283/600: open fd = 3 2024-09-12T03:35:33.626 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.626 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.626 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.626 INFO:tasks.workunit.client.0.smithi064.stdout:284/600: open fd = 3 2024-09-12T03:35:33.626 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.626 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.626 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.626 INFO:tasks.workunit.client.0.smithi064.stdout:285/600: open fd = 3 2024-09-12T03:35:33.626 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.626 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.626 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.626 INFO:tasks.workunit.client.0.smithi064.stdout:286/600: open fd = 3 2024-09-12T03:35:33.626 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.626 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.627 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.627 INFO:tasks.workunit.client.0.smithi064.stdout:287/600: open fd = 3 2024-09-12T03:35:33.627 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.627 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.627 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.627 INFO:tasks.workunit.client.0.smithi064.stdout:288/600: open fd = 3 2024-09-12T03:35:33.627 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.627 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.627 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.627 INFO:tasks.workunit.client.0.smithi064.stdout:289/600: open fd = 3 2024-09-12T03:35:33.627 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.627 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.627 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.627 INFO:tasks.workunit.client.0.smithi064.stdout:290/600: open fd = 3 2024-09-12T03:35:33.628 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.628 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.628 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.628 INFO:tasks.workunit.client.0.smithi064.stdout:291/600: open fd = 3 2024-09-12T03:35:33.628 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.628 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.628 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.628 INFO:tasks.workunit.client.0.smithi064.stdout:292/600: open fd = 3 2024-09-12T03:35:33.628 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.628 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.628 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.628 INFO:tasks.workunit.client.0.smithi064.stdout:293/600: open fd = 3 2024-09-12T03:35:33.628 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.628 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.629 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.629 INFO:tasks.workunit.client.0.smithi064.stdout:294/600: open fd = 3 2024-09-12T03:35:33.629 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.629 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.629 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.629 INFO:tasks.workunit.client.0.smithi064.stdout:295/600: open fd = 3 2024-09-12T03:35:33.629 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.629 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.629 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.629 INFO:tasks.workunit.client.0.smithi064.stdout:296/600: open fd = 3 2024-09-12T03:35:33.630 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.630 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.630 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.630 INFO:tasks.workunit.client.0.smithi064.stdout:297/600: open fd = 3 2024-09-12T03:35:33.630 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.630 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.630 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.630 INFO:tasks.workunit.client.0.smithi064.stdout:298/600: open fd = 3 2024-09-12T03:35:33.630 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.630 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.630 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.630 INFO:tasks.workunit.client.0.smithi064.stdout:299/600: open fd = 3 2024-09-12T03:35:33.630 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.630 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.631 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.631 INFO:tasks.workunit.client.0.smithi064.stdout:300/600: open fd = 3 2024-09-12T03:35:33.631 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.631 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.631 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.631 INFO:tasks.workunit.client.0.smithi064.stdout:301/600: open fd = 3 2024-09-12T03:35:33.631 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.631 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.631 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.631 INFO:tasks.workunit.client.0.smithi064.stdout:302/600: open fd = 3 2024-09-12T03:35:33.631 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.631 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.631 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.631 INFO:tasks.workunit.client.0.smithi064.stdout:303/600: open fd = 3 2024-09-12T03:35:33.632 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.632 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.632 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.632 INFO:tasks.workunit.client.0.smithi064.stdout:304/600: open fd = 3 2024-09-12T03:35:33.632 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.632 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.632 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.632 INFO:tasks.workunit.client.0.smithi064.stdout:305/600: open fd = 3 2024-09-12T03:35:33.632 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.632 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.632 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.632 INFO:tasks.workunit.client.0.smithi064.stdout:306/600: open fd = 3 2024-09-12T03:35:33.632 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.633 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.633 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.633 INFO:tasks.workunit.client.0.smithi064.stdout:307/600: open fd = 3 2024-09-12T03:35:33.633 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.633 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.633 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.633 INFO:tasks.workunit.client.0.smithi064.stdout:308/600: open fd = 3 2024-09-12T03:35:33.633 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.633 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.633 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.633 INFO:tasks.workunit.client.0.smithi064.stdout:309/600: open fd = 3 2024-09-12T03:35:33.633 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.633 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.633 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.634 INFO:tasks.workunit.client.0.smithi064.stdout:310/600: open fd = 3 2024-09-12T03:35:33.634 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.634 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.634 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.634 INFO:tasks.workunit.client.0.smithi064.stdout:311/600: open fd = 3 2024-09-12T03:35:33.634 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.634 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.634 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:35:33.634 INFO:tasks.workunit.client.0.smithi064.stdout:312/600: open fd = 3 2024-09-12T03:35:33.634 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.634 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:35:33.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:35:33 smithi115 ceph-mon[34801]: pgmap v191: 129 pgs: 129 active+clean; 1.6 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:35:33.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:35:33 smithi195 ceph-mon[34786]: pgmap v191: 129 pgs: 129 active+clean; 1.6 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:35:33.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:35:33 smithi064 ceph-mon[30952]: pgmap v191: 129 pgs: 129 active+clean; 1.6 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:35:35.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:35:35 smithi115 ceph-mon[34801]: pgmap v192: 129 pgs: 129 active+clean; 1.6 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:35:35.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:35:35 smithi195 ceph-mon[34786]: pgmap v192: 129 pgs: 129 active+clean; 1.6 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:35:35.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:35:35 smithi064 ceph-mon[30952]: pgmap v192: 129 pgs: 129 active+clean; 1.6 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:35:37.574 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:35:37 smithi064 ceph-mon[30952]: pgmap v193: 129 pgs: 129 active+clean; 1.6 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:35:37.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:35:37 smithi115 ceph-mon[34801]: pgmap v193: 129 pgs: 129 active+clean; 1.6 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:35:37.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:35:37 smithi195 ceph-mon[34786]: pgmap v193: 129 pgs: 129 active+clean; 1.6 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:35:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.357+0000 7f4766711640 1 -- 172.21.15.64:0/3802132265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4760103680 msgr2=0x7f476010bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:35:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.357+0000 7f4766711640 1 --2- 172.21.15.64:0/3802132265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4760103680 0x7f476010bb60 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f4754009900 tx=0x7f475402a480 comp rx=0 tx=0).stop 2024-09-12T03:35:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.357+0000 7f4766711640 1 -- 172.21.15.64:0/3802132265 shutdown_connections 2024-09-12T03:35:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.357+0000 7f4766711640 1 --2- 172.21.15.64:0/3802132265 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f476010c130 0x7f476010e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.357+0000 7f4766711640 1 --2- 172.21.15.64:0/3802132265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4760103680 0x7f476010bb60 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.357+0000 7f4766711640 1 --2- 172.21.15.64:0/3802132265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4760102d60 0x7f4760103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.357+0000 7f4766711640 1 -- 172.21.15.64:0/3802132265 >> 172.21.15.64:0/3802132265 conn(0x7f47600fc6b0 msgr2=0x7f47600fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:35:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.358+0000 7f4766711640 1 -- 172.21.15.64:0/3802132265 shutdown_connections 2024-09-12T03:35:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.358+0000 7f4766711640 1 -- 172.21.15.64:0/3802132265 wait complete. 2024-09-12T03:35:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.358+0000 7f4766711640 1 Processor -- start 2024-09-12T03:35:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.358+0000 7f4766711640 1 -- start start 2024-09-12T03:35:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.359+0000 7f4766711640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4760102d60 0x7f47601982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.359+0000 7f4766711640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4760103680 0x7f4760198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.359+0000 7f4766711640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f476010c130 0x7f476019cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.359+0000 7f4766711640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4760111db0 con 0x7f4760102d60 2024-09-12T03:35:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.359+0000 7f4766711640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4760111c30 con 0x7f4760103680 2024-09-12T03:35:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.359+0000 7f4766711640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4760111f30 con 0x7f476010c130 2024-09-12T03:35:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.359+0000 7f475ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4760102d60 0x7f47601982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.359+0000 7f475ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4760102d60 0x7f47601982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37782/0 (socket says 172.21.15.64:37782) 2024-09-12T03:35:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.359+0000 7f475ffff640 1 -- 172.21.15.64:0/1791472098 learned_addr learned my addr 172.21.15.64:0/1791472098 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:35:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.359+0000 7f475f7fe640 1 --2- 172.21.15.64:0/1791472098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4760103680 0x7f4760198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.359+0000 7f4764cae640 1 --2- 172.21.15.64:0/1791472098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f476010c130 0x7f476019cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.359+0000 7f475ffff640 1 -- 172.21.15.64:0/1791472098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f476010c130 msgr2=0x7f476019cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:35:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.359+0000 7f475ffff640 1 --2- 172.21.15.64:0/1791472098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f476010c130 0x7f476019cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.359+0000 7f475ffff640 1 -- 172.21.15.64:0/1791472098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4760103680 msgr2=0x7f4760198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:35:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.359+0000 7f475ffff640 1 --2- 172.21.15.64:0/1791472098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4760103680 0x7f4760198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.359+0000 7f475ffff640 1 -- 172.21.15.64:0/1791472098 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f476019d2b0 con 0x7f4760102d60 2024-09-12T03:35:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.359+0000 7f4764cae640 1 --2- 172.21.15.64:0/1791472098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f476010c130 0x7f476019cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:35:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.359+0000 7f475ffff640 1 --2- 172.21.15.64:0/1791472098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4760102d60 0x7f47601982c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f475000d950 tx=0x7f475000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:35:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.360+0000 7f475d7fa640 1 -- 172.21.15.64:0/1791472098 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4750017070 con 0x7f4760102d60 2024-09-12T03:35:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.360+0000 7f4766711640 1 -- 172.21.15.64:0/1791472098 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f476019d5a0 con 0x7f4760102d60 2024-09-12T03:35:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.360+0000 7f475d7fa640 1 -- 172.21.15.64:0/1791472098 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4750009c30 con 0x7f4760102d60 2024-09-12T03:35:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.360+0000 7f475d7fa640 1 -- 172.21.15.64:0/1791472098 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475000f620 con 0x7f4760102d60 2024-09-12T03:35:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.360+0000 7f4766711640 1 -- 172.21.15.64:0/1791472098 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47601a4e30 con 0x7f4760102d60 2024-09-12T03:35:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.362+0000 7f475d7fa640 1 -- 172.21.15.64:0/1791472098 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f47500109a0 con 0x7f4760102d60 2024-09-12T03:35:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.363+0000 7f47694a8740 1 Processor -- start 2024-09-12T03:35:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.363+0000 7f47694a8740 1 -- start start 2024-09-12T03:35:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.363+0000 7f47694a8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9c9679af0 0x55c9c9679ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.363+0000 7f47694a8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9c967a410 0x55c9c9684770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.363+0000 7f47694a8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9c9684cb0 0x55c9c96870a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.363+0000 7f47694a8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c9c95de2f0 con 0x55c9c9684cb0 2024-09-12T03:35:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.363+0000 7f47694a8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c9c9586270 con 0x55c9c9679af0 2024-09-12T03:35:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.363+0000 7f47694a8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c9c95da2b0 con 0x55c9c967a410 2024-09-12T03:35:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.363+0000 7f475f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9c9684cb0 0x55c9c96870a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.363+0000 7f475f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9c9684cb0 0x55c9c96870a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37788/0 (socket says 172.21.15.64:37788) 2024-09-12T03:35:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.363+0000 7f475f7fe640 1 -- 172.21.15.64:0/1713290742 learned_addr learned my addr 172.21.15.64:0/1713290742 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:35:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.363+0000 7f4764cae640 1 --2- 172.21.15.64:0/1713290742 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9c9679af0 0x55c9c9679ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.363+0000 7f475d7fa640 1 --2- 172.21.15.64:0/1791472098 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f47480a2da0 0x7f47480a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.363+0000 7f475ffff640 1 --2- 172.21.15.64:0/1713290742 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9c967a410 0x55c9c9684770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.363+0000 7f475f7fe640 1 -- 172.21.15.64:0/1713290742 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9c967a410 msgr2=0x55c9c9684770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:35:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.363+0000 7f475f7fe640 1 --2- 172.21.15.64:0/1713290742 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9c967a410 0x55c9c9684770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.363+0000 7f475f7fe640 1 -- 172.21.15.64:0/1713290742 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9c9679af0 msgr2=0x55c9c9679ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:35:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.363+0000 7f475f7fe640 1 --2- 172.21.15.64:0/1713290742 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9c9679af0 0x55c9c9679ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.363+0000 7f4764cae640 1 --2- 172.21.15.64:0/1791472098 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f47480a2da0 0x7f47480a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.363+0000 7f475f7fe640 1 -- 172.21.15.64:0/1713290742 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9c9585340 con 0x55c9c9684cb0 2024-09-12T03:35:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.363+0000 7f475d7fa640 1 -- 172.21.15.64:0/1791472098 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f47500cf450 con 0x7f4760102d60 2024-09-12T03:35:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.364+0000 7f475f7fe640 1 --2- 172.21.15.64:0/1713290742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9c9684cb0 0x55c9c96870a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4754009900 tx=0x7f4754032b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f1dfb885663fdb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:35:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.364+0000 7f4764cae640 1 --2- 172.21.15.64:0/1791472098 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f47480a2da0 0x7f47480a5260 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f47580092f0 tx=0x7f4758004ab0 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:35:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.364+0000 7f47467fc640 1 -- 172.21.15.64:0/1713290742 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4754040070 con 0x55c9c9684cb0 2024-09-12T03:35:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.364+0000 7f47467fc640 1 -- 172.21.15.64:0/1713290742 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4754004030 con 0x55c9c9684cb0 2024-09-12T03:35:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.364+0000 7f47467fc640 1 -- 172.21.15.64:0/1713290742 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4754035040 con 0x55c9c9684cb0 2024-09-12T03:35:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.365+0000 7f47694a8740 1 -- 172.21.15.64:0/1713290742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9c9684cb0 msgr2=0x55c9c96870a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:35:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.365+0000 7f47694a8740 1 --2- 172.21.15.64:0/1713290742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9c9684cb0 0x55c9c96870a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4754009900 tx=0x7f4754032b70 comp rx=0 tx=0).stop 2024-09-12T03:35:39.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.365+0000 7f47694a8740 1 -- 172.21.15.64:0/1713290742 shutdown_connections 2024-09-12T03:35:39.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.365+0000 7f47694a8740 1 --2- 172.21.15.64:0/1713290742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9c9684cb0 0x55c9c96870a0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:39.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.365+0000 7f47694a8740 1 --2- 172.21.15.64:0/1713290742 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9c967a410 0x55c9c9684770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:39.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.365+0000 7f47694a8740 1 --2- 172.21.15.64:0/1713290742 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9c9679af0 0x55c9c9679ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:39.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.365+0000 7f47694a8740 1 -- 172.21.15.64:0/1713290742 >> 172.21.15.64:0/1713290742 conn(0x55c9c9676060 msgr2=0x55c9c9676440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:35:39.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.365+0000 7f47694a8740 1 -- 172.21.15.64:0/1713290742 shutdown_connections 2024-09-12T03:35:39.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.365+0000 7f47694a8740 1 -- 172.21.15.64:0/1713290742 wait complete. 2024-09-12T03:35:39.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.365+0000 7f47694a8740 1 -- start start 2024-09-12T03:35:39.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.365+0000 7f47694a8740 1 Processor -- start 2024-09-12T03:35:39.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.366+0000 7f47694a8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9c9679af0 0x55c9c9681be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:39.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.366+0000 7f47694a8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9c967a410 0x55c9c9682120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:39.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.366+0000 7f47694a8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9c9684cb0 0x55c9c9682660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.366+0000 7f47694a8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c9c9687880 con 0x55c9c967a410 2024-09-12T03:35:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.366+0000 7f47694a8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c9c9567290 con 0x55c9c9684cb0 2024-09-12T03:35:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.366+0000 7f47694a8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c9c9687a00 con 0x55c9c9679af0 2024-09-12T03:35:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.366+0000 7f47694a8740 10 client.0 _ll_register_callbacks cb 0x55c9c9598140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:35:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.366+0000 7f475f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9c967a410 0x55c9c9682120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.366+0000 7f475f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9c967a410 0x55c9c9682120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37802/0 (socket says 172.21.15.64:37802) 2024-09-12T03:35:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.366+0000 7f475f7fe640 1 -- 172.21.15.64:0/3364300121 learned_addr learned my addr 172.21.15.64:0/3364300121 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:35:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.366+0000 7f4764cae640 1 --2- 172.21.15.64:0/3364300121 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9c9684cb0 0x55c9c9682660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.366+0000 7f475ffff640 1 --2- 172.21.15.64:0/3364300121 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9c9679af0 0x55c9c9681be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.366+0000 7f475f7fe640 1 -- 172.21.15.64:0/3364300121 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9c9679af0 msgr2=0x55c9c9681be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:35:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.366+0000 7f475f7fe640 1 --2- 172.21.15.64:0/3364300121 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9c9679af0 0x55c9c9681be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.366+0000 7f475f7fe640 1 -- 172.21.15.64:0/3364300121 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9c9684cb0 msgr2=0x55c9c9682660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:35:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.366+0000 7f475f7fe640 1 --2- 172.21.15.64:0/3364300121 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9c9684cb0 0x55c9c9682660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.366+0000 7f475f7fe640 1 -- 172.21.15.64:0/3364300121 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9c92f1210 con 0x55c9c967a410 2024-09-12T03:35:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.366+0000 7f475ffff640 1 --2- 172.21.15.64:0/3364300121 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9c9679af0 0x55c9c9681be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:35:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.366+0000 7f475f7fe640 1 --2- 172.21.15.64:0/3364300121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9c967a410 0x55c9c9682120 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f4754006010 tx=0x7f47540041e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc0d1bef9152646 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:35:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.366+0000 7f473f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:35:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.367+0000 7f473f7fe640 1 -- 172.21.15.64:0/3364300121 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4754040070 con 0x55c9c967a410 2024-09-12T03:35:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.367+0000 7f473f7fe640 1 -- 172.21.15.64:0/3364300121 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f47540054e0 con 0x55c9c967a410 2024-09-12T03:35:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.367+0000 7f47694a8740 1 -- 172.21.15.64:0/3364300121 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4754004030 con 0x55c9c967a410 2024-09-12T03:35:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.367+0000 7f473f7fe640 1 -- 172.21.15.64:0/3364300121 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4754056050 con 0x55c9c967a410 2024-09-12T03:35:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.367+0000 7f47694a8740 10 client.15864 fetch_fsmap learned FSMap version 25 2024-09-12T03:35:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.367+0000 7f47694a8740 1 -- 172.21.15.64:0/3364300121 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c9c93bd620 con 0x55c9c967a410 2024-09-12T03:35:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.367+0000 7f473f7fe640 1 -- 172.21.15.64:0/3364300121 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4754004030 con 0x55c9c967a410 2024-09-12T03:35:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.367+0000 7f473f7fe640 1 -- 172.21.15.64:0/3364300121 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4754003cb0 con 0x55c9c967a410 2024-09-12T03:35:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.367+0000 7f47694a8740 10 client.15864 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:35:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.367+0000 7f47694a8740 10 client.15864 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:35:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.367+0000 7f47694a8740 20 client.15864 populate_metadata read hostname 'smithi064' 2024-09-12T03:35:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.367+0000 7f47694a8740 1 --2- 172.21.15.64:0/3364300121 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55c9c9694b00 0x55c9c9696ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.367+0000 7f47694a8740 4 client.15864 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:35:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.367+0000 7f47694a8740 1 -- 172.21.15.64:0/3364300121 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9c95eba20 con 0x55c9c9694b00 2024-09-12T03:35:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.368+0000 7f475ffff640 1 --2- 172.21.15.64:0/3364300121 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55c9c9694b00 0x55c9c9696ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.368+0000 7f475ffff640 1 --2- 172.21.15.64:0/3364300121 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55c9c9694b00 0x55c9c9696ef0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f9c1f870743d5bf server_cookie=1bd53248a45c424e in_seq=0 out_seq=0 2024-09-12T03:35:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.368+0000 7f473f7fe640 10 client.15864 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:35:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.369+0000 7f473f7fe640 1 -- 172.21.15.64:0/3364300121 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c9c95eba20 con 0x55c9c9694b00 2024-09-12T03:35:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.369+0000 7f473f7fe640 10 client.15864 handle_command_reply: tid=0 2024-09-12T03:35:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.369+0000 7f47694a8740 1 client.15864 shutdown 2024-09-12T03:35:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.369+0000 7f47694a8740 1 -- 172.21.15.64:0/3364300121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9c967a410 msgr2=0x55c9c9682120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:35:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.369+0000 7f47694a8740 1 --2- 172.21.15.64:0/3364300121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9c967a410 0x55c9c9682120 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f4754006010 tx=0x7f47540041e0 comp rx=0 tx=0).stop 2024-09-12T03:35:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.369+0000 7f47694a8740 1 -- 172.21.15.64:0/3364300121 shutdown_connections 2024-09-12T03:35:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.369+0000 7f47694a8740 1 --2- 172.21.15.64:0/3364300121 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55c9c9694b00 0x55c9c9696ef0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.369+0000 7f47694a8740 1 --2- 172.21.15.64:0/3364300121 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9c9684cb0 0x55c9c9682660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.369+0000 7f47694a8740 1 --2- 172.21.15.64:0/3364300121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9c967a410 0x55c9c9682120 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.369+0000 7f47694a8740 1 --2- 172.21.15.64:0/3364300121 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9c9679af0 0x55c9c9681be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.369+0000 7f47694a8740 1 -- 172.21.15.64:0/3364300121 >> 172.21.15.64:0/3364300121 conn(0x55c9c9676060 msgr2=0x55c9c967bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:35:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.369+0000 7f473f7fe640 0 client.15864 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:35:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.369+0000 7f47694a8740 1 -- 172.21.15.64:0/3364300121 shutdown_connections 2024-09-12T03:35:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.369+0000 7f47694a8740 1 -- 172.21.15.64:0/3364300121 wait complete. 2024-09-12T03:35:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.369+0000 7f47694a8740 20 client.15864 trim_cache size 0 max 16384 2024-09-12T03:35:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.384+0000 7f47694a8740 1 Processor -- start 2024-09-12T03:35:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.384+0000 7f47694a8740 1 -- start start 2024-09-12T03:35:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.384+0000 7f47694a8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9c9679af0 0x55c9c9682490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.384+0000 7f47694a8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9c967a410 0x55c9c96750b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.384+0000 7f47694a8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9c9684cb0 0x55c9c969c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.384+0000 7f47694a8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f47601b18a0 con 0x55c9c9679af0 2024-09-12T03:35:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.384+0000 7f47694a8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f47601af170 con 0x55c9c9684cb0 2024-09-12T03:35:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.384+0000 7f47694a8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f47601acfc0 con 0x55c9c967a410 2024-09-12T03:35:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.384+0000 7f475ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9c9679af0 0x55c9c9682490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.384+0000 7f475ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9c9679af0 0x55c9c9682490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37810/0 (socket says 172.21.15.64:37810) 2024-09-12T03:35:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.384+0000 7f475ffff640 1 -- 172.21.15.64:0/1486692041 learned_addr learned my addr 172.21.15.64:0/1486692041 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:35:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.384+0000 7f475ffff640 1 -- 172.21.15.64:0/1486692041 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9c967a410 msgr2=0x55c9c96750b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:35:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.384+0000 7f475ffff640 1 --2- 172.21.15.64:0/1486692041 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9c967a410 0x55c9c96750b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.384+0000 7f475ffff640 1 -- 172.21.15.64:0/1486692041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9c9684cb0 msgr2=0x55c9c969c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:35:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.384+0000 7f475ffff640 1 --2- 172.21.15.64:0/1486692041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9c9684cb0 0x55c9c969c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.384+0000 7f475ffff640 1 -- 172.21.15.64:0/1486692041 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9c96702f0 con 0x55c9c9679af0 2024-09-12T03:35:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.385+0000 7f475ffff640 1 --2- 172.21.15.64:0/1486692041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9c9679af0 0x55c9c9682490 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f4750097f70 tx=0x7f4750086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ee4a7a05004d44 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:35:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.385+0000 7f4745ffb640 1 -- 172.21.15.64:0/1486692041 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4750020040 con 0x55c9c9679af0 2024-09-12T03:35:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.385+0000 7f4745ffb640 1 -- 172.21.15.64:0/1486692041 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f475009f950 con 0x55c9c9679af0 2024-09-12T03:35:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.385+0000 7f4745ffb640 1 -- 172.21.15.64:0/1486692041 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4750097c10 con 0x55c9c9679af0 2024-09-12T03:35:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.385+0000 7f47694a8740 1 -- 172.21.15.64:0/1486692041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9c9679af0 msgr2=0x55c9c9682490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:35:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.385+0000 7f47694a8740 1 --2- 172.21.15.64:0/1486692041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9c9679af0 0x55c9c9682490 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f4750097f70 tx=0x7f4750086040 comp rx=0 tx=0).stop 2024-09-12T03:35:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.385+0000 7f47694a8740 1 -- 172.21.15.64:0/1486692041 shutdown_connections 2024-09-12T03:35:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.385+0000 7f47694a8740 1 --2- 172.21.15.64:0/1486692041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9c9684cb0 0x55c9c969c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.385+0000 7f47694a8740 1 --2- 172.21.15.64:0/1486692041 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9c967a410 0x55c9c96750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.385+0000 7f47694a8740 1 --2- 172.21.15.64:0/1486692041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9c9679af0 0x55c9c9682490 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.385+0000 7f47694a8740 1 -- 172.21.15.64:0/1486692041 >> 172.21.15.64:0/1486692041 conn(0x55c9c9676060 msgr2=0x55c9c96749a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:35:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.385+0000 7f47694a8740 1 -- 172.21.15.64:0/1486692041 shutdown_connections 2024-09-12T03:35:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.385+0000 7f47694a8740 1 -- 172.21.15.64:0/1486692041 wait complete. 2024-09-12T03:35:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.386+0000 7f47694a8740 1 -- start start 2024-09-12T03:35:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.386+0000 7f47694a8740 1 Processor -- start 2024-09-12T03:35:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.386+0000 7f47694a8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9c9679af0 0x55c9c967ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.386+0000 7f47694a8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9c967a410 0x55c9c967f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.386+0000 7f47694a8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9c9684cb0 0x55c9c967f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.386+0000 7f47694a8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f47601ad630 con 0x55c9c967a410 2024-09-12T03:35:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.386+0000 7f47694a8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f47601aeaa0 con 0x55c9c9684cb0 2024-09-12T03:35:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.386+0000 7f47694a8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f47601add00 con 0x55c9c9679af0 2024-09-12T03:35:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.386+0000 7f47694a8740 10 client.0 _ll_register_callbacks cb 0x7f4754032ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:35:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.386+0000 7f475f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9c967a410 0x55c9c967f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.386+0000 7f475f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9c967a410 0x55c9c967f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37812/0 (socket says 172.21.15.64:37812) 2024-09-12T03:35:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.386+0000 7f475f7fe640 1 -- 172.21.15.64:0/141996008 learned_addr learned my addr 172.21.15.64:0/141996008 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:35:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.386+0000 7f475ffff640 1 --2- 172.21.15.64:0/141996008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9c9679af0 0x55c9c967ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.387+0000 7f4764cae640 1 --2- 172.21.15.64:0/141996008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9c9684cb0 0x55c9c967f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.387+0000 7f475f7fe640 1 -- 172.21.15.64:0/141996008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9c9679af0 msgr2=0x55c9c967ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:35:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.387+0000 7f475f7fe640 1 --2- 172.21.15.64:0/141996008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9c9679af0 0x55c9c967ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.387+0000 7f475f7fe640 1 -- 172.21.15.64:0/141996008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9c9684cb0 msgr2=0x55c9c967f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:35:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.387+0000 7f475f7fe640 1 --2- 172.21.15.64:0/141996008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9c9684cb0 0x55c9c967f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.387+0000 7f475f7fe640 1 -- 172.21.15.64:0/141996008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9c96fe5c0 con 0x55c9c967a410 2024-09-12T03:35:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.387+0000 7f475ffff640 1 --2- 172.21.15.64:0/141996008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9c9679af0 0x55c9c967ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:35:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.387+0000 7f4764cae640 1 --2- 172.21.15.64:0/141996008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9c9684cb0 0x55c9c967f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:35:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.387+0000 7f475f7fe640 1 --2- 172.21.15.64:0/141996008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9c967a410 0x55c9c967f250 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f47540047a0 tx=0x7f4754004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de5f3b3582d6f28d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:35:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.387+0000 7f473f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:35:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.387+0000 7f473f7fe640 1 -- 172.21.15.64:0/141996008 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475405f060 con 0x55c9c967a410 2024-09-12T03:35:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.387+0000 7f473f7fe640 1 -- 172.21.15.64:0/141996008 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f47540051f0 con 0x55c9c967a410 2024-09-12T03:35:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.387+0000 7f47694a8740 1 -- 172.21.15.64:0/141996008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f475009f950 con 0x55c9c967a410 2024-09-12T03:35:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.387+0000 7f473f7fe640 1 -- 172.21.15.64:0/141996008 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4754040070 con 0x55c9c967a410 2024-09-12T03:35:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.387+0000 7f47694a8740 10 client.15870 fetch_fsmap learned FSMap version 25 2024-09-12T03:35:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.387+0000 7f47694a8740 1 -- 172.21.15.64:0/141996008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c9c96ff8b0 con 0x55c9c967a410 2024-09-12T03:35:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.387+0000 7f473f7fe640 1 -- 172.21.15.64:0/141996008 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f475009f950 con 0x55c9c967a410 2024-09-12T03:35:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.388+0000 7f473f7fe640 1 -- 172.21.15.64:0/141996008 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4754032720 con 0x55c9c967a410 2024-09-12T03:35:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.388+0000 7f47694a8740 10 client.15870 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:35:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.388+0000 7f47694a8740 10 client.15870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:35:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.388+0000 7f47694a8740 20 client.15870 populate_metadata read hostname 'smithi064' 2024-09-12T03:35:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.388+0000 7f47694a8740 1 --2- 172.21.15.64:0/141996008 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55c9c9700000 0x55c9c9702420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.388+0000 7f47694a8740 4 client.15870 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:35:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.388+0000 7f47694a8740 1 -- 172.21.15.64:0/141996008 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c9c95e5d70 con 0x55c9c9700000 2024-09-12T03:35:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.388+0000 7f475ffff640 1 --2- 172.21.15.64:0/141996008 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55c9c9700000 0x55c9c9702420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.388+0000 7f475ffff640 1 --2- 172.21.15.64:0/141996008 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55c9c9700000 0x55c9c9702420 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63372bd46961d54b server_cookie=ad135862edd4f457 in_seq=0 out_seq=0 2024-09-12T03:35:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.388+0000 7f473f7fe640 10 client.15870 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:35:39.403 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.403+0000 7f473f7fe640 1 -- 172.21.15.64:0/141996008 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c9c95e5d70 con 0x55c9c9700000 2024-09-12T03:35:39.403 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.403+0000 7f473f7fe640 10 client.15870 handle_command_reply: tid=0 2024-09-12T03:35:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.403+0000 7f47694a8740 1 client.15870 shutdown 2024-09-12T03:35:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.403+0000 7f47694a8740 1 -- 172.21.15.64:0/141996008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9c967a410 msgr2=0x55c9c967f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:35:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.403+0000 7f47694a8740 1 --2- 172.21.15.64:0/141996008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9c967a410 0x55c9c967f250 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f47540047a0 tx=0x7f4754004cc0 comp rx=0 tx=0).stop 2024-09-12T03:35:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.403+0000 7f47694a8740 1 -- 172.21.15.64:0/141996008 shutdown_connections 2024-09-12T03:35:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.403+0000 7f47694a8740 1 --2- 172.21.15.64:0/141996008 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55c9c9700000 0x55c9c9702420 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.403+0000 7f47694a8740 1 --2- 172.21.15.64:0/141996008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9c9684cb0 0x55c9c967f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.403+0000 7f473f7fe640 0 client.15870 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:35:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.403+0000 7f47694a8740 1 --2- 172.21.15.64:0/141996008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9c967a410 0x55c9c967f250 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.403+0000 7f47694a8740 1 --2- 172.21.15.64:0/141996008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9c9679af0 0x55c9c967ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.403+0000 7f47694a8740 1 -- 172.21.15.64:0/141996008 >> 172.21.15.64:0/141996008 conn(0x55c9c9676060 msgr2=0x55c9c967bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:35:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.403+0000 7f47694a8740 1 -- 172.21.15.64:0/141996008 shutdown_connections 2024-09-12T03:35:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.403+0000 7f47694a8740 1 -- 172.21.15.64:0/141996008 wait complete. 2024-09-12T03:35:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.403+0000 7f47694a8740 20 client.15870 trim_cache size 0 max 16384 2024-09-12T03:35:39.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:35:39.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:35:39.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:35:39.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:35:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.404+0000 7f4766711640 1 -- 172.21.15.64:0/1791472098 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f47480a2da0 msgr2=0x7f47480a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:35:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.404+0000 7f4766711640 1 --2- 172.21.15.64:0/1791472098 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f47480a2da0 0x7f47480a5260 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f47580092f0 tx=0x7f4758004ab0 comp rx=0 tx=0).stop 2024-09-12T03:35:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.404+0000 7f4766711640 1 -- 172.21.15.64:0/1791472098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4760102d60 msgr2=0x7f47601982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:35:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.404+0000 7f4766711640 1 --2- 172.21.15.64:0/1791472098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4760102d60 0x7f47601982c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f475000d950 tx=0x7f475000de10 comp rx=0 tx=0).stop 2024-09-12T03:35:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.404+0000 7f4766711640 1 -- 172.21.15.64:0/1791472098 shutdown_connections 2024-09-12T03:35:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.404+0000 7f4766711640 1 --2- 172.21.15.64:0/1791472098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f476010c130 0x7f476019cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.404+0000 7f4766711640 1 --2- 172.21.15.64:0/1791472098 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f47480a2da0 0x7f47480a5260 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.404+0000 7f4766711640 1 --2- 172.21.15.64:0/1791472098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4760103680 0x7f4760198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.404+0000 7f4766711640 1 --2- 172.21.15.64:0/1791472098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4760102d60 0x7f47601982c0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.404+0000 7f4766711640 1 -- 172.21.15.64:0/1791472098 >> 172.21.15.64:0/1791472098 conn(0x7f47600fc6b0 msgr2=0x7f47600fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:35:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.404+0000 7f4766711640 1 -- 172.21.15.64:0/1791472098 shutdown_connections 2024-09-12T03:35:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:39.404+0000 7f4766711640 1 -- 172.21.15.64:0/1791472098 wait complete. 2024-09-12T03:35:39.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:35:39 smithi115 ceph-mon[34801]: pgmap v194: 129 pgs: 129 active+clean; 1.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-09-12T03:35:39.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:35:39 smithi195 ceph-mon[34786]: pgmap v194: 129 pgs: 129 active+clean; 1.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-09-12T03:35:39.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:35:39 smithi064 ceph-mon[30952]: pgmap v194: 129 pgs: 129 active+clean; 1.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-09-12T03:35:41.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:35:41 smithi115 ceph-mon[34801]: pgmap v195: 129 pgs: 129 active+clean; 1.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-12T03:35:41.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:35:41 smithi195 ceph-mon[34786]: pgmap v195: 129 pgs: 129 active+clean; 1.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-12T03:35:41.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:35:41 smithi064 ceph-mon[30952]: pgmap v195: 129 pgs: 129 active+clean; 1.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-12T03:35:43.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:35:43 smithi115 ceph-mon[34801]: pgmap v196: 129 pgs: 129 active+clean; 1.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 17 op/s 2024-09-12T03:35:43.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:35:43 smithi195 ceph-mon[34786]: pgmap v196: 129 pgs: 129 active+clean; 1.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 17 op/s 2024-09-12T03:35:43.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:35:43 smithi064 ceph-mon[30952]: pgmap v196: 129 pgs: 129 active+clean; 1.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 17 op/s 2024-09-12T03:35:45.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:35:45 smithi115 ceph-mon[34801]: pgmap v197: 129 pgs: 129 active+clean; 1.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-09-12T03:35:45.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:35:45 smithi195 ceph-mon[34786]: pgmap v197: 129 pgs: 129 active+clean; 1.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-09-12T03:35:45.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:35:45 smithi064 ceph-mon[30952]: pgmap v197: 129 pgs: 129 active+clean; 1.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-09-12T03:35:47.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:35:47 smithi115 ceph-mon[34801]: pgmap v198: 129 pgs: 129 active+clean; 1.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-09-12T03:35:47.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:35:47 smithi195 ceph-mon[34786]: pgmap v198: 129 pgs: 129 active+clean; 1.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-09-12T03:35:47.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:35:47 smithi064 ceph-mon[30952]: pgmap v198: 129 pgs: 129 active+clean; 1.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-09-12T03:35:49.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:35:49 smithi115 ceph-mon[34801]: pgmap v199: 129 pgs: 129 active+clean; 2.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 17 op/s 2024-09-12T03:35:49.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:35:49 smithi195 ceph-mon[34786]: pgmap v199: 129 pgs: 129 active+clean; 2.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 17 op/s 2024-09-12T03:35:49.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:35:49 smithi064 ceph-mon[30952]: pgmap v199: 129 pgs: 129 active+clean; 2.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 17 op/s 2024-09-12T03:35:51.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:35:51 smithi115 ceph-mon[34801]: pgmap v200: 129 pgs: 129 active+clean; 2.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-12T03:35:51.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:35:51 smithi195 ceph-mon[34786]: pgmap v200: 129 pgs: 129 active+clean; 2.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-12T03:35:51.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:35:51 smithi064 ceph-mon[30952]: pgmap v200: 129 pgs: 129 active+clean; 2.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-12T03:35:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.504+0000 7f56ae186640 1 -- 172.21.15.64:0/705112283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56a810c110 msgr2=0x7f56a810e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:35:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.504+0000 7f56ae186640 1 --2- 172.21.15.64:0/705112283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56a810c110 0x7f56a810e590 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f56a0009ca0 tx=0x7f56a002a950 comp rx=0 tx=0).stop 2024-09-12T03:35:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.504+0000 7f56ae186640 1 -- 172.21.15.64:0/705112283 shutdown_connections 2024-09-12T03:35:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.504+0000 7f56ae186640 1 --2- 172.21.15.64:0/705112283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56a810c110 0x7f56a810e590 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.504+0000 7f56ae186640 1 --2- 172.21.15.64:0/705112283 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56a81035e0 0x7f56a810bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.504+0000 7f56ae186640 1 --2- 172.21.15.64:0/705112283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56a8102cc0 0x7f56a81030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.504+0000 7f56ae186640 1 -- 172.21.15.64:0/705112283 >> 172.21.15.64:0/705112283 conn(0x7f56a80fc6b0 msgr2=0x7f56a80fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:35:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.504+0000 7f56ae186640 1 -- 172.21.15.64:0/705112283 shutdown_connections 2024-09-12T03:35:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.504+0000 7f56ae186640 1 -- 172.21.15.64:0/705112283 wait complete. 2024-09-12T03:35:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.505+0000 7f56ae186640 1 Processor -- start 2024-09-12T03:35:52.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.505+0000 7f56ae186640 1 -- start start 2024-09-12T03:35:52.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.505+0000 7f56ae186640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56a8102cc0 0x7f56a81981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:52.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.505+0000 7f56ae186640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56a81035e0 0x7f56a8198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:52.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.505+0000 7f56ae186640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56a810c110 0x7f56a819ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:52.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.505+0000 7f56ae186640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f56a8111da0 con 0x7f56a8102cc0 2024-09-12T03:35:52.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.505+0000 7f56ae186640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f56a8111c20 con 0x7f56a810c110 2024-09-12T03:35:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.505+0000 7f56ae186640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f56a8111f20 con 0x7f56a81035e0 2024-09-12T03:35:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.505+0000 7f56a77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56a8102cc0 0x7f56a81981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.505+0000 7f56a77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56a8102cc0 0x7f56a81981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49578/0 (socket says 172.21.15.64:49578) 2024-09-12T03:35:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.505+0000 7f56a77fe640 1 -- 172.21.15.64:0/2898946149 learned_addr learned my addr 172.21.15.64:0/2898946149 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:35:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.505+0000 7f56a7fff640 1 --2- 172.21.15.64:0/2898946149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56a810c110 0x7f56a819ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.506+0000 7f56a6ffd640 1 --2- 172.21.15.64:0/2898946149 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56a81035e0 0x7f56a8198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.506+0000 7f56a77fe640 1 -- 172.21.15.64:0/2898946149 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56a81035e0 msgr2=0x7f56a8198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:35:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.506+0000 7f56a77fe640 1 --2- 172.21.15.64:0/2898946149 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56a81035e0 0x7f56a8198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.506+0000 7f56a77fe640 1 -- 172.21.15.64:0/2898946149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56a810c110 msgr2=0x7f56a819ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:35:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.506+0000 7f56a77fe640 1 --2- 172.21.15.64:0/2898946149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56a810c110 0x7f56a819ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.506+0000 7f56a77fe640 1 -- 172.21.15.64:0/2898946149 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a819d1c0 con 0x7f56a8102cc0 2024-09-12T03:35:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.506+0000 7f56a6ffd640 1 --2- 172.21.15.64:0/2898946149 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56a81035e0 0x7f56a8198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:35:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.506+0000 7f56a7fff640 1 --2- 172.21.15.64:0/2898946149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56a810c110 0x7f56a819ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:35:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.506+0000 7f56a77fe640 1 --2- 172.21.15.64:0/2898946149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56a8102cc0 0x7f56a81981d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f569800e9f0 tx=0x7f569800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:35:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.506+0000 7f56a4ff9640 1 -- 172.21.15.64:0/2898946149 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569800cd80 con 0x7f56a8102cc0 2024-09-12T03:35:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.506+0000 7f56a4ff9640 1 -- 172.21.15.64:0/2898946149 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56980108e0 con 0x7f56a8102cc0 2024-09-12T03:35:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.506+0000 7f56ae186640 1 -- 172.21.15.64:0/2898946149 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56a819d4b0 con 0x7f56a8102cc0 2024-09-12T03:35:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.506+0000 7f56a4ff9640 1 -- 172.21.15.64:0/2898946149 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5698018920 con 0x7f56a8102cc0 2024-09-12T03:35:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.506+0000 7f56ae186640 1 -- 172.21.15.64:0/2898946149 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56a81a4d40 con 0x7f56a8102cc0 2024-09-12T03:35:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.509+0000 7f56a4ff9640 1 -- 172.21.15.64:0/2898946149 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f5698010a90 con 0x7f56a8102cc0 2024-09-12T03:35:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.509+0000 7f56b0ff6740 1 Processor -- start 2024-09-12T03:35:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.509+0000 7f56b0ff6740 1 -- start start 2024-09-12T03:35:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.509+0000 7f56b0ff6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555618697af0 0x555618697ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.509+0000 7f56b0ff6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555618698410 0x5556186a2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.509+0000 7f56b0ff6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556186a2cb0 0x5556186a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.509+0000 7f56b0ff6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5556185fc2f0 con 0x5556186a2cb0 2024-09-12T03:35:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.509+0000 7f56b0ff6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5556185a4270 con 0x555618698410 2024-09-12T03:35:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.509+0000 7f56b0ff6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5556185f82b0 con 0x555618697af0 2024-09-12T03:35:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.509+0000 7f56a4ff9640 1 --2- 172.21.15.64:0/2898946149 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f56900a2e20 0x7f56900a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.510+0000 7f56a6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556186a2cb0 0x5556186a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.510+0000 7f56a6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556186a2cb0 0x5556186a50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49584/0 (socket says 172.21.15.64:49584) 2024-09-12T03:35:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.510+0000 7f56a6ffd640 1 -- 172.21.15.64:0/2606936037 learned_addr learned my addr 172.21.15.64:0/2606936037 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:35:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.510+0000 7f56a7fff640 1 --2- 172.21.15.64:0/2898946149 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f56900a2e20 0x7f56900a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.510+0000 7f56a77fe640 1 --2- 172.21.15.64:0/2606936037 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555618698410 0x5556186a2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.510+0000 7f56a7fff640 1 --2- 172.21.15.64:0/2606936037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555618697af0 0x555618697ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.510+0000 7f56a4ff9640 1 -- 172.21.15.64:0/2898946149 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f56980cfbb0 con 0x7f56a8102cc0 2024-09-12T03:35:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.510+0000 7f56a6ffd640 1 -- 172.21.15.64:0/2606936037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555618697af0 msgr2=0x555618697ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:35:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.510+0000 7f56a6ffd640 1 --2- 172.21.15.64:0/2606936037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555618697af0 0x555618697ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.510+0000 7f56a6ffd640 1 -- 172.21.15.64:0/2606936037 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555618698410 msgr2=0x5556186a2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:35:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.510+0000 7f56a6ffd640 1 --2- 172.21.15.64:0/2606936037 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555618698410 0x5556186a2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.510+0000 7f56a6ffd640 1 -- 172.21.15.64:0/2606936037 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556185a3340 con 0x5556186a2cb0 2024-09-12T03:35:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.510+0000 7f56a77fe640 1 --2- 172.21.15.64:0/2606936037 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555618698410 0x5556186a2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:35:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.510+0000 7f56a7fff640 1 --2- 172.21.15.64:0/2898946149 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f56900a2e20 0x7f56900a52e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f56a0009a60 tx=0x7f56a0005d80 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:35:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.510+0000 7f56a6ffd640 1 --2- 172.21.15.64:0/2606936037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556186a2cb0 0x5556186a50a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f569c00bb70 tx=0x7f569c00d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6863bbe2b5faea4b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:35:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.510+0000 7f568dffb640 1 -- 172.21.15.64:0/2606936037 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569c017620 con 0x5556186a2cb0 2024-09-12T03:35:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.511+0000 7f568dffb640 1 -- 172.21.15.64:0/2606936037 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f569c017c70 con 0x5556186a2cb0 2024-09-12T03:35:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.511+0000 7f568dffb640 1 -- 172.21.15.64:0/2606936037 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569c00fad0 con 0x5556186a2cb0 2024-09-12T03:35:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.511+0000 7f56b0ff6740 1 -- 172.21.15.64:0/2606936037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556186a2cb0 msgr2=0x5556186a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:35:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.511+0000 7f56b0ff6740 1 --2- 172.21.15.64:0/2606936037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556186a2cb0 0x5556186a50a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f569c00bb70 tx=0x7f569c00d710 comp rx=0 tx=0).stop 2024-09-12T03:35:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.511+0000 7f56b0ff6740 1 -- 172.21.15.64:0/2606936037 shutdown_connections 2024-09-12T03:35:52.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.511+0000 7f56b0ff6740 1 --2- 172.21.15.64:0/2606936037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556186a2cb0 0x5556186a50a0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:52.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.511+0000 7f56b0ff6740 1 --2- 172.21.15.64:0/2606936037 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555618698410 0x5556186a2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:52.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.511+0000 7f56b0ff6740 1 --2- 172.21.15.64:0/2606936037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555618697af0 0x555618697ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:52.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.511+0000 7f56b0ff6740 1 -- 172.21.15.64:0/2606936037 >> 172.21.15.64:0/2606936037 conn(0x555618694060 msgr2=0x555618694440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:35:52.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.511+0000 7f56b0ff6740 1 -- 172.21.15.64:0/2606936037 shutdown_connections 2024-09-12T03:35:52.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.511+0000 7f56b0ff6740 1 -- 172.21.15.64:0/2606936037 wait complete. 2024-09-12T03:35:52.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.512+0000 7f56b0ff6740 1 -- start start 2024-09-12T03:35:52.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.512+0000 7f56b0ff6740 1 Processor -- start 2024-09-12T03:35:52.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.512+0000 7f56b0ff6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555618697af0 0x55561869fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:52.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.512+0000 7f56b0ff6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555618698410 0x5556186a0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:52.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.512+0000 7f56b0ff6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5556186a2cb0 0x5556186a0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:52.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.512+0000 7f56b0ff6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5556186a5880 con 0x555618697af0 2024-09-12T03:35:52.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.512+0000 7f56b0ff6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555618585290 con 0x555618698410 2024-09-12T03:35:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.512+0000 7f56b0ff6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5556186a5a00 con 0x5556186a2cb0 2024-09-12T03:35:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.512+0000 7f56b0ff6740 10 client.0 _ll_register_callbacks cb 0x5556185b6140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:35:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.512+0000 7f56a77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555618697af0 0x55561869fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.512+0000 7f56a77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555618697af0 0x55561869fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49586/0 (socket says 172.21.15.64:49586) 2024-09-12T03:35:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.512+0000 7f56a77fe640 1 -- 172.21.15.64:0/192291930 learned_addr learned my addr 172.21.15.64:0/192291930 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:35:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.513+0000 7f56a6ffd640 1 --2- 172.21.15.64:0/192291930 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555618698410 0x5556186a0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.513+0000 7f56a7fff640 1 --2- 172.21.15.64:0/192291930 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5556186a2cb0 0x5556186a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.513+0000 7f56a77fe640 1 -- 172.21.15.64:0/192291930 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5556186a2cb0 msgr2=0x5556186a0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:35:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.513+0000 7f56a77fe640 1 --2- 172.21.15.64:0/192291930 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5556186a2cb0 0x5556186a0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.513+0000 7f56a77fe640 1 -- 172.21.15.64:0/192291930 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555618698410 msgr2=0x5556186a0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:35:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.513+0000 7f56a77fe640 1 --2- 172.21.15.64:0/192291930 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555618698410 0x5556186a0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.513+0000 7f56a77fe640 1 -- 172.21.15.64:0/192291930 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55561830f210 con 0x555618697af0 2024-09-12T03:35:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.513+0000 7f56a6ffd640 1 --2- 172.21.15.64:0/192291930 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555618698410 0x5556186a0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:35:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.513+0000 7f56a77fe640 1 --2- 172.21.15.64:0/192291930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555618697af0 0x55561869fad0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f569800c6a0 tx=0x7f5698014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=461fa70777a06d32 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:35:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.513+0000 7f5686ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:35:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.513+0000 7f5686ffd640 1 -- 172.21.15.64:0/192291930 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56980a46e0 con 0x555618697af0 2024-09-12T03:35:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.513+0000 7f5686ffd640 1 -- 172.21.15.64:0/192291930 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f5698010430 con 0x555618697af0 2024-09-12T03:35:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.513+0000 7f56b0ff6740 1 -- 172.21.15.64:0/192291930 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f569c017c70 con 0x555618697af0 2024-09-12T03:35:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.513+0000 7f5686ffd640 1 -- 172.21.15.64:0/192291930 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f569809c400 con 0x555618697af0 2024-09-12T03:35:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.513+0000 7f56b0ff6740 10 client.15882 fetch_fsmap learned FSMap version 25 2024-09-12T03:35:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.513+0000 7f56b0ff6740 1 -- 172.21.15.64:0/192291930 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5556183db620 con 0x555618697af0 2024-09-12T03:35:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.513+0000 7f5686ffd640 1 -- 172.21.15.64:0/192291930 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f569c017c70 con 0x555618697af0 2024-09-12T03:35:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.513+0000 7f5686ffd640 1 -- 172.21.15.64:0/192291930 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5698097a50 con 0x555618697af0 2024-09-12T03:35:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.514+0000 7f56b0ff6740 10 client.15882 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:35:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.514+0000 7f56b0ff6740 10 client.15882 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:35:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.514+0000 7f56b0ff6740 20 client.15882 populate_metadata read hostname 'smithi064' 2024-09-12T03:35:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.514+0000 7f56b0ff6740 1 --2- 172.21.15.64:0/192291930 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5556186b2b30 0x5556186b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.514+0000 7f56b0ff6740 4 client.15882 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:35:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.514+0000 7f56b0ff6740 1 -- 172.21.15.64:0/192291930 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555618609a20 con 0x5556186b2b30 2024-09-12T03:35:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.514+0000 7f56a6ffd640 1 --2- 172.21.15.64:0/192291930 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5556186b2b30 0x5556186b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.514+0000 7f5686ffd640 10 client.15882 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:35:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.514+0000 7f56a6ffd640 1 --2- 172.21.15.64:0/192291930 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5556186b2b30 0x5556186b4f60 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e5bb993767be8fe server_cookie=ef8324ec57d179ae in_seq=0 out_seq=0 2024-09-12T03:35:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.515+0000 7f5686ffd640 1 -- 172.21.15.64:0/192291930 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555618609a20 con 0x5556186b2b30 2024-09-12T03:35:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.515+0000 7f5686ffd640 10 client.15882 handle_command_reply: tid=0 2024-09-12T03:35:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.515+0000 7f56b0ff6740 1 client.15882 shutdown 2024-09-12T03:35:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.515+0000 7f56b0ff6740 1 -- 172.21.15.64:0/192291930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555618697af0 msgr2=0x55561869fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:35:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.515+0000 7f56b0ff6740 1 --2- 172.21.15.64:0/192291930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555618697af0 0x55561869fad0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f569800c6a0 tx=0x7f5698014600 comp rx=0 tx=0).stop 2024-09-12T03:35:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.515+0000 7f56b0ff6740 1 -- 172.21.15.64:0/192291930 shutdown_connections 2024-09-12T03:35:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.515+0000 7f56b0ff6740 1 --2- 172.21.15.64:0/192291930 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5556186b2b30 0x5556186b4f60 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.515+0000 7f56b0ff6740 1 --2- 172.21.15.64:0/192291930 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5556186a2cb0 0x5556186a0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.515+0000 7f56b0ff6740 1 --2- 172.21.15.64:0/192291930 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555618698410 0x5556186a0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.515+0000 7f56b0ff6740 1 --2- 172.21.15.64:0/192291930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555618697af0 0x55561869fad0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.515+0000 7f5686ffd640 0 client.15882 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:35:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.515+0000 7f56b0ff6740 1 -- 172.21.15.64:0/192291930 >> 172.21.15.64:0/192291930 conn(0x555618694060 msgr2=0x555618699d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:35:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.515+0000 7f56b0ff6740 1 -- 172.21.15.64:0/192291930 shutdown_connections 2024-09-12T03:35:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.515+0000 7f56b0ff6740 1 -- 172.21.15.64:0/192291930 wait complete. 2024-09-12T03:35:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.515+0000 7f56b0ff6740 20 client.15882 trim_cache size 0 max 16384 2024-09-12T03:35:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.530+0000 7f56b0ff6740 1 Processor -- start 2024-09-12T03:35:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.530+0000 7f56b0ff6740 1 -- start start 2024-09-12T03:35:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.530+0000 7f56b0ff6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555618697af0 0x55561869e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.530+0000 7f56b0ff6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555618698410 0x5556186a0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.530+0000 7f56b0ff6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5556186a2cb0 0x5556186ba840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.530+0000 7f56b0ff6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f56a81b17a0 con 0x555618697af0 2024-09-12T03:35:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.530+0000 7f56b0ff6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f56a81af070 con 0x555618698410 2024-09-12T03:35:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.530+0000 7f56b0ff6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f56a81acec0 con 0x5556186a2cb0 2024-09-12T03:35:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.530+0000 7f56a77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555618697af0 0x55561869e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.530+0000 7f56a77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555618697af0 0x55561869e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49588/0 (socket says 172.21.15.64:49588) 2024-09-12T03:35:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.530+0000 7f56a77fe640 1 -- 172.21.15.64:0/3843967795 learned_addr learned my addr 172.21.15.64:0/3843967795 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:35:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.531+0000 7f56a6ffd640 1 --2- 172.21.15.64:0/3843967795 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555618698410 0x5556186a0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.531+0000 7f56a7fff640 1 --2- 172.21.15.64:0/3843967795 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5556186a2cb0 0x5556186ba840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.531+0000 7f56a77fe640 1 -- 172.21.15.64:0/3843967795 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5556186a2cb0 msgr2=0x5556186ba840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:35:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.531+0000 7f56a77fe640 1 --2- 172.21.15.64:0/3843967795 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5556186a2cb0 0x5556186ba840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.531+0000 7f56a77fe640 1 -- 172.21.15.64:0/3843967795 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555618698410 msgr2=0x5556186a0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:35:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.531+0000 7f56a77fe640 1 --2- 172.21.15.64:0/3843967795 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555618698410 0x5556186a0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.531+0000 7f56a77fe640 1 -- 172.21.15.64:0/3843967795 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555618693e90 con 0x555618697af0 2024-09-12T03:35:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.531+0000 7f56a77fe640 1 --2- 172.21.15.64:0/3843967795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555618697af0 0x55561869e380 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f569809bc90 tx=0x7f5698000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=808c44496f7f6780 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:35:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.531+0000 7f568d7fa640 1 -- 172.21.15.64:0/3843967795 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569809bd00 con 0x555618697af0 2024-09-12T03:35:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.531+0000 7f568d7fa640 1 -- 172.21.15.64:0/3843967795 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56980a00b0 con 0x555618697af0 2024-09-12T03:35:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.531+0000 7f568d7fa640 1 -- 172.21.15.64:0/3843967795 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56980a4bc0 con 0x555618697af0 2024-09-12T03:35:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.532+0000 7f56b0ff6740 1 -- 172.21.15.64:0/3843967795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555618697af0 msgr2=0x55561869e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:35:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.532+0000 7f56b0ff6740 1 --2- 172.21.15.64:0/3843967795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555618697af0 0x55561869e380 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f569809bc90 tx=0x7f5698000bc0 comp rx=0 tx=0).stop 2024-09-12T03:35:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.532+0000 7f56b0ff6740 1 -- 172.21.15.64:0/3843967795 shutdown_connections 2024-09-12T03:35:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.532+0000 7f56b0ff6740 1 --2- 172.21.15.64:0/3843967795 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5556186a2cb0 0x5556186ba840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.532+0000 7f56b0ff6740 1 --2- 172.21.15.64:0/3843967795 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555618698410 0x5556186a0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.532+0000 7f56b0ff6740 1 --2- 172.21.15.64:0/3843967795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555618697af0 0x55561869e380 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.532+0000 7f56b0ff6740 1 -- 172.21.15.64:0/3843967795 >> 172.21.15.64:0/3843967795 conn(0x555618694060 msgr2=0x5556186a1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:35:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.532+0000 7f56b0ff6740 1 -- 172.21.15.64:0/3843967795 shutdown_connections 2024-09-12T03:35:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.532+0000 7f56b0ff6740 1 -- 172.21.15.64:0/3843967795 wait complete. 2024-09-12T03:35:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.532+0000 7f56b0ff6740 1 -- start start 2024-09-12T03:35:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.532+0000 7f56b0ff6740 1 Processor -- start 2024-09-12T03:35:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.533+0000 7f56b0ff6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555618697af0 0x55561869cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.533+0000 7f56b0ff6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555618698410 0x55561869d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.533+0000 7f56b0ff6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5556186a2cb0 0x55561869d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.533+0000 7f56b0ff6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f56a81ad530 con 0x555618698410 2024-09-12T03:35:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.533+0000 7f56b0ff6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f56a81ae9a0 con 0x555618697af0 2024-09-12T03:35:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.533+0000 7f56b0ff6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f56a81adc00 con 0x5556186a2cb0 2024-09-12T03:35:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.533+0000 7f56b0ff6740 10 client.0 _ll_register_callbacks cb 0x7f5698097930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:35:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.533+0000 7f56a6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555618698410 0x55561869d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.533+0000 7f56a6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555618698410 0x55561869d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49590/0 (socket says 172.21.15.64:49590) 2024-09-12T03:35:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.533+0000 7f56a6ffd640 1 -- 172.21.15.64:0/4158919685 learned_addr learned my addr 172.21.15.64:0/4158919685 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:35:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.533+0000 7f56a77fe640 1 --2- 172.21.15.64:0/4158919685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555618697af0 0x55561869cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.533+0000 7f56a7fff640 1 --2- 172.21.15.64:0/4158919685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5556186a2cb0 0x55561869d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.533+0000 7f56a6ffd640 1 -- 172.21.15.64:0/4158919685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5556186a2cb0 msgr2=0x55561869d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:35:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.533+0000 7f56a6ffd640 1 --2- 172.21.15.64:0/4158919685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5556186a2cb0 0x55561869d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.533+0000 7f56a6ffd640 1 -- 172.21.15.64:0/4158919685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555618697af0 msgr2=0x55561869cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:35:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.533+0000 7f56a6ffd640 1 --2- 172.21.15.64:0/4158919685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555618697af0 0x55561869cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.533+0000 7f56a6ffd640 1 -- 172.21.15.64:0/4158919685 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556186a9300 con 0x555618698410 2024-09-12T03:35:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.533+0000 7f56a7fff640 1 --2- 172.21.15.64:0/4158919685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5556186a2cb0 0x55561869d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:35:52.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.533+0000 7f56a77fe640 1 --2- 172.21.15.64:0/4158919685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555618697af0 0x55561869cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:35:52.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.533+0000 7f56a6ffd640 1 --2- 172.21.15.64:0/4158919685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555618698410 0x55561869d070 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f569c00bb10 tx=0x7f569c00b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c4bcdbad8317dea server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:35:52.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.533+0000 7f5686ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:35:52.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.533+0000 7f5686ffd640 1 -- 172.21.15.64:0/4158919685 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569c02e020 con 0x555618698410 2024-09-12T03:35:52.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.534+0000 7f5686ffd640 1 -- 172.21.15.64:0/4158919685 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9495+0+0 (secure 0 0 0) 0x7f569c005240 con 0x555618698410 2024-09-12T03:35:52.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.534+0000 7f56b0ff6740 1 -- 172.21.15.64:0/4158919685 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56980a00b0 con 0x555618698410 2024-09-12T03:35:52.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.534+0000 7f5686ffd640 1 -- 172.21.15.64:0/4158919685 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f569c039050 con 0x555618698410 2024-09-12T03:35:52.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.534+0000 7f56b0ff6740 10 client.15888 fetch_fsmap learned FSMap version 25 2024-09-12T03:35:52.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.534+0000 7f56b0ff6740 1 -- 172.21.15.64:0/4158919685 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55561871db70 con 0x555618698410 2024-09-12T03:35:52.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.534+0000 7f5686ffd640 1 -- 172.21.15.64:0/4158919685 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56980a00b0 con 0x555618698410 2024-09-12T03:35:52.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.534+0000 7f5686ffd640 1 -- 172.21.15.64:0/4158919685 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f569c004660 con 0x555618698410 2024-09-12T03:35:52.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.534+0000 7f56b0ff6740 10 client.15888 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:35:52.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.534+0000 7f56b0ff6740 10 client.15888 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:35:52.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.534+0000 7f56b0ff6740 20 client.15888 populate_metadata read hostname 'smithi064' 2024-09-12T03:35:52.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.534+0000 7f56b0ff6740 1 --2- 172.21.15.64:0/4158919685 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55561871e0c0 0x5556187204e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:35:52.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.534+0000 7f56b0ff6740 4 client.15888 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:35:52.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.534+0000 7f56b0ff6740 1 -- 172.21.15.64:0/4158919685 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555618603d70 con 0x55561871e0c0 2024-09-12T03:35:52.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.534+0000 7f56a77fe640 1 --2- 172.21.15.64:0/4158919685 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55561871e0c0 0x5556187204e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:35:52.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.535+0000 7f5686ffd640 10 client.15888 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:35:52.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.535+0000 7f56a77fe640 1 --2- 172.21.15.64:0/4158919685 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55561871e0c0 0x5556187204e0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc45373f174ae94a server_cookie=40f4f5c3a0935e3d in_seq=0 out_seq=0 2024-09-12T03:35:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.549+0000 7f5686ffd640 1 -- 172.21.15.64:0/4158919685 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555618603d70 con 0x55561871e0c0 2024-09-12T03:35:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.549+0000 7f5686ffd640 10 client.15888 handle_command_reply: tid=0 2024-09-12T03:35:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.549+0000 7f56b0ff6740 1 client.15888 shutdown 2024-09-12T03:35:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.549+0000 7f56b0ff6740 1 -- 172.21.15.64:0/4158919685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555618698410 msgr2=0x55561869d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:35:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.549+0000 7f56b0ff6740 1 --2- 172.21.15.64:0/4158919685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555618698410 0x55561869d070 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f569c00bb10 tx=0x7f569c00b400 comp rx=0 tx=0).stop 2024-09-12T03:35:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.549+0000 7f56b0ff6740 1 -- 172.21.15.64:0/4158919685 shutdown_connections 2024-09-12T03:35:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.549+0000 7f56b0ff6740 1 --2- 172.21.15.64:0/4158919685 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55561871e0c0 0x5556187204e0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.549+0000 7f56b0ff6740 1 --2- 172.21.15.64:0/4158919685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5556186a2cb0 0x55561869d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.549+0000 7f56b0ff6740 1 --2- 172.21.15.64:0/4158919685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555618698410 0x55561869d070 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.549+0000 7f5686ffd640 0 client.15888 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:35:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.549+0000 7f56b0ff6740 1 --2- 172.21.15.64:0/4158919685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555618697af0 0x55561869cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.549+0000 7f56b0ff6740 1 -- 172.21.15.64:0/4158919685 >> 172.21.15.64:0/4158919685 conn(0x555618694060 msgr2=0x555618699550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:35:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.549+0000 7f56b0ff6740 1 -- 172.21.15.64:0/4158919685 shutdown_connections 2024-09-12T03:35:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.549+0000 7f56b0ff6740 1 -- 172.21.15.64:0/4158919685 wait complete. 2024-09-12T03:35:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.549+0000 7f56b0ff6740 20 client.15888 trim_cache size 0 max 16384 2024-09-12T03:35:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:35:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:35:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:35:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:35:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.549+0000 7f56ae186640 1 -- 172.21.15.64:0/2898946149 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f56900a2e20 msgr2=0x7f56900a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:35:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.550+0000 7f56ae186640 1 --2- 172.21.15.64:0/2898946149 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f56900a2e20 0x7f56900a52e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f56a0009a60 tx=0x7f56a0005d80 comp rx=0 tx=0).stop 2024-09-12T03:35:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.550+0000 7f56ae186640 1 -- 172.21.15.64:0/2898946149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56a8102cc0 msgr2=0x7f56a81981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:35:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.550+0000 7f56ae186640 1 --2- 172.21.15.64:0/2898946149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56a8102cc0 0x7f56a81981d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f569800e9f0 tx=0x7f569800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:35:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.550+0000 7f56ae186640 1 -- 172.21.15.64:0/2898946149 shutdown_connections 2024-09-12T03:35:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.550+0000 7f56ae186640 1 --2- 172.21.15.64:0/2898946149 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f56900a2e20 0x7f56900a52e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.550+0000 7f56ae186640 1 --2- 172.21.15.64:0/2898946149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56a810c110 0x7f56a819ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.550+0000 7f56ae186640 1 --2- 172.21.15.64:0/2898946149 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56a81035e0 0x7f56a8198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.550+0000 7f56ae186640 1 --2- 172.21.15.64:0/2898946149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56a8102cc0 0x7f56a81981d0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:35:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.550+0000 7f56ae186640 1 -- 172.21.15.64:0/2898946149 >> 172.21.15.64:0/2898946149 conn(0x7f56a80fc6b0 msgr2=0x7f56a80fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:35:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.550+0000 7f56ae186640 1 -- 172.21.15.64:0/2898946149 shutdown_connections 2024-09-12T03:35:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:35:52.550+0000 7f56ae186640 1 -- 172.21.15.64:0/2898946149 wait complete. 2024-09-12T03:35:53.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:35:53 smithi115 ceph-mon[34801]: pgmap v201: 129 pgs: 129 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 17 op/s 2024-09-12T03:35:53.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:35:53 smithi195 ceph-mon[34786]: pgmap v201: 129 pgs: 129 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 17 op/s 2024-09-12T03:35:53.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:35:53 smithi064 ceph-mon[30952]: pgmap v201: 129 pgs: 129 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 17 op/s 2024-09-12T03:35:55.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:35:55 smithi115 ceph-mon[34801]: pgmap v202: 129 pgs: 129 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 10 op/s 2024-09-12T03:35:55.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:35:55 smithi195 ceph-mon[34786]: pgmap v202: 129 pgs: 129 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 10 op/s 2024-09-12T03:35:55.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:35:55 smithi064 ceph-mon[30952]: pgmap v202: 129 pgs: 129 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 10 op/s 2024-09-12T03:35:57.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:35:57 smithi115 ceph-mon[34801]: pgmap v203: 129 pgs: 129 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 10 op/s 2024-09-12T03:35:57.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:35:57 smithi195 ceph-mon[34786]: pgmap v203: 129 pgs: 129 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 10 op/s 2024-09-12T03:35:57.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:35:57 smithi064 ceph-mon[30952]: pgmap v203: 129 pgs: 129 active+clean; 2.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 10 op/s 2024-09-12T03:35:59.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:35:59 smithi115 ceph-mon[34801]: pgmap v204: 129 pgs: 129 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 55 KiB/s wr, 18 op/s 2024-09-12T03:35:59.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:35:59 smithi195 ceph-mon[34786]: pgmap v204: 129 pgs: 129 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 55 KiB/s wr, 18 op/s 2024-09-12T03:35:59.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:35:59 smithi064 ceph-mon[30952]: pgmap v204: 129 pgs: 129 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 55 KiB/s wr, 18 op/s 2024-09-12T03:36:01.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:36:01 smithi115 ceph-mon[34801]: pgmap v205: 129 pgs: 129 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:36:01.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:36:01 smithi195 ceph-mon[34786]: pgmap v205: 129 pgs: 129 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:36:01.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:36:01 smithi064 ceph-mon[30952]: pgmap v205: 129 pgs: 129 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:36:03.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:36:03 smithi115 ceph-mon[34801]: pgmap v206: 129 pgs: 129 active+clean; 2.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 103 KiB/s wr, 35 op/s 2024-09-12T03:36:03.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:36:03 smithi195 ceph-mon[34786]: pgmap v206: 129 pgs: 129 active+clean; 2.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 103 KiB/s wr, 35 op/s 2024-09-12T03:36:03.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:36:03 smithi064 ceph-mon[30952]: pgmap v206: 129 pgs: 129 active+clean; 2.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 103 KiB/s wr, 35 op/s 2024-09-12T03:36:04.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:36:04 smithi115 ceph-mon[34801]: osdmap e94: 12 total, 12 up, 12 in 2024-09-12T03:36:04.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:36:04 smithi195 ceph-mon[34786]: osdmap e94: 12 total, 12 up, 12 in 2024-09-12T03:36:04.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:36:04 smithi064 ceph-mon[30952]: osdmap e94: 12 total, 12 up, 12 in 2024-09-12T03:36:05.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:36:05 smithi115 ceph-mon[34801]: pgmap v208: 129 pgs: 129 active+clean; 2.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 103 KiB/s wr, 36 op/s 2024-09-12T03:36:05.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.649+0000 7f300aab7640 1 -- 172.21.15.64:0/1144216255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3004101390 msgr2=0x7f300410d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:05.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.649+0000 7f300aab7640 1 --2- 172.21.15.64:0/1144216255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3004101390 0x7f300410d8c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc009cd0 tx=0x7f2ffc02a8e0 comp rx=0 tx=0).stop 2024-09-12T03:36:05.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.649+0000 7f300aab7640 1 -- 172.21.15.64:0/1144216255 shutdown_connections 2024-09-12T03:36:05.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.649+0000 7f300aab7640 1 --2- 172.21.15.64:0/1144216255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3004101390 0x7f300410d8c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.649+0000 7f300aab7640 1 --2- 172.21.15.64:0/1144216255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30041009f0 0x7f3004100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.649+0000 7f300aab7640 1 --2- 172.21.15.64:0/1144216255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3004106a00 0x7f3004106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.649+0000 7f300aab7640 1 -- 172.21.15.64:0/1144216255 >> 172.21.15.64:0/1144216255 conn(0x7f30040fc6b0 msgr2=0x7f30040fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:36:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.649+0000 7f300aab7640 1 -- 172.21.15.64:0/1144216255 shutdown_connections 2024-09-12T03:36:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.649+0000 7f300aab7640 1 -- 172.21.15.64:0/1144216255 wait complete. 2024-09-12T03:36:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.649+0000 7f300aab7640 1 Processor -- start 2024-09-12T03:36:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.649+0000 7f300aab7640 1 -- start start 2024-09-12T03:36:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.650+0000 7f300aab7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30041009f0 0x7f3004197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.650+0000 7f300aab7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3004101390 0x7f30041976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.650+0000 7f300aab7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3004106a00 0x7f300419ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.650+0000 7f300aab7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3004110d40 con 0x7f3004106a00 2024-09-12T03:36:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.650+0000 7f300aab7640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3004110bc0 con 0x7f30041009f0 2024-09-12T03:36:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.650+0000 7f300aab7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3004110ec0 con 0x7f3004101390 2024-09-12T03:36:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.650+0000 7f3009054640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3004106a00 0x7f300419ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.650+0000 7f3009054640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3004106a00 0x7f300419ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59848/0 (socket says 172.21.15.64:59848) 2024-09-12T03:36:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.650+0000 7f3009054640 1 -- 172.21.15.64:0/3192099941 learned_addr learned my addr 172.21.15.64:0/3192099941 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:36:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.650+0000 7f3003fff640 1 --2- 172.21.15.64:0/3192099941 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3004101390 0x7f30041976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.650+0000 7f3008853640 1 --2- 172.21.15.64:0/3192099941 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30041009f0 0x7f3004197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.650+0000 7f3009054640 1 -- 172.21.15.64:0/3192099941 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3004101390 msgr2=0x7f30041976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.650+0000 7f3009054640 1 --2- 172.21.15.64:0/3192099941 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3004101390 0x7f30041976d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.650+0000 7f3009054640 1 -- 172.21.15.64:0/3192099941 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30041009f0 msgr2=0x7f3004197190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.650+0000 7f3009054640 1 --2- 172.21.15.64:0/3192099941 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30041009f0 0x7f3004197190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.650+0000 7f3009054640 1 -- 172.21.15.64:0/3192099941 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f300419c180 con 0x7f3004106a00 2024-09-12T03:36:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.650+0000 7f3009054640 1 --2- 172.21.15.64:0/3192099941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3004106a00 0x7f300419ba50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc005e30 tx=0x7f2ffc003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:36:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.651+0000 7f3001ffb640 1 -- 172.21.15.64:0/3192099941 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc04a070 con 0x7f3004106a00 2024-09-12T03:36:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.651+0000 7f3001ffb640 1 -- 172.21.15.64:0/3192099941 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ffc031d00 con 0x7f3004106a00 2024-09-12T03:36:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.651+0000 7f300aab7640 1 -- 172.21.15.64:0/3192099941 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f300419c410 con 0x7f3004106a00 2024-09-12T03:36:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.651+0000 7f3001ffb640 1 -- 172.21.15.64:0/3192099941 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc030890 con 0x7f3004106a00 2024-09-12T03:36:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.651+0000 7f300aab7640 1 -- 172.21.15.64:0/3192099941 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f300406a300 con 0x7f3004106a00 2024-09-12T03:36:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.653+0000 7f3001ffb640 1 -- 172.21.15.64:0/3192099941 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f2ffc030a30 con 0x7f3004106a00 2024-09-12T03:36:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.654+0000 7f300d852740 1 Processor -- start 2024-09-12T03:36:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.654+0000 7f300d852740 1 -- start start 2024-09-12T03:36:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.654+0000 7f300d852740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579bcba6af0 0x5579bcba6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.654+0000 7f300d852740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579bcba7410 0x5579bcbb1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.654+0000 7f300d852740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579bcbb1cb0 0x5579bcbb40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.654+0000 7f3001ffb640 1 --2- 172.21.15.64:0/3192099941 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f2ff40a2e20 0x7f2ff40a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.654+0000 7f300d852740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5579bcb0b2f0 con 0x5579bcba6af0 2024-09-12T03:36:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.654+0000 7f300d852740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5579bcab3270 con 0x5579bcbb1cb0 2024-09-12T03:36:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.654+0000 7f300d852740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5579bcb072b0 con 0x5579bcba7410 2024-09-12T03:36:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.654+0000 7f3003fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579bcba6af0 0x5579bcba6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.654+0000 7f3001ffb640 1 -- 172.21.15.64:0/3192099941 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9671+0+0 (secure 0 0 0) 0x7f2ffc02e020 con 0x7f3004106a00 2024-09-12T03:36:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.654+0000 7f3003fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579bcba6af0 0x5579bcba6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59858/0 (socket says 172.21.15.64:59858) 2024-09-12T03:36:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.654+0000 7f3003fff640 1 -- 172.21.15.64:0/2328182326 learned_addr learned my addr 172.21.15.64:0/2328182326 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:36:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.654+0000 7f3003fff640 1 --2- 172.21.15.64:0/3192099941 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f2ff40a2e20 0x7f2ff40a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.654+0000 7f3009054640 1 --2- 172.21.15.64:0/2328182326 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579bcbb1cb0 0x5579bcbb40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.654+0000 7f3008853640 1 --2- 172.21.15.64:0/2328182326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579bcba7410 0x5579bcbb1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.655+0000 7f3003fff640 1 -- 172.21.15.64:0/2328182326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579bcba7410 msgr2=0x5579bcbb1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:36:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.655+0000 7f3003fff640 1 --2- 172.21.15.64:0/2328182326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579bcba7410 0x5579bcbb1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.655+0000 7f3003fff640 1 -- 172.21.15.64:0/2328182326 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579bcbb1cb0 msgr2=0x5579bcbb40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:36:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.655+0000 7f3003fff640 1 --2- 172.21.15.64:0/2328182326 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579bcbb1cb0 0x5579bcbb40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.655+0000 7f3003fff640 1 -- 172.21.15.64:0/2328182326 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579bcab2340 con 0x5579bcba6af0 2024-09-12T03:36:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.655+0000 7f3008853640 1 --2- 172.21.15.64:0/2328182326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579bcba7410 0x5579bcbb1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:36:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.655+0000 7f3003fff640 1 --2- 172.21.15.64:0/2328182326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579bcba6af0 0x5579bcba6ed0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0011ae0 tx=0x7f2ff000f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=881079dc432124b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:36:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.655+0000 7f3003fff640 1 --2- 172.21.15.64:0/3192099941 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f2ff40a2e20 0x7f2ff40a52e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2ff000f8c0 tx=0x7f2ff001e000 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:36:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.655+0000 7f2feeffd640 1 -- 172.21.15.64:0/2328182326 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff0011100 con 0x5579bcba6af0 2024-09-12T03:36:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.655+0000 7f2feeffd640 1 -- 172.21.15.64:0/2328182326 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ff00112a0 con 0x5579bcba6af0 2024-09-12T03:36:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.655+0000 7f2feeffd640 1 -- 172.21.15.64:0/2328182326 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff001de40 con 0x5579bcba6af0 2024-09-12T03:36:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.656+0000 7f300d852740 1 -- 172.21.15.64:0/2328182326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579bcba6af0 msgr2=0x5579bcba6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.656+0000 7f300d852740 1 --2- 172.21.15.64:0/2328182326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579bcba6af0 0x5579bcba6ed0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0011ae0 tx=0x7f2ff000f3c0 comp rx=0 tx=0).stop 2024-09-12T03:36:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.656+0000 7f300d852740 1 -- 172.21.15.64:0/2328182326 shutdown_connections 2024-09-12T03:36:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.656+0000 7f300d852740 1 --2- 172.21.15.64:0/2328182326 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579bcbb1cb0 0x5579bcbb40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.656+0000 7f300d852740 1 --2- 172.21.15.64:0/2328182326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579bcba7410 0x5579bcbb1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.656+0000 7f300d852740 1 --2- 172.21.15.64:0/2328182326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579bcba6af0 0x5579bcba6ed0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.656+0000 7f300d852740 1 -- 172.21.15.64:0/2328182326 >> 172.21.15.64:0/2328182326 conn(0x5579bcba3060 msgr2=0x5579bcba3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:36:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.656+0000 7f300d852740 1 -- 172.21.15.64:0/2328182326 shutdown_connections 2024-09-12T03:36:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.656+0000 7f300d852740 1 -- 172.21.15.64:0/2328182326 wait complete. 2024-09-12T03:36:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.656+0000 7f300d852740 1 -- start start 2024-09-12T03:36:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.656+0000 7f300d852740 1 Processor -- start 2024-09-12T03:36:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.657+0000 7f300d852740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579bcba6af0 0x5579bcbaec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.657+0000 7f300d852740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579bcba7410 0x5579bcbaf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.657+0000 7f300d852740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579bcbb1cb0 0x5579bcbaf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.657+0000 7f300d852740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5579bcbb4880 con 0x5579bcbb1cb0 2024-09-12T03:36:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.657+0000 7f300d852740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5579bca94290 con 0x5579bcba7410 2024-09-12T03:36:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.657+0000 7f300d852740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5579bcbb4a00 con 0x5579bcba6af0 2024-09-12T03:36:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.657+0000 7f300d852740 10 client.0 _ll_register_callbacks cb 0x5579bcac5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:36:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.657+0000 7f3009054640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579bcbb1cb0 0x5579bcbaf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.657+0000 7f3009054640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579bcbb1cb0 0x5579bcbaf700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59872/0 (socket says 172.21.15.64:59872) 2024-09-12T03:36:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.657+0000 7f3009054640 1 -- 172.21.15.64:0/2265168417 learned_addr learned my addr 172.21.15.64:0/2265168417 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:36:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.657+0000 7f3003fff640 1 --2- 172.21.15.64:0/2265168417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579bcba7410 0x5579bcbaf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.657+0000 7f3008853640 1 --2- 172.21.15.64:0/2265168417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579bcba6af0 0x5579bcbaec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.657+0000 7f3009054640 1 -- 172.21.15.64:0/2265168417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579bcba6af0 msgr2=0x5579bcbaec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:36:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.657+0000 7f3009054640 1 --2- 172.21.15.64:0/2265168417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579bcba6af0 0x5579bcbaec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.657+0000 7f3009054640 1 -- 172.21.15.64:0/2265168417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579bcba7410 msgr2=0x5579bcbaf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:36:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.657+0000 7f3009054640 1 --2- 172.21.15.64:0/2265168417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579bcba7410 0x5579bcbaf1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.657+0000 7f3009054640 1 -- 172.21.15.64:0/2265168417 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579bc81e210 con 0x5579bcbb1cb0 2024-09-12T03:36:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.658+0000 7f3009054640 1 --2- 172.21.15.64:0/2265168417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579bcbb1cb0 0x5579bcbaf700 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc0ba720 tx=0x7f2ffc0bdc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be915fcf7a0fb125 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:36:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.658+0000 7f2fe3fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:36:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.658+0000 7f2fe3fff640 1 -- 172.21.15.64:0/2265168417 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc0098c0 con 0x5579bcbb1cb0 2024-09-12T03:36:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.658+0000 7f2fe3fff640 1 -- 172.21.15.64:0/2265168417 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9671+0+0 (secure 0 0 0) 0x7f2ffc0bd1a0 con 0x5579bcbb1cb0 2024-09-12T03:36:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.658+0000 7f300d852740 1 -- 172.21.15.64:0/2265168417 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ff00112a0 con 0x5579bcbb1cb0 2024-09-12T03:36:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.658+0000 7f2fe3fff640 1 -- 172.21.15.64:0/2265168417 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2ffc0be450 con 0x5579bcbb1cb0 2024-09-12T03:36:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.658+0000 7f300d852740 10 client.15900 fetch_fsmap learned FSMap version 25 2024-09-12T03:36:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.658+0000 7f300d852740 1 -- 172.21.15.64:0/2265168417 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5579bc8ea620 con 0x5579bcbb1cb0 2024-09-12T03:36:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.658+0000 7f2fe3fff640 1 -- 172.21.15.64:0/2265168417 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ff00112a0 con 0x5579bcbb1cb0 2024-09-12T03:36:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.658+0000 7f2fe3fff640 1 -- 172.21.15.64:0/2265168417 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2ffc030070 con 0x5579bcbb1cb0 2024-09-12T03:36:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.658+0000 7f300d852740 10 client.15900 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:36:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.658+0000 7f300d852740 10 client.15900 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:36:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.658+0000 7f300d852740 20 client.15900 populate_metadata read hostname 'smithi064' 2024-09-12T03:36:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.658+0000 7f300d852740 1 --2- 172.21.15.64:0/2265168417 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5579bcbc1d10 0x5579bcbc4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.658+0000 7f300d852740 4 client.15900 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:36:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.658+0000 7f300d852740 1 -- 172.21.15.64:0/2265168417 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579bcb18a20 con 0x5579bcbc1d10 2024-09-12T03:36:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.659+0000 7f3008853640 1 --2- 172.21.15.64:0/2265168417 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5579bcbc1d10 0x5579bcbc4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.659+0000 7f3008853640 1 --2- 172.21.15.64:0/2265168417 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5579bcbc1d10 0x5579bcbc4120 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9295378dba921af3 server_cookie=3b7dda4099111a1d in_seq=0 out_seq=0 2024-09-12T03:36:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.659+0000 7f2fe3fff640 10 client.15900 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:36:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.660+0000 7f2fe3fff640 1 -- 172.21.15.64:0/2265168417 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5579bcb18a20 con 0x5579bcbc1d10 2024-09-12T03:36:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.660+0000 7f2fe3fff640 10 client.15900 handle_command_reply: tid=0 2024-09-12T03:36:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.660+0000 7f300d852740 1 client.15900 shutdown 2024-09-12T03:36:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.660+0000 7f300d852740 1 -- 172.21.15.64:0/2265168417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579bcbb1cb0 msgr2=0x5579bcbaf700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.660+0000 7f300d852740 1 --2- 172.21.15.64:0/2265168417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579bcbb1cb0 0x5579bcbaf700 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc0ba720 tx=0x7f2ffc0bdc20 comp rx=0 tx=0).stop 2024-09-12T03:36:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.660+0000 7f300d852740 1 -- 172.21.15.64:0/2265168417 shutdown_connections 2024-09-12T03:36:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.660+0000 7f300d852740 1 --2- 172.21.15.64:0/2265168417 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5579bcbc1d10 0x5579bcbc4120 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.660+0000 7f300d852740 1 --2- 172.21.15.64:0/2265168417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579bcbb1cb0 0x5579bcbaf700 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.660+0000 7f300d852740 1 --2- 172.21.15.64:0/2265168417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579bcba7410 0x5579bcbaf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.660+0000 7f2fe3fff640 0 client.15900 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:36:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.660+0000 7f300d852740 1 --2- 172.21.15.64:0/2265168417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579bcba6af0 0x5579bcbaec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.660+0000 7f300d852740 1 -- 172.21.15.64:0/2265168417 >> 172.21.15.64:0/2265168417 conn(0x5579bcba3060 msgr2=0x5579bcba21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:36:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.660+0000 7f300d852740 1 -- 172.21.15.64:0/2265168417 shutdown_connections 2024-09-12T03:36:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.660+0000 7f300d852740 1 -- 172.21.15.64:0/2265168417 wait complete. 2024-09-12T03:36:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.660+0000 7f300d852740 20 client.15900 trim_cache size 0 max 16384 2024-09-12T03:36:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.675+0000 7f300d852740 1 Processor -- start 2024-09-12T03:36:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.675+0000 7f300d852740 1 -- start start 2024-09-12T03:36:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.675+0000 7f300d852740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579bcba6af0 0x5579bcbaf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.675+0000 7f300d852740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579bcba7410 0x5579bcbc9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.675+0000 7f300d852740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579bcbb1cb0 0x5579bcbc9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.675+0000 7f300d852740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f30041af7f0 con 0x5579bcbb1cb0 2024-09-12T03:36:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.675+0000 7f300d852740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f30041ad0c0 con 0x5579bcba6af0 2024-09-12T03:36:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.675+0000 7f300d852740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f30041aaf30 con 0x5579bcba7410 2024-09-12T03:36:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.675+0000 7f3009054640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579bcbb1cb0 0x5579bcbc9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.675+0000 7f3009054640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579bcbb1cb0 0x5579bcbc9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59882/0 (socket says 172.21.15.64:59882) 2024-09-12T03:36:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.675+0000 7f3009054640 1 -- 172.21.15.64:0/615216300 learned_addr learned my addr 172.21.15.64:0/615216300 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:36:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.675+0000 7f3008853640 1 --2- 172.21.15.64:0/615216300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579bcba6af0 0x5579bcbaf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.676+0000 7f3003fff640 1 --2- 172.21.15.64:0/615216300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579bcba7410 0x5579bcbc9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.676+0000 7f3009054640 1 -- 172.21.15.64:0/615216300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579bcba7410 msgr2=0x5579bcbc9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:36:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.676+0000 7f3009054640 1 --2- 172.21.15.64:0/615216300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579bcba7410 0x5579bcbc9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.676+0000 7f3009054640 1 -- 172.21.15.64:0/615216300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579bcba6af0 msgr2=0x5579bcbaf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:36:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.676+0000 7f3009054640 1 --2- 172.21.15.64:0/615216300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579bcba6af0 0x5579bcbaf530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.676+0000 7f3009054640 1 -- 172.21.15.64:0/615216300 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579bcbc4770 con 0x5579bcbb1cb0 2024-09-12T03:36:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.676+0000 7f3003fff640 1 --2- 172.21.15.64:0/615216300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579bcba7410 0x5579bcbc9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:36:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.676+0000 7f3009054640 1 --2- 172.21.15.64:0/615216300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579bcbb1cb0 0x5579bcbc9f40 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc0bdce0 tx=0x7f2ffc0bc260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7dbf9b6b6619daf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:36:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.676+0000 7f2fee7fc640 1 -- 172.21.15.64:0/615216300 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc0098c0 con 0x5579bcbb1cb0 2024-09-12T03:36:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.676+0000 7f2fee7fc640 1 -- 172.21.15.64:0/615216300 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ffc036040 con 0x5579bcbb1cb0 2024-09-12T03:36:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.676+0000 7f2fee7fc640 1 -- 172.21.15.64:0/615216300 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc0ba3c0 con 0x5579bcbb1cb0 2024-09-12T03:36:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.677+0000 7f300d852740 1 -- 172.21.15.64:0/615216300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579bcbb1cb0 msgr2=0x5579bcbc9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.677+0000 7f300d852740 1 --2- 172.21.15.64:0/615216300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579bcbb1cb0 0x5579bcbc9f40 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc0bdce0 tx=0x7f2ffc0bc260 comp rx=0 tx=0).stop 2024-09-12T03:36:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.677+0000 7f300d852740 1 -- 172.21.15.64:0/615216300 shutdown_connections 2024-09-12T03:36:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.677+0000 7f300d852740 1 --2- 172.21.15.64:0/615216300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579bcbb1cb0 0x5579bcbc9f40 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.677+0000 7f300d852740 1 --2- 172.21.15.64:0/615216300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579bcba7410 0x5579bcbc9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.677+0000 7f300d852740 1 --2- 172.21.15.64:0/615216300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579bcba6af0 0x5579bcbaf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.677+0000 7f300d852740 1 -- 172.21.15.64:0/615216300 >> 172.21.15.64:0/615216300 conn(0x5579bcba3060 msgr2=0x5579bcbb0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:36:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.677+0000 7f300d852740 1 -- 172.21.15.64:0/615216300 shutdown_connections 2024-09-12T03:36:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.677+0000 7f300d852740 1 -- 172.21.15.64:0/615216300 wait complete. 2024-09-12T03:36:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.677+0000 7f300d852740 1 -- start start 2024-09-12T03:36:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.677+0000 7f300d852740 1 Processor -- start 2024-09-12T03:36:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.677+0000 7f300d852740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579bcba6af0 0x5579bcbb9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.677+0000 7f300d852740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579bcba7410 0x5579bcbba3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.677+0000 7f300d852740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579bcbb1cb0 0x5579bcbb6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.677+0000 7f300d852740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f30041ab580 con 0x5579bcba6af0 2024-09-12T03:36:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.677+0000 7f300d852740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f30041ac9f0 con 0x5579bcbb1cb0 2024-09-12T03:36:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.677+0000 7f300d852740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f30041abc50 con 0x5579bcba7410 2024-09-12T03:36:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.678+0000 7f300d852740 10 client.0 _ll_register_callbacks cb 0x7f2ffc0ba0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:36:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.678+0000 7f3008853640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579bcba6af0 0x5579bcbb9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.678+0000 7f3008853640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579bcba6af0 0x5579bcbb9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59884/0 (socket says 172.21.15.64:59884) 2024-09-12T03:36:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.678+0000 7f3008853640 1 -- 172.21.15.64:0/3920297759 learned_addr learned my addr 172.21.15.64:0/3920297759 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:36:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.678+0000 7f3009054640 1 --2- 172.21.15.64:0/3920297759 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579bcbb1cb0 0x5579bcbb6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.678+0000 7f3003fff640 1 --2- 172.21.15.64:0/3920297759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579bcba7410 0x5579bcbba3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.678+0000 7f3008853640 1 -- 172.21.15.64:0/3920297759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579bcba7410 msgr2=0x5579bcbba3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:36:05.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.678+0000 7f3008853640 1 --2- 172.21.15.64:0/3920297759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579bcba7410 0x5579bcbba3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:05.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.678+0000 7f3008853640 1 -- 172.21.15.64:0/3920297759 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579bcbb1cb0 msgr2=0x5579bcbb6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:36:05.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.678+0000 7f3008853640 1 --2- 172.21.15.64:0/3920297759 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579bcbb1cb0 0x5579bcbb6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:05.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.678+0000 7f3008853640 1 -- 172.21.15.64:0/3920297759 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579bcbafa70 con 0x5579bcba6af0 2024-09-12T03:36:05.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.678+0000 7f3009054640 1 --2- 172.21.15.64:0/3920297759 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579bcbb1cb0 0x5579bcbb6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:36:05.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.678+0000 7f3008853640 1 --2- 172.21.15.64:0/3920297759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579bcba6af0 0x5579bcbb9e80 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8005d80 tx=0x7f2ff801e7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5f0804bc92f13d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:36:05.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.678+0000 7f2fe3fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:36:05.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.678+0000 7f2fe3fff640 1 -- 172.21.15.64:0/3920297759 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8014730 con 0x5579bcba6af0 2024-09-12T03:36:05.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.678+0000 7f2fe3fff640 1 -- 172.21.15.64:0/3920297759 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9671+0+0 (secure 0 0 0) 0x7f2ff80148d0 con 0x5579bcba6af0 2024-09-12T03:36:05.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.678+0000 7f300d852740 1 -- 172.21.15.64:0/3920297759 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ffc036040 con 0x5579bcba6af0 2024-09-12T03:36:05.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.678+0000 7f2fe3fff640 1 -- 172.21.15.64:0/3920297759 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2ff8014ad0 con 0x5579bcba6af0 2024-09-12T03:36:05.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.678+0000 7f300d852740 10 client.15906 fetch_fsmap learned FSMap version 25 2024-09-12T03:36:05.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.678+0000 7f300d852740 1 -- 172.21.15.64:0/3920297759 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5579bcb9e2d0 con 0x5579bcba6af0 2024-09-12T03:36:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.679+0000 7f2fe3fff640 1 -- 172.21.15.64:0/3920297759 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ffc036040 con 0x5579bcba6af0 2024-09-12T03:36:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.679+0000 7f2fe3fff640 1 -- 172.21.15.64:0/3920297759 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2ff8025420 con 0x5579bcba6af0 2024-09-12T03:36:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.679+0000 7f300d852740 10 client.15906 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:36:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.679+0000 7f300d852740 10 client.15906 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:36:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.679+0000 7f300d852740 20 client.15906 populate_metadata read hostname 'smithi064' 2024-09-12T03:36:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.679+0000 7f300d852740 1 --2- 172.21.15.64:0/3920297759 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5579bcc2d7b0 0x5579bcc2fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.679+0000 7f300d852740 4 client.15906 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:36:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.679+0000 7f300d852740 1 -- 172.21.15.64:0/3920297759 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5579bcb12d70 con 0x5579bcc2d7b0 2024-09-12T03:36:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.679+0000 7f3003fff640 1 --2- 172.21.15.64:0/3920297759 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5579bcc2d7b0 0x5579bcc2fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.679+0000 7f3003fff640 1 --2- 172.21.15.64:0/3920297759 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5579bcc2d7b0 0x5579bcc2fbd0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4aedd13a5ad0bdde server_cookie=ae741be257d7b71c in_seq=0 out_seq=0 2024-09-12T03:36:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.679+0000 7f2fe3fff640 10 client.15906 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:36:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.694+0000 7f2fe3fff640 1 -- 172.21.15.64:0/3920297759 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5579bcb12d70 con 0x5579bcc2d7b0 2024-09-12T03:36:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.694+0000 7f2fe3fff640 10 client.15906 handle_command_reply: tid=0 2024-09-12T03:36:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.694+0000 7f300d852740 1 client.15906 shutdown 2024-09-12T03:36:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.694+0000 7f300d852740 1 -- 172.21.15.64:0/3920297759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579bcba6af0 msgr2=0x5579bcbb9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.694+0000 7f300d852740 1 --2- 172.21.15.64:0/3920297759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579bcba6af0 0x5579bcbb9e80 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8005d80 tx=0x7f2ff801e7c0 comp rx=0 tx=0).stop 2024-09-12T03:36:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.694+0000 7f300d852740 1 -- 172.21.15.64:0/3920297759 shutdown_connections 2024-09-12T03:36:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.694+0000 7f300d852740 1 --2- 172.21.15.64:0/3920297759 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5579bcc2d7b0 0x5579bcc2fbd0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.694+0000 7f300d852740 1 --2- 172.21.15.64:0/3920297759 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5579bcbb1cb0 0x5579bcbb6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.694+0000 7f300d852740 1 --2- 172.21.15.64:0/3920297759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579bcba7410 0x5579bcbba3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.694+0000 7f2fe3fff640 0 client.15906 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:36:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.694+0000 7f300d852740 1 --2- 172.21.15.64:0/3920297759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579bcba6af0 0x5579bcbb9e80 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.694+0000 7f300d852740 1 -- 172.21.15.64:0/3920297759 >> 172.21.15.64:0/3920297759 conn(0x5579bcba3060 msgr2=0x5579bcba19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:36:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.695+0000 7f300d852740 1 -- 172.21.15.64:0/3920297759 shutdown_connections 2024-09-12T03:36:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.695+0000 7f300d852740 1 -- 172.21.15.64:0/3920297759 wait complete. 2024-09-12T03:36:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.695+0000 7f300d852740 20 client.15906 trim_cache size 0 max 16384 2024-09-12T03:36:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:36:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:36:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:36:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:36:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.695+0000 7f300aab7640 1 -- 172.21.15.64:0/3192099941 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f2ff40a2e20 msgr2=0x7f2ff40a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.695+0000 7f300aab7640 1 --2- 172.21.15.64:0/3192099941 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f2ff40a2e20 0x7f2ff40a52e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2ff000f8c0 tx=0x7f2ff001e000 comp rx=0 tx=0).stop 2024-09-12T03:36:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.695+0000 7f300aab7640 1 -- 172.21.15.64:0/3192099941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3004106a00 msgr2=0x7f300419ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.695+0000 7f300aab7640 1 --2- 172.21.15.64:0/3192099941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3004106a00 0x7f300419ba50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc005e30 tx=0x7f2ffc003ba0 comp rx=0 tx=0).stop 2024-09-12T03:36:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.695+0000 7f300aab7640 1 -- 172.21.15.64:0/3192099941 shutdown_connections 2024-09-12T03:36:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.695+0000 7f300aab7640 1 --2- 172.21.15.64:0/3192099941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3004106a00 0x7f300419ba50 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.695+0000 7f300aab7640 1 --2- 172.21.15.64:0/3192099941 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3004101390 0x7f30041976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.695+0000 7f300aab7640 1 --2- 172.21.15.64:0/3192099941 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f2ff40a2e20 0x7f2ff40a52e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.695+0000 7f300aab7640 1 --2- 172.21.15.64:0/3192099941 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30041009f0 0x7f3004197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.695+0000 7f300aab7640 1 -- 172.21.15.64:0/3192099941 >> 172.21.15.64:0/3192099941 conn(0x7f30040fc6b0 msgr2=0x7f30040fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:36:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.696+0000 7f300aab7640 1 -- 172.21.15.64:0/3192099941 shutdown_connections 2024-09-12T03:36:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:05.696+0000 7f300aab7640 1 -- 172.21.15.64:0/3192099941 wait complete. 2024-09-12T03:36:05.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:36:05 smithi195 ceph-mon[34786]: pgmap v208: 129 pgs: 129 active+clean; 2.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 103 KiB/s wr, 36 op/s 2024-09-12T03:36:05.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:36:05 smithi064 ceph-mon[30952]: pgmap v208: 129 pgs: 129 active+clean; 2.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 103 KiB/s wr, 36 op/s 2024-09-12T03:36:07.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:36:07 smithi115 ceph-mon[34801]: pgmap v209: 129 pgs: 129 active+clean; 2.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 103 KiB/s wr, 36 op/s 2024-09-12T03:36:07.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:36:07 smithi195 ceph-mon[34786]: pgmap v209: 129 pgs: 129 active+clean; 2.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 103 KiB/s wr, 36 op/s 2024-09-12T03:36:07.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:36:07 smithi064 ceph-mon[30952]: pgmap v209: 129 pgs: 129 active+clean; 2.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 103 KiB/s wr, 36 op/s 2024-09-12T03:36:09.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:36:09 smithi115 ceph-mon[34801]: pgmap v210: 129 pgs: 129 active+clean; 2.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 106 KiB/s wr, 36 op/s 2024-09-12T03:36:09.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:36:09 smithi195 ceph-mon[34786]: pgmap v210: 129 pgs: 129 active+clean; 2.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 106 KiB/s wr, 36 op/s 2024-09-12T03:36:09.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:36:09 smithi064 ceph-mon[30952]: pgmap v210: 129 pgs: 129 active+clean; 2.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 106 KiB/s wr, 36 op/s 2024-09-12T03:36:11.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:36:11 smithi115 ceph-mon[34801]: pgmap v211: 129 pgs: 129 active+clean; 2.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 106 KiB/s wr, 36 op/s 2024-09-12T03:36:11.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:36:11 smithi115 ceph-mon[34801]: osdmap e95: 12 total, 12 up, 12 in 2024-09-12T03:36:11.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:36:11 smithi195 ceph-mon[34786]: pgmap v211: 129 pgs: 129 active+clean; 2.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 106 KiB/s wr, 36 op/s 2024-09-12T03:36:11.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:36:11 smithi195 ceph-mon[34786]: osdmap e95: 12 total, 12 up, 12 in 2024-09-12T03:36:11.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:36:11 smithi064 ceph-mon[30952]: pgmap v211: 129 pgs: 129 active+clean; 2.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 106 KiB/s wr, 36 op/s 2024-09-12T03:36:11.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:36:11 smithi064 ceph-mon[30952]: osdmap e95: 12 total, 12 up, 12 in 2024-09-12T03:36:13.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:36:13 smithi115 ceph-mon[34801]: pgmap v213: 129 pgs: 129 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 59 KiB/s wr, 18 op/s 2024-09-12T03:36:13.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:36:13 smithi195 ceph-mon[34786]: pgmap v213: 129 pgs: 129 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 59 KiB/s wr, 18 op/s 2024-09-12T03:36:13.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:36:13 smithi064 ceph-mon[30952]: pgmap v213: 129 pgs: 129 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 59 KiB/s wr, 18 op/s 2024-09-12T03:36:15.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:36:15 smithi115 ceph-mon[34801]: pgmap v214: 129 pgs: 129 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-09-12T03:36:15.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:36:15 smithi195 ceph-mon[34786]: pgmap v214: 129 pgs: 129 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-09-12T03:36:15.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:36:15 smithi064 ceph-mon[30952]: pgmap v214: 129 pgs: 129 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-09-12T03:36:17.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:36:17 smithi115 ceph-mon[34801]: pgmap v215: 129 pgs: 129 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-09-12T03:36:17.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:36:17 smithi195 ceph-mon[34786]: pgmap v215: 129 pgs: 129 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-09-12T03:36:17.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:36:17 smithi064 ceph-mon[30952]: pgmap v215: 129 pgs: 129 active+clean; 3.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-09-12T03:36:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.796+0000 7fcb6106e640 1 -- 172.21.15.64:0/1444348817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb5c101390 msgr2=0x7fcb5c10d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.796+0000 7fcb6106e640 1 --2- 172.21.15.64:0/1444348817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb5c101390 0x7fcb5c10d8c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fcb54009cd0 tx=0x7fcb5402a8e0 comp rx=0 tx=0).stop 2024-09-12T03:36:18.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.796+0000 7fcb6106e640 1 -- 172.21.15.64:0/1444348817 shutdown_connections 2024-09-12T03:36:18.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.796+0000 7fcb6106e640 1 --2- 172.21.15.64:0/1444348817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb5c101390 0x7fcb5c10d8c0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:18.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.796+0000 7fcb6106e640 1 --2- 172.21.15.64:0/1444348817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb5c1009f0 0x7fcb5c100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:18.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.796+0000 7fcb6106e640 1 --2- 172.21.15.64:0/1444348817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb5c106a00 0x7fcb5c106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:18.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.796+0000 7fcb6106e640 1 -- 172.21.15.64:0/1444348817 >> 172.21.15.64:0/1444348817 conn(0x7fcb5c0fc6b0 msgr2=0x7fcb5c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:36:18.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.796+0000 7fcb6106e640 1 -- 172.21.15.64:0/1444348817 shutdown_connections 2024-09-12T03:36:18.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.796+0000 7fcb6106e640 1 -- 172.21.15.64:0/1444348817 wait complete. 2024-09-12T03:36:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.797+0000 7fcb6106e640 1 Processor -- start 2024-09-12T03:36:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.797+0000 7fcb6106e640 1 -- start start 2024-09-12T03:36:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.798+0000 7fcb6106e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb5c1009f0 0x7fcb5c1971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.798+0000 7fcb6106e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb5c101390 0x7fcb5c1976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.798+0000 7fcb6106e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb5c106a00 0x7fcb5c19ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.798+0000 7fcb6106e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c110cf0 con 0x7fcb5c101390 2024-09-12T03:36:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.798+0000 7fcb6106e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c110b70 con 0x7fcb5c1009f0 2024-09-12T03:36:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.798+0000 7fcb6106e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c110e70 con 0x7fcb5c106a00 2024-09-12T03:36:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.798+0000 7fcb5a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb5c101390 0x7fcb5c1976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.798+0000 7fcb5a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb5c101390 0x7fcb5c1976e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52110/0 (socket says 172.21.15.64:52110) 2024-09-12T03:36:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.798+0000 7fcb5a59c640 1 -- 172.21.15.64:0/925248032 learned_addr learned my addr 172.21.15.64:0/925248032 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:36:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.798+0000 7fcb5b59e640 1 --2- 172.21.15.64:0/925248032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb5c106a00 0x7fcb5c19ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.798+0000 7fcb5ad9d640 1 --2- 172.21.15.64:0/925248032 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb5c1009f0 0x7fcb5c1971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.798+0000 7fcb5a59c640 1 -- 172.21.15.64:0/925248032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb5c106a00 msgr2=0x7fcb5c19ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.798+0000 7fcb5a59c640 1 --2- 172.21.15.64:0/925248032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb5c106a00 0x7fcb5c19ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.798+0000 7fcb5a59c640 1 -- 172.21.15.64:0/925248032 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb5c1009f0 msgr2=0x7fcb5c1971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.798+0000 7fcb5a59c640 1 --2- 172.21.15.64:0/925248032 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb5c1009f0 0x7fcb5c1971a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.798+0000 7fcb5a59c640 1 -- 172.21.15.64:0/925248032 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb5c19c190 con 0x7fcb5c101390 2024-09-12T03:36:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.798+0000 7fcb5a59c640 1 --2- 172.21.15.64:0/925248032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb5c101390 0x7fcb5c1976e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400d950 tx=0x7fcb4400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:36:18.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.799+0000 7fcb53fff640 1 -- 172.21.15.64:0/925248032 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb44017070 con 0x7fcb5c101390 2024-09-12T03:36:18.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.799+0000 7fcb53fff640 1 -- 172.21.15.64:0/925248032 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb44009c30 con 0x7fcb5c101390 2024-09-12T03:36:18.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.799+0000 7fcb6106e640 1 -- 172.21.15.64:0/925248032 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb5c19c480 con 0x7fcb5c101390 2024-09-12T03:36:18.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.799+0000 7fcb53fff640 1 -- 172.21.15.64:0/925248032 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4400f620 con 0x7fcb5c101390 2024-09-12T03:36:18.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.799+0000 7fcb6106e640 1 -- 172.21.15.64:0/925248032 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb5c1a3d10 con 0x7fcb5c101390 2024-09-12T03:36:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.801+0000 7fcb53fff640 1 -- 172.21.15.64:0/925248032 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fcb440109a0 con 0x7fcb5c101390 2024-09-12T03:36:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.801+0000 7fcb63e08740 1 Processor -- start 2024-09-12T03:36:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.801+0000 7fcb63e08740 1 -- start start 2024-09-12T03:36:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.802+0000 7fcb63e08740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559576a4d970 0x559576a4dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.802+0000 7fcb63e08740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559576a4e290 0x559576a5a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.802+0000 7fcb63e08740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559576a5aec0 0x559576a5d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.802+0000 7fcb63e08740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5595769b22f0 con 0x559576a5aec0 2024-09-12T03:36:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.802+0000 7fcb63e08740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55957695a270 con 0x559576a4e290 2024-09-12T03:36:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.802+0000 7fcb63e08740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5595769ae2b0 con 0x559576a4d970 2024-09-12T03:36:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.802+0000 7fcb5a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559576a5aec0 0x559576a5d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.802+0000 7fcb53fff640 1 --2- 172.21.15.64:0/925248032 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fcb480a2da0 0x7fcb480a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.802+0000 7fcb5a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559576a5aec0 0x559576a5d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52126/0 (socket says 172.21.15.64:52126) 2024-09-12T03:36:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.802+0000 7fcb5a59c640 1 -- 172.21.15.64:0/1412249303 learned_addr learned my addr 172.21.15.64:0/1412249303 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:36:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.802+0000 7fcb5b59e640 1 --2- 172.21.15.64:0/1412249303 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559576a4d970 0x559576a4dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.802+0000 7fcb5ad9d640 1 --2- 172.21.15.64:0/1412249303 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559576a4e290 0x559576a5a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.802+0000 7fcb5b59e640 1 --2- 172.21.15.64:0/925248032 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fcb480a2da0 0x7fcb480a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.802+0000 7fcb53fff640 1 -- 172.21.15.64:0/925248032 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9583+0+0 (secure 0 0 0) 0x7fcb440cf450 con 0x7fcb5c101390 2024-09-12T03:36:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.803+0000 7fcb5a59c640 1 -- 172.21.15.64:0/1412249303 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559576a4d970 msgr2=0x559576a4dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:36:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.803+0000 7fcb5a59c640 1 --2- 172.21.15.64:0/1412249303 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559576a4d970 0x559576a4dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.803+0000 7fcb5a59c640 1 -- 172.21.15.64:0/1412249303 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559576a4e290 msgr2=0x559576a5a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:36:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.803+0000 7fcb5a59c640 1 --2- 172.21.15.64:0/1412249303 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559576a4e290 0x559576a5a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.803+0000 7fcb5a59c640 1 -- 172.21.15.64:0/1412249303 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559576959340 con 0x559576a5aec0 2024-09-12T03:36:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.803+0000 7fcb5b59e640 1 --2- 172.21.15.64:0/1412249303 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559576a4d970 0x559576a4dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:36:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.803+0000 7fcb5a59c640 1 --2- 172.21.15.64:0/1412249303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559576a5aec0 0x559576a5d2b0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fcb4401d000 tx=0x7fcb44005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=674cb7209de44319 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:36:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.803+0000 7fcb5b59e640 1 --2- 172.21.15.64:0/925248032 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fcb480a2da0 0x7fcb480a5260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcb54004af0 tx=0x7fcb54005da0 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:36:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.803+0000 7fcb517fa640 1 -- 172.21.15.64:0/1412249303 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb44020070 con 0x559576a5aec0 2024-09-12T03:36:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.803+0000 7fcb517fa640 1 -- 172.21.15.64:0/1412249303 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb440a4160 con 0x559576a5aec0 2024-09-12T03:36:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.803+0000 7fcb517fa640 1 -- 172.21.15.64:0/1412249303 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb440106c0 con 0x559576a5aec0 2024-09-12T03:36:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.804+0000 7fcb63e08740 1 -- 172.21.15.64:0/1412249303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559576a5aec0 msgr2=0x559576a5d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.804+0000 7fcb63e08740 1 --2- 172.21.15.64:0/1412249303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559576a5aec0 0x559576a5d2b0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fcb4401d000 tx=0x7fcb44005880 comp rx=0 tx=0).stop 2024-09-12T03:36:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.804+0000 7fcb63e08740 1 -- 172.21.15.64:0/1412249303 shutdown_connections 2024-09-12T03:36:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.804+0000 7fcb63e08740 1 --2- 172.21.15.64:0/1412249303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559576a5aec0 0x559576a5d2b0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.804+0000 7fcb63e08740 1 --2- 172.21.15.64:0/1412249303 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559576a4e290 0x559576a5a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.804+0000 7fcb63e08740 1 --2- 172.21.15.64:0/1412249303 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559576a4d970 0x559576a4dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.804+0000 7fcb63e08740 1 -- 172.21.15.64:0/1412249303 >> 172.21.15.64:0/1412249303 conn(0x559576a4a060 msgr2=0x559576a4a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:36:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.804+0000 7fcb63e08740 1 -- 172.21.15.64:0/1412249303 shutdown_connections 2024-09-12T03:36:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.804+0000 7fcb63e08740 1 -- 172.21.15.64:0/1412249303 wait complete. 2024-09-12T03:36:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.804+0000 7fcb63e08740 1 -- start start 2024-09-12T03:36:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.804+0000 7fcb63e08740 1 Processor -- start 2024-09-12T03:36:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.805+0000 7fcb63e08740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559576a4d970 0x559576a57e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.805+0000 7fcb63e08740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559576a4e290 0x559576a58380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.805+0000 7fcb63e08740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559576a5aec0 0x559576a588c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.805+0000 7fcb63e08740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559576a5da30 con 0x559576a5aec0 2024-09-12T03:36:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.805+0000 7fcb63e08740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55957693b290 con 0x559576a4d970 2024-09-12T03:36:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.805+0000 7fcb63e08740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559576a5dbb0 con 0x559576a4e290 2024-09-12T03:36:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.805+0000 7fcb63e08740 10 client.0 _ll_register_callbacks cb 0x55957696c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:36:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.805+0000 7fcb5b59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559576a5aec0 0x559576a588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.805+0000 7fcb5b59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559576a5aec0 0x559576a588c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52128/0 (socket says 172.21.15.64:52128) 2024-09-12T03:36:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.805+0000 7fcb5b59e640 1 -- 172.21.15.64:0/599032571 learned_addr learned my addr 172.21.15.64:0/599032571 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:36:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.805+0000 7fcb5ad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559576a4d970 0x559576a57e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.805+0000 7fcb5a59c640 1 --2- 172.21.15.64:0/599032571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559576a4e290 0x559576a58380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.805+0000 7fcb5b59e640 1 -- 172.21.15.64:0/599032571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559576a4e290 msgr2=0x559576a58380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:36:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.805+0000 7fcb5b59e640 1 --2- 172.21.15.64:0/599032571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559576a4e290 0x559576a58380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.805+0000 7fcb5b59e640 1 -- 172.21.15.64:0/599032571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559576a4d970 msgr2=0x559576a57e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:36:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.805+0000 7fcb5b59e640 1 --2- 172.21.15.64:0/599032571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559576a4d970 0x559576a57e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.805+0000 7fcb5b59e640 1 -- 172.21.15.64:0/599032571 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595766c5210 con 0x559576a5aec0 2024-09-12T03:36:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.805+0000 7fcb5a59c640 1 --2- 172.21.15.64:0/599032571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559576a4e290 0x559576a58380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:36:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.805+0000 7fcb5b59e640 1 --2- 172.21.15.64:0/599032571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559576a5aec0 0x559576a588c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fcb54009cd0 tx=0x7fcb540095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb34cad401477fd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:36:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.805+0000 7fcb427fc640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:36:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.806+0000 7fcb427fc640 1 -- 172.21.15.64:0/599032571 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb54050070 con 0x559576a5aec0 2024-09-12T03:36:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.806+0000 7fcb427fc640 1 -- 172.21.15.64:0/599032571 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9583+0+0 (secure 0 0 0) 0x7fcb54004690 con 0x559576a5aec0 2024-09-12T03:36:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.806+0000 7fcb63e08740 1 -- 172.21.15.64:0/599032571 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb440a4160 con 0x559576a5aec0 2024-09-12T03:36:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.806+0000 7fcb427fc640 1 -- 172.21.15.64:0/599032571 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcb5405f050 con 0x559576a5aec0 2024-09-12T03:36:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.806+0000 7fcb63e08740 10 client.15918 fetch_fsmap learned FSMap version 25 2024-09-12T03:36:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.806+0000 7fcb63e08740 1 -- 172.21.15.64:0/599032571 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559576791620 con 0x559576a5aec0 2024-09-12T03:36:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.806+0000 7fcb427fc640 1 -- 172.21.15.64:0/599032571 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb440a4160 con 0x559576a5aec0 2024-09-12T03:36:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.806+0000 7fcb427fc640 1 -- 172.21.15.64:0/599032571 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fcb54048040 con 0x559576a5aec0 2024-09-12T03:36:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.806+0000 7fcb63e08740 10 client.15918 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:36:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.806+0000 7fcb63e08740 10 client.15918 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:36:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.806+0000 7fcb63e08740 20 client.15918 populate_metadata read hostname 'smithi064' 2024-09-12T03:36:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.806+0000 7fcb63e08740 1 --2- 172.21.15.64:0/599032571 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x559576a6ae30 0x559576a6d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.806+0000 7fcb63e08740 4 client.15918 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:36:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.806+0000 7fcb63e08740 1 -- 172.21.15.64:0/599032571 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595769bfa20 con 0x559576a6ae30 2024-09-12T03:36:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.806+0000 7fcb5ad9d640 1 --2- 172.21.15.64:0/599032571 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x559576a6ae30 0x559576a6d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.807+0000 7fcb427fc640 10 client.15918 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:36:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.807+0000 7fcb5ad9d640 1 --2- 172.21.15.64:0/599032571 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x559576a6ae30 0x559576a6d280 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=305215b278d551f6 server_cookie=88c16edf83a8c2ed in_seq=0 out_seq=0 2024-09-12T03:36:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.807+0000 7fcb427fc640 1 -- 172.21.15.64:0/599032571 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5595769bfa20 con 0x559576a6ae30 2024-09-12T03:36:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.807+0000 7fcb427fc640 10 client.15918 handle_command_reply: tid=0 2024-09-12T03:36:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.808+0000 7fcb63e08740 1 client.15918 shutdown 2024-09-12T03:36:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.808+0000 7fcb63e08740 1 -- 172.21.15.64:0/599032571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559576a5aec0 msgr2=0x559576a588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.808+0000 7fcb63e08740 1 --2- 172.21.15.64:0/599032571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559576a5aec0 0x559576a588c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fcb54009cd0 tx=0x7fcb540095c0 comp rx=0 tx=0).stop 2024-09-12T03:36:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.808+0000 7fcb63e08740 1 -- 172.21.15.64:0/599032571 shutdown_connections 2024-09-12T03:36:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.808+0000 7fcb63e08740 1 --2- 172.21.15.64:0/599032571 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x559576a6ae30 0x559576a6d280 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.808+0000 7fcb63e08740 1 --2- 172.21.15.64:0/599032571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559576a5aec0 0x559576a588c0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.808+0000 7fcb63e08740 1 --2- 172.21.15.64:0/599032571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559576a4e290 0x559576a58380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.808+0000 7fcb427fc640 0 client.15918 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:36:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.808+0000 7fcb63e08740 1 --2- 172.21.15.64:0/599032571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559576a4d970 0x559576a57e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.808+0000 7fcb63e08740 1 -- 172.21.15.64:0/599032571 >> 172.21.15.64:0/599032571 conn(0x559576a4a060 msgr2=0x559576a4fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:36:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.808+0000 7fcb63e08740 1 -- 172.21.15.64:0/599032571 shutdown_connections 2024-09-12T03:36:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.808+0000 7fcb63e08740 1 -- 172.21.15.64:0/599032571 wait complete. 2024-09-12T03:36:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.808+0000 7fcb63e08740 20 client.15918 trim_cache size 0 max 16384 2024-09-12T03:36:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.822+0000 7fcb63e08740 1 Processor -- start 2024-09-12T03:36:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.823+0000 7fcb63e08740 1 -- start start 2024-09-12T03:36:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.823+0000 7fcb63e08740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559576a4d970 0x559576a72be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.823+0000 7fcb63e08740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559576a4e290 0x559576a73120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.823+0000 7fcb63e08740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559576a5aec0 0x559576a55080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.823+0000 7fcb63e08740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c1af7b0 con 0x559576a5aec0 2024-09-12T03:36:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.823+0000 7fcb63e08740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c1ad080 con 0x559576a4e290 2024-09-12T03:36:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.823+0000 7fcb63e08740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c1aaef0 con 0x559576a4d970 2024-09-12T03:36:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.823+0000 7fcb5b59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559576a5aec0 0x559576a55080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.823+0000 7fcb5b59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559576a5aec0 0x559576a55080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52140/0 (socket says 172.21.15.64:52140) 2024-09-12T03:36:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.823+0000 7fcb5b59e640 1 -- 172.21.15.64:0/4279730757 learned_addr learned my addr 172.21.15.64:0/4279730757 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:36:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.823+0000 7fcb5a59c640 1 --2- 172.21.15.64:0/4279730757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559576a4e290 0x559576a73120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.823+0000 7fcb5ad9d640 1 --2- 172.21.15.64:0/4279730757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559576a4d970 0x559576a72be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.823+0000 7fcb5b59e640 1 -- 172.21.15.64:0/4279730757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559576a4d970 msgr2=0x559576a72be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:36:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.823+0000 7fcb5b59e640 1 --2- 172.21.15.64:0/4279730757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559576a4d970 0x559576a72be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.823+0000 7fcb5b59e640 1 -- 172.21.15.64:0/4279730757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559576a4e290 msgr2=0x559576a73120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:36:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.823+0000 7fcb5b59e640 1 --2- 172.21.15.64:0/4279730757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559576a4e290 0x559576a73120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.823+0000 7fcb5b59e640 1 -- 172.21.15.64:0/4279730757 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559576a56dd0 con 0x559576a5aec0 2024-09-12T03:36:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.824+0000 7fcb5b59e640 1 --2- 172.21.15.64:0/4279730757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559576a5aec0 0x559576a55080 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fcb5402ae00 tx=0x7fcb54031430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5850768ab3ecc65e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:36:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.824+0000 7fcb50ff9640 1 -- 172.21.15.64:0/4279730757 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb54050070 con 0x559576a5aec0 2024-09-12T03:36:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.824+0000 7fcb50ff9640 1 -- 172.21.15.64:0/4279730757 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb54036070 con 0x559576a5aec0 2024-09-12T03:36:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.824+0000 7fcb50ff9640 1 -- 172.21.15.64:0/4279730757 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb54030800 con 0x559576a5aec0 2024-09-12T03:36:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.824+0000 7fcb63e08740 1 -- 172.21.15.64:0/4279730757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559576a5aec0 msgr2=0x559576a55080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.824+0000 7fcb63e08740 1 --2- 172.21.15.64:0/4279730757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559576a5aec0 0x559576a55080 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fcb5402ae00 tx=0x7fcb54031430 comp rx=0 tx=0).stop 2024-09-12T03:36:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.824+0000 7fcb63e08740 1 -- 172.21.15.64:0/4279730757 shutdown_connections 2024-09-12T03:36:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.824+0000 7fcb63e08740 1 --2- 172.21.15.64:0/4279730757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559576a5aec0 0x559576a55080 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.824+0000 7fcb63e08740 1 --2- 172.21.15.64:0/4279730757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559576a4e290 0x559576a73120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.824+0000 7fcb63e08740 1 --2- 172.21.15.64:0/4279730757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559576a4d970 0x559576a72be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.825+0000 7fcb63e08740 1 -- 172.21.15.64:0/4279730757 >> 172.21.15.64:0/4279730757 conn(0x559576a4a060 msgr2=0x559576a59ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:36:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.825+0000 7fcb63e08740 1 -- 172.21.15.64:0/4279730757 shutdown_connections 2024-09-12T03:36:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.825+0000 7fcb63e08740 1 -- 172.21.15.64:0/4279730757 wait complete. 2024-09-12T03:36:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.825+0000 7fcb63e08740 1 -- start start 2024-09-12T03:36:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.825+0000 7fcb63e08740 1 Processor -- start 2024-09-12T03:36:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.825+0000 7fcb63e08740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559576a4d970 0x559576a5f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.825+0000 7fcb63e08740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559576a4e290 0x559576a5f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.825+0000 7fcb63e08740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559576a5aec0 0x559576acebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.825+0000 7fcb63e08740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c1ab540 con 0x559576a4e290 2024-09-12T03:36:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.825+0000 7fcb63e08740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c1ac9b0 con 0x559576a5aec0 2024-09-12T03:36:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.825+0000 7fcb63e08740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c1abc10 con 0x559576a4d970 2024-09-12T03:36:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.825+0000 7fcb63e08740 10 client.0 _ll_register_callbacks cb 0x7fcb54044040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:36:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.825+0000 7fcb5a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559576a4e290 0x559576a5f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.825+0000 7fcb5a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559576a4e290 0x559576a5f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52150/0 (socket says 172.21.15.64:52150) 2024-09-12T03:36:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.825+0000 7fcb5a59c640 1 -- 172.21.15.64:0/4040651229 learned_addr learned my addr 172.21.15.64:0/4040651229 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:36:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.826+0000 7fcb5ad9d640 1 --2- 172.21.15.64:0/4040651229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559576a4d970 0x559576a5f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.826+0000 7fcb5a59c640 1 -- 172.21.15.64:0/4040651229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559576a4d970 msgr2=0x559576a5f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:36:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.826+0000 7fcb5a59c640 1 --2- 172.21.15.64:0/4040651229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559576a4d970 0x559576a5f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.826+0000 7fcb5a59c640 1 -- 172.21.15.64:0/4040651229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559576a5aec0 msgr2=0x559576acebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:36:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.826+0000 7fcb5a59c640 1 --2- 172.21.15.64:0/4040651229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559576a5aec0 0x559576acebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.826+0000 7fcb5a59c640 1 -- 172.21.15.64:0/4040651229 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559576a67800 con 0x559576a4e290 2024-09-12T03:36:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.826+0000 7fcb5ad9d640 1 --2- 172.21.15.64:0/4040651229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559576a4d970 0x559576a5f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:36:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.826+0000 7fcb5a59c640 1 --2- 172.21.15.64:0/4040651229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559576a4e290 0x559576a5f980 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fcb4409b530 tx=0x7fcb44097e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8afd5211ce3b07ab server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:36:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.826+0000 7fcb427fc640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:36:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.826+0000 7fcb427fc640 1 -- 172.21.15.64:0/4040651229 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb44020040 con 0x559576a4e290 2024-09-12T03:36:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.826+0000 7fcb427fc640 1 -- 172.21.15.64:0/4040651229 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9583+0+0 (secure 0 0 0) 0x7fcb440998a0 con 0x559576a4e290 2024-09-12T03:36:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.826+0000 7fcb63e08740 1 -- 172.21.15.64:0/4040651229 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb54036070 con 0x559576a4e290 2024-09-12T03:36:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.826+0000 7fcb427fc640 1 -- 172.21.15.64:0/4040651229 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcb440a4160 con 0x559576a4e290 2024-09-12T03:36:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.826+0000 7fcb63e08740 10 client.15924 fetch_fsmap learned FSMap version 25 2024-09-12T03:36:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.826+0000 7fcb63e08740 1 -- 172.21.15.64:0/4040651229 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559576ad6430 con 0x559576a4e290 2024-09-12T03:36:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.826+0000 7fcb427fc640 1 -- 172.21.15.64:0/4040651229 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb54036070 con 0x559576a4e290 2024-09-12T03:36:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.827+0000 7fcb427fc640 1 -- 172.21.15.64:0/4040651229 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fcb44010460 con 0x559576a4e290 2024-09-12T03:36:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.827+0000 7fcb63e08740 10 client.15924 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:36:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.827+0000 7fcb63e08740 10 client.15924 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:36:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.827+0000 7fcb63e08740 20 client.15924 populate_metadata read hostname 'smithi064' 2024-09-12T03:36:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.827+0000 7fcb63e08740 1 --2- 172.21.15.64:0/4040651229 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x559576ad66b0 0x559576ad8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.827+0000 7fcb63e08740 4 client.15924 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:36:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.827+0000 7fcb63e08740 1 -- 172.21.15.64:0/4040651229 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5595769b9d70 con 0x559576ad66b0 2024-09-12T03:36:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.827+0000 7fcb5ad9d640 1 --2- 172.21.15.64:0/4040651229 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x559576ad66b0 0x559576ad8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.827+0000 7fcb427fc640 10 client.15924 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:36:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.827+0000 7fcb5ad9d640 1 --2- 172.21.15.64:0/4040651229 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x559576ad66b0 0x559576ad8aa0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a64c35dc1199dd5d server_cookie=2e11d2ccf00688d0 in_seq=0 out_seq=0 2024-09-12T03:36:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.842+0000 7fcb427fc640 1 -- 172.21.15.64:0/4040651229 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5595769b9d70 con 0x559576ad66b0 2024-09-12T03:36:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.842+0000 7fcb427fc640 10 client.15924 handle_command_reply: tid=0 2024-09-12T03:36:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.842+0000 7fcb63e08740 1 client.15924 shutdown 2024-09-12T03:36:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.842+0000 7fcb63e08740 1 -- 172.21.15.64:0/4040651229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559576a4e290 msgr2=0x559576a5f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.842+0000 7fcb63e08740 1 --2- 172.21.15.64:0/4040651229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559576a4e290 0x559576a5f980 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fcb4409b530 tx=0x7fcb44097e40 comp rx=0 tx=0).stop 2024-09-12T03:36:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.842+0000 7fcb63e08740 1 -- 172.21.15.64:0/4040651229 shutdown_connections 2024-09-12T03:36:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.842+0000 7fcb63e08740 1 --2- 172.21.15.64:0/4040651229 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x559576ad66b0 0x559576ad8aa0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.842+0000 7fcb63e08740 1 --2- 172.21.15.64:0/4040651229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559576a5aec0 0x559576acebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.842+0000 7fcb63e08740 1 --2- 172.21.15.64:0/4040651229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559576a4e290 0x559576a5f980 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.842+0000 7fcb63e08740 1 --2- 172.21.15.64:0/4040651229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559576a4d970 0x559576a5f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.842+0000 7fcb427fc640 0 client.15924 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:36:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.842+0000 7fcb63e08740 1 -- 172.21.15.64:0/4040651229 >> 172.21.15.64:0/4040651229 conn(0x559576a4a060 msgr2=0x559576a57e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:36:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.842+0000 7fcb63e08740 1 -- 172.21.15.64:0/4040651229 shutdown_connections 2024-09-12T03:36:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.842+0000 7fcb63e08740 1 -- 172.21.15.64:0/4040651229 wait complete. 2024-09-12T03:36:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.842+0000 7fcb63e08740 20 client.15924 trim_cache size 0 max 16384 2024-09-12T03:36:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:36:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:36:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:36:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:36:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.843+0000 7fcb6106e640 1 -- 172.21.15.64:0/925248032 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fcb480a2da0 msgr2=0x7fcb480a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.843+0000 7fcb6106e640 1 --2- 172.21.15.64:0/925248032 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fcb480a2da0 0x7fcb480a5260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcb54004af0 tx=0x7fcb54005da0 comp rx=0 tx=0).stop 2024-09-12T03:36:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.843+0000 7fcb6106e640 1 -- 172.21.15.64:0/925248032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb5c101390 msgr2=0x7fcb5c1976e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.843+0000 7fcb6106e640 1 --2- 172.21.15.64:0/925248032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb5c101390 0x7fcb5c1976e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400d950 tx=0x7fcb4400de10 comp rx=0 tx=0).stop 2024-09-12T03:36:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.843+0000 7fcb6106e640 1 -- 172.21.15.64:0/925248032 shutdown_connections 2024-09-12T03:36:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.843+0000 7fcb6106e640 1 --2- 172.21.15.64:0/925248032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb5c106a00 0x7fcb5c19ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.843+0000 7fcb6106e640 1 --2- 172.21.15.64:0/925248032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb5c101390 0x7fcb5c1976e0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.843+0000 7fcb6106e640 1 --2- 172.21.15.64:0/925248032 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fcb480a2da0 0x7fcb480a5260 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.843+0000 7fcb6106e640 1 --2- 172.21.15.64:0/925248032 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb5c1009f0 0x7fcb5c1971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.843+0000 7fcb6106e640 1 -- 172.21.15.64:0/925248032 >> 172.21.15.64:0/925248032 conn(0x7fcb5c0fc6b0 msgr2=0x7fcb5c0fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:36:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.843+0000 7fcb6106e640 1 -- 172.21.15.64:0/925248032 shutdown_connections 2024-09-12T03:36:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:18.843+0000 7fcb6106e640 1 -- 172.21.15.64:0/925248032 wait complete. 2024-09-12T03:36:19.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:36:19 smithi115 ceph-mon[34801]: pgmap v216: 129 pgs: 129 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-12T03:36:19.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:36:19 smithi195 ceph-mon[34786]: pgmap v216: 129 pgs: 129 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-12T03:36:19.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:36:19 smithi064 ceph-mon[30952]: pgmap v216: 129 pgs: 129 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-12T03:36:21.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:36:21 smithi195 ceph-mon[34786]: pgmap v217: 129 pgs: 129 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-12T03:36:21.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:36:21 smithi064 ceph-mon[30952]: pgmap v217: 129 pgs: 129 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-12T03:36:21.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:36:21 smithi115 ceph-mon[34801]: pgmap v217: 129 pgs: 129 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-12T03:36:23.695 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:36:23 smithi115 ceph-mon[34801]: pgmap v218: 129 pgs: 129 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 63 KiB/s wr, 20 op/s 2024-09-12T03:36:23.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:36:23 smithi195 ceph-mon[34786]: pgmap v218: 129 pgs: 129 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 63 KiB/s wr, 20 op/s 2024-09-12T03:36:23.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:36:23 smithi064 ceph-mon[30952]: pgmap v218: 129 pgs: 129 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 63 KiB/s wr, 20 op/s 2024-09-12T03:36:24.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:36:24 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:36:24.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:36:24 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:36:24.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:36:24 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:36:24.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:36:24 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:36:24.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:36:24 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:36:24.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:36:24 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:36:24.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:36:24 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:36:24.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:36:24 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:36:24.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:36:24 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:36:24.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:36:24 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:36:24.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:36:24 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:36:24.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:36:24 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:36:25.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:36:25 smithi195 ceph-mon[34786]: pgmap v219: 129 pgs: 129 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:36:25.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:36:25 smithi064 ceph-mon[30952]: pgmap v219: 129 pgs: 129 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:36:25.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:36:25 smithi115 ceph-mon[34801]: pgmap v219: 129 pgs: 129 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:36:27.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:36:27 smithi195 ceph-mon[34786]: pgmap v220: 129 pgs: 129 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:36:27.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:36:27 smithi064 ceph-mon[30952]: pgmap v220: 129 pgs: 129 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:36:27.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:36:27 smithi115 ceph-mon[34801]: pgmap v220: 129 pgs: 129 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:36:29.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:36:29 smithi195 ceph-mon[34786]: pgmap v221: 129 pgs: 129 active+clean; 3.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:36:29.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:36:29 smithi064 ceph-mon[30952]: pgmap v221: 129 pgs: 129 active+clean; 3.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:36:29.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:36:29 smithi115 ceph-mon[34801]: pgmap v221: 129 pgs: 129 active+clean; 3.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:36:31.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:36:31 smithi195 ceph-mon[34786]: pgmap v222: 129 pgs: 129 active+clean; 3.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-12T03:36:31.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:36:31 smithi064 ceph-mon[30952]: pgmap v222: 129 pgs: 129 active+clean; 3.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-12T03:36:31.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:36:31 smithi115 ceph-mon[34801]: pgmap v222: 129 pgs: 129 active+clean; 3.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-12T03:36:31.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.943+0000 7fa958a01640 1 -- 172.21.15.64:0/4149197893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9541009b0 msgr2=0x7fa954100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:31.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.943+0000 7fa958a01640 1 --2- 172.21.15.64:0/4149197893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9541009b0 0x7fa954100e10 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa944009930 tx=0x7fa94402a420 comp rx=0 tx=0).stop 2024-09-12T03:36:31.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.943+0000 7fa958a01640 1 -- 172.21.15.64:0/4149197893 shutdown_connections 2024-09-12T03:36:31.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.943+0000 7fa958a01640 1 --2- 172.21.15.64:0/4149197893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa954101350 0x7fa95410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:31.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.943+0000 7fa958a01640 1 --2- 172.21.15.64:0/4149197893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9541009b0 0x7fa954100e10 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.943+0000 7fa958a01640 1 --2- 172.21.15.64:0/4149197893 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9541069c0 0x7fa954106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.943+0000 7fa958a01640 1 -- 172.21.15.64:0/4149197893 >> 172.21.15.64:0/4149197893 conn(0x7fa954077990 msgr2=0x7fa9540fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:36:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.943+0000 7fa958a01640 1 -- 172.21.15.64:0/4149197893 shutdown_connections 2024-09-12T03:36:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.943+0000 7fa958a01640 1 -- 172.21.15.64:0/4149197893 wait complete. 2024-09-12T03:36:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.944+0000 7fa958a01640 1 Processor -- start 2024-09-12T03:36:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.944+0000 7fa958a01640 1 -- start start 2024-09-12T03:36:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.944+0000 7fa958a01640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9541009b0 0x7fa954198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.944+0000 7fa958a01640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa954101350 0x7fa954198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.944+0000 7fa958a01640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9541069c0 0x7fa95419cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.944+0000 7fa958a01640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa954110cb0 con 0x7fa9541069c0 2024-09-12T03:36:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.944+0000 7fa958a01640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa954110b30 con 0x7fa9541009b0 2024-09-12T03:36:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.944+0000 7fa958a01640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa954110e30 con 0x7fa954101350 2024-09-12T03:36:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.945+0000 7fa952d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9541069c0 0x7fa95419cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.945+0000 7fa951d9b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa954101350 0x7fa954198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.945+0000 7fa952d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9541069c0 0x7fa95419cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47142/0 (socket says 172.21.15.64:47142) 2024-09-12T03:36:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.945+0000 7fa952d9d640 1 -- 172.21.15.64:0/3129180391 learned_addr learned my addr 172.21.15.64:0/3129180391 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:36:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.945+0000 7fa95259c640 1 --2- 172.21.15.64:0/3129180391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9541009b0 0x7fa954198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.945+0000 7fa952d9d640 1 -- 172.21.15.64:0/3129180391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa954101350 msgr2=0x7fa954198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.945+0000 7fa952d9d640 1 --2- 172.21.15.64:0/3129180391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa954101350 0x7fa954198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.945+0000 7fa952d9d640 1 -- 172.21.15.64:0/3129180391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9541009b0 msgr2=0x7fa954198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.945+0000 7fa952d9d640 1 --2- 172.21.15.64:0/3129180391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9541009b0 0x7fa954198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.945+0000 7fa952d9d640 1 -- 172.21.15.64:0/3129180391 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa95419d230 con 0x7fa9541069c0 2024-09-12T03:36:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.945+0000 7fa95259c640 1 --2- 172.21.15.64:0/3129180391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9541009b0 0x7fa954198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:36:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.945+0000 7fa952d9d640 1 --2- 172.21.15.64:0/3129180391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9541069c0 0x7fa95419cb00 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa94c00cce0 tx=0x7fa94c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:36:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.945+0000 7fa94b7fe640 1 -- 172.21.15.64:0/3129180391 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c00f450 con 0x7fa9541069c0 2024-09-12T03:36:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.945+0000 7fa94b7fe640 1 -- 172.21.15.64:0/3129180391 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa94c00faa0 con 0x7fa9541069c0 2024-09-12T03:36:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.945+0000 7fa958a01640 1 -- 172.21.15.64:0/3129180391 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa95419d520 con 0x7fa9541069c0 2024-09-12T03:36:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.945+0000 7fa94b7fe640 1 -- 172.21.15.64:0/3129180391 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c018b70 con 0x7fa9541069c0 2024-09-12T03:36:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.945+0000 7fa958a01640 1 -- 172.21.15.64:0/3129180391 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9541a4db0 con 0x7fa9541069c0 2024-09-12T03:36:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.948+0000 7fa94b7fe640 1 -- 172.21.15.64:0/3129180391 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fa94c018d10 con 0x7fa9541069c0 2024-09-12T03:36:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.948+0000 7fa95b873740 1 Processor -- start 2024-09-12T03:36:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.948+0000 7fa95b873740 1 -- start start 2024-09-12T03:36:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.948+0000 7fa95b873740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f56e7fbaf0 0x55f56e7fbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.948+0000 7fa95b873740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f56e7fc410 0x55f56e806770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.948+0000 7fa95b873740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f56e806cb0 0x55f56e8090a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.948+0000 7fa95b873740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f56e7602f0 con 0x55f56e806cb0 2024-09-12T03:36:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.948+0000 7fa95b873740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f56e708270 con 0x55f56e7fc410 2024-09-12T03:36:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.948+0000 7fa95b873740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f56e75c2b0 con 0x55f56e7fbaf0 2024-09-12T03:36:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.949+0000 7fa951d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f56e806cb0 0x55f56e8090a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.949+0000 7fa951d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f56e806cb0 0x55f56e8090a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47156/0 (socket says 172.21.15.64:47156) 2024-09-12T03:36:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.949+0000 7fa951d9b640 1 -- 172.21.15.64:0/3573775055 learned_addr learned my addr 172.21.15.64:0/3573775055 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:36:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.949+0000 7fa95259c640 1 --2- 172.21.15.64:0/3573775055 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f56e7fc410 0x55f56e806770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.949+0000 7fa94b7fe640 1 --2- 172.21.15.64:0/3129180391 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fa9400a2da0 0x7fa9400a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.949+0000 7fa951d9b640 1 --2- 172.21.15.64:0/3573775055 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f56e7fbaf0 0x55f56e7fbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.949+0000 7fa952d9d640 1 --2- 172.21.15.64:0/3129180391 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fa9400a2da0 0x7fa9400a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.949+0000 7fa951d9b640 1 -- 172.21.15.64:0/3573775055 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f56e7fbaf0 msgr2=0x55f56e7fbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:36:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.949+0000 7fa951d9b640 1 --2- 172.21.15.64:0/3573775055 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f56e7fbaf0 0x55f56e7fbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.949+0000 7fa951d9b640 1 -- 172.21.15.64:0/3573775055 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f56e7fc410 msgr2=0x55f56e806770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:36:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.949+0000 7fa951d9b640 1 --2- 172.21.15.64:0/3573775055 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f56e7fc410 0x55f56e806770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.949+0000 7fa951d9b640 1 -- 172.21.15.64:0/3573775055 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f56e707340 con 0x55f56e806cb0 2024-09-12T03:36:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.949+0000 7fa94b7fe640 1 -- 172.21.15.64:0/3129180391 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9583+0+0 (secure 0 0 0) 0x7fa94c0cfcc0 con 0x7fa9541069c0 2024-09-12T03:36:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.949+0000 7fa951d9b640 1 --2- 172.21.15.64:0/3573775055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f56e806cb0 0x55f56e8090a0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa944009930 tx=0x7fa944005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc709c1b387252ff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:36:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.949+0000 7fa948ff9640 1 -- 172.21.15.64:0/3573775055 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa944035070 con 0x55f56e806cb0 2024-09-12T03:36:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.949+0000 7fa952d9d640 1 --2- 172.21.15.64:0/3129180391 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fa9400a2da0 0x7fa9400a5260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa94c098220 tx=0x7fa94c06ee30 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:36:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.950+0000 7fa948ff9640 1 -- 172.21.15.64:0/3573775055 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9440316e0 con 0x55f56e806cb0 2024-09-12T03:36:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.950+0000 7fa948ff9640 1 -- 172.21.15.64:0/3573775055 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9440047a0 con 0x55f56e806cb0 2024-09-12T03:36:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.950+0000 7fa95b873740 1 -- 172.21.15.64:0/3573775055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f56e806cb0 msgr2=0x55f56e8090a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.950+0000 7fa95b873740 1 --2- 172.21.15.64:0/3573775055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f56e806cb0 0x55f56e8090a0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa944009930 tx=0x7fa944005fd0 comp rx=0 tx=0).stop 2024-09-12T03:36:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.950+0000 7fa95b873740 1 -- 172.21.15.64:0/3573775055 shutdown_connections 2024-09-12T03:36:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.950+0000 7fa95b873740 1 --2- 172.21.15.64:0/3573775055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f56e806cb0 0x55f56e8090a0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.950+0000 7fa95b873740 1 --2- 172.21.15.64:0/3573775055 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f56e7fc410 0x55f56e806770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.950+0000 7fa95b873740 1 --2- 172.21.15.64:0/3573775055 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f56e7fbaf0 0x55f56e7fbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.950+0000 7fa95b873740 1 -- 172.21.15.64:0/3573775055 >> 172.21.15.64:0/3573775055 conn(0x55f56e7f8060 msgr2=0x55f56e7f8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:36:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.950+0000 7fa95b873740 1 -- 172.21.15.64:0/3573775055 shutdown_connections 2024-09-12T03:36:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.950+0000 7fa95b873740 1 -- 172.21.15.64:0/3573775055 wait complete. 2024-09-12T03:36:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.951+0000 7fa95b873740 1 -- start start 2024-09-12T03:36:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.951+0000 7fa95b873740 1 Processor -- start 2024-09-12T03:36:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.951+0000 7fa95b873740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f56e7fbaf0 0x55f56e803bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.951+0000 7fa95b873740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f56e7fc410 0x55f56e804110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.951+0000 7fa95b873740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f56e806cb0 0x55f56e804650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.951+0000 7fa95b873740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f56e809880 con 0x55f56e7fc410 2024-09-12T03:36:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.951+0000 7fa95b873740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f56e6e9290 con 0x55f56e806cb0 2024-09-12T03:36:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.951+0000 7fa95b873740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f56e809a00 con 0x55f56e7fbaf0 2024-09-12T03:36:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.951+0000 7fa95b873740 10 client.0 _ll_register_callbacks cb 0x55f56e71a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:36:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.951+0000 7fa95259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f56e7fc410 0x55f56e804110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.951+0000 7fa95259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f56e7fc410 0x55f56e804110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47172/0 (socket says 172.21.15.64:47172) 2024-09-12T03:36:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.951+0000 7fa95259c640 1 -- 172.21.15.64:0/1418621939 learned_addr learned my addr 172.21.15.64:0/1418621939 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:36:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.952+0000 7fa951d9b640 1 --2- 172.21.15.64:0/1418621939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f56e806cb0 0x55f56e804650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.952+0000 7fa95259c640 1 -- 172.21.15.64:0/1418621939 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f56e7fbaf0 msgr2=0x55f56e803bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:36:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.952+0000 7fa95259c640 1 --2- 172.21.15.64:0/1418621939 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f56e7fbaf0 0x55f56e803bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.952+0000 7fa95259c640 1 -- 172.21.15.64:0/1418621939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f56e806cb0 msgr2=0x55f56e804650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:36:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.952+0000 7fa95259c640 1 --2- 172.21.15.64:0/1418621939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f56e806cb0 0x55f56e804650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.952+0000 7fa95259c640 1 -- 172.21.15.64:0/1418621939 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f56e473210 con 0x55f56e7fc410 2024-09-12T03:36:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.952+0000 7fa95259c640 1 --2- 172.21.15.64:0/1418621939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f56e7fc410 0x55f56e804110 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa93c00bf70 tx=0x7fa93c0128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2942b8fe11f4210b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:36:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.952+0000 7fa939ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:36:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.952+0000 7fa939ffb640 1 -- 172.21.15.64:0/1418621939 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c01f060 con 0x55f56e7fc410 2024-09-12T03:36:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.952+0000 7fa939ffb640 1 -- 172.21.15.64:0/1418621939 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9583+0+0 (secure 0 0 0) 0x7fa93c010960 con 0x55f56e7fc410 2024-09-12T03:36:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.952+0000 7fa95b873740 1 -- 172.21.15.64:0/1418621939 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9440316e0 con 0x55f56e7fc410 2024-09-12T03:36:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.952+0000 7fa939ffb640 1 -- 172.21.15.64:0/1418621939 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa93c026050 con 0x55f56e7fc410 2024-09-12T03:36:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.952+0000 7fa95b873740 10 client.15936 fetch_fsmap learned FSMap version 25 2024-09-12T03:36:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.952+0000 7fa95b873740 1 -- 172.21.15.64:0/1418621939 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f56e53f620 con 0x55f56e7fc410 2024-09-12T03:36:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.952+0000 7fa939ffb640 1 -- 172.21.15.64:0/1418621939 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9440316e0 con 0x55f56e7fc410 2024-09-12T03:36:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.953+0000 7fa939ffb640 1 -- 172.21.15.64:0/1418621939 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa93c019030 con 0x55f56e7fc410 2024-09-12T03:36:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.953+0000 7fa95b873740 10 client.15936 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:36:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.953+0000 7fa95b873740 10 client.15936 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:36:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.953+0000 7fa95b873740 20 client.15936 populate_metadata read hostname 'smithi064' 2024-09-12T03:36:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.953+0000 7fa95b873740 1 --2- 172.21.15.64:0/1418621939 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55f56e816d40 0x55f56e819130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.953+0000 7fa95b873740 4 client.15936 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:36:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.953+0000 7fa95b873740 1 -- 172.21.15.64:0/1418621939 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f56e76da20 con 0x55f56e816d40 2024-09-12T03:36:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.953+0000 7fa951d9b640 1 --2- 172.21.15.64:0/1418621939 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55f56e816d40 0x55f56e819130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.953+0000 7fa951d9b640 1 --2- 172.21.15.64:0/1418621939 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55f56e816d40 0x55f56e819130 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69b3a946fd8ff831 server_cookie=1acf926977ecac26 in_seq=0 out_seq=0 2024-09-12T03:36:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.953+0000 7fa939ffb640 10 client.15936 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:36:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.954+0000 7fa939ffb640 1 -- 172.21.15.64:0/1418621939 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f56e76da20 con 0x55f56e816d40 2024-09-12T03:36:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.954+0000 7fa939ffb640 10 client.15936 handle_command_reply: tid=0 2024-09-12T03:36:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.954+0000 7fa95b873740 1 client.15936 shutdown 2024-09-12T03:36:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.954+0000 7fa95b873740 1 -- 172.21.15.64:0/1418621939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f56e7fc410 msgr2=0x55f56e804110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.954+0000 7fa95b873740 1 --2- 172.21.15.64:0/1418621939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f56e7fc410 0x55f56e804110 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa93c00bf70 tx=0x7fa93c0128e0 comp rx=0 tx=0).stop 2024-09-12T03:36:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.954+0000 7fa95b873740 1 -- 172.21.15.64:0/1418621939 shutdown_connections 2024-09-12T03:36:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.954+0000 7fa95b873740 1 --2- 172.21.15.64:0/1418621939 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55f56e816d40 0x55f56e819130 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.954+0000 7fa95b873740 1 --2- 172.21.15.64:0/1418621939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f56e806cb0 0x55f56e804650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.954+0000 7fa95b873740 1 --2- 172.21.15.64:0/1418621939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f56e7fc410 0x55f56e804110 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.954+0000 7fa95b873740 1 --2- 172.21.15.64:0/1418621939 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f56e7fbaf0 0x55f56e803bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.954+0000 7fa95b873740 1 -- 172.21.15.64:0/1418621939 >> 172.21.15.64:0/1418621939 conn(0x55f56e7f8060 msgr2=0x55f56e7fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:36:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.954+0000 7fa939ffb640 0 client.15936 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:36:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.954+0000 7fa95b873740 1 -- 172.21.15.64:0/1418621939 shutdown_connections 2024-09-12T03:36:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.954+0000 7fa95b873740 1 -- 172.21.15.64:0/1418621939 wait complete. 2024-09-12T03:36:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.954+0000 7fa95b873740 20 client.15936 trim_cache size 0 max 16384 2024-09-12T03:36:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.969+0000 7fa95b873740 1 Processor -- start 2024-09-12T03:36:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.969+0000 7fa95b873740 1 -- start start 2024-09-12T03:36:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.970+0000 7fa95b873740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f56e7fbaf0 0x55f56e804480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.970+0000 7fa95b873740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f56e7fc410 0x55f56e81ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.970+0000 7fa95b873740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f56e806cb0 0x55f56e81ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.970+0000 7fa95b873740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa9541b1860 con 0x55f56e7fbaf0 2024-09-12T03:36:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.970+0000 7fa95b873740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa9541af130 con 0x55f56e806cb0 2024-09-12T03:36:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.970+0000 7fa95b873740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa9541acfa0 con 0x55f56e7fc410 2024-09-12T03:36:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.970+0000 7fa951d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f56e7fbaf0 0x55f56e804480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.970+0000 7fa951d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f56e7fbaf0 0x55f56e804480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47174/0 (socket says 172.21.15.64:47174) 2024-09-12T03:36:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.970+0000 7fa951d9b640 1 -- 172.21.15.64:0/3866583 learned_addr learned my addr 172.21.15.64:0/3866583 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:36:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.970+0000 7fa951d9b640 1 -- 172.21.15.64:0/3866583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f56e7fc410 msgr2=0x55f56e81ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:36:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.970+0000 7fa951d9b640 1 --2- 172.21.15.64:0/3866583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f56e7fc410 0x55f56e81ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.970+0000 7fa951d9b640 1 -- 172.21.15.64:0/3866583 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f56e806cb0 msgr2=0x55f56e81ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:36:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.970+0000 7fa951d9b640 1 --2- 172.21.15.64:0/3866583 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f56e806cb0 0x55f56e81ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.970+0000 7fa951d9b640 1 -- 172.21.15.64:0/3866583 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f56e7f7e90 con 0x55f56e7fbaf0 2024-09-12T03:36:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.970+0000 7fa951d9b640 1 --2- 172.21.15.64:0/3866583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f56e7fbaf0 0x55f56e804480 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa944009900 tx=0x7fa9440052b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b62ce8a1a4101b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:36:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.970+0000 7fa93bfff640 1 -- 172.21.15.64:0/3866583 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa944035040 con 0x55f56e7fbaf0 2024-09-12T03:36:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.970+0000 7fa93bfff640 1 -- 172.21.15.64:0/3866583 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa944051030 con 0x55f56e7fbaf0 2024-09-12T03:36:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.970+0000 7fa93bfff640 1 -- 172.21.15.64:0/3866583 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa944004450 con 0x55f56e7fbaf0 2024-09-12T03:36:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.971+0000 7fa95b873740 1 -- 172.21.15.64:0/3866583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f56e7fbaf0 msgr2=0x55f56e804480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.971+0000 7fa95b873740 1 --2- 172.21.15.64:0/3866583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f56e7fbaf0 0x55f56e804480 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa944009900 tx=0x7fa9440052b0 comp rx=0 tx=0).stop 2024-09-12T03:36:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.971+0000 7fa95b873740 1 -- 172.21.15.64:0/3866583 shutdown_connections 2024-09-12T03:36:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.971+0000 7fa95b873740 1 --2- 172.21.15.64:0/3866583 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f56e806cb0 0x55f56e81ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.971+0000 7fa95b873740 1 --2- 172.21.15.64:0/3866583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f56e7fc410 0x55f56e81ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.971+0000 7fa95b873740 1 --2- 172.21.15.64:0/3866583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f56e7fbaf0 0x55f56e804480 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.971+0000 7fa95b873740 1 -- 172.21.15.64:0/3866583 >> 172.21.15.64:0/3866583 conn(0x55f56e7f8060 msgr2=0x55f56e805c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:36:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.971+0000 7fa95b873740 1 -- 172.21.15.64:0/3866583 shutdown_connections 2024-09-12T03:36:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.971+0000 7fa95b873740 1 -- 172.21.15.64:0/3866583 wait complete. 2024-09-12T03:36:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.971+0000 7fa95b873740 1 -- start start 2024-09-12T03:36:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.971+0000 7fa95b873740 1 Processor -- start 2024-09-12T03:36:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.971+0000 7fa95b873740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f56e7fbaf0 0x55f56e80f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.971+0000 7fa95b873740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f56e7fc410 0x55f56e80a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.971+0000 7fa95b873740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f56e806cb0 0x55f56e80a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.972+0000 7fa95b873740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa9541ad5f0 con 0x55f56e7fc410 2024-09-12T03:36:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.972+0000 7fa95b873740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa9541aea60 con 0x55f56e7fbaf0 2024-09-12T03:36:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.972+0000 7fa95b873740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa9541adcc0 con 0x55f56e806cb0 2024-09-12T03:36:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.972+0000 7fa95b873740 10 client.0 _ll_register_callbacks cb 0x7fa93c00b890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:36:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.972+0000 7fa95259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f56e7fc410 0x55f56e80a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.972+0000 7fa95259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f56e7fc410 0x55f56e80a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47176/0 (socket says 172.21.15.64:47176) 2024-09-12T03:36:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.972+0000 7fa95259c640 1 -- 172.21.15.64:0/4161616122 learned_addr learned my addr 172.21.15.64:0/4161616122 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:36:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.972+0000 7fa951d9b640 1 --2- 172.21.15.64:0/4161616122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f56e7fbaf0 0x55f56e80f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.972+0000 7fa95259c640 1 -- 172.21.15.64:0/4161616122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f56e806cb0 msgr2=0x55f56e80a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:36:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.972+0000 7fa95259c640 1 --2- 172.21.15.64:0/4161616122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f56e806cb0 0x55f56e80a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.972+0000 7fa95259c640 1 -- 172.21.15.64:0/4161616122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f56e7fbaf0 msgr2=0x55f56e80f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:36:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.972+0000 7fa95259c640 1 --2- 172.21.15.64:0/4161616122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f56e7fbaf0 0x55f56e80f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.972+0000 7fa95259c640 1 -- 172.21.15.64:0/4161616122 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f56e80aec0 con 0x55f56e7fc410 2024-09-12T03:36:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.972+0000 7fa951d9b640 1 --2- 172.21.15.64:0/4161616122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f56e7fbaf0 0x55f56e80f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:36:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.972+0000 7fa95259c640 1 --2- 172.21.15.64:0/4161616122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f56e7fc410 0x55f56e80a440 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x55f56e803440 tx=0x7fa93c00d350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ac7b5214f8fba5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:36:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.972+0000 7fa939ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:36:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.972+0000 7fa939ffb640 1 -- 172.21.15.64:0/4161616122 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c005450 con 0x55f56e7fc410 2024-09-12T03:36:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.972+0000 7fa95b873740 1 -- 172.21.15.64:0/4161616122 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa944051030 con 0x55f56e7fc410 2024-09-12T03:36:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.972+0000 7fa939ffb640 1 -- 172.21.15.64:0/4161616122 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9583+0+0 (secure 0 0 0) 0x7fa93c0055f0 con 0x55f56e7fc410 2024-09-12T03:36:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.972+0000 7fa939ffb640 1 -- 172.21.15.64:0/4161616122 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa93c01f020 con 0x55f56e7fc410 2024-09-12T03:36:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.972+0000 7fa95b873740 10 client.15942 fetch_fsmap learned FSMap version 25 2024-09-12T03:36:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.972+0000 7fa95b873740 1 -- 172.21.15.64:0/4161616122 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f56e881740 con 0x55f56e7fc410 2024-09-12T03:36:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.973+0000 7fa939ffb640 1 -- 172.21.15.64:0/4161616122 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa944051030 con 0x55f56e7fc410 2024-09-12T03:36:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.973+0000 7fa939ffb640 1 -- 172.21.15.64:0/4161616122 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa93c019070 con 0x55f56e7fc410 2024-09-12T03:36:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.973+0000 7fa95b873740 10 client.15942 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:36:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.973+0000 7fa95b873740 10 client.15942 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:36:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.973+0000 7fa95b873740 20 client.15942 populate_metadata read hostname 'smithi064' 2024-09-12T03:36:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.973+0000 7fa95b873740 1 --2- 172.21.15.64:0/4161616122 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55f56e881ef0 0x55f56e884310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.973+0000 7fa95b873740 4 client.15942 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:36:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.973+0000 7fa95b873740 1 -- 172.21.15.64:0/4161616122 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f56e767d70 con 0x55f56e881ef0 2024-09-12T03:36:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.973+0000 7fa951d9b640 1 --2- 172.21.15.64:0/4161616122 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55f56e881ef0 0x55f56e884310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.973+0000 7fa951d9b640 1 --2- 172.21.15.64:0/4161616122 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55f56e881ef0 0x55f56e884310 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fabd4927573c22b server_cookie=2e5a7aa032e80518 in_seq=0 out_seq=0 2024-09-12T03:36:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.973+0000 7fa939ffb640 10 client.15942 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:36:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.988+0000 7fa939ffb640 1 -- 172.21.15.64:0/4161616122 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f56e767d70 con 0x55f56e881ef0 2024-09-12T03:36:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.988+0000 7fa939ffb640 10 client.15942 handle_command_reply: tid=0 2024-09-12T03:36:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.988+0000 7fa95b873740 1 client.15942 shutdown 2024-09-12T03:36:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.988+0000 7fa95b873740 1 -- 172.21.15.64:0/4161616122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f56e7fc410 msgr2=0x55f56e80a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.988+0000 7fa95b873740 1 --2- 172.21.15.64:0/4161616122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f56e7fc410 0x55f56e80a440 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x55f56e803440 tx=0x7fa93c00d350 comp rx=0 tx=0).stop 2024-09-12T03:36:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.988+0000 7fa95b873740 1 -- 172.21.15.64:0/4161616122 shutdown_connections 2024-09-12T03:36:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.988+0000 7fa95b873740 1 --2- 172.21.15.64:0/4161616122 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55f56e881ef0 0x55f56e884310 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.988+0000 7fa95b873740 1 --2- 172.21.15.64:0/4161616122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f56e806cb0 0x55f56e80a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.988+0000 7fa95b873740 1 --2- 172.21.15.64:0/4161616122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f56e7fc410 0x55f56e80a440 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.988+0000 7fa939ffb640 0 client.15942 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:36:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.988+0000 7fa95b873740 1 --2- 172.21.15.64:0/4161616122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f56e7fbaf0 0x55f56e80f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.988+0000 7fa95b873740 1 -- 172.21.15.64:0/4161616122 >> 172.21.15.64:0/4161616122 conn(0x55f56e7f8060 msgr2=0x55f56e7f69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:36:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.988+0000 7fa95b873740 1 -- 172.21.15.64:0/4161616122 shutdown_connections 2024-09-12T03:36:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.988+0000 7fa95b873740 1 -- 172.21.15.64:0/4161616122 wait complete. 2024-09-12T03:36:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.988+0000 7fa95b873740 20 client.15942 trim_cache size 0 max 16384 2024-09-12T03:36:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:36:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:36:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:36:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:36:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.989+0000 7fa958a01640 1 -- 172.21.15.64:0/3129180391 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fa9400a2da0 msgr2=0x7fa9400a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.989+0000 7fa958a01640 1 --2- 172.21.15.64:0/3129180391 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fa9400a2da0 0x7fa9400a5260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa94c098220 tx=0x7fa94c06ee30 comp rx=0 tx=0).stop 2024-09-12T03:36:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.989+0000 7fa958a01640 1 -- 172.21.15.64:0/3129180391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9541069c0 msgr2=0x7fa95419cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.989+0000 7fa958a01640 1 --2- 172.21.15.64:0/3129180391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9541069c0 0x7fa95419cb00 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa94c00cce0 tx=0x7fa94c00a620 comp rx=0 tx=0).stop 2024-09-12T03:36:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.989+0000 7fa958a01640 1 -- 172.21.15.64:0/3129180391 shutdown_connections 2024-09-12T03:36:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.989+0000 7fa958a01640 1 --2- 172.21.15.64:0/3129180391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9541069c0 0x7fa95419cb00 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.989+0000 7fa958a01640 1 --2- 172.21.15.64:0/3129180391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa954101350 0x7fa954198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.989+0000 7fa958a01640 1 --2- 172.21.15.64:0/3129180391 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fa9400a2da0 0x7fa9400a5260 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.989+0000 7fa958a01640 1 --2- 172.21.15.64:0/3129180391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9541009b0 0x7fa954198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.989+0000 7fa958a01640 1 -- 172.21.15.64:0/3129180391 >> 172.21.15.64:0/3129180391 conn(0x7fa954077990 msgr2=0x7fa954069b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:36:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.989+0000 7fa958a01640 1 -- 172.21.15.64:0/3129180391 shutdown_connections 2024-09-12T03:36:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:31.989+0000 7fa958a01640 1 -- 172.21.15.64:0/3129180391 wait complete. 2024-09-12T03:36:33.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:36:33 smithi195 ceph-mon[34786]: pgmap v223: 129 pgs: 129 active+clean; 3.9 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:36:33.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:36:33 smithi064 ceph-mon[30952]: pgmap v223: 129 pgs: 129 active+clean; 3.9 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:36:33.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:36:33 smithi115 ceph-mon[34801]: pgmap v223: 129 pgs: 129 active+clean; 3.9 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:36:35.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:36:35 smithi195 ceph-mon[34786]: pgmap v224: 129 pgs: 129 active+clean; 3.9 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-12T03:36:35.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:36:35 smithi064 ceph-mon[30952]: pgmap v224: 129 pgs: 129 active+clean; 3.9 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-12T03:36:35.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:36:35 smithi115 ceph-mon[34801]: pgmap v224: 129 pgs: 129 active+clean; 3.9 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-12T03:36:36.154 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.155 INFO:tasks.workunit.client.0.smithi064.stdout:313/600: open fd = 3 2024-09-12T03:36:36.155 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.155 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.155 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.155 INFO:tasks.workunit.client.0.smithi064.stdout:314/600: open fd = 3 2024-09-12T03:36:36.155 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.155 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.155 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.155 INFO:tasks.workunit.client.0.smithi064.stdout:315/600: open fd = 3 2024-09-12T03:36:36.155 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.155 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.155 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.155 INFO:tasks.workunit.client.0.smithi064.stdout:316/600: open fd = 3 2024-09-12T03:36:36.156 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.156 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.156 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.156 INFO:tasks.workunit.client.0.smithi064.stdout:317/600: open fd = 3 2024-09-12T03:36:36.156 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.156 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.156 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.156 INFO:tasks.workunit.client.0.smithi064.stdout:318/600: open fd = 3 2024-09-12T03:36:36.156 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.156 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.156 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.156 INFO:tasks.workunit.client.0.smithi064.stdout:319/600: open fd = 3 2024-09-12T03:36:36.156 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.157 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.157 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.157 INFO:tasks.workunit.client.0.smithi064.stdout:320/600: open fd = 3 2024-09-12T03:36:36.157 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.157 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.157 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.157 INFO:tasks.workunit.client.0.smithi064.stdout:321/600: open fd = 3 2024-09-12T03:36:36.157 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.157 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.157 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.157 INFO:tasks.workunit.client.0.smithi064.stdout:322/600: open fd = 3 2024-09-12T03:36:36.157 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.157 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.157 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.158 INFO:tasks.workunit.client.0.smithi064.stdout:323/600: open fd = 3 2024-09-12T03:36:36.158 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.158 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.158 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.158 INFO:tasks.workunit.client.0.smithi064.stdout:324/600: open fd = 3 2024-09-12T03:36:36.158 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.158 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.158 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.158 INFO:tasks.workunit.client.0.smithi064.stdout:325/600: open fd = 3 2024-09-12T03:36:36.158 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.158 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.158 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.158 INFO:tasks.workunit.client.0.smithi064.stdout:326/600: open fd = 3 2024-09-12T03:36:36.158 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.158 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.159 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.159 INFO:tasks.workunit.client.0.smithi064.stdout:327/600: open fd = 3 2024-09-12T03:36:36.159 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.159 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.159 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.159 INFO:tasks.workunit.client.0.smithi064.stdout:328/600: open fd = 3 2024-09-12T03:36:36.159 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.159 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.159 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.159 INFO:tasks.workunit.client.0.smithi064.stdout:329/600: open fd = 3 2024-09-12T03:36:36.159 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.159 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.159 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.159 INFO:tasks.workunit.client.0.smithi064.stdout:330/600: open fd = 3 2024-09-12T03:36:36.160 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.160 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.160 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.160 INFO:tasks.workunit.client.0.smithi064.stdout:331/600: open fd = 3 2024-09-12T03:36:36.160 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.160 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.160 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.160 INFO:tasks.workunit.client.0.smithi064.stdout:332/600: open fd = 3 2024-09-12T03:36:36.160 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.160 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.160 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.160 INFO:tasks.workunit.client.0.smithi064.stdout:333/600: open fd = 3 2024-09-12T03:36:36.160 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.161 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.161 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.161 INFO:tasks.workunit.client.0.smithi064.stdout:334/600: open fd = 3 2024-09-12T03:36:36.161 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.161 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.161 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.161 INFO:tasks.workunit.client.0.smithi064.stdout:335/600: open fd = 3 2024-09-12T03:36:36.161 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.161 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.161 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.161 INFO:tasks.workunit.client.0.smithi064.stdout:336/600: open fd = 3 2024-09-12T03:36:36.161 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.161 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.162 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.162 INFO:tasks.workunit.client.0.smithi064.stdout:337/600: open fd = 3 2024-09-12T03:36:36.162 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.162 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.162 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.162 INFO:tasks.workunit.client.0.smithi064.stdout:338/600: open fd = 3 2024-09-12T03:36:36.162 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.162 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.162 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.162 INFO:tasks.workunit.client.0.smithi064.stdout:339/600: open fd = 3 2024-09-12T03:36:36.162 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.162 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.162 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.163 INFO:tasks.workunit.client.0.smithi064.stdout:340/600: open fd = 3 2024-09-12T03:36:36.163 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.163 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.163 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.163 INFO:tasks.workunit.client.0.smithi064.stdout:341/600: open fd = 3 2024-09-12T03:36:36.163 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.163 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.163 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.163 INFO:tasks.workunit.client.0.smithi064.stdout:342/600: open fd = 3 2024-09-12T03:36:36.164 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.164 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.164 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.164 INFO:tasks.workunit.client.0.smithi064.stdout:343/600: open fd = 3 2024-09-12T03:36:36.164 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.164 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.164 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.164 INFO:tasks.workunit.client.0.smithi064.stdout:344/600: open fd = 3 2024-09-12T03:36:36.164 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.164 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.164 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.164 INFO:tasks.workunit.client.0.smithi064.stdout:345/600: open fd = 3 2024-09-12T03:36:36.164 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.165 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.165 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.165 INFO:tasks.workunit.client.0.smithi064.stdout:346/600: open fd = 3 2024-09-12T03:36:36.165 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.165 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.165 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.165 INFO:tasks.workunit.client.0.smithi064.stdout:347/600: open fd = 3 2024-09-12T03:36:36.165 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.165 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.165 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.165 INFO:tasks.workunit.client.0.smithi064.stdout:348/600: open fd = 3 2024-09-12T03:36:36.165 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.165 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.165 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.166 INFO:tasks.workunit.client.0.smithi064.stdout:349/600: open fd = 3 2024-09-12T03:36:36.166 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.166 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.166 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.166 INFO:tasks.workunit.client.0.smithi064.stdout:350/600: open fd = 3 2024-09-12T03:36:36.166 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.167 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.167 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.167 INFO:tasks.workunit.client.0.smithi064.stdout:351/600: open fd = 3 2024-09-12T03:36:36.167 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.167 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.167 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.168 INFO:tasks.workunit.client.0.smithi064.stdout:352/600: open fd = 3 2024-09-12T03:36:36.168 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.168 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.168 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.168 INFO:tasks.workunit.client.0.smithi064.stdout:353/600: open fd = 3 2024-09-12T03:36:36.168 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.168 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.168 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.168 INFO:tasks.workunit.client.0.smithi064.stdout:354/600: open fd = 3 2024-09-12T03:36:36.168 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.168 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.168 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.168 INFO:tasks.workunit.client.0.smithi064.stdout:355/600: open fd = 3 2024-09-12T03:36:36.169 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.169 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.169 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.169 INFO:tasks.workunit.client.0.smithi064.stdout:356/600: open fd = 3 2024-09-12T03:36:36.169 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.169 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.169 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.169 INFO:tasks.workunit.client.0.smithi064.stdout:357/600: open fd = 3 2024-09-12T03:36:36.169 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.169 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.169 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.169 INFO:tasks.workunit.client.0.smithi064.stdout:358/600: open fd = 3 2024-09-12T03:36:36.169 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.169 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.170 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.170 INFO:tasks.workunit.client.0.smithi064.stdout:359/600: open fd = 3 2024-09-12T03:36:36.170 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.170 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.170 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.170 INFO:tasks.workunit.client.0.smithi064.stdout:360/600: open fd = 3 2024-09-12T03:36:36.170 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.170 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.170 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.170 INFO:tasks.workunit.client.0.smithi064.stdout:361/600: open fd = 3 2024-09-12T03:36:36.170 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.170 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.170 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.170 INFO:tasks.workunit.client.0.smithi064.stdout:362/600: open fd = 3 2024-09-12T03:36:36.170 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.171 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.171 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.171 INFO:tasks.workunit.client.0.smithi064.stdout:363/600: open fd = 3 2024-09-12T03:36:36.171 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.171 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.171 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.171 INFO:tasks.workunit.client.0.smithi064.stdout:364/600: open fd = 3 2024-09-12T03:36:36.171 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.171 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.171 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.171 INFO:tasks.workunit.client.0.smithi064.stdout:365/600: open fd = 3 2024-09-12T03:36:36.171 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.171 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.171 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.172 INFO:tasks.workunit.client.0.smithi064.stdout:366/600: open fd = 3 2024-09-12T03:36:36.172 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.172 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.172 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.172 INFO:tasks.workunit.client.0.smithi064.stdout:367/600: open fd = 3 2024-09-12T03:36:36.172 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.172 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.172 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.172 INFO:tasks.workunit.client.0.smithi064.stdout:368/600: open fd = 3 2024-09-12T03:36:36.172 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.172 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.172 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.172 INFO:tasks.workunit.client.0.smithi064.stdout:369/600: open fd = 3 2024-09-12T03:36:36.173 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.173 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.173 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.173 INFO:tasks.workunit.client.0.smithi064.stdout:370/600: open fd = 3 2024-09-12T03:36:36.173 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.173 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.173 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.173 INFO:tasks.workunit.client.0.smithi064.stdout:371/600: open fd = 3 2024-09-12T03:36:36.173 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.173 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.173 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.174 INFO:tasks.workunit.client.0.smithi064.stdout:372/600: open fd = 3 2024-09-12T03:36:36.174 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.174 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.174 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.174 INFO:tasks.workunit.client.0.smithi064.stdout:373/600: open fd = 3 2024-09-12T03:36:36.174 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.174 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.174 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:36.174 INFO:tasks.workunit.client.0.smithi064.stdout:374/600: open fd = 3 2024-09-12T03:36:36.174 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.174 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:36:36.175 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:36:37.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:36:37 smithi195 ceph-mon[34786]: pgmap v225: 129 pgs: 129 active+clean; 3.9 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-12T03:36:37.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:36:37 smithi064 ceph-mon[30952]: pgmap v225: 129 pgs: 129 active+clean; 3.9 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-12T03:36:37.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:36:37 smithi115 ceph-mon[34801]: pgmap v225: 129 pgs: 129 active+clean; 3.9 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-12T03:36:39.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:36:39 smithi195 ceph-mon[34786]: pgmap v226: 129 pgs: 129 active+clean; 4.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:36:39.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:36:39 smithi064 ceph-mon[30952]: pgmap v226: 129 pgs: 129 active+clean; 4.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:36:39.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:36:39 smithi115 ceph-mon[34801]: pgmap v226: 129 pgs: 129 active+clean; 4.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:36:41.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:36:41 smithi195 ceph-mon[34786]: pgmap v227: 129 pgs: 129 active+clean; 4.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:36:41.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:36:41 smithi064 ceph-mon[30952]: pgmap v227: 129 pgs: 129 active+clean; 4.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:36:41.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:36:41 smithi115 ceph-mon[34801]: pgmap v227: 129 pgs: 129 active+clean; 4.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:36:43.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:36:43 smithi195 ceph-mon[34786]: pgmap v228: 129 pgs: 129 active+clean; 4.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-09-12T03:36:43.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:36:43 smithi064 ceph-mon[30952]: pgmap v228: 129 pgs: 129 active+clean; 4.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-09-12T03:36:43.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:36:43 smithi115 ceph-mon[34801]: pgmap v228: 129 pgs: 129 active+clean; 4.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-09-12T03:36:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.089+0000 7f0c5300e640 1 -- 172.21.15.64:0/1169635572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c4c1080f0 msgr2=0x7f0c4c10ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.089+0000 7f0c5300e640 1 --2- 172.21.15.64:0/1169635572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c4c1080f0 0x7f0c4c10ea10 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0c44009ca0 tx=0x7f0c4402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:36:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.089+0000 7f0c5300e640 1 -- 172.21.15.64:0/1169635572 shutdown_connections 2024-09-12T03:36:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.089+0000 7f0c5300e640 1 --2- 172.21.15.64:0/1169635572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c4c1080f0 0x7f0c4c10ea10 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.089+0000 7f0c5300e640 1 --2- 172.21.15.64:0/1169635572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c4c101250 0x7f0c4c107990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.089+0000 7f0c5300e640 1 --2- 172.21.15.64:0/1169635572 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c4c1008a0 0x7f0c4c100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.089+0000 7f0c5300e640 1 -- 172.21.15.64:0/1169635572 >> 172.21.15.64:0/1169635572 conn(0x7f0c4c075ef0 msgr2=0x7f0c4c076300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:36:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.089+0000 7f0c5300e640 1 -- 172.21.15.64:0/1169635572 shutdown_connections 2024-09-12T03:36:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.089+0000 7f0c5300e640 1 -- 172.21.15.64:0/1169635572 wait complete. 2024-09-12T03:36:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.090+0000 7f0c5300e640 1 Processor -- start 2024-09-12T03:36:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.090+0000 7f0c5300e640 1 -- start start 2024-09-12T03:36:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.090+0000 7f0c5300e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c4c1008a0 0x7f0c4c101810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.090+0000 7f0c5300e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c4c1080f0 0x7f0c4c101d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.090+0000 7f0c5300e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c4c102290 0x7f0c4c1a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.090+0000 7f0c5300e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4c111dd0 con 0x7f0c4c102290 2024-09-12T03:36:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.090+0000 7f0c5300e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4c111c50 con 0x7f0c4c1080f0 2024-09-12T03:36:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.090+0000 7f0c5300e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4c111f50 con 0x7f0c4c1008a0 2024-09-12T03:36:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.090+0000 7f0c515ab640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c4c102290 0x7f0c4c1a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.090+0000 7f0c515ab640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c4c102290 0x7f0c4c1a6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39440/0 (socket says 172.21.15.64:39440) 2024-09-12T03:36:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.090+0000 7f0c515ab640 1 -- 172.21.15.64:0/4026833862 learned_addr learned my addr 172.21.15.64:0/4026833862 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:36:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.090+0000 7f0c50daa640 1 --2- 172.21.15.64:0/4026833862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c4c1008a0 0x7f0c4c101810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.090+0000 7f0c4bfff640 1 --2- 172.21.15.64:0/4026833862 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c4c1080f0 0x7f0c4c101d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.090+0000 7f0c515ab640 1 -- 172.21.15.64:0/4026833862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c4c1008a0 msgr2=0x7f0c4c101810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.090+0000 7f0c515ab640 1 --2- 172.21.15.64:0/4026833862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c4c1008a0 0x7f0c4c101810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.090+0000 7f0c515ab640 1 -- 172.21.15.64:0/4026833862 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c4c1080f0 msgr2=0x7f0c4c101d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.090+0000 7f0c515ab640 1 --2- 172.21.15.64:0/4026833862 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c4c1080f0 0x7f0c4c101d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.090+0000 7f0c515ab640 1 -- 172.21.15.64:0/4026833862 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c4c1a6a30 con 0x7f0c4c102290 2024-09-12T03:36:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.091+0000 7f0c515ab640 1 --2- 172.21.15.64:0/4026833862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c4c102290 0x7f0c4c1a6330 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0c44009dc0 tx=0x7f0c44032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:36:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.091+0000 7f0c49ffb640 1 -- 172.21.15.64:0/4026833862 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c44003cf0 con 0x7f0c4c102290 2024-09-12T03:36:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.091+0000 7f0c49ffb640 1 -- 172.21.15.64:0/4026833862 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c440044f0 con 0x7f0c4c102290 2024-09-12T03:36:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.091+0000 7f0c5300e640 1 -- 172.21.15.64:0/4026833862 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c4c1a6cc0 con 0x7f0c4c102290 2024-09-12T03:36:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.091+0000 7f0c49ffb640 1 -- 172.21.15.64:0/4026833862 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c44030b80 con 0x7f0c4c102290 2024-09-12T03:36:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.091+0000 7f0c5300e640 1 -- 172.21.15.64:0/4026833862 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c4c1a7210 con 0x7f0c4c102290 2024-09-12T03:36:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.094+0000 7f0c49ffb640 1 -- 172.21.15.64:0/4026833862 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0c440046a0 con 0x7f0c4c102290 2024-09-12T03:36:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.094+0000 7f0c55e85740 1 Processor -- start 2024-09-12T03:36:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.095+0000 7f0c55e85740 1 -- start start 2024-09-12T03:36:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.095+0000 7f0c55e85740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7a9a38af0 0x55f7a9a38ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.095+0000 7f0c55e85740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7a9a39410 0x55f7a9a43770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.095+0000 7f0c55e85740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f7a9a43cb0 0x55f7a9a460a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.095+0000 7f0c55e85740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f7a999d2f0 con 0x55f7a9a39410 2024-09-12T03:36:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.095+0000 7f0c55e85740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f7a9945270 con 0x55f7a9a38af0 2024-09-12T03:36:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.095+0000 7f0c55e85740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f7a99992b0 con 0x55f7a9a43cb0 2024-09-12T03:36:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.095+0000 7f0c50daa640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7a9a39410 0x55f7a9a43770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.095+0000 7f0c50daa640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7a9a39410 0x55f7a9a43770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39456/0 (socket says 172.21.15.64:39456) 2024-09-12T03:36:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.095+0000 7f0c50daa640 1 -- 172.21.15.64:0/3442359466 learned_addr learned my addr 172.21.15.64:0/3442359466 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:36:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.095+0000 7f0c4bfff640 1 --2- 172.21.15.64:0/3442359466 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7a9a38af0 0x55f7a9a38ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.095+0000 7f0c4bfff640 1 --2- 172.21.15.64:0/3442359466 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f7a9a43cb0 0x55f7a9a460a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.095+0000 7f0c50daa640 1 -- 172.21.15.64:0/3442359466 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f7a9a43cb0 msgr2=0x55f7a9a460a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:36:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.095+0000 7f0c50daa640 1 --2- 172.21.15.64:0/3442359466 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f7a9a43cb0 0x55f7a9a460a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.095+0000 7f0c50daa640 1 -- 172.21.15.64:0/3442359466 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7a9a38af0 msgr2=0x55f7a9a38ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:36:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.095+0000 7f0c50daa640 1 --2- 172.21.15.64:0/3442359466 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7a9a38af0 0x55f7a9a38ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.095+0000 7f0c49ffb640 1 --2- 172.21.15.64:0/4026833862 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f0c340a2dd0 0x7f0c340a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.095+0000 7f0c50daa640 1 -- 172.21.15.64:0/3442359466 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7a9944340 con 0x55f7a9a39410 2024-09-12T03:36:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.095+0000 7f0c515ab640 1 --2- 172.21.15.64:0/4026833862 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f0c340a2dd0 0x7f0c340a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.096+0000 7f0c50daa640 1 --2- 172.21.15.64:0/3442359466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7a9a39410 0x55f7a9a43770 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c00ad00 tx=0x7f0c3c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54daa814eb38b2b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:36:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.096+0000 7f0c49ffb640 1 -- 172.21.15.64:0/4026833862 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9583+0+0 (secure 0 0 0) 0x7f0c440f0cb0 con 0x7f0c4c102290 2024-09-12T03:36:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.096+0000 7f0c32ffd640 1 -- 172.21.15.64:0/3442359466 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c01d070 con 0x55f7a9a39410 2024-09-12T03:36:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.096+0000 7f0c32ffd640 1 -- 172.21.15.64:0/3442359466 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c3c018af0 con 0x55f7a9a39410 2024-09-12T03:36:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.096+0000 7f0c32ffd640 1 -- 172.21.15.64:0/3442359466 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c010ba0 con 0x55f7a9a39410 2024-09-12T03:36:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.096+0000 7f0c55e85740 1 -- 172.21.15.64:0/3442359466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7a9a39410 msgr2=0x55f7a9a43770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.096+0000 7f0c515ab640 1 --2- 172.21.15.64:0/4026833862 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f0c340a2dd0 0x7f0c340a5290 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0c440ee1a0 tx=0x7f0c4408ff30 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:36:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.096+0000 7f0c55e85740 1 --2- 172.21.15.64:0/3442359466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7a9a39410 0x55f7a9a43770 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c00ad00 tx=0x7f0c3c00f790 comp rx=0 tx=0).stop 2024-09-12T03:36:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.096+0000 7f0c55e85740 1 -- 172.21.15.64:0/3442359466 shutdown_connections 2024-09-12T03:36:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.097+0000 7f0c55e85740 1 --2- 172.21.15.64:0/3442359466 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f7a9a43cb0 0x55f7a9a460a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.097+0000 7f0c55e85740 1 --2- 172.21.15.64:0/3442359466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7a9a39410 0x55f7a9a43770 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.097+0000 7f0c55e85740 1 --2- 172.21.15.64:0/3442359466 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7a9a38af0 0x55f7a9a38ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.097+0000 7f0c55e85740 1 -- 172.21.15.64:0/3442359466 >> 172.21.15.64:0/3442359466 conn(0x55f7a9a35060 msgr2=0x55f7a9a35440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:36:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.097+0000 7f0c55e85740 1 -- 172.21.15.64:0/3442359466 shutdown_connections 2024-09-12T03:36:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.097+0000 7f0c55e85740 1 -- 172.21.15.64:0/3442359466 wait complete. 2024-09-12T03:36:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.097+0000 7f0c55e85740 1 -- start start 2024-09-12T03:36:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.097+0000 7f0c55e85740 1 Processor -- start 2024-09-12T03:36:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.097+0000 7f0c55e85740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7a9a38af0 0x55f7a9a40b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.097+0000 7f0c55e85740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f7a9a39410 0x55f7a9a410a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.097+0000 7f0c55e85740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7a9a43cb0 0x55f7a9a415e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.097+0000 7f0c55e85740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f7a9a46880 con 0x55f7a9a38af0 2024-09-12T03:36:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.097+0000 7f0c55e85740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f7a9926290 con 0x55f7a9a43cb0 2024-09-12T03:36:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.097+0000 7f0c55e85740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f7a9a46a00 con 0x55f7a9a39410 2024-09-12T03:36:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.098+0000 7f0c55e85740 10 client.0 _ll_register_callbacks cb 0x55f7a9957140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:36:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.098+0000 7f0c4bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7a9a38af0 0x55f7a9a40b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.098+0000 7f0c4bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7a9a38af0 0x55f7a9a40b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39470/0 (socket says 172.21.15.64:39470) 2024-09-12T03:36:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.098+0000 7f0c4bfff640 1 -- 172.21.15.64:0/276628697 learned_addr learned my addr 172.21.15.64:0/276628697 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:36:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.098+0000 7f0c4bfff640 1 --2- 172.21.15.64:0/276628697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7a9a43cb0 0x55f7a9a415e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.098+0000 7f0c50daa640 1 --2- 172.21.15.64:0/276628697 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f7a9a39410 0x55f7a9a410a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.098+0000 7f0c4bfff640 1 -- 172.21.15.64:0/276628697 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f7a9a39410 msgr2=0x55f7a9a410a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:36:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.098+0000 7f0c4bfff640 1 --2- 172.21.15.64:0/276628697 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f7a9a39410 0x55f7a9a410a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.098+0000 7f0c4bfff640 1 -- 172.21.15.64:0/276628697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7a9a43cb0 msgr2=0x55f7a9a415e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:36:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.098+0000 7f0c4bfff640 1 --2- 172.21.15.64:0/276628697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7a9a43cb0 0x55f7a9a415e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.098+0000 7f0c4bfff640 1 -- 172.21.15.64:0/276628697 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7a96b0210 con 0x55f7a9a38af0 2024-09-12T03:36:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.098+0000 7f0c4bfff640 1 --2- 172.21.15.64:0/276628697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7a9a38af0 0x55f7a9a40b60 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0c4000ce40 tx=0x7f0c4000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd18ba80d098bb72 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:36:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.098+0000 7f0c2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:36:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.098+0000 7f0c2bfff640 1 -- 172.21.15.64:0/276628697 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c40014110 con 0x55f7a9a38af0 2024-09-12T03:36:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.098+0000 7f0c2bfff640 1 -- 172.21.15.64:0/276628697 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9583+0+0 (secure 0 0 0) 0x7f0c40005020 con 0x55f7a9a38af0 2024-09-12T03:36:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.098+0000 7f0c55e85740 1 -- 172.21.15.64:0/276628697 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c3c018af0 con 0x55f7a9a38af0 2024-09-12T03:36:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.099+0000 7f0c2bfff640 1 -- 172.21.15.64:0/276628697 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0c40026050 con 0x55f7a9a38af0 2024-09-12T03:36:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.099+0000 7f0c55e85740 10 client.15954 fetch_fsmap learned FSMap version 25 2024-09-12T03:36:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.099+0000 7f0c55e85740 1 -- 172.21.15.64:0/276628697 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f7a977c620 con 0x55f7a9a38af0 2024-09-12T03:36:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.099+0000 7f0c2bfff640 1 -- 172.21.15.64:0/276628697 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c3c018af0 con 0x55f7a9a38af0 2024-09-12T03:36:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.099+0000 7f0c2bfff640 1 -- 172.21.15.64:0/276628697 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f0c40004cd0 con 0x55f7a9a38af0 2024-09-12T03:36:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.099+0000 7f0c55e85740 10 client.15954 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:36:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.099+0000 7f0c55e85740 10 client.15954 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:36:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.099+0000 7f0c55e85740 20 client.15954 populate_metadata read hostname 'smithi064' 2024-09-12T03:36:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.099+0000 7f0c55e85740 1 --2- 172.21.15.64:0/276628697 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55f7a9a53c80 0x55f7a9a56070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.099+0000 7f0c55e85740 4 client.15954 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:36:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.099+0000 7f0c55e85740 1 -- 172.21.15.64:0/276628697 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7a99aaa20 con 0x55f7a9a53c80 2024-09-12T03:36:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.099+0000 7f0c50daa640 1 --2- 172.21.15.64:0/276628697 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55f7a9a53c80 0x55f7a9a56070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.100+0000 7f0c50daa640 1 --2- 172.21.15.64:0/276628697 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55f7a9a53c80 0x55f7a9a56070 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ded6b0bbc682bf69 server_cookie=8d1e6efa5d98a035 in_seq=0 out_seq=0 2024-09-12T03:36:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.100+0000 7f0c2bfff640 10 client.15954 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:36:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.101+0000 7f0c2bfff640 1 -- 172.21.15.64:0/276628697 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f7a99aaa20 con 0x55f7a9a53c80 2024-09-12T03:36:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.101+0000 7f0c2bfff640 10 client.15954 handle_command_reply: tid=0 2024-09-12T03:36:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.101+0000 7f0c55e85740 1 client.15954 shutdown 2024-09-12T03:36:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.101+0000 7f0c55e85740 1 -- 172.21.15.64:0/276628697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7a9a38af0 msgr2=0x55f7a9a40b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.101+0000 7f0c55e85740 1 --2- 172.21.15.64:0/276628697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7a9a38af0 0x55f7a9a40b60 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0c4000ce40 tx=0x7f0c4000c760 comp rx=0 tx=0).stop 2024-09-12T03:36:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.101+0000 7f0c55e85740 1 -- 172.21.15.64:0/276628697 shutdown_connections 2024-09-12T03:36:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.101+0000 7f0c55e85740 1 --2- 172.21.15.64:0/276628697 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55f7a9a53c80 0x55f7a9a56070 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.101+0000 7f0c55e85740 1 --2- 172.21.15.64:0/276628697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7a9a43cb0 0x55f7a9a415e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.101+0000 7f0c55e85740 1 --2- 172.21.15.64:0/276628697 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f7a9a39410 0x55f7a9a410a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.101+0000 7f0c55e85740 1 --2- 172.21.15.64:0/276628697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7a9a38af0 0x55f7a9a40b60 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.101+0000 7f0c2bfff640 0 client.15954 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:36:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.101+0000 7f0c55e85740 1 -- 172.21.15.64:0/276628697 >> 172.21.15.64:0/276628697 conn(0x55f7a9a35060 msgr2=0x55f7a9a3adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:36:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.101+0000 7f0c55e85740 1 -- 172.21.15.64:0/276628697 shutdown_connections 2024-09-12T03:36:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.101+0000 7f0c55e85740 1 -- 172.21.15.64:0/276628697 wait complete. 2024-09-12T03:36:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.101+0000 7f0c55e85740 20 client.15954 trim_cache size 0 max 16384 2024-09-12T03:36:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.116+0000 7f0c55e85740 1 Processor -- start 2024-09-12T03:36:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.116+0000 7f0c55e85740 1 -- start start 2024-09-12T03:36:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.116+0000 7f0c55e85740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7a9a38af0 0x55f7a9a3b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.116+0000 7f0c55e85740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f7a9a39410 0x55f7a9a41410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.116+0000 7f0c55e85740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7a9a43cb0 0x55f7a9a5b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.116+0000 7f0c55e85740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4c1b3ab0 con 0x55f7a9a43cb0 2024-09-12T03:36:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.116+0000 7f0c55e85740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4c1b1380 con 0x55f7a9a38af0 2024-09-12T03:36:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.116+0000 7f0c55e85740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4c1af1f0 con 0x55f7a9a39410 2024-09-12T03:36:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.116+0000 7f0c4bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7a9a43cb0 0x55f7a9a5b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.116+0000 7f0c4bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7a9a43cb0 0x55f7a9a5b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39480/0 (socket says 172.21.15.64:39480) 2024-09-12T03:36:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.116+0000 7f0c4bfff640 1 -- 172.21.15.64:0/411125627 learned_addr learned my addr 172.21.15.64:0/411125627 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:36:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.116+0000 7f0c50daa640 1 --2- 172.21.15.64:0/411125627 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f7a9a39410 0x55f7a9a41410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.116+0000 7f0c4bfff640 1 --2- 172.21.15.64:0/411125627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7a9a38af0 0x55f7a9a3b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.116+0000 7f0c4bfff640 1 -- 172.21.15.64:0/411125627 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f7a9a39410 msgr2=0x55f7a9a41410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:36:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.116+0000 7f0c4bfff640 1 --2- 172.21.15.64:0/411125627 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f7a9a39410 0x55f7a9a41410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.116+0000 7f0c4bfff640 1 -- 172.21.15.64:0/411125627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7a9a38af0 msgr2=0x55f7a9a3b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:36:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.116+0000 7f0c4bfff640 1 --2- 172.21.15.64:0/411125627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7a9a38af0 0x55f7a9a3b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.116+0000 7f0c4bfff640 1 -- 172.21.15.64:0/411125627 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7a9a34e90 con 0x55f7a9a43cb0 2024-09-12T03:36:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.117+0000 7f0c4bfff640 1 --2- 172.21.15.64:0/411125627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7a9a43cb0 0x55f7a9a5b980 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f0c400059f0 tx=0x7f0c4000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eac830ea4f0fda4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:36:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.117+0000 7f0c327fc640 1 -- 172.21.15.64:0/411125627 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c40014110 con 0x55f7a9a43cb0 2024-09-12T03:36:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.117+0000 7f0c327fc640 1 -- 172.21.15.64:0/411125627 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c40012b10 con 0x55f7a9a43cb0 2024-09-12T03:36:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.117+0000 7f0c327fc640 1 -- 172.21.15.64:0/411125627 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c400208c0 con 0x55f7a9a43cb0 2024-09-12T03:36:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.117+0000 7f0c55e85740 1 -- 172.21.15.64:0/411125627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7a9a43cb0 msgr2=0x55f7a9a5b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.117+0000 7f0c55e85740 1 --2- 172.21.15.64:0/411125627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7a9a43cb0 0x55f7a9a5b980 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f0c400059f0 tx=0x7f0c4000cd70 comp rx=0 tx=0).stop 2024-09-12T03:36:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.117+0000 7f0c55e85740 1 -- 172.21.15.64:0/411125627 shutdown_connections 2024-09-12T03:36:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.117+0000 7f0c55e85740 1 --2- 172.21.15.64:0/411125627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7a9a43cb0 0x55f7a9a5b980 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.117+0000 7f0c55e85740 1 --2- 172.21.15.64:0/411125627 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f7a9a39410 0x55f7a9a41410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.117+0000 7f0c55e85740 1 --2- 172.21.15.64:0/411125627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7a9a38af0 0x55f7a9a3b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.117+0000 7f0c55e85740 1 -- 172.21.15.64:0/411125627 >> 172.21.15.64:0/411125627 conn(0x55f7a9a35060 msgr2=0x55f7a9a42be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:36:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.118+0000 7f0c55e85740 1 -- 172.21.15.64:0/411125627 shutdown_connections 2024-09-12T03:36:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.118+0000 7f0c55e85740 1 -- 172.21.15.64:0/411125627 wait complete. 2024-09-12T03:36:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.118+0000 7f0c55e85740 1 -- start start 2024-09-12T03:36:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.118+0000 7f0c55e85740 1 Processor -- start 2024-09-12T03:36:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.118+0000 7f0c55e85740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f7a9a38af0 0x55f7a9a4c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.118+0000 7f0c55e85740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7a9a39410 0x55f7a9a3d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.118+0000 7f0c55e85740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7a9a43cb0 0x55f7a9a3dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.118+0000 7f0c55e85740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4c1af840 con 0x55f7a9a39410 2024-09-12T03:36:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.118+0000 7f0c55e85740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4c1b0cb0 con 0x55f7a9a43cb0 2024-09-12T03:36:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.118+0000 7f0c55e85740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4c1aff10 con 0x55f7a9a38af0 2024-09-12T03:36:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.118+0000 7f0c55e85740 10 client.0 _ll_register_callbacks cb 0x7f0c40014990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:36:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.118+0000 7f0c50daa640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7a9a39410 0x55f7a9a3d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.118+0000 7f0c50daa640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7a9a39410 0x55f7a9a3d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39494/0 (socket says 172.21.15.64:39494) 2024-09-12T03:36:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.118+0000 7f0c50daa640 1 -- 172.21.15.64:0/206941695 learned_addr learned my addr 172.21.15.64:0/206941695 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:36:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.119+0000 7f0c4bfff640 1 --2- 172.21.15.64:0/206941695 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f7a9a38af0 0x55f7a9a4c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.119+0000 7f0c4bfff640 1 --2- 172.21.15.64:0/206941695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7a9a43cb0 0x55f7a9a3dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.119+0000 7f0c50daa640 1 -- 172.21.15.64:0/206941695 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f7a9a38af0 msgr2=0x55f7a9a4c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:36:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.119+0000 7f0c50daa640 1 --2- 172.21.15.64:0/206941695 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f7a9a38af0 0x55f7a9a4c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.119+0000 7f0c50daa640 1 -- 172.21.15.64:0/206941695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7a9a43cb0 msgr2=0x55f7a9a3dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:36:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.119+0000 7f0c50daa640 1 --2- 172.21.15.64:0/206941695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7a9a43cb0 0x55f7a9a3dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.119+0000 7f0c50daa640 1 -- 172.21.15.64:0/206941695 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7a9a3e250 con 0x55f7a9a39410 2024-09-12T03:36:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.119+0000 7f0c4bfff640 1 --2- 172.21.15.64:0/206941695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7a9a43cb0 0x55f7a9a3dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:36:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.119+0000 7f0c50daa640 1 --2- 172.21.15.64:0/206941695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7a9a39410 0x55f7a9a3d6a0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c00ad00 tx=0x7f0c3c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db8f565dd0a482e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:36:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.119+0000 7f0c2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:36:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.119+0000 7f0c2bfff640 1 -- 172.21.15.64:0/206941695 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c01d040 con 0x55f7a9a39410 2024-09-12T03:36:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.119+0000 7f0c55e85740 1 -- 172.21.15.64:0/206941695 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c40012b10 con 0x55f7a9a39410 2024-09-12T03:36:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.119+0000 7f0c2bfff640 1 -- 172.21.15.64:0/206941695 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9583+0+0 (secure 0 0 0) 0x7f0c3c0185e0 con 0x55f7a9a39410 2024-09-12T03:36:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.119+0000 7f0c2bfff640 1 -- 172.21.15.64:0/206941695 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0c3c03a050 con 0x55f7a9a39410 2024-09-12T03:36:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.119+0000 7f0c55e85740 10 client.15960 fetch_fsmap learned FSMap version 25 2024-09-12T03:36:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.119+0000 7f0c55e85740 1 -- 172.21.15.64:0/206941695 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f7a9abe810 con 0x55f7a9a39410 2024-09-12T03:36:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.120+0000 7f0c2bfff640 1 -- 172.21.15.64:0/206941695 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c40012b10 con 0x55f7a9a39410 2024-09-12T03:36:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.120+0000 7f0c2bfff640 1 -- 172.21.15.64:0/206941695 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f0c3c010830 con 0x55f7a9a39410 2024-09-12T03:36:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.120+0000 7f0c55e85740 10 client.15960 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:36:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.120+0000 7f0c55e85740 10 client.15960 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:36:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.120+0000 7f0c55e85740 20 client.15960 populate_metadata read hostname 'smithi064' 2024-09-12T03:36:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.120+0000 7f0c55e85740 1 --2- 172.21.15.64:0/206941695 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55f7a9abecb0 0x55f7a9ac10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.120+0000 7f0c55e85740 4 client.15960 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:36:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.120+0000 7f0c55e85740 1 -- 172.21.15.64:0/206941695 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f7a99a4d70 con 0x55f7a9abecb0 2024-09-12T03:36:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.120+0000 7f0c4bfff640 1 --2- 172.21.15.64:0/206941695 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55f7a9abecb0 0x55f7a9ac10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:45.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.120+0000 7f0c2bfff640 10 client.15960 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:36:45.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.120+0000 7f0c4bfff640 1 --2- 172.21.15.64:0/206941695 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55f7a9abecb0 0x55f7a9ac10a0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26a1be4fef28353 server_cookie=ac98e9e5a3e6cb12 in_seq=0 out_seq=0 2024-09-12T03:36:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.136+0000 7f0c2bfff640 1 -- 172.21.15.64:0/206941695 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f7a99a4d70 con 0x55f7a9abecb0 2024-09-12T03:36:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.136+0000 7f0c2bfff640 10 client.15960 handle_command_reply: tid=0 2024-09-12T03:36:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.136+0000 7f0c55e85740 1 client.15960 shutdown 2024-09-12T03:36:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.136+0000 7f0c55e85740 1 -- 172.21.15.64:0/206941695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7a9a39410 msgr2=0x55f7a9a3d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.136+0000 7f0c55e85740 1 --2- 172.21.15.64:0/206941695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7a9a39410 0x55f7a9a3d6a0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c00ad00 tx=0x7f0c3c005e80 comp rx=0 tx=0).stop 2024-09-12T03:36:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.136+0000 7f0c55e85740 1 -- 172.21.15.64:0/206941695 shutdown_connections 2024-09-12T03:36:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.136+0000 7f0c55e85740 1 --2- 172.21.15.64:0/206941695 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55f7a9abecb0 0x55f7a9ac10a0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.136+0000 7f0c55e85740 1 --2- 172.21.15.64:0/206941695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7a9a43cb0 0x55f7a9a3dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.136+0000 7f0c2bfff640 0 client.15960 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:36:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.136+0000 7f0c55e85740 1 --2- 172.21.15.64:0/206941695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7a9a39410 0x55f7a9a3d6a0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.136+0000 7f0c55e85740 1 --2- 172.21.15.64:0/206941695 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f7a9a38af0 0x55f7a9a4c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.136+0000 7f0c55e85740 1 -- 172.21.15.64:0/206941695 >> 172.21.15.64:0/206941695 conn(0x55f7a9a35060 msgr2=0x55f7a9a339a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:36:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.136+0000 7f0c55e85740 1 -- 172.21.15.64:0/206941695 shutdown_connections 2024-09-12T03:36:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.136+0000 7f0c55e85740 1 -- 172.21.15.64:0/206941695 wait complete. 2024-09-12T03:36:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.136+0000 7f0c55e85740 20 client.15960 trim_cache size 0 max 16384 2024-09-12T03:36:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:36:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:36:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:36:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:36:45.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.137+0000 7f0c5300e640 1 -- 172.21.15.64:0/4026833862 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f0c340a2dd0 msgr2=0x7f0c340a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:45.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.137+0000 7f0c5300e640 1 --2- 172.21.15.64:0/4026833862 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f0c340a2dd0 0x7f0c340a5290 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0c440ee1a0 tx=0x7f0c4408ff30 comp rx=0 tx=0).stop 2024-09-12T03:36:45.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.137+0000 7f0c5300e640 1 -- 172.21.15.64:0/4026833862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c4c102290 msgr2=0x7f0c4c1a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:45.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.137+0000 7f0c5300e640 1 --2- 172.21.15.64:0/4026833862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c4c102290 0x7f0c4c1a6330 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0c44009dc0 tx=0x7f0c44032ff0 comp rx=0 tx=0).stop 2024-09-12T03:36:45.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.137+0000 7f0c5300e640 1 -- 172.21.15.64:0/4026833862 shutdown_connections 2024-09-12T03:36:45.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.137+0000 7f0c5300e640 1 --2- 172.21.15.64:0/4026833862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c4c102290 0x7f0c4c1a6330 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:45.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.137+0000 7f0c5300e640 1 --2- 172.21.15.64:0/4026833862 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f0c340a2dd0 0x7f0c340a5290 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:45.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.137+0000 7f0c5300e640 1 --2- 172.21.15.64:0/4026833862 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c4c1080f0 0x7f0c4c101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:45.140 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.137+0000 7f0c5300e640 1 --2- 172.21.15.64:0/4026833862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c4c1008a0 0x7f0c4c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:45.140 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.137+0000 7f0c5300e640 1 -- 172.21.15.64:0/4026833862 >> 172.21.15.64:0/4026833862 conn(0x7f0c4c075ef0 msgr2=0x7f0c4c0ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:36:45.140 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.137+0000 7f0c5300e640 1 -- 172.21.15.64:0/4026833862 shutdown_connections 2024-09-12T03:36:45.140 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:45.137+0000 7f0c5300e640 1 -- 172.21.15.64:0/4026833862 wait complete. 2024-09-12T03:36:45.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:36:45 smithi195 ceph-mon[34786]: pgmap v229: 129 pgs: 129 active+clean; 4.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:36:45.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:36:45 smithi064 ceph-mon[30952]: pgmap v229: 129 pgs: 129 active+clean; 4.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:36:45.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:36:45 smithi115 ceph-mon[34801]: pgmap v229: 129 pgs: 129 active+clean; 4.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:36:47.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:36:47 smithi195 ceph-mon[34786]: pgmap v230: 129 pgs: 129 active+clean; 4.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:36:47.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:36:47 smithi064 ceph-mon[30952]: pgmap v230: 129 pgs: 129 active+clean; 4.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:36:47.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:36:47 smithi115 ceph-mon[34801]: pgmap v230: 129 pgs: 129 active+clean; 4.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:36:49.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:36:49 smithi195 ceph-mon[34786]: pgmap v231: 129 pgs: 129 active+clean; 4.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 63 KiB/s wr, 20 op/s 2024-09-12T03:36:49.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:36:49 smithi064 ceph-mon[30952]: pgmap v231: 129 pgs: 129 active+clean; 4.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 63 KiB/s wr, 20 op/s 2024-09-12T03:36:49.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:36:49 smithi115 ceph-mon[34801]: pgmap v231: 129 pgs: 129 active+clean; 4.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 63 KiB/s wr, 20 op/s 2024-09-12T03:36:51.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:36:51 smithi195 ceph-mon[34786]: pgmap v232: 129 pgs: 129 active+clean; 4.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:36:51.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:36:51 smithi064 ceph-mon[30952]: pgmap v232: 129 pgs: 129 active+clean; 4.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:36:51.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:36:51 smithi115 ceph-mon[34801]: pgmap v232: 129 pgs: 129 active+clean; 4.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:36:53.710 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:36:53 smithi195 ceph-mon[34786]: pgmap v233: 129 pgs: 129 active+clean; 4.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:36:53.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:36:53 smithi064 ceph-mon[30952]: pgmap v233: 129 pgs: 129 active+clean; 4.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:36:53.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:36:53 smithi115 ceph-mon[34801]: pgmap v233: 129 pgs: 129 active+clean; 4.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:36:55.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:36:55 smithi195 ceph-mon[34786]: pgmap v234: 129 pgs: 129 active+clean; 4.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-12T03:36:55.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:36:55 smithi064 ceph-mon[30952]: pgmap v234: 129 pgs: 129 active+clean; 4.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-12T03:36:55.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:36:55 smithi115 ceph-mon[34801]: pgmap v234: 129 pgs: 129 active+clean; 4.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-12T03:36:57.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:36:57 smithi195 ceph-mon[34786]: pgmap v235: 129 pgs: 129 active+clean; 4.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-12T03:36:57.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:36:57 smithi064 ceph-mon[30952]: pgmap v235: 129 pgs: 129 active+clean; 4.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-12T03:36:57.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:36:57 smithi115 ceph-mon[34801]: pgmap v235: 129 pgs: 129 active+clean; 4.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-12T03:36:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.236+0000 7fd214de4640 1 -- 172.21.15.64:0/3437228210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd210102cc0 msgr2=0x7fd2101030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.236+0000 7fd214de4640 1 --2- 172.21.15.64:0/3437228210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd210102cc0 0x7fd2101030a0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd200009900 tx=0x7fd20002a480 comp rx=0 tx=0).stop 2024-09-12T03:36:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.236+0000 7fd214de4640 1 -- 172.21.15.64:0/3437228210 shutdown_connections 2024-09-12T03:36:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.236+0000 7fd214de4640 1 --2- 172.21.15.64:0/3437228210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd21010c110 0x7fd21010e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.236+0000 7fd214de4640 1 --2- 172.21.15.64:0/3437228210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2101035e0 0x7fd21010bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.236+0000 7fd214de4640 1 --2- 172.21.15.64:0/3437228210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd210102cc0 0x7fd2101030a0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.236+0000 7fd214de4640 1 -- 172.21.15.64:0/3437228210 >> 172.21.15.64:0/3437228210 conn(0x7fd2100fc6b0 msgr2=0x7fd2100fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:36:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.236+0000 7fd214de4640 1 -- 172.21.15.64:0/3437228210 shutdown_connections 2024-09-12T03:36:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.236+0000 7fd214de4640 1 -- 172.21.15.64:0/3437228210 wait complete. 2024-09-12T03:36:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.237+0000 7fd214de4640 1 Processor -- start 2024-09-12T03:36:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.237+0000 7fd214de4640 1 -- start start 2024-09-12T03:36:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.237+0000 7fd214de4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd210102cc0 0x7fd21019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.237+0000 7fd214de4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2101035e0 0x7fd21019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.237+0000 7fd214de4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd21010c110 0x7fd21019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.237+0000 7fd214de4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd210111de0 con 0x7fd2101035e0 2024-09-12T03:36:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.237+0000 7fd214de4640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd210111c60 con 0x7fd21010c110 2024-09-12T03:36:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.237+0000 7fd214de4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd210111f60 con 0x7fd210102cc0 2024-09-12T03:36:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.237+0000 7fd20dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2101035e0 0x7fd21019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.237+0000 7fd20dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2101035e0 0x7fd21019aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36352/0 (socket says 172.21.15.64:36352) 2024-09-12T03:36:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.237+0000 7fd20dd9b640 1 -- 172.21.15.64:0/3786547946 learned_addr learned my addr 172.21.15.64:0/3786547946 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:36:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.237+0000 7fd20e59c640 1 --2- 172.21.15.64:0/3786547946 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd210102cc0 0x7fd21019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.238+0000 7fd20ed9d640 1 --2- 172.21.15.64:0/3786547946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd21010c110 0x7fd21019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.238+0000 7fd20dd9b640 1 -- 172.21.15.64:0/3786547946 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd210102cc0 msgr2=0x7fd21019a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.238+0000 7fd20dd9b640 1 --2- 172.21.15.64:0/3786547946 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd210102cc0 0x7fd21019a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.238+0000 7fd20dd9b640 1 -- 172.21.15.64:0/3786547946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd21010c110 msgr2=0x7fd21019ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.238+0000 7fd20dd9b640 1 --2- 172.21.15.64:0/3786547946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd21010c110 0x7fd21019ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.238+0000 7fd20dd9b640 1 -- 172.21.15.64:0/3786547946 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd21019f4b0 con 0x7fd2101035e0 2024-09-12T03:36:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.238+0000 7fd20dd9b640 1 --2- 172.21.15.64:0/3786547946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2101035e0 0x7fd21019aa00 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd20400c940 tx=0x7fd20400ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:36:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.238+0000 7fd1ff7fe640 1 -- 172.21.15.64:0/3786547946 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd204004220 con 0x7fd2101035e0 2024-09-12T03:36:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.238+0000 7fd214de4640 1 -- 172.21.15.64:0/3786547946 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd21019f7a0 con 0x7fd2101035e0 2024-09-12T03:36:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.238+0000 7fd1ff7fe640 1 -- 172.21.15.64:0/3786547946 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2040043c0 con 0x7fd2101035e0 2024-09-12T03:36:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.238+0000 7fd1ff7fe640 1 -- 172.21.15.64:0/3786547946 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd204004a00 con 0x7fd2101035e0 2024-09-12T03:36:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.238+0000 7fd214de4640 1 -- 172.21.15.64:0/3786547946 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2101a7030 con 0x7fd2101035e0 2024-09-12T03:36:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.241+0000 7fd1ff7fe640 1 -- 172.21.15.64:0/3786547946 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fd2040079b0 con 0x7fd2101035e0 2024-09-12T03:36:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.241+0000 7fd217c5a740 1 Processor -- start 2024-09-12T03:36:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.242+0000 7fd217c5a740 1 -- start start 2024-09-12T03:36:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.242+0000 7fd217c5a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558201250af0 0x558201250ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.242+0000 7fd217c5a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558201251410 0x55820125b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.242+0000 7fd217c5a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55820125bcb0 0x55820125e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.242+0000 7fd217c5a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5582011b52f0 con 0x558201251410 2024-09-12T03:36:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.242+0000 7fd217c5a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55820115d270 con 0x558201250af0 2024-09-12T03:36:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.242+0000 7fd217c5a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5582011b12b0 con 0x55820125bcb0 2024-09-12T03:36:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.242+0000 7fd20e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558201251410 0x55820125b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.242+0000 7fd20e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558201251410 0x55820125b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36368/0 (socket says 172.21.15.64:36368) 2024-09-12T03:36:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.242+0000 7fd20e59c640 1 -- 172.21.15.64:0/2731518448 learned_addr learned my addr 172.21.15.64:0/2731518448 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:36:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.242+0000 7fd20ed9d640 1 --2- 172.21.15.64:0/2731518448 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558201250af0 0x558201250ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.242+0000 7fd20dd9b640 1 --2- 172.21.15.64:0/2731518448 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55820125bcb0 0x55820125e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.242+0000 7fd20e59c640 1 -- 172.21.15.64:0/2731518448 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55820125bcb0 msgr2=0x55820125e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:36:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.242+0000 7fd20e59c640 1 --2- 172.21.15.64:0/2731518448 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55820125bcb0 0x55820125e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.242+0000 7fd20e59c640 1 -- 172.21.15.64:0/2731518448 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558201250af0 msgr2=0x558201250ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:36:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.242+0000 7fd20e59c640 1 --2- 172.21.15.64:0/2731518448 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558201250af0 0x558201250ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.242+0000 7fd20e59c640 1 -- 172.21.15.64:0/2731518448 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55820115c340 con 0x558201251410 2024-09-12T03:36:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.242+0000 7fd1ff7fe640 1 --2- 172.21.15.64:0/3786547946 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fd1f80a2da0 0x7fd1f80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.243+0000 7fd20ed9d640 1 --2- 172.21.15.64:0/3786547946 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fd1f80a2da0 0x7fd1f80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.243+0000 7fd1ff7fe640 1 -- 172.21.15.64:0/3786547946 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9583+0+0 (secure 0 0 0) 0x7fd2040cf860 con 0x7fd2101035e0 2024-09-12T03:36:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.243+0000 7fd20e59c640 1 --2- 172.21.15.64:0/2731518448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558201251410 0x55820125b770 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd200009900 tx=0x7fd200031dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fa595c261824106 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:36:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.243+0000 7fd1fcff9640 1 -- 172.21.15.64:0/2731518448 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd200040070 con 0x558201251410 2024-09-12T03:36:58.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.243+0000 7fd1fcff9640 1 -- 172.21.15.64:0/2731518448 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2000316a0 con 0x558201251410 2024-09-12T03:36:58.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.243+0000 7fd1fcff9640 1 -- 172.21.15.64:0/2731518448 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd200035040 con 0x558201251410 2024-09-12T03:36:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.244+0000 7fd217c5a740 1 -- 172.21.15.64:0/2731518448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558201251410 msgr2=0x55820125b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.244+0000 7fd217c5a740 1 --2- 172.21.15.64:0/2731518448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558201251410 0x55820125b770 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd200009900 tx=0x7fd200031dc0 comp rx=0 tx=0).stop 2024-09-12T03:36:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.244+0000 7fd20ed9d640 1 --2- 172.21.15.64:0/3786547946 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fd1f80a2da0 0x7fd1f80a5260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd20800b4d0 tx=0x7fd20800e040 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:36:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.244+0000 7fd217c5a740 1 -- 172.21.15.64:0/2731518448 shutdown_connections 2024-09-12T03:36:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.244+0000 7fd217c5a740 1 --2- 172.21.15.64:0/2731518448 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55820125bcb0 0x55820125e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.244+0000 7fd217c5a740 1 --2- 172.21.15.64:0/2731518448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558201251410 0x55820125b770 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.244+0000 7fd217c5a740 1 --2- 172.21.15.64:0/2731518448 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558201250af0 0x558201250ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.244+0000 7fd217c5a740 1 -- 172.21.15.64:0/2731518448 >> 172.21.15.64:0/2731518448 conn(0x55820124d060 msgr2=0x55820124d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:36:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.244+0000 7fd217c5a740 1 -- 172.21.15.64:0/2731518448 shutdown_connections 2024-09-12T03:36:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.244+0000 7fd217c5a740 1 -- 172.21.15.64:0/2731518448 wait complete. 2024-09-12T03:36:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.244+0000 7fd217c5a740 1 -- start start 2024-09-12T03:36:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.244+0000 7fd217c5a740 1 Processor -- start 2024-09-12T03:36:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.245+0000 7fd217c5a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558201250af0 0x558201258c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.245+0000 7fd217c5a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558201251410 0x558201259150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.245+0000 7fd217c5a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55820125bcb0 0x558201259690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.245+0000 7fd217c5a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55820125e880 con 0x558201251410 2024-09-12T03:36:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.245+0000 7fd217c5a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55820113e290 con 0x558201250af0 2024-09-12T03:36:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.245+0000 7fd217c5a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55820125ea00 con 0x55820125bcb0 2024-09-12T03:36:58.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.245+0000 7fd217c5a740 10 client.0 _ll_register_callbacks cb 0x55820116f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:36:58.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.245+0000 7fd20dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558201251410 0x558201259150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:58.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.245+0000 7fd20dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558201251410 0x558201259150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36380/0 (socket says 172.21.15.64:36380) 2024-09-12T03:36:58.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.245+0000 7fd20dd9b640 1 -- 172.21.15.64:0/1489236663 learned_addr learned my addr 172.21.15.64:0/1489236663 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:36:58.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.245+0000 7fd20ed9d640 1 --2- 172.21.15.64:0/1489236663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55820125bcb0 0x558201259690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:58.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.245+0000 7fd20e59c640 1 --2- 172.21.15.64:0/1489236663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558201250af0 0x558201258c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:58.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.245+0000 7fd20dd9b640 1 -- 172.21.15.64:0/1489236663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55820125bcb0 msgr2=0x558201259690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:36:58.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.245+0000 7fd20dd9b640 1 --2- 172.21.15.64:0/1489236663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55820125bcb0 0x558201259690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:58.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.245+0000 7fd20dd9b640 1 -- 172.21.15.64:0/1489236663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558201250af0 msgr2=0x558201258c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:36:58.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.245+0000 7fd20dd9b640 1 --2- 172.21.15.64:0/1489236663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558201250af0 0x558201258c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:58.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.245+0000 7fd20dd9b640 1 -- 172.21.15.64:0/1489236663 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558200ec8210 con 0x558201251410 2024-09-12T03:36:58.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.245+0000 7fd20ed9d640 1 --2- 172.21.15.64:0/1489236663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55820125bcb0 0x558201259690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:36:58.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.245+0000 7fd20dd9b640 1 --2- 172.21.15.64:0/1489236663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558201251410 0x558201259150 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fd2040058c0 tx=0x7fd20409b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cdea0496461a9aa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:36:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.245+0000 7fd1f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:36:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.245+0000 7fd1f5ffb640 1 -- 172.21.15.64:0/1489236663 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20401d040 con 0x558201251410 2024-09-12T03:36:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.245+0000 7fd217c5a740 1 -- 172.21.15.64:0/1489236663 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2000316a0 con 0x558201251410 2024-09-12T03:36:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.245+0000 7fd1f5ffb640 1 -- 172.21.15.64:0/1489236663 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9583+0+0 (secure 0 0 0) 0x7fd2040a4140 con 0x558201251410 2024-09-12T03:36:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.246+0000 7fd1f5ffb640 1 -- 172.21.15.64:0/1489236663 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd20409b960 con 0x558201251410 2024-09-12T03:36:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.246+0000 7fd217c5a740 10 client.15972 fetch_fsmap learned FSMap version 25 2024-09-12T03:36:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.246+0000 7fd217c5a740 1 -- 172.21.15.64:0/1489236663 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558200f94620 con 0x558201251410 2024-09-12T03:36:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.246+0000 7fd1f5ffb640 1 -- 172.21.15.64:0/1489236663 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2000316a0 con 0x558201251410 2024-09-12T03:36:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.246+0000 7fd1f5ffb640 1 -- 172.21.15.64:0/1489236663 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd2040860c0 con 0x558201251410 2024-09-12T03:36:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.246+0000 7fd217c5a740 10 client.15972 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:36:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.246+0000 7fd217c5a740 10 client.15972 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:36:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.246+0000 7fd217c5a740 20 client.15972 populate_metadata read hostname 'smithi064' 2024-09-12T03:36:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.246+0000 7fd217c5a740 1 --2- 172.21.15.64:0/1489236663 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55820126bc70 0x55820126e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.246+0000 7fd217c5a740 4 client.15972 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:36:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.246+0000 7fd217c5a740 1 -- 172.21.15.64:0/1489236663 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582011c2a20 con 0x55820126bc70 2024-09-12T03:36:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.246+0000 7fd20e59c640 1 --2- 172.21.15.64:0/1489236663 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55820126bc70 0x55820126e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.247+0000 7fd20e59c640 1 --2- 172.21.15.64:0/1489236663 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55820126bc70 0x55820126e060 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=252ae4b5316f64be server_cookie=531c040b5c2c69ac in_seq=0 out_seq=0 2024-09-12T03:36:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.247+0000 7fd1f5ffb640 10 client.15972 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:36:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.247+0000 7fd1f5ffb640 1 -- 172.21.15.64:0/1489236663 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5582011c2a20 con 0x55820126bc70 2024-09-12T03:36:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.247+0000 7fd1f5ffb640 10 client.15972 handle_command_reply: tid=0 2024-09-12T03:36:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.247+0000 7fd217c5a740 1 client.15972 shutdown 2024-09-12T03:36:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.248+0000 7fd217c5a740 1 -- 172.21.15.64:0/1489236663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558201251410 msgr2=0x558201259150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.248+0000 7fd217c5a740 1 --2- 172.21.15.64:0/1489236663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558201251410 0x558201259150 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fd2040058c0 tx=0x7fd20409b560 comp rx=0 tx=0).stop 2024-09-12T03:36:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.248+0000 7fd217c5a740 1 -- 172.21.15.64:0/1489236663 shutdown_connections 2024-09-12T03:36:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.248+0000 7fd217c5a740 1 --2- 172.21.15.64:0/1489236663 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55820126bc70 0x55820126e060 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.248+0000 7fd217c5a740 1 --2- 172.21.15.64:0/1489236663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55820125bcb0 0x558201259690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.248+0000 7fd217c5a740 1 --2- 172.21.15.64:0/1489236663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558201251410 0x558201259150 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.248+0000 7fd1f5ffb640 0 client.15972 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:36:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.248+0000 7fd217c5a740 1 --2- 172.21.15.64:0/1489236663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558201250af0 0x558201258c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.248+0000 7fd217c5a740 1 -- 172.21.15.64:0/1489236663 >> 172.21.15.64:0/1489236663 conn(0x55820124d060 msgr2=0x558201252dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:36:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.248+0000 7fd217c5a740 1 -- 172.21.15.64:0/1489236663 shutdown_connections 2024-09-12T03:36:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.248+0000 7fd217c5a740 1 -- 172.21.15.64:0/1489236663 wait complete. 2024-09-12T03:36:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.248+0000 7fd217c5a740 20 client.15972 trim_cache size 0 max 16384 2024-09-12T03:36:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.263+0000 7fd217c5a740 1 Processor -- start 2024-09-12T03:36:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.263+0000 7fd217c5a740 1 -- start start 2024-09-12T03:36:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.263+0000 7fd217c5a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558201250af0 0x5582012594c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.263+0000 7fd217c5a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558201251410 0x55820124c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.263+0000 7fd217c5a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55820125bcb0 0x558201273930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.263+0000 7fd217c5a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd2101b3ba0 con 0x558201251410 2024-09-12T03:36:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.263+0000 7fd217c5a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd2101b1470 con 0x558201250af0 2024-09-12T03:36:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.263+0000 7fd217c5a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd2101af2e0 con 0x55820125bcb0 2024-09-12T03:36:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.263+0000 7fd20dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558201251410 0x55820124c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.263+0000 7fd20dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558201251410 0x55820124c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36388/0 (socket says 172.21.15.64:36388) 2024-09-12T03:36:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.263+0000 7fd20dd9b640 1 -- 172.21.15.64:0/2752073026 learned_addr learned my addr 172.21.15.64:0/2752073026 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:36:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.263+0000 7fd20e59c640 1 --2- 172.21.15.64:0/2752073026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558201250af0 0x5582012594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.263+0000 7fd20dd9b640 1 -- 172.21.15.64:0/2752073026 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55820125bcb0 msgr2=0x558201273930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:36:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.263+0000 7fd20ed9d640 1 --2- 172.21.15.64:0/2752073026 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55820125bcb0 0x558201273930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.263+0000 7fd20dd9b640 1 --2- 172.21.15.64:0/2752073026 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55820125bcb0 0x558201273930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.263+0000 7fd20dd9b640 1 -- 172.21.15.64:0/2752073026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558201250af0 msgr2=0x5582012594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:36:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.263+0000 7fd20dd9b640 1 --2- 172.21.15.64:0/2752073026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558201250af0 0x5582012594c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.263+0000 7fd20dd9b640 1 -- 172.21.15.64:0/2752073026 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55820125e400 con 0x558201251410 2024-09-12T03:36:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.263+0000 7fd20e59c640 1 --2- 172.21.15.64:0/2752073026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558201250af0 0x5582012594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:36:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.263+0000 7fd20dd9b640 1 --2- 172.21.15.64:0/2752073026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558201251410 0x55820124c0b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fd204097dc0 tx=0x7fd20409b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a19e575ec3d47 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:36:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.264+0000 7fd1f7fff640 1 -- 172.21.15.64:0/2752073026 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20401d050 con 0x558201251410 2024-09-12T03:36:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.264+0000 7fd1f7fff640 1 -- 172.21.15.64:0/2752073026 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2040cfb60 con 0x558201251410 2024-09-12T03:36:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.264+0000 7fd1f7fff640 1 -- 172.21.15.64:0/2752073026 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd204098130 con 0x558201251410 2024-09-12T03:36:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.264+0000 7fd217c5a740 1 -- 172.21.15.64:0/2752073026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558201251410 msgr2=0x55820124c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.264+0000 7fd217c5a740 1 --2- 172.21.15.64:0/2752073026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558201251410 0x55820124c0b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fd204097dc0 tx=0x7fd20409b5d0 comp rx=0 tx=0).stop 2024-09-12T03:36:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.264+0000 7fd217c5a740 1 -- 172.21.15.64:0/2752073026 shutdown_connections 2024-09-12T03:36:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.264+0000 7fd217c5a740 1 --2- 172.21.15.64:0/2752073026 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55820125bcb0 0x558201273930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.264+0000 7fd217c5a740 1 --2- 172.21.15.64:0/2752073026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558201251410 0x55820124c0b0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.264+0000 7fd217c5a740 1 --2- 172.21.15.64:0/2752073026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558201250af0 0x5582012594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:58.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.264+0000 7fd217c5a740 1 -- 172.21.15.64:0/2752073026 >> 172.21.15.64:0/2752073026 conn(0x55820124d060 msgr2=0x55820124b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:36:58.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.264+0000 7fd217c5a740 1 -- 172.21.15.64:0/2752073026 shutdown_connections 2024-09-12T03:36:58.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.264+0000 7fd217c5a740 1 -- 172.21.15.64:0/2752073026 wait complete. 2024-09-12T03:36:58.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.264+0000 7fd217c5a740 1 -- start start 2024-09-12T03:36:58.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.265+0000 7fd217c5a740 1 Processor -- start 2024-09-12T03:36:58.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.265+0000 7fd217c5a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558201250af0 0x55820125fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:58.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.265+0000 7fd217c5a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558201251410 0x55820125ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:58.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.265+0000 7fd217c5a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55820125bcb0 0x5582012604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:58.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.265+0000 7fd217c5a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd2101af930 con 0x55820125bcb0 2024-09-12T03:36:58.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.265+0000 7fd217c5a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd2101b0da0 con 0x558201251410 2024-09-12T03:36:58.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.265+0000 7fd217c5a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd2101b0000 con 0x558201250af0 2024-09-12T03:36:58.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.265+0000 7fd217c5a740 10 client.0 _ll_register_callbacks cb 0x7fd204004570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:36:58.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.265+0000 7fd20ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55820125bcb0 0x5582012604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:58.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.265+0000 7fd20ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55820125bcb0 0x5582012604b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36400/0 (socket says 172.21.15.64:36400) 2024-09-12T03:36:58.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.265+0000 7fd20ed9d640 1 -- 172.21.15.64:0/3108968320 learned_addr learned my addr 172.21.15.64:0/3108968320 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:36:58.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.265+0000 7fd20e59c640 1 --2- 172.21.15.64:0/3108968320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558201250af0 0x55820125fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:58.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.265+0000 7fd20dd9b640 1 --2- 172.21.15.64:0/3108968320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558201251410 0x55820125ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:58.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.265+0000 7fd20ed9d640 1 -- 172.21.15.64:0/3108968320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558201250af0 msgr2=0x55820125fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:36:58.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.265+0000 7fd20ed9d640 1 --2- 172.21.15.64:0/3108968320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558201250af0 0x55820125fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:58.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.265+0000 7fd20ed9d640 1 -- 172.21.15.64:0/3108968320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558201251410 msgr2=0x55820125ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:36:58.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.265+0000 7fd20ed9d640 1 --2- 172.21.15.64:0/3108968320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558201251410 0x55820125ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:58.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.265+0000 7fd20ed9d640 1 -- 172.21.15.64:0/3108968320 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582012d5950 con 0x55820125bcb0 2024-09-12T03:36:58.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.265+0000 7fd20dd9b640 1 --2- 172.21.15.64:0/3108968320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558201251410 0x55820125ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:36:58.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.266+0000 7fd20ed9d640 1 --2- 172.21.15.64:0/3108968320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55820125bcb0 0x5582012604b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd208018c50 tx=0x7fd208004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c98f35dc8d229 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:36:58.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.266+0000 7fd1f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:36:58.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.266+0000 7fd1f5ffb640 1 -- 172.21.15.64:0/3108968320 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd208029060 con 0x55820125bcb0 2024-09-12T03:36:58.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.266+0000 7fd1f5ffb640 1 -- 172.21.15.64:0/3108968320 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9583+0+0 (secure 0 0 0) 0x7fd208024600 con 0x55820125bcb0 2024-09-12T03:36:58.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.266+0000 7fd217c5a740 1 -- 172.21.15.64:0/3108968320 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2040cfb60 con 0x55820125bcb0 2024-09-12T03:36:58.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.266+0000 7fd1f5ffb640 1 -- 172.21.15.64:0/3108968320 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd208030050 con 0x55820125bcb0 2024-09-12T03:36:58.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.266+0000 7fd217c5a740 10 client.15978 fetch_fsmap learned FSMap version 25 2024-09-12T03:36:58.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.266+0000 7fd217c5a740 1 -- 172.21.15.64:0/3108968320 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5582012d6ca0 con 0x55820125bcb0 2024-09-12T03:36:58.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.266+0000 7fd1f5ffb640 1 -- 172.21.15.64:0/3108968320 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2040cfb60 con 0x55820125bcb0 2024-09-12T03:36:58.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.266+0000 7fd1f5ffb640 1 -- 172.21.15.64:0/3108968320 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd208023750 con 0x55820125bcb0 2024-09-12T03:36:58.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.266+0000 7fd217c5a740 10 client.15978 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:36:58.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.266+0000 7fd217c5a740 10 client.15978 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:36:58.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.266+0000 7fd217c5a740 20 client.15978 populate_metadata read hostname 'smithi064' 2024-09-12T03:36:58.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.266+0000 7fd217c5a740 1 --2- 172.21.15.64:0/3108968320 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5582012d73f0 0x5582012d9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:36:58.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.266+0000 7fd217c5a740 4 client.15978 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:36:58.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.266+0000 7fd217c5a740 1 -- 172.21.15.64:0/3108968320 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5582011bcd70 con 0x5582012d73f0 2024-09-12T03:36:58.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.267+0000 7fd20e59c640 1 --2- 172.21.15.64:0/3108968320 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5582012d73f0 0x5582012d9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:36:58.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.267+0000 7fd1f5ffb640 10 client.15978 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:36:58.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.267+0000 7fd20e59c640 1 --2- 172.21.15.64:0/3108968320 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5582012d73f0 0x5582012d9810 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6a2a5dd05e6d169 server_cookie=b9eb46bea338fc1c in_seq=0 out_seq=0 2024-09-12T03:36:58.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.281+0000 7fd1f5ffb640 1 -- 172.21.15.64:0/3108968320 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5582011bcd70 con 0x5582012d73f0 2024-09-12T03:36:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.281+0000 7fd1f5ffb640 10 client.15978 handle_command_reply: tid=0 2024-09-12T03:36:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.281+0000 7fd217c5a740 1 client.15978 shutdown 2024-09-12T03:36:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.281+0000 7fd217c5a740 1 -- 172.21.15.64:0/3108968320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55820125bcb0 msgr2=0x5582012604b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.281+0000 7fd217c5a740 1 --2- 172.21.15.64:0/3108968320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55820125bcb0 0x5582012604b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd208018c50 tx=0x7fd208004b50 comp rx=0 tx=0).stop 2024-09-12T03:36:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.281+0000 7fd217c5a740 1 -- 172.21.15.64:0/3108968320 shutdown_connections 2024-09-12T03:36:58.285 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.281+0000 7fd217c5a740 1 --2- 172.21.15.64:0/3108968320 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5582012d73f0 0x5582012d9810 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:58.285 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.281+0000 7fd217c5a740 1 --2- 172.21.15.64:0/3108968320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55820125bcb0 0x5582012604b0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:58.285 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.281+0000 7fd217c5a740 1 --2- 172.21.15.64:0/3108968320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558201251410 0x55820125ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:58.285 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.281+0000 7fd1f5ffb640 0 client.15978 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:36:58.285 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.281+0000 7fd217c5a740 1 --2- 172.21.15.64:0/3108968320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558201250af0 0x55820125fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.281+0000 7fd217c5a740 1 -- 172.21.15.64:0/3108968320 >> 172.21.15.64:0/3108968320 conn(0x55820124d060 msgr2=0x558201252d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:36:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.282+0000 7fd217c5a740 1 -- 172.21.15.64:0/3108968320 shutdown_connections 2024-09-12T03:36:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.282+0000 7fd217c5a740 1 -- 172.21.15.64:0/3108968320 wait complete. 2024-09-12T03:36:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.282+0000 7fd217c5a740 20 client.15978 trim_cache size 0 max 16384 2024-09-12T03:36:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:36:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:36:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:36:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:36:58.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.282+0000 7fd214de4640 1 -- 172.21.15.64:0/3786547946 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fd1f80a2da0 msgr2=0x7fd1f80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:58.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.282+0000 7fd214de4640 1 --2- 172.21.15.64:0/3786547946 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fd1f80a2da0 0x7fd1f80a5260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd20800b4d0 tx=0x7fd20800e040 comp rx=0 tx=0).stop 2024-09-12T03:36:58.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.282+0000 7fd214de4640 1 -- 172.21.15.64:0/3786547946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2101035e0 msgr2=0x7fd21019aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:36:58.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.282+0000 7fd214de4640 1 --2- 172.21.15.64:0/3786547946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2101035e0 0x7fd21019aa00 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd20400c940 tx=0x7fd20400ce00 comp rx=0 tx=0).stop 2024-09-12T03:36:58.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.282+0000 7fd214de4640 1 -- 172.21.15.64:0/3786547946 shutdown_connections 2024-09-12T03:36:58.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.282+0000 7fd214de4640 1 --2- 172.21.15.64:0/3786547946 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fd1f80a2da0 0x7fd1f80a5260 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:58.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.282+0000 7fd214de4640 1 --2- 172.21.15.64:0/3786547946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd21010c110 0x7fd21019ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:58.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.282+0000 7fd214de4640 1 --2- 172.21.15.64:0/3786547946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2101035e0 0x7fd21019aa00 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:58.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.282+0000 7fd214de4640 1 --2- 172.21.15.64:0/3786547946 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd210102cc0 0x7fd21019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:36:58.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.282+0000 7fd214de4640 1 -- 172.21.15.64:0/3786547946 >> 172.21.15.64:0/3786547946 conn(0x7fd2100fc6b0 msgr2=0x7fd2100fca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:36:58.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.282+0000 7fd214de4640 1 -- 172.21.15.64:0/3786547946 shutdown_connections 2024-09-12T03:36:58.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:36:58.282+0000 7fd214de4640 1 -- 172.21.15.64:0/3786547946 wait complete. 2024-09-12T03:36:59.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:36:59 smithi064 ceph-mon[30952]: pgmap v236: 129 pgs: 129 active+clean; 4.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:36:59.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:36:59 smithi115 ceph-mon[34801]: pgmap v236: 129 pgs: 129 active+clean; 4.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:36:59.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:36:59 smithi195 ceph-mon[34786]: pgmap v236: 129 pgs: 129 active+clean; 4.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:37:01.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:37:01 smithi064 ceph-mon[30952]: pgmap v237: 129 pgs: 129 active+clean; 4.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-12T03:37:01.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:37:01 smithi115 ceph-mon[34801]: pgmap v237: 129 pgs: 129 active+clean; 4.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-12T03:37:01.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:37:01 smithi195 ceph-mon[34786]: pgmap v237: 129 pgs: 129 active+clean; 4.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-12T03:37:03.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:37:03 smithi064 ceph-mon[30952]: pgmap v238: 129 pgs: 129 active+clean; 5.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 102 KiB/s wr, 35 op/s 2024-09-12T03:37:03.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:37:03 smithi115 ceph-mon[34801]: pgmap v238: 129 pgs: 129 active+clean; 5.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 102 KiB/s wr, 35 op/s 2024-09-12T03:37:03.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:37:03 smithi195 ceph-mon[34786]: pgmap v238: 129 pgs: 129 active+clean; 5.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 102 KiB/s wr, 35 op/s 2024-09-12T03:37:04.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:37:04 smithi064 ceph-mon[30952]: osdmap e96: 12 total, 12 up, 12 in 2024-09-12T03:37:04.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:37:04 smithi115 ceph-mon[34801]: osdmap e96: 12 total, 12 up, 12 in 2024-09-12T03:37:04.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:37:04 smithi195 ceph-mon[34786]: osdmap e96: 12 total, 12 up, 12 in 2024-09-12T03:37:05.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:37:05 smithi064 ceph-mon[30952]: pgmap v240: 129 pgs: 129 active+clean; 5.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 104 KiB/s wr, 36 op/s 2024-09-12T03:37:05.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:37:05 smithi115 ceph-mon[34801]: pgmap v240: 129 pgs: 129 active+clean; 5.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 104 KiB/s wr, 36 op/s 2024-09-12T03:37:05.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:37:05 smithi195 ceph-mon[34786]: pgmap v240: 129 pgs: 129 active+clean; 5.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 104 KiB/s wr, 36 op/s 2024-09-12T03:37:07.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:37:07 smithi064 ceph-mon[30952]: pgmap v241: 129 pgs: 129 active+clean; 5.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 104 KiB/s wr, 36 op/s 2024-09-12T03:37:07.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:37:07 smithi115 ceph-mon[34801]: pgmap v241: 129 pgs: 129 active+clean; 5.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 104 KiB/s wr, 36 op/s 2024-09-12T03:37:07.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:37:07 smithi195 ceph-mon[34786]: pgmap v241: 129 pgs: 129 active+clean; 5.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 104 KiB/s wr, 36 op/s 2024-09-12T03:37:09.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:37:09 smithi064 ceph-mon[30952]: pgmap v242: 129 pgs: 129 active+clean; 5.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 105 KiB/s wr, 36 op/s 2024-09-12T03:37:09.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:37:09 smithi115 ceph-mon[34801]: pgmap v242: 129 pgs: 129 active+clean; 5.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 105 KiB/s wr, 36 op/s 2024-09-12T03:37:09.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:37:09 smithi195 ceph-mon[34786]: pgmap v242: 129 pgs: 129 active+clean; 5.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 105 KiB/s wr, 36 op/s 2024-09-12T03:37:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.384+0000 7fea0e9a6640 1 -- 172.21.15.64:0/68480335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea0810c110 msgr2=0x7fea0810e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.384+0000 7fea0e9a6640 1 --2- 172.21.15.64:0/68480335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea0810c110 0x7fea0810e560 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fea00009ca0 tx=0x7fea0002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:37:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.384+0000 7fea0e9a6640 1 -- 172.21.15.64:0/68480335 shutdown_connections 2024-09-12T03:37:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.384+0000 7fea0e9a6640 1 --2- 172.21.15.64:0/68480335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea0810c110 0x7fea0810e560 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.384+0000 7fea0e9a6640 1 --2- 172.21.15.64:0/68480335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea081035c0 0x7fea0810baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.384+0000 7fea0e9a6640 1 --2- 172.21.15.64:0/68480335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fea08102c10 0x7fea08102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.384+0000 7fea0e9a6640 1 -- 172.21.15.64:0/68480335 >> 172.21.15.64:0/68480335 conn(0x7fea080fc6b0 msgr2=0x7fea080fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:11.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.384+0000 7fea0e9a6640 1 -- 172.21.15.64:0/68480335 shutdown_connections 2024-09-12T03:37:11.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.384+0000 7fea0e9a6640 1 -- 172.21.15.64:0/68480335 wait complete. 2024-09-12T03:37:11.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.384+0000 7fea0e9a6640 1 Processor -- start 2024-09-12T03:37:11.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.384+0000 7fea0e9a6640 1 -- start start 2024-09-12T03:37:11.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.385+0000 7fea0e9a6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea08102c10 0x7fea081008f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:11.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.385+0000 7fea0e9a6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fea081035c0 0x7fea08100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:11.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.385+0000 7fea0e9a6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea0810c110 0x7fea08101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:11.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.385+0000 7fea0e9a6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fea08111d70 con 0x7fea0810c110 2024-09-12T03:37:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.385+0000 7fea0e9a6640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fea08111bf0 con 0x7fea08102c10 2024-09-12T03:37:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.385+0000 7fea0e9a6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fea08111ef0 con 0x7fea081035c0 2024-09-12T03:37:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.385+0000 7fea0cf43640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea0810c110 0x7fea08101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.385+0000 7fea0cf43640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea0810c110 0x7fea08101370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43768/0 (socket says 172.21.15.64:43768) 2024-09-12T03:37:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.385+0000 7fea0cf43640 1 -- 172.21.15.64:0/3084363248 learned_addr learned my addr 172.21.15.64:0/3084363248 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:37:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.385+0000 7fea07fff640 1 --2- 172.21.15.64:0/3084363248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea08102c10 0x7fea081008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.385+0000 7fea077fe640 1 --2- 172.21.15.64:0/3084363248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fea081035c0 0x7fea08100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.385+0000 7fea0cf43640 1 -- 172.21.15.64:0/3084363248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fea081035c0 msgr2=0x7fea08100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.385+0000 7fea0cf43640 1 --2- 172.21.15.64:0/3084363248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fea081035c0 0x7fea08100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.385+0000 7fea0cf43640 1 -- 172.21.15.64:0/3084363248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea08102c10 msgr2=0x7fea081008f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.385+0000 7fea0cf43640 1 --2- 172.21.15.64:0/3084363248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea08102c10 0x7fea081008f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.385+0000 7fea0cf43640 1 -- 172.21.15.64:0/3084363248 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea081a66a0 con 0x7fea0810c110 2024-09-12T03:37:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.385+0000 7fea077fe640 1 --2- 172.21.15.64:0/3084363248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fea081035c0 0x7fea08100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:37:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.386+0000 7fea0cf43640 1 --2- 172.21.15.64:0/3084363248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea0810c110 0x7fea08101370 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fea00009dc0 tx=0x7fea00032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.386+0000 7fea057fa640 1 -- 172.21.15.64:0/3084363248 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea00003cf0 con 0x7fea0810c110 2024-09-12T03:37:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.386+0000 7fea0e9a6640 1 -- 172.21.15.64:0/3084363248 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea081a6930 con 0x7fea0810c110 2024-09-12T03:37:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.386+0000 7fea057fa640 1 -- 172.21.15.64:0/3084363248 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea000044f0 con 0x7fea0810c110 2024-09-12T03:37:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.386+0000 7fea057fa640 1 -- 172.21.15.64:0/3084363248 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea00030bf0 con 0x7fea0810c110 2024-09-12T03:37:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.386+0000 7fea0e9a6640 1 -- 172.21.15.64:0/3084363248 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea081a6e80 con 0x7fea0810c110 2024-09-12T03:37:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.389+0000 7fea11822740 1 Processor -- start 2024-09-12T03:37:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.389+0000 7fea057fa640 1 -- 172.21.15.64:0/3084363248 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fea00004040 con 0x7fea0810c110 2024-09-12T03:37:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.389+0000 7fea11822740 1 -- start start 2024-09-12T03:37:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.390+0000 7fea11822740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5860cdaf0 0x55c5860cded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.390+0000 7fea11822740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5860ce410 0x55c5860d8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.390+0000 7fea11822740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c5860d8cb0 0x55c5860db0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.390+0000 7fea11822740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c5860322f0 con 0x55c5860cdaf0 2024-09-12T03:37:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.390+0000 7fea11822740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c585fda270 con 0x55c5860ce410 2024-09-12T03:37:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.390+0000 7fea11822740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c58602e2b0 con 0x55c5860d8cb0 2024-09-12T03:37:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.390+0000 7fea077fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5860cdaf0 0x55c5860cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.390+0000 7fea077fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5860cdaf0 0x55c5860cded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43782/0 (socket says 172.21.15.64:43782) 2024-09-12T03:37:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.390+0000 7fea077fe640 1 -- 172.21.15.64:0/941120851 learned_addr learned my addr 172.21.15.64:0/941120851 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:37:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.390+0000 7fea07fff640 1 --2- 172.21.15.64:0/941120851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5860ce410 0x55c5860d8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.390+0000 7fea077fe640 1 -- 172.21.15.64:0/941120851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c5860d8cb0 msgr2=0x55c5860db0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:37:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.390+0000 7fea077fe640 1 --2- 172.21.15.64:0/941120851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c5860d8cb0 0x55c5860db0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.390+0000 7fea077fe640 1 -- 172.21.15.64:0/941120851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5860ce410 msgr2=0x55c5860d8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:37:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.390+0000 7fea077fe640 1 --2- 172.21.15.64:0/941120851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5860ce410 0x55c5860d8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.390+0000 7fea057fa640 1 --2- 172.21.15.64:0/3084363248 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fe9f00a2da0 0x7fe9f00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.390+0000 7fea077fe640 1 -- 172.21.15.64:0/941120851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c585fd9340 con 0x55c5860cdaf0 2024-09-12T03:37:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.390+0000 7fea057fa640 1 -- 172.21.15.64:0/3084363248 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9583+0+0 (secure 0 0 0) 0x7fea000f0e60 con 0x7fea0810c110 2024-09-12T03:37:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.390+0000 7fea077fe640 1 --2- 172.21.15.64:0/3084363248 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fe9f00a2da0 0x7fe9f00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.390+0000 7fea077fe640 1 --2- 172.21.15.64:0/941120851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5860cdaf0 0x55c5860cded0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc009c30 tx=0x7fe9fc0106e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ce6fb4e6d9b691 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.391+0000 7fe9ee7fc640 1 -- 172.21.15.64:0/941120851 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc01f7c0 con 0x55c5860cdaf0 2024-09-12T03:37:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.391+0000 7fe9ee7fc640 1 -- 172.21.15.64:0/941120851 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe9fc012da0 con 0x55c5860cdaf0 2024-09-12T03:37:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.391+0000 7fe9ee7fc640 1 -- 172.21.15.64:0/941120851 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc01ecf0 con 0x55c5860cdaf0 2024-09-12T03:37:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.391+0000 7fea11822740 1 -- 172.21.15.64:0/941120851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5860cdaf0 msgr2=0x55c5860cded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.391+0000 7fea077fe640 1 --2- 172.21.15.64:0/3084363248 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fe9f00a2da0 0x7fe9f00a5260 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc0058c0 tx=0x7fe9fc021000 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.391+0000 7fea11822740 1 --2- 172.21.15.64:0/941120851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5860cdaf0 0x55c5860cded0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc009c30 tx=0x7fe9fc0106e0 comp rx=0 tx=0).stop 2024-09-12T03:37:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.391+0000 7fea11822740 1 -- 172.21.15.64:0/941120851 shutdown_connections 2024-09-12T03:37:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.391+0000 7fea11822740 1 --2- 172.21.15.64:0/941120851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c5860d8cb0 0x55c5860db0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.391+0000 7fea11822740 1 --2- 172.21.15.64:0/941120851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5860ce410 0x55c5860d8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.391+0000 7fea11822740 1 --2- 172.21.15.64:0/941120851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5860cdaf0 0x55c5860cded0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.391+0000 7fea11822740 1 -- 172.21.15.64:0/941120851 >> 172.21.15.64:0/941120851 conn(0x55c5860ca060 msgr2=0x55c5860ca440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.392+0000 7fea11822740 1 -- 172.21.15.64:0/941120851 shutdown_connections 2024-09-12T03:37:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.392+0000 7fea11822740 1 -- 172.21.15.64:0/941120851 wait complete. 2024-09-12T03:37:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.392+0000 7fea11822740 1 -- start start 2024-09-12T03:37:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.392+0000 7fea11822740 1 Processor -- start 2024-09-12T03:37:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.392+0000 7fea11822740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5860cdaf0 0x55c5860d5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.392+0000 7fea11822740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c5860ce410 0x55c5860d6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.392+0000 7fea11822740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5860d8cb0 0x55c5860d65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.392+0000 7fea11822740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c5860db880 con 0x55c5860cdaf0 2024-09-12T03:37:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.392+0000 7fea11822740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c585fbb290 con 0x55c5860d8cb0 2024-09-12T03:37:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.392+0000 7fea11822740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c5860dba00 con 0x55c5860ce410 2024-09-12T03:37:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.393+0000 7fea11822740 10 client.0 _ll_register_callbacks cb 0x55c585fec140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:37:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.393+0000 7fea07fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5860cdaf0 0x55c5860d5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.393+0000 7fea07fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5860cdaf0 0x55c5860d5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43794/0 (socket says 172.21.15.64:43794) 2024-09-12T03:37:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.393+0000 7fea07fff640 1 -- 172.21.15.64:0/667195106 learned_addr learned my addr 172.21.15.64:0/667195106 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:37:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.393+0000 7fea077fe640 1 --2- 172.21.15.64:0/667195106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c5860ce410 0x55c5860d6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.393+0000 7fea0cf43640 1 --2- 172.21.15.64:0/667195106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5860d8cb0 0x55c5860d65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.393+0000 7fea07fff640 1 -- 172.21.15.64:0/667195106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c5860ce410 msgr2=0x55c5860d6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:37:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.393+0000 7fea07fff640 1 --2- 172.21.15.64:0/667195106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c5860ce410 0x55c5860d6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.393+0000 7fea07fff640 1 -- 172.21.15.64:0/667195106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5860d8cb0 msgr2=0x55c5860d65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:37:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.393+0000 7fea07fff640 1 --2- 172.21.15.64:0/667195106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5860d8cb0 0x55c5860d65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.393+0000 7fea07fff640 1 -- 172.21.15.64:0/667195106 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c585d45210 con 0x55c5860cdaf0 2024-09-12T03:37:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.393+0000 7fea0cf43640 1 --2- 172.21.15.64:0/667195106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5860d8cb0 0x55c5860d65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:37:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.393+0000 7fea07fff640 1 --2- 172.21.15.64:0/667195106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5860cdaf0 0x55c5860d5b50 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8010840 tx=0x7fe9f8010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd14ac7a5c3fedc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.393+0000 7fe9e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:37:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.393+0000 7fe9e77fe640 1 -- 172.21.15.64:0/667195106 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f800ed40 con 0x55c5860cdaf0 2024-09-12T03:37:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.393+0000 7fe9e77fe640 1 -- 172.21.15.64:0/667195106 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9583+0+0 (secure 0 0 0) 0x7fe9f801c480 con 0x55c5860cdaf0 2024-09-12T03:37:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.393+0000 7fea11822740 1 -- 172.21.15.64:0/667195106 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9fc012da0 con 0x55c5860cdaf0 2024-09-12T03:37:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.394+0000 7fe9e77fe640 1 -- 172.21.15.64:0/667195106 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe9f8025050 con 0x55c5860cdaf0 2024-09-12T03:37:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.394+0000 7fea11822740 10 client.15990 fetch_fsmap learned FSMap version 25 2024-09-12T03:37:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.394+0000 7fea11822740 1 -- 172.21.15.64:0/667195106 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c585e11620 con 0x55c5860cdaf0 2024-09-12T03:37:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.394+0000 7fe9e77fe640 1 -- 172.21.15.64:0/667195106 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe9fc012da0 con 0x55c5860cdaf0 2024-09-12T03:37:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.394+0000 7fe9e77fe640 1 -- 172.21.15.64:0/667195106 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe9f80115f0 con 0x55c5860cdaf0 2024-09-12T03:37:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.394+0000 7fea11822740 10 client.15990 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:37:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.394+0000 7fea11822740 10 client.15990 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:37:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.394+0000 7fea11822740 20 client.15990 populate_metadata read hostname 'smithi064' 2024-09-12T03:37:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.394+0000 7fea11822740 1 --2- 172.21.15.64:0/667195106 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55c5860e8ca0 0x55c5860eb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.394+0000 7fea11822740 4 client.15990 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:37:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.394+0000 7fea11822740 1 -- 172.21.15.64:0/667195106 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c58603fa20 con 0x55c5860e8ca0 2024-09-12T03:37:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.394+0000 7fea077fe640 1 --2- 172.21.15.64:0/667195106 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55c5860e8ca0 0x55c5860eb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.395+0000 7fe9e77fe640 10 client.15990 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:37:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.395+0000 7fea077fe640 1 --2- 172.21.15.64:0/667195106 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55c5860e8ca0 0x55c5860eb0d0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd59a09419183e70 server_cookie=2dda3dcb6eb5b1a8 in_seq=0 out_seq=0 2024-09-12T03:37:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.396+0000 7fe9e77fe640 1 -- 172.21.15.64:0/667195106 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c58603fa20 con 0x55c5860e8ca0 2024-09-12T03:37:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.396+0000 7fe9e77fe640 10 client.15990 handle_command_reply: tid=0 2024-09-12T03:37:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.396+0000 7fea11822740 1 client.15990 shutdown 2024-09-12T03:37:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.396+0000 7fea11822740 1 -- 172.21.15.64:0/667195106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5860cdaf0 msgr2=0x55c5860d5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.396+0000 7fea11822740 1 --2- 172.21.15.64:0/667195106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5860cdaf0 0x55c5860d5b50 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8010840 tx=0x7fe9f8010d00 comp rx=0 tx=0).stop 2024-09-12T03:37:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.396+0000 7fea11822740 1 -- 172.21.15.64:0/667195106 shutdown_connections 2024-09-12T03:37:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.396+0000 7fea11822740 1 --2- 172.21.15.64:0/667195106 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55c5860e8ca0 0x55c5860eb0d0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.396+0000 7fea11822740 1 --2- 172.21.15.64:0/667195106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5860d8cb0 0x55c5860d65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.396+0000 7fea11822740 1 --2- 172.21.15.64:0/667195106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c5860ce410 0x55c5860d6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.396+0000 7fea11822740 1 --2- 172.21.15.64:0/667195106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5860cdaf0 0x55c5860d5b50 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.396+0000 7fe9e77fe640 0 client.15990 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:37:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.396+0000 7fea11822740 1 -- 172.21.15.64:0/667195106 >> 172.21.15.64:0/667195106 conn(0x55c5860ca060 msgr2=0x55c5860cfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.396+0000 7fea11822740 1 -- 172.21.15.64:0/667195106 shutdown_connections 2024-09-12T03:37:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.396+0000 7fea11822740 1 -- 172.21.15.64:0/667195106 wait complete. 2024-09-12T03:37:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.396+0000 7fea11822740 20 client.15990 trim_cache size 0 max 16384 2024-09-12T03:37:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.411+0000 7fea11822740 1 Processor -- start 2024-09-12T03:37:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.411+0000 7fea11822740 1 -- start start 2024-09-12T03:37:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.411+0000 7fea11822740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5860cdaf0 0x55c5860d0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.411+0000 7fea11822740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c5860ce410 0x55c5860d6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.411+0000 7fea11822740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5860d8cb0 0x55c5860f09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.411+0000 7fea11822740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fea081b3a90 con 0x55c5860cdaf0 2024-09-12T03:37:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.411+0000 7fea11822740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fea081b1360 con 0x55c5860d8cb0 2024-09-12T03:37:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.411+0000 7fea11822740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fea081af1d0 con 0x55c5860ce410 2024-09-12T03:37:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.411+0000 7fea07fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5860cdaf0 0x55c5860d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.411+0000 7fea07fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5860cdaf0 0x55c5860d0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43804/0 (socket says 172.21.15.64:43804) 2024-09-12T03:37:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.411+0000 7fea07fff640 1 -- 172.21.15.64:0/3130182560 learned_addr learned my addr 172.21.15.64:0/3130182560 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:37:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.411+0000 7fea07fff640 1 -- 172.21.15.64:0/3130182560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c5860ce410 msgr2=0x55c5860d6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:37:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.411+0000 7fea07fff640 1 --2- 172.21.15.64:0/3130182560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c5860ce410 0x55c5860d6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.411+0000 7fea07fff640 1 -- 172.21.15.64:0/3130182560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5860d8cb0 msgr2=0x55c5860f09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:37:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.411+0000 7fea07fff640 1 --2- 172.21.15.64:0/3130182560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5860d8cb0 0x55c5860f09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.411+0000 7fea07fff640 1 -- 172.21.15.64:0/3130182560 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5860c9e90 con 0x55c5860cdaf0 2024-09-12T03:37:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.412+0000 7fea07fff640 1 --2- 172.21.15.64:0/3130182560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5860cdaf0 0x55c5860d0560 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8000c00 tx=0x7fe9f8010490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d9fce19f6d31eb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.412+0000 7fe9edffb640 1 -- 172.21.15.64:0/3130182560 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f802d020 con 0x55c5860cdaf0 2024-09-12T03:37:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.412+0000 7fe9edffb640 1 -- 172.21.15.64:0/3130182560 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe9f800c620 con 0x55c5860cdaf0 2024-09-12T03:37:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.412+0000 7fe9edffb640 1 -- 172.21.15.64:0/3130182560 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f801c6e0 con 0x55c5860cdaf0 2024-09-12T03:37:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.412+0000 7fea11822740 1 -- 172.21.15.64:0/3130182560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5860cdaf0 msgr2=0x55c5860d0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.412+0000 7fea11822740 1 --2- 172.21.15.64:0/3130182560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5860cdaf0 0x55c5860d0560 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8000c00 tx=0x7fe9f8010490 comp rx=0 tx=0).stop 2024-09-12T03:37:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.412+0000 7fea11822740 1 -- 172.21.15.64:0/3130182560 shutdown_connections 2024-09-12T03:37:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.412+0000 7fea11822740 1 --2- 172.21.15.64:0/3130182560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5860d8cb0 0x55c5860f09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.412+0000 7fea11822740 1 --2- 172.21.15.64:0/3130182560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c5860ce410 0x55c5860d6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.412+0000 7fea11822740 1 --2- 172.21.15.64:0/3130182560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5860cdaf0 0x55c5860d0560 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.412+0000 7fea11822740 1 -- 172.21.15.64:0/3130182560 >> 172.21.15.64:0/3130182560 conn(0x55c5860ca060 msgr2=0x55c5860d7bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.412+0000 7fea11822740 1 -- 172.21.15.64:0/3130182560 shutdown_connections 2024-09-12T03:37:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.413+0000 7fea11822740 1 -- 172.21.15.64:0/3130182560 wait complete. 2024-09-12T03:37:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.413+0000 7fea11822740 1 -- start start 2024-09-12T03:37:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.413+0000 7fea11822740 1 Processor -- start 2024-09-12T03:37:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.413+0000 7fea11822740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5860cdaf0 0x55c5860e1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.413+0000 7fea11822740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c5860ce410 0x55c5860d28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.413+0000 7fea11822740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5860d8cb0 0x55c5860d2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.413+0000 7fea11822740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fea081af820 con 0x55c5860cdaf0 2024-09-12T03:37:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.413+0000 7fea11822740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fea081b0c90 con 0x55c5860d8cb0 2024-09-12T03:37:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.413+0000 7fea11822740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fea081afef0 con 0x55c5860ce410 2024-09-12T03:37:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.413+0000 7fea11822740 10 client.0 _ll_register_callbacks cb 0x7fe9f800c450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:37:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.413+0000 7fea07fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5860cdaf0 0x55c5860e1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.413+0000 7fea07fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5860cdaf0 0x55c5860e1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43818/0 (socket says 172.21.15.64:43818) 2024-09-12T03:37:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.413+0000 7fea07fff640 1 -- 172.21.15.64:0/3149103068 learned_addr learned my addr 172.21.15.64:0/3149103068 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:37:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.414+0000 7fea0cf43640 1 --2- 172.21.15.64:0/3149103068 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5860d8cb0 0x55c5860d2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.414+0000 7fea077fe640 1 --2- 172.21.15.64:0/3149103068 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c5860ce410 0x55c5860d28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.414+0000 7fea07fff640 1 -- 172.21.15.64:0/3149103068 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c5860ce410 msgr2=0x55c5860d28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:37:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.414+0000 7fea07fff640 1 --2- 172.21.15.64:0/3149103068 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c5860ce410 0x55c5860d28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.414+0000 7fea07fff640 1 -- 172.21.15.64:0/3149103068 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5860d8cb0 msgr2=0x55c5860d2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:37:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.414+0000 7fea07fff640 1 --2- 172.21.15.64:0/3149103068 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5860d8cb0 0x55c5860d2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.414+0000 7fea07fff640 1 -- 172.21.15.64:0/3149103068 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5860c52d0 con 0x55c5860cdaf0 2024-09-12T03:37:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.414+0000 7fea07fff640 1 --2- 172.21.15.64:0/3149103068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5860cdaf0 0x55c5860e1420 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800c5f0 tx=0x7fe9f8011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0659cd77cdf5a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.414+0000 7fe9e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:37:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.414+0000 7fe9e77fe640 1 -- 172.21.15.64:0/3149103068 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8037060 con 0x55c5860cdaf0 2024-09-12T03:37:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.414+0000 7fe9e77fe640 1 -- 172.21.15.64:0/3149103068 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9583+0+0 (secure 0 0 0) 0x7fe9f8005040 con 0x55c5860cdaf0 2024-09-12T03:37:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.414+0000 7fea11822740 1 -- 172.21.15.64:0/3149103068 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9f800c620 con 0x55c5860cdaf0 2024-09-12T03:37:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.414+0000 7fe9e77fe640 1 -- 172.21.15.64:0/3149103068 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe9f803e050 con 0x55c5860cdaf0 2024-09-12T03:37:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.414+0000 7fea11822740 10 client.15996 fetch_fsmap learned FSMap version 25 2024-09-12T03:37:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.414+0000 7fea11822740 1 -- 172.21.15.64:0/3149103068 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c586153850 con 0x55c5860cdaf0 2024-09-12T03:37:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.415+0000 7fe9e77fe640 1 -- 172.21.15.64:0/3149103068 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe9f800c620 con 0x55c5860cdaf0 2024-09-12T03:37:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.415+0000 7fe9e77fe640 1 -- 172.21.15.64:0/3149103068 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe9f8003c60 con 0x55c5860cdaf0 2024-09-12T03:37:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.415+0000 7fea11822740 10 client.15996 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:37:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.415+0000 7fea11822740 10 client.15996 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:37:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.415+0000 7fea11822740 20 client.15996 populate_metadata read hostname 'smithi064' 2024-09-12T03:37:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.415+0000 7fea11822740 1 --2- 172.21.15.64:0/3149103068 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55c586153eb0 0x55c5861562a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.415+0000 7fea11822740 4 client.15996 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:37:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.415+0000 7fea11822740 1 -- 172.21.15.64:0/3149103068 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c586039d70 con 0x55c586153eb0 2024-09-12T03:37:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.415+0000 7fea077fe640 1 --2- 172.21.15.64:0/3149103068 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55c586153eb0 0x55c5861562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.415+0000 7fea077fe640 1 --2- 172.21.15.64:0/3149103068 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55c586153eb0 0x55c5861562a0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46655f3e390b90ba server_cookie=7f58c29dbb40e68b in_seq=0 out_seq=0 2024-09-12T03:37:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.415+0000 7fe9e77fe640 10 client.15996 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:37:11.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.433+0000 7fe9e77fe640 1 -- 172.21.15.64:0/3149103068 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c586039d70 con 0x55c586153eb0 2024-09-12T03:37:11.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.433+0000 7fe9e77fe640 10 client.15996 handle_command_reply: tid=0 2024-09-12T03:37:11.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.433+0000 7fea11822740 1 client.15996 shutdown 2024-09-12T03:37:11.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.433+0000 7fea11822740 1 -- 172.21.15.64:0/3149103068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5860cdaf0 msgr2=0x55c5860e1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:11.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.433+0000 7fea11822740 1 --2- 172.21.15.64:0/3149103068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5860cdaf0 0x55c5860e1420 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800c5f0 tx=0x7fe9f8011de0 comp rx=0 tx=0).stop 2024-09-12T03:37:11.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.434+0000 7fea11822740 1 -- 172.21.15.64:0/3149103068 shutdown_connections 2024-09-12T03:37:11.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.434+0000 7fea11822740 1 --2- 172.21.15.64:0/3149103068 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55c586153eb0 0x55c5861562a0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:11.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.434+0000 7fea11822740 1 --2- 172.21.15.64:0/3149103068 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5860d8cb0 0x55c5860d2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:11.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.434+0000 7fea11822740 1 --2- 172.21.15.64:0/3149103068 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c5860ce410 0x55c5860d28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.434+0000 7fea11822740 1 --2- 172.21.15.64:0/3149103068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5860cdaf0 0x55c5860e1420 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.434+0000 7fe9e77fe640 0 client.15996 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:37:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.434+0000 7fea11822740 1 -- 172.21.15.64:0/3149103068 >> 172.21.15.64:0/3149103068 conn(0x55c5860ca060 msgr2=0x55c5860c89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.434+0000 7fea11822740 1 -- 172.21.15.64:0/3149103068 shutdown_connections 2024-09-12T03:37:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.434+0000 7fea11822740 1 -- 172.21.15.64:0/3149103068 wait complete. 2024-09-12T03:37:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.434+0000 7fea11822740 20 client.15996 trim_cache size 0 max 16384 2024-09-12T03:37:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:37:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:37:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:37:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:37:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.434+0000 7fea0e9a6640 1 -- 172.21.15.64:0/3084363248 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fe9f00a2da0 msgr2=0x7fe9f00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.434+0000 7fea0e9a6640 1 --2- 172.21.15.64:0/3084363248 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fe9f00a2da0 0x7fe9f00a5260 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc0058c0 tx=0x7fe9fc021000 comp rx=0 tx=0).stop 2024-09-12T03:37:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.434+0000 7fea0e9a6640 1 -- 172.21.15.64:0/3084363248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea0810c110 msgr2=0x7fea08101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.434+0000 7fea0e9a6640 1 --2- 172.21.15.64:0/3084363248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea0810c110 0x7fea08101370 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fea00009dc0 tx=0x7fea00032ff0 comp rx=0 tx=0).stop 2024-09-12T03:37:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.434+0000 7fea0e9a6640 1 -- 172.21.15.64:0/3084363248 shutdown_connections 2024-09-12T03:37:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.434+0000 7fea0e9a6640 1 --2- 172.21.15.64:0/3084363248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea0810c110 0x7fea08101370 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.434+0000 7fea0e9a6640 1 --2- 172.21.15.64:0/3084363248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fea081035c0 0x7fea08100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.434+0000 7fea0e9a6640 1 --2- 172.21.15.64:0/3084363248 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fe9f00a2da0 0x7fe9f00a5260 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.434+0000 7fea0e9a6640 1 --2- 172.21.15.64:0/3084363248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea08102c10 0x7fea081008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.434+0000 7fea0e9a6640 1 -- 172.21.15.64:0/3084363248 >> 172.21.15.64:0/3084363248 conn(0x7fea080fc6b0 msgr2=0x7fea080fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.435+0000 7fea0e9a6640 1 -- 172.21.15.64:0/3084363248 shutdown_connections 2024-09-12T03:37:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:11.435+0000 7fea0e9a6640 1 -- 172.21.15.64:0/3084363248 wait complete. 2024-09-12T03:37:11.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:37:11 smithi064 ceph-mon[30952]: pgmap v243: 129 pgs: 129 active+clean; 5.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 105 KiB/s wr, 36 op/s 2024-09-12T03:37:11.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:37:11 smithi064 ceph-mon[30952]: osdmap e97: 12 total, 12 up, 12 in 2024-09-12T03:37:11.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:37:11 smithi115 ceph-mon[34801]: pgmap v243: 129 pgs: 129 active+clean; 5.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 105 KiB/s wr, 36 op/s 2024-09-12T03:37:11.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:37:11 smithi115 ceph-mon[34801]: osdmap e97: 12 total, 12 up, 12 in 2024-09-12T03:37:11.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:37:11 smithi195 ceph-mon[34786]: pgmap v243: 129 pgs: 129 active+clean; 5.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 105 KiB/s wr, 36 op/s 2024-09-12T03:37:11.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:37:11 smithi195 ceph-mon[34786]: osdmap e97: 12 total, 12 up, 12 in 2024-09-12T03:37:13.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:37:13 smithi064 ceph-mon[30952]: pgmap v245: 129 pgs: 129 active+clean; 1.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 64 KiB/s wr, 20 op/s 2024-09-12T03:37:13.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:37:13 smithi115 ceph-mon[34801]: pgmap v245: 129 pgs: 129 active+clean; 1.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 64 KiB/s wr, 20 op/s 2024-09-12T03:37:13.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:37:13 smithi195 ceph-mon[34786]: pgmap v245: 129 pgs: 129 active+clean; 1.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 64 KiB/s wr, 20 op/s 2024-09-12T03:37:15.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:37:15 smithi064 ceph-mon[30952]: pgmap v246: 129 pgs: 129 active+clean; 1.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 53 KiB/s wr, 16 op/s 2024-09-12T03:37:15.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:37:15 smithi115 ceph-mon[34801]: pgmap v246: 129 pgs: 129 active+clean; 1.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 53 KiB/s wr, 16 op/s 2024-09-12T03:37:15.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:37:15 smithi195 ceph-mon[34786]: pgmap v246: 129 pgs: 129 active+clean; 1.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 53 KiB/s wr, 16 op/s 2024-09-12T03:37:17.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:37:17 smithi064 ceph-mon[30952]: pgmap v247: 129 pgs: 129 active+clean; 1.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 53 KiB/s wr, 16 op/s 2024-09-12T03:37:17.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:37:17 smithi115 ceph-mon[34801]: pgmap v247: 129 pgs: 129 active+clean; 1.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 53 KiB/s wr, 16 op/s 2024-09-12T03:37:17.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:37:17 smithi195 ceph-mon[34786]: pgmap v247: 129 pgs: 129 active+clean; 1.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 53 KiB/s wr, 16 op/s 2024-09-12T03:37:19.758 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:37:19 smithi195 ceph-mon[34786]: pgmap v248: 129 pgs: 129 active+clean; 1.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 50 KiB/s wr, 16 op/s 2024-09-12T03:37:19.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:37:19 smithi064 ceph-mon[30952]: pgmap v248: 129 pgs: 129 active+clean; 1.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 50 KiB/s wr, 16 op/s 2024-09-12T03:37:19.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:37:19 smithi115 ceph-mon[34801]: pgmap v248: 129 pgs: 129 active+clean; 1.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 50 KiB/s wr, 16 op/s 2024-09-12T03:37:21.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:37:21 smithi064 ceph-mon[30952]: pgmap v249: 129 pgs: 129 active+clean; 1.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 50 KiB/s wr, 16 op/s 2024-09-12T03:37:21.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:37:21 smithi115 ceph-mon[34801]: pgmap v249: 129 pgs: 129 active+clean; 1.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 50 KiB/s wr, 16 op/s 2024-09-12T03:37:21.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:37:21 smithi195 ceph-mon[34786]: pgmap v249: 129 pgs: 129 active+clean; 1.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 50 KiB/s wr, 16 op/s 2024-09-12T03:37:23.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:37:23 smithi064 ceph-mon[30952]: pgmap v250: 129 pgs: 129 active+clean; 2.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 66 KiB/s wr, 21 op/s 2024-09-12T03:37:23.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:37:23 smithi115 ceph-mon[34801]: pgmap v250: 129 pgs: 129 active+clean; 2.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 66 KiB/s wr, 21 op/s 2024-09-12T03:37:23.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:37:23 smithi195 ceph-mon[34786]: pgmap v250: 129 pgs: 129 active+clean; 2.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 66 KiB/s wr, 21 op/s 2024-09-12T03:37:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.533+0000 7f483cd66640 1 -- 172.21.15.64:0/4096168935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48381069c0 msgr2=0x7f4838106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.533+0000 7f483cd66640 1 --2- 172.21.15.64:0/4096168935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48381069c0 0x7f4838106da0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f4828009930 tx=0x7f482802a420 comp rx=0 tx=0).stop 2024-09-12T03:37:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.534+0000 7f483cd66640 1 -- 172.21.15.64:0/4096168935 shutdown_connections 2024-09-12T03:37:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.534+0000 7f483cd66640 1 --2- 172.21.15.64:0/4096168935 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4838101350 0x7f483810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.534+0000 7f483cd66640 1 --2- 172.21.15.64:0/4096168935 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48381009b0 0x7f4838100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.534+0000 7f483cd66640 1 --2- 172.21.15.64:0/4096168935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48381069c0 0x7f4838106da0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.534+0000 7f483cd66640 1 -- 172.21.15.64:0/4096168935 >> 172.21.15.64:0/4096168935 conn(0x7f4838077990 msgr2=0x7f48380fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.534+0000 7f483cd66640 1 -- 172.21.15.64:0/4096168935 shutdown_connections 2024-09-12T03:37:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.534+0000 7f483cd66640 1 -- 172.21.15.64:0/4096168935 wait complete. 2024-09-12T03:37:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.534+0000 7f483cd66640 1 Processor -- start 2024-09-12T03:37:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.534+0000 7f483cd66640 1 -- start start 2024-09-12T03:37:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.535+0000 7f483cd66640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48381009b0 0x7f4838197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.535+0000 7f483cd66640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4838101350 0x7f48381976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.535+0000 7f483cd66640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48381069c0 0x7f483819ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.535+0000 7f483cd66640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4838110cb0 con 0x7f48381069c0 2024-09-12T03:37:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.535+0000 7f483cd66640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4838110b30 con 0x7f4838101350 2024-09-12T03:37:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.535+0000 7f483cd66640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4838110e30 con 0x7f48381009b0 2024-09-12T03:37:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.535+0000 7f4836d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48381069c0 0x7f483819ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.535+0000 7f4835d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4838101350 0x7f48381976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.535+0000 7f4836d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48381069c0 0x7f483819ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55198/0 (socket says 172.21.15.64:55198) 2024-09-12T03:37:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.535+0000 7f4836d9d640 1 -- 172.21.15.64:0/3402263877 learned_addr learned my addr 172.21.15.64:0/3402263877 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:37:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.535+0000 7f483659c640 1 --2- 172.21.15.64:0/3402263877 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48381009b0 0x7f4838197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.535+0000 7f4835d9b640 1 -- 172.21.15.64:0/3402263877 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48381009b0 msgr2=0x7f4838197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.535+0000 7f4835d9b640 1 --2- 172.21.15.64:0/3402263877 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48381009b0 0x7f4838197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.535+0000 7f4835d9b640 1 -- 172.21.15.64:0/3402263877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48381069c0 msgr2=0x7f483819ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.535+0000 7f4835d9b640 1 --2- 172.21.15.64:0/3402263877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48381069c0 0x7f483819ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.535+0000 7f4835d9b640 1 -- 172.21.15.64:0/3402263877 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f483819c160 con 0x7f4838101350 2024-09-12T03:37:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.535+0000 7f483659c640 1 --2- 172.21.15.64:0/3402263877 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48381009b0 0x7f4838197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:37:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.535+0000 7f4836d9d640 1 --2- 172.21.15.64:0/3402263877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48381069c0 0x7f483819ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:37:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.535+0000 7f4835d9b640 1 --2- 172.21.15.64:0/3402263877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4838101350 0x7f48381976b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f482c00e9f0 tx=0x7f482c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.536+0000 7f48277fe640 1 -- 172.21.15.64:0/3402263877 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c00ccc0 con 0x7f4838101350 2024-09-12T03:37:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.536+0000 7f483cd66640 1 -- 172.21.15.64:0/3402263877 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f483819c450 con 0x7f4838101350 2024-09-12T03:37:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.536+0000 7f48277fe640 1 -- 172.21.15.64:0/3402263877 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f482c0108e0 con 0x7f4838101350 2024-09-12T03:37:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.536+0000 7f48277fe640 1 -- 172.21.15.64:0/3402263877 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c018880 con 0x7f4838101350 2024-09-12T03:37:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.536+0000 7f483cd66640 1 -- 172.21.15.64:0/3402263877 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48381a3ce0 con 0x7f4838101350 2024-09-12T03:37:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.538+0000 7f483fbdd740 1 Processor -- start 2024-09-12T03:37:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.538+0000 7f483fbdd740 1 -- start start 2024-09-12T03:37:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.538+0000 7f48277fe640 1 -- 172.21.15.64:0/3402263877 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f482c010a90 con 0x7f4838101350 2024-09-12T03:37:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.539+0000 7f483fbdd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595acbfb930 0x5595acbfbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.539+0000 7f483fbdd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595acbfc250 0x5595acc08980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.539+0000 7f483fbdd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595acc08ec0 0x5595acc0b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.539+0000 7f483fbdd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5595acb602f0 con 0x5595acbfb930 2024-09-12T03:37:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.539+0000 7f483fbdd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5595acb08270 con 0x5595acc08ec0 2024-09-12T03:37:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.539+0000 7f483fbdd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5595acb5c2b0 con 0x5595acbfc250 2024-09-12T03:37:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.539+0000 7f4836d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595acbfb930 0x5595acbfbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.539+0000 7f4836d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595acbfb930 0x5595acbfbd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55214/0 (socket says 172.21.15.64:55214) 2024-09-12T03:37:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.539+0000 7f4836d9d640 1 -- 172.21.15.64:0/3797080026 learned_addr learned my addr 172.21.15.64:0/3797080026 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:37:24.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.539+0000 7f483659c640 1 --2- 172.21.15.64:0/3797080026 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595acbfc250 0x5595acc08980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:24.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.539+0000 7f4835d9b640 1 --2- 172.21.15.64:0/3797080026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595acc08ec0 0x5595acc0b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:24.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.539+0000 7f4836d9d640 1 -- 172.21.15.64:0/3797080026 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595acbfc250 msgr2=0x5595acc08980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:37:24.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.539+0000 7f4836d9d640 1 --2- 172.21.15.64:0/3797080026 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595acbfc250 0x5595acc08980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:24.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.539+0000 7f4836d9d640 1 -- 172.21.15.64:0/3797080026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595acc08ec0 msgr2=0x5595acc0b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:37:24.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.539+0000 7f4836d9d640 1 --2- 172.21.15.64:0/3797080026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595acc08ec0 0x5595acc0b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:24.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.539+0000 7f4836d9d640 1 -- 172.21.15.64:0/3797080026 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595acb07340 con 0x5595acbfb930 2024-09-12T03:37:24.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.539+0000 7f483659c640 1 --2- 172.21.15.64:0/3797080026 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595acbfc250 0x5595acc08980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:37:24.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.539+0000 7f48277fe640 1 --2- 172.21.15.64:0/3402263877 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f48200a2da0 0x7f48200a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:24.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.539+0000 7f4836d9d640 1 --2- 172.21.15.64:0/3797080026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595acbfb930 0x5595acbfbd10 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f4830011880 tx=0x7f4830011d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a064f696077a5ff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.540+0000 7f483659c640 1 --2- 172.21.15.64:0/3402263877 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f48200a2da0 0x7f48200a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.540+0000 7f48277fe640 1 -- 172.21.15.64:0/3402263877 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9583+0+0 (secure 0 0 0) 0x7f482c0cfbc0 con 0x7f4838101350 2024-09-12T03:37:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.540+0000 7f4824ff9640 1 -- 172.21.15.64:0/3797080026 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4830011310 con 0x5595acbfb930 2024-09-12T03:37:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.540+0000 7f4824ff9640 1 -- 172.21.15.64:0/3797080026 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f483000fe50 con 0x5595acbfb930 2024-09-12T03:37:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.540+0000 7f4824ff9640 1 -- 172.21.15.64:0/3797080026 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48300193e0 con 0x5595acbfb930 2024-09-12T03:37:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.541+0000 7f483659c640 1 --2- 172.21.15.64:0/3402263877 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f48200a2da0 0x7f48200a5260 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4828009900 tx=0x7f4828005e50 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.541+0000 7f483fbdd740 1 -- 172.21.15.64:0/3797080026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595acbfb930 msgr2=0x5595acbfbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.541+0000 7f483fbdd740 1 --2- 172.21.15.64:0/3797080026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595acbfb930 0x5595acbfbd10 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f4830011880 tx=0x7f4830011d40 comp rx=0 tx=0).stop 2024-09-12T03:37:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.541+0000 7f483fbdd740 1 -- 172.21.15.64:0/3797080026 shutdown_connections 2024-09-12T03:37:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.541+0000 7f483fbdd740 1 --2- 172.21.15.64:0/3797080026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595acc08ec0 0x5595acc0b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.541+0000 7f483fbdd740 1 --2- 172.21.15.64:0/3797080026 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595acbfc250 0x5595acc08980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.541+0000 7f483fbdd740 1 --2- 172.21.15.64:0/3797080026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595acbfb930 0x5595acbfbd10 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.541+0000 7f483fbdd740 1 -- 172.21.15.64:0/3797080026 >> 172.21.15.64:0/3797080026 conn(0x5595acbf8060 msgr2=0x5595acbf8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.541+0000 7f483fbdd740 1 -- 172.21.15.64:0/3797080026 shutdown_connections 2024-09-12T03:37:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.541+0000 7f483fbdd740 1 -- 172.21.15.64:0/3797080026 wait complete. 2024-09-12T03:37:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.542+0000 7f483fbdd740 1 -- start start 2024-09-12T03:37:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.542+0000 7f483fbdd740 1 Processor -- start 2024-09-12T03:37:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.542+0000 7f483fbdd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595acbfb930 0x5595acc05de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.542+0000 7f483fbdd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595acbfc250 0x5595acc06320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.542+0000 7f483fbdd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595acc08ec0 0x5595acc06860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.542+0000 7f483fbdd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5595acc0ba90 con 0x5595acc08ec0 2024-09-12T03:37:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.542+0000 7f483fbdd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5595acae9290 con 0x5595acbfc250 2024-09-12T03:37:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.542+0000 7f483fbdd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5595acc0bc10 con 0x5595acbfb930 2024-09-12T03:37:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.542+0000 7f483fbdd740 10 client.0 _ll_register_callbacks cb 0x5595acb1a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:37:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.542+0000 7f4836d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595acc08ec0 0x5595acc06860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.542+0000 7f4836d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595acc08ec0 0x5595acc06860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55220/0 (socket says 172.21.15.64:55220) 2024-09-12T03:37:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.542+0000 7f4835d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595acbfc250 0x5595acc06320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.542+0000 7f4836d9d640 1 -- 172.21.15.64:0/861597438 learned_addr learned my addr 172.21.15.64:0/861597438 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:37:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.542+0000 7f483659c640 1 --2- 172.21.15.64:0/861597438 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595acbfb930 0x5595acc05de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.543+0000 7f4836d9d640 1 -- 172.21.15.64:0/861597438 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595acbfb930 msgr2=0x5595acc05de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:37:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.543+0000 7f4836d9d640 1 --2- 172.21.15.64:0/861597438 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595acbfb930 0x5595acc05de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.543+0000 7f4836d9d640 1 -- 172.21.15.64:0/861597438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595acbfc250 msgr2=0x5595acc06320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:37:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.543+0000 7f4836d9d640 1 --2- 172.21.15.64:0/861597438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595acbfc250 0x5595acc06320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.543+0000 7f4836d9d640 1 -- 172.21.15.64:0/861597438 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595ac873210 con 0x5595acc08ec0 2024-09-12T03:37:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.543+0000 7f4835d9b640 1 --2- 172.21.15.64:0/861597438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595acbfc250 0x5595acc06320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:37:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.543+0000 7f4836d9d640 1 --2- 172.21.15.64:0/861597438 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595acc08ec0 0x5595acc06860 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f48300049d0 tx=0x7f483000fa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46a38e139cfc5b31 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.543+0000 7f481dffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:37:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.543+0000 7f481dffb640 1 -- 172.21.15.64:0/861597438 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4830010430 con 0x5595acc08ec0 2024-09-12T03:37:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.543+0000 7f481dffb640 1 -- 172.21.15.64:0/861597438 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9583+0+0 (secure 0 0 0) 0x7f48300105d0 con 0x5595acc08ec0 2024-09-12T03:37:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.543+0000 7f483fbdd740 1 -- 172.21.15.64:0/861597438 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f483000fe50 con 0x5595acc08ec0 2024-09-12T03:37:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.543+0000 7f481dffb640 1 -- 172.21.15.64:0/861597438 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4830031050 con 0x5595acc08ec0 2024-09-12T03:37:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.543+0000 7f483fbdd740 10 client.16008 fetch_fsmap learned FSMap version 25 2024-09-12T03:37:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.543+0000 7f483fbdd740 1 -- 172.21.15.64:0/861597438 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5595ac93f620 con 0x5595acc08ec0 2024-09-12T03:37:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.543+0000 7f481dffb640 1 -- 172.21.15.64:0/861597438 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f483000fe50 con 0x5595acc08ec0 2024-09-12T03:37:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.544+0000 7f481dffb640 1 -- 172.21.15.64:0/861597438 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4830018070 con 0x5595acc08ec0 2024-09-12T03:37:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.544+0000 7f483fbdd740 10 client.16008 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:37:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.544+0000 7f483fbdd740 10 client.16008 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:37:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.544+0000 7f483fbdd740 20 client.16008 populate_metadata read hostname 'smithi064' 2024-09-12T03:37:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.544+0000 7f483fbdd740 1 --2- 172.21.15.64:0/861597438 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5595acc18d30 0x5595acc1b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.544+0000 7f483fbdd740 4 client.16008 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:37:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.544+0000 7f483fbdd740 1 -- 172.21.15.64:0/861597438 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595acb6da20 con 0x5595acc18d30 2024-09-12T03:37:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.544+0000 7f483659c640 1 --2- 172.21.15.64:0/861597438 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5595acc18d30 0x5595acc1b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.544+0000 7f483659c640 1 --2- 172.21.15.64:0/861597438 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5595acc18d30 0x5595acc1b120 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32fcf7802fc6c299 server_cookie=5f59ee2694e1cbd in_seq=0 out_seq=0 2024-09-12T03:37:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.544+0000 7f481dffb640 10 client.16008 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:37:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.545+0000 7f481dffb640 1 -- 172.21.15.64:0/861597438 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5595acb6da20 con 0x5595acc18d30 2024-09-12T03:37:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.545+0000 7f481dffb640 10 client.16008 handle_command_reply: tid=0 2024-09-12T03:37:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.545+0000 7f483fbdd740 1 client.16008 shutdown 2024-09-12T03:37:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.545+0000 7f483fbdd740 1 -- 172.21.15.64:0/861597438 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595acc08ec0 msgr2=0x5595acc06860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.545+0000 7f483fbdd740 1 --2- 172.21.15.64:0/861597438 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595acc08ec0 0x5595acc06860 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f48300049d0 tx=0x7f483000fa20 comp rx=0 tx=0).stop 2024-09-12T03:37:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.545+0000 7f483fbdd740 1 -- 172.21.15.64:0/861597438 shutdown_connections 2024-09-12T03:37:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.545+0000 7f483fbdd740 1 --2- 172.21.15.64:0/861597438 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5595acc18d30 0x5595acc1b120 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.545+0000 7f483fbdd740 1 --2- 172.21.15.64:0/861597438 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595acc08ec0 0x5595acc06860 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.545+0000 7f483fbdd740 1 --2- 172.21.15.64:0/861597438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595acbfc250 0x5595acc06320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.545+0000 7f483fbdd740 1 --2- 172.21.15.64:0/861597438 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595acbfb930 0x5595acc05de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.545+0000 7f481dffb640 0 client.16008 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:37:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.545+0000 7f483fbdd740 1 -- 172.21.15.64:0/861597438 >> 172.21.15.64:0/861597438 conn(0x5595acbf8060 msgr2=0x5595acbf71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.545+0000 7f483fbdd740 1 -- 172.21.15.64:0/861597438 shutdown_connections 2024-09-12T03:37:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.545+0000 7f483fbdd740 1 -- 172.21.15.64:0/861597438 wait complete. 2024-09-12T03:37:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.545+0000 7f483fbdd740 20 client.16008 trim_cache size 0 max 16384 2024-09-12T03:37:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.560+0000 7f483fbdd740 1 Processor -- start 2024-09-12T03:37:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.560+0000 7f483fbdd740 1 -- start start 2024-09-12T03:37:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.560+0000 7f483fbdd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595acbfb930 0x5595acc06310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.560+0000 7f483fbdd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595acbfc250 0x5595acc06850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.560+0000 7f483fbdd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595acc08ec0 0x5595acc20860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.560+0000 7f483fbdd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f48381af730 con 0x5595acbfb930 2024-09-12T03:37:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.560+0000 7f483fbdd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f48381ad000 con 0x5595acc08ec0 2024-09-12T03:37:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.560+0000 7f483fbdd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f48381aae70 con 0x5595acbfc250 2024-09-12T03:37:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.560+0000 7f483659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595acbfb930 0x5595acc06310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.560+0000 7f483659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595acbfb930 0x5595acc06310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55234/0 (socket says 172.21.15.64:55234) 2024-09-12T03:37:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.560+0000 7f483659c640 1 -- 172.21.15.64:0/2499522052 learned_addr learned my addr 172.21.15.64:0/2499522052 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:37:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.560+0000 7f4835d9b640 1 --2- 172.21.15.64:0/2499522052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595acbfc250 0x5595acc06850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.560+0000 7f4836d9d640 1 --2- 172.21.15.64:0/2499522052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595acc08ec0 0x5595acc20860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.561+0000 7f483659c640 1 -- 172.21.15.64:0/2499522052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595acbfc250 msgr2=0x5595acc06850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:37:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.561+0000 7f483659c640 1 --2- 172.21.15.64:0/2499522052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595acbfc250 0x5595acc06850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.561+0000 7f483659c640 1 -- 172.21.15.64:0/2499522052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595acc08ec0 msgr2=0x5595acc20860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:37:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.561+0000 7f483659c640 1 --2- 172.21.15.64:0/2499522052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595acc08ec0 0x5595acc20860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.561+0000 7f483659c640 1 -- 172.21.15.64:0/2499522052 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595acbfc080 con 0x5595acbfb930 2024-09-12T03:37:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.561+0000 7f483659c640 1 --2- 172.21.15.64:0/2499522052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595acbfb930 0x5595acc06310 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4828031a70 tx=0x7f482803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff40f9b30892e05a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.561+0000 7f481ffff640 1 -- 172.21.15.64:0/2499522052 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4828043070 con 0x5595acbfb930 2024-09-12T03:37:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.561+0000 7f481ffff640 1 -- 172.21.15.64:0/2499522052 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4828058070 con 0x5595acbfb930 2024-09-12T03:37:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.561+0000 7f481ffff640 1 -- 172.21.15.64:0/2499522052 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4828035040 con 0x5595acbfb930 2024-09-12T03:37:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.562+0000 7f483fbdd740 1 -- 172.21.15.64:0/2499522052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595acbfb930 msgr2=0x5595acc06310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.562+0000 7f483fbdd740 1 --2- 172.21.15.64:0/2499522052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595acbfb930 0x5595acc06310 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4828031a70 tx=0x7f482803f000 comp rx=0 tx=0).stop 2024-09-12T03:37:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.562+0000 7f483fbdd740 1 -- 172.21.15.64:0/2499522052 shutdown_connections 2024-09-12T03:37:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.562+0000 7f483fbdd740 1 --2- 172.21.15.64:0/2499522052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595acc08ec0 0x5595acc20860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.562+0000 7f483fbdd740 1 --2- 172.21.15.64:0/2499522052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595acbfc250 0x5595acc06850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.562+0000 7f483fbdd740 1 --2- 172.21.15.64:0/2499522052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595acbfb930 0x5595acc06310 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.562+0000 7f483fbdd740 1 -- 172.21.15.64:0/2499522052 >> 172.21.15.64:0/2499522052 conn(0x5595acbf8060 msgr2=0x5595acc07e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.562+0000 7f483fbdd740 1 -- 172.21.15.64:0/2499522052 shutdown_connections 2024-09-12T03:37:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.562+0000 7f483fbdd740 1 -- 172.21.15.64:0/2499522052 wait complete. 2024-09-12T03:37:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.562+0000 7f483fbdd740 1 -- start start 2024-09-12T03:37:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.563+0000 7f483fbdd740 1 Processor -- start 2024-09-12T03:37:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.563+0000 7f483fbdd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595acbfb930 0x5595acc111b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.563+0000 7f483fbdd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595acbfc250 0x5595acc116f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.563+0000 7f483fbdd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595acc08ec0 0x5595acc0c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.563+0000 7f483fbdd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f48381ab4c0 con 0x5595acbfc250 2024-09-12T03:37:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.563+0000 7f483fbdd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f48381ac930 con 0x5595acbfb930 2024-09-12T03:37:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.563+0000 7f483fbdd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f48381abb90 con 0x5595acc08ec0 2024-09-12T03:37:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.563+0000 7f483fbdd740 10 client.0 _ll_register_callbacks cb 0x7f4830011720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:37:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.563+0000 7f4835d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595acbfc250 0x5595acc116f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.563+0000 7f483659c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595acbfb930 0x5595acc111b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.563+0000 7f4835d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595acbfc250 0x5595acc116f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55246/0 (socket says 172.21.15.64:55246) 2024-09-12T03:37:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.563+0000 7f4835d9b640 1 -- 172.21.15.64:0/4083265501 learned_addr learned my addr 172.21.15.64:0/4083265501 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:37:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.563+0000 7f4836d9d640 1 --2- 172.21.15.64:0/4083265501 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595acc08ec0 0x5595acc0c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.563+0000 7f4835d9b640 1 -- 172.21.15.64:0/4083265501 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595acc08ec0 msgr2=0x5595acc0c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:37:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.563+0000 7f4835d9b640 1 --2- 172.21.15.64:0/4083265501 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595acc08ec0 0x5595acc0c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.563+0000 7f4835d9b640 1 -- 172.21.15.64:0/4083265501 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595acbfb930 msgr2=0x5595acc111b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:37:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.563+0000 7f4835d9b640 1 --2- 172.21.15.64:0/4083265501 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595acbfb930 0x5595acc111b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.563+0000 7f4835d9b640 1 -- 172.21.15.64:0/4083265501 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595acc0cde0 con 0x5595acbfc250 2024-09-12T03:37:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.563+0000 7f483659c640 1 --2- 172.21.15.64:0/4083265501 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595acbfb930 0x5595acc111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:37:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.564+0000 7f4835d9b640 1 --2- 172.21.15.64:0/4083265501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595acbfc250 0x5595acc116f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f482c0981d0 tx=0x7f482c09bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b1503c8b52ef93 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.564+0000 7f481dffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:37:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.564+0000 7f481dffb640 1 -- 172.21.15.64:0/4083265501 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c09b6d0 con 0x5595acbfc250 2024-09-12T03:37:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.564+0000 7f481dffb640 1 -- 172.21.15.64:0/4083265501 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9583+0+0 (secure 0 0 0) 0x7f482c010430 con 0x5595acbfc250 2024-09-12T03:37:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.564+0000 7f483fbdd740 1 -- 172.21.15.64:0/4083265501 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4828058070 con 0x5595acbfc250 2024-09-12T03:37:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.564+0000 7f481dffb640 1 -- 172.21.15.64:0/4083265501 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f482c010e00 con 0x5595acbfc250 2024-09-12T03:37:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.564+0000 7f483fbdd740 10 client.16014 fetch_fsmap learned FSMap version 25 2024-09-12T03:37:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.564+0000 7f483fbdd740 1 -- 172.21.15.64:0/4083265501 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5595acc83460 con 0x5595acbfc250 2024-09-12T03:37:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.564+0000 7f481dffb640 1 -- 172.21.15.64:0/4083265501 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4828058070 con 0x5595acbfc250 2024-09-12T03:37:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.564+0000 7f481dffb640 1 -- 172.21.15.64:0/4083265501 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f482c07acb0 con 0x5595acbfc250 2024-09-12T03:37:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.564+0000 7f483fbdd740 10 client.16014 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:37:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.564+0000 7f483fbdd740 10 client.16014 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:37:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.564+0000 7f483fbdd740 20 client.16014 populate_metadata read hostname 'smithi064' 2024-09-12T03:37:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.564+0000 7f483fbdd740 1 --2- 172.21.15.64:0/4083265501 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5595acc83be0 0x5595acc86000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.564+0000 7f483fbdd740 4 client.16014 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:37:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.564+0000 7f483fbdd740 1 -- 172.21.15.64:0/4083265501 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5595acb67d70 con 0x5595acc83be0 2024-09-12T03:37:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.565+0000 7f483659c640 1 --2- 172.21.15.64:0/4083265501 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5595acc83be0 0x5595acc86000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.565+0000 7f481dffb640 10 client.16014 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:37:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.565+0000 7f483659c640 1 --2- 172.21.15.64:0/4083265501 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5595acc83be0 0x5595acc86000 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c104ac7d2fa1299c server_cookie=cb9f6c9d70b0319 in_seq=0 out_seq=0 2024-09-12T03:37:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.580+0000 7f481dffb640 1 -- 172.21.15.64:0/4083265501 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5595acb67d70 con 0x5595acc83be0 2024-09-12T03:37:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.580+0000 7f481dffb640 10 client.16014 handle_command_reply: tid=0 2024-09-12T03:37:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.580+0000 7f483fbdd740 1 client.16014 shutdown 2024-09-12T03:37:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.580+0000 7f483fbdd740 1 -- 172.21.15.64:0/4083265501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595acbfc250 msgr2=0x5595acc116f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.580+0000 7f483fbdd740 1 --2- 172.21.15.64:0/4083265501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595acbfc250 0x5595acc116f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f482c0981d0 tx=0x7f482c09bb40 comp rx=0 tx=0).stop 2024-09-12T03:37:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.580+0000 7f483fbdd740 1 -- 172.21.15.64:0/4083265501 shutdown_connections 2024-09-12T03:37:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.580+0000 7f483fbdd740 1 --2- 172.21.15.64:0/4083265501 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5595acc83be0 0x5595acc86000 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.580+0000 7f483fbdd740 1 --2- 172.21.15.64:0/4083265501 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5595acc08ec0 0x5595acc0c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.580+0000 7f483fbdd740 1 --2- 172.21.15.64:0/4083265501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595acbfc250 0x5595acc116f0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.580+0000 7f483fbdd740 1 --2- 172.21.15.64:0/4083265501 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5595acbfb930 0x5595acc111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.580+0000 7f483fbdd740 1 -- 172.21.15.64:0/4083265501 >> 172.21.15.64:0/4083265501 conn(0x5595acbf8060 msgr2=0x5595acbf69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.580+0000 7f481dffb640 0 client.16014 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:37:24.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.580+0000 7f483fbdd740 1 -- 172.21.15.64:0/4083265501 shutdown_connections 2024-09-12T03:37:24.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.580+0000 7f483fbdd740 1 -- 172.21.15.64:0/4083265501 wait complete. 2024-09-12T03:37:24.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.580+0000 7f483fbdd740 20 client.16014 trim_cache size 0 max 16384 2024-09-12T03:37:24.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:37:24.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:37:24.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:37:24.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:37:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.581+0000 7f483cd66640 1 -- 172.21.15.64:0/3402263877 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f48200a2da0 msgr2=0x7f48200a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.581+0000 7f483cd66640 1 --2- 172.21.15.64:0/3402263877 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f48200a2da0 0x7f48200a5260 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4828009900 tx=0x7f4828005e50 comp rx=0 tx=0).stop 2024-09-12T03:37:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.581+0000 7f483cd66640 1 -- 172.21.15.64:0/3402263877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4838101350 msgr2=0x7f48381976b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.581+0000 7f483cd66640 1 --2- 172.21.15.64:0/3402263877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4838101350 0x7f48381976b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f482c00e9f0 tx=0x7f482c00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:37:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.581+0000 7f483cd66640 1 -- 172.21.15.64:0/3402263877 shutdown_connections 2024-09-12T03:37:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.581+0000 7f483cd66640 1 --2- 172.21.15.64:0/3402263877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48381069c0 0x7f483819ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.581+0000 7f483cd66640 1 --2- 172.21.15.64:0/3402263877 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f48200a2da0 0x7f48200a5260 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:24.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.581+0000 7f483cd66640 1 --2- 172.21.15.64:0/3402263877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4838101350 0x7f48381976b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:24.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.581+0000 7f483cd66640 1 --2- 172.21.15.64:0/3402263877 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48381009b0 0x7f4838197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:24.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.581+0000 7f483cd66640 1 -- 172.21.15.64:0/3402263877 >> 172.21.15.64:0/3402263877 conn(0x7f4838077990 msgr2=0x7f48381038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:24.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.581+0000 7f483cd66640 1 -- 172.21.15.64:0/3402263877 shutdown_connections 2024-09-12T03:37:24.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:24.581+0000 7f483cd66640 1 -- 172.21.15.64:0/3402263877 wait complete. 2024-09-12T03:37:24.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:37:24 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:37:24.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:37:24 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:37:24.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:37:24 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:37:25.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:37:25 smithi064 ceph-mon[30952]: pgmap v251: 129 pgs: 129 active+clean; 2.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-12T03:37:25.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:37:25 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:37:25.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:37:25 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:37:25.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:37:25 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:37:25.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:37:25 smithi115 ceph-mon[34801]: pgmap v251: 129 pgs: 129 active+clean; 2.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-12T03:37:25.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:37:25 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:37:25.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:37:25 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:37:25.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:37:25 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:37:25.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:37:25 smithi195 ceph-mon[34786]: pgmap v251: 129 pgs: 129 active+clean; 2.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-12T03:37:25.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:37:25 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:37:25.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:37:25 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:37:25.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:37:25 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:37:27.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:37:27 smithi115 ceph-mon[34801]: pgmap v252: 129 pgs: 129 active+clean; 2.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-12T03:37:27.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:37:27 smithi195 ceph-mon[34786]: pgmap v252: 129 pgs: 129 active+clean; 2.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-12T03:37:27.521 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:37:27 smithi064 ceph-mon[30952]: pgmap v252: 129 pgs: 129 active+clean; 2.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-12T03:37:29.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:37:29 smithi115 ceph-mon[34801]: pgmap v253: 129 pgs: 129 active+clean; 2.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-09-12T03:37:29.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:37:29 smithi195 ceph-mon[34786]: pgmap v253: 129 pgs: 129 active+clean; 2.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-09-12T03:37:29.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:37:29 smithi064 ceph-mon[30952]: pgmap v253: 129 pgs: 129 active+clean; 2.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-09-12T03:37:31.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:37:31 smithi115 ceph-mon[34801]: pgmap v254: 129 pgs: 129 active+clean; 2.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-12T03:37:31.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:37:31 smithi195 ceph-mon[34786]: pgmap v254: 129 pgs: 129 active+clean; 2.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-12T03:37:31.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:37:31 smithi064 ceph-mon[30952]: pgmap v254: 129 pgs: 129 active+clean; 2.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-12T03:37:33.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:37:33 smithi115 ceph-mon[34801]: pgmap v255: 129 pgs: 129 active+clean; 2.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 57 KiB/s wr, 17 op/s 2024-09-12T03:37:33.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:37:33 smithi195 ceph-mon[34786]: pgmap v255: 129 pgs: 129 active+clean; 2.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 57 KiB/s wr, 17 op/s 2024-09-12T03:37:33.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:37:33 smithi064 ceph-mon[30952]: pgmap v255: 129 pgs: 129 active+clean; 2.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 57 KiB/s wr, 17 op/s 2024-09-12T03:37:35.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:37:35 smithi115 ceph-mon[34801]: pgmap v256: 129 pgs: 129 active+clean; 2.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-09-12T03:37:35.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:37:35 smithi195 ceph-mon[34786]: pgmap v256: 129 pgs: 129 active+clean; 2.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-09-12T03:37:35.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:37:35 smithi064 ceph-mon[30952]: pgmap v256: 129 pgs: 129 active+clean; 2.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-09-12T03:37:37.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:37:37 smithi115 ceph-mon[34801]: pgmap v257: 129 pgs: 129 active+clean; 2.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-09-12T03:37:37.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:37:37 smithi195 ceph-mon[34786]: pgmap v257: 129 pgs: 129 active+clean; 2.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-09-12T03:37:37.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:37:37 smithi064 ceph-mon[30952]: pgmap v257: 129 pgs: 129 active+clean; 2.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-09-12T03:37:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.681+0000 7fab392dd640 1 -- 172.21.15.64:0/3369946159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab34101ae0 msgr2=0x7fab34101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.681+0000 7fab392dd640 1 --2- 172.21.15.64:0/3369946159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab34101ae0 0x7fab34101f40 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fab28009930 tx=0x7fab2802a420 comp rx=0 tx=0).stop 2024-09-12T03:37:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.681+0000 7fab392dd640 1 -- 172.21.15.64:0/3369946159 shutdown_connections 2024-09-12T03:37:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.681+0000 7fab392dd640 1 --2- 172.21.15.64:0/3369946159 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab34102480 0x7fab3410e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.681+0000 7fab392dd640 1 --2- 172.21.15.64:0/3369946159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab34101ae0 0x7fab34101f40 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.681+0000 7fab392dd640 1 --2- 172.21.15.64:0/3369946159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab34107b80 0x7fab34107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.681+0000 7fab392dd640 1 -- 172.21.15.64:0/3369946159 >> 172.21.15.64:0/3369946159 conn(0x7fab340fd7e0 msgr2=0x7fab340ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.681+0000 7fab392dd640 1 -- 172.21.15.64:0/3369946159 shutdown_connections 2024-09-12T03:37:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.681+0000 7fab392dd640 1 -- 172.21.15.64:0/3369946159 wait complete. 2024-09-12T03:37:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.681+0000 7fab392dd640 1 Processor -- start 2024-09-12T03:37:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.682+0000 7fab392dd640 1 -- start start 2024-09-12T03:37:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.682+0000 7fab392dd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab34101ae0 0x7fab340744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.682+0000 7fab392dd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab34102480 0x7fab34074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.682+0000 7fab392dd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab34107b80 0x7fab34074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.682+0000 7fab392dd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fab34111da0 con 0x7fab34107b80 2024-09-12T03:37:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.682+0000 7fab392dd640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fab34111c20 con 0x7fab34101ae0 2024-09-12T03:37:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.682+0000 7fab392dd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fab34111f20 con 0x7fab34102480 2024-09-12T03:37:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.682+0000 7fab337fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab34107b80 0x7fab34074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.682+0000 7fab337fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab34107b80 0x7fab34074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55674/0 (socket says 172.21.15.64:55674) 2024-09-12T03:37:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.682+0000 7fab337fe640 1 -- 172.21.15.64:0/1588921079 learned_addr learned my addr 172.21.15.64:0/1588921079 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:37:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.682+0000 7fab327fc640 1 --2- 172.21.15.64:0/1588921079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab34102480 0x7fab34074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.682+0000 7fab32ffd640 1 --2- 172.21.15.64:0/1588921079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab34101ae0 0x7fab340744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.682+0000 7fab337fe640 1 -- 172.21.15.64:0/1588921079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab34102480 msgr2=0x7fab34074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.682+0000 7fab337fe640 1 --2- 172.21.15.64:0/1588921079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab34102480 0x7fab34074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.682+0000 7fab337fe640 1 -- 172.21.15.64:0/1588921079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab34101ae0 msgr2=0x7fab340744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.682+0000 7fab337fe640 1 --2- 172.21.15.64:0/1588921079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab34101ae0 0x7fab340744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.682+0000 7fab337fe640 1 -- 172.21.15.64:0/1588921079 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab341a6730 con 0x7fab34107b80 2024-09-12T03:37:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.682+0000 7fab32ffd640 1 --2- 172.21.15.64:0/1588921079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab34101ae0 0x7fab340744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:37:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.683+0000 7fab337fe640 1 --2- 172.21.15.64:0/1588921079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab34107b80 0x7fab34074f70 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fab2c00eb10 tx=0x7fab2c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.683+0000 7fab1bfff640 1 -- 172.21.15.64:0/1588921079 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c004b20 con 0x7fab34107b80 2024-09-12T03:37:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.683+0000 7fab1bfff640 1 -- 172.21.15.64:0/1588921079 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab2c004cc0 con 0x7fab34107b80 2024-09-12T03:37:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.683+0000 7fab392dd640 1 -- 172.21.15.64:0/1588921079 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab341a6a20 con 0x7fab34107b80 2024-09-12T03:37:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.683+0000 7fab1bfff640 1 -- 172.21.15.64:0/1588921079 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c010690 con 0x7fab34107b80 2024-09-12T03:37:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.683+0000 7fab392dd640 1 -- 172.21.15.64:0/1588921079 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab341a6ef0 con 0x7fab34107b80 2024-09-12T03:37:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.685+0000 7fab1bfff640 1 -- 172.21.15.64:0/1588921079 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fab2c005020 con 0x7fab34107b80 2024-09-12T03:37:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.686+0000 7fab3c075740 1 Processor -- start 2024-09-12T03:37:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.686+0000 7fab3c075740 1 -- start start 2024-09-12T03:37:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.686+0000 7fab3c075740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631afe5baf0 0x5631afe5bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.686+0000 7fab3c075740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631afe5c410 0x5631afe66770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.686+0000 7fab3c075740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5631afe66cb0 0x5631afe690a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.686+0000 7fab3c075740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5631afdc02f0 con 0x5631afe5c410 2024-09-12T03:37:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.686+0000 7fab3c075740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5631afd68270 con 0x5631afe5baf0 2024-09-12T03:37:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.686+0000 7fab3c075740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5631afdbc2b0 con 0x5631afe66cb0 2024-09-12T03:37:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.686+0000 7fab1bfff640 1 --2- 172.21.15.64:0/1588921079 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fab1c0a2da0 0x7fab1c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.686+0000 7fab32ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631afe5c410 0x5631afe66770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.686+0000 7fab32ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631afe5c410 0x5631afe66770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55676/0 (socket says 172.21.15.64:55676) 2024-09-12T03:37:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.686+0000 7fab32ffd640 1 -- 172.21.15.64:0/103774124 learned_addr learned my addr 172.21.15.64:0/103774124 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:37:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.686+0000 7fab337fe640 1 --2- 172.21.15.64:0/1588921079 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fab1c0a2da0 0x7fab1c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.686+0000 7fab327fc640 1 --2- 172.21.15.64:0/103774124 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5631afe66cb0 0x5631afe690a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.687+0000 7fab327fc640 1 --2- 172.21.15.64:0/103774124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631afe5baf0 0x5631afe5bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.687+0000 7fab1bfff640 1 -- 172.21.15.64:0/1588921079 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9583+0+0 (secure 0 0 0) 0x7fab2c0cfeb0 con 0x7fab34107b80 2024-09-12T03:37:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.687+0000 7fab32ffd640 1 -- 172.21.15.64:0/103774124 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5631afe66cb0 msgr2=0x5631afe690a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:37:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.687+0000 7fab32ffd640 1 --2- 172.21.15.64:0/103774124 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5631afe66cb0 0x5631afe690a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.687+0000 7fab32ffd640 1 -- 172.21.15.64:0/103774124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631afe5baf0 msgr2=0x5631afe5bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:37:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.687+0000 7fab32ffd640 1 --2- 172.21.15.64:0/103774124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631afe5baf0 0x5631afe5bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.687+0000 7fab32ffd640 1 -- 172.21.15.64:0/103774124 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631afd67340 con 0x5631afe5c410 2024-09-12T03:37:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.687+0000 7fab327fc640 1 --2- 172.21.15.64:0/103774124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631afe5baf0 0x5631afe5bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:37:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.687+0000 7fab337fe640 1 --2- 172.21.15.64:0/1588921079 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fab1c0a2da0 0x7fab1c0a5260 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fab2c09bf40 tx=0x7fab2c091440 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.687+0000 7fab32ffd640 1 --2- 172.21.15.64:0/103774124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631afe5c410 0x5631afe66770 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fab24008be0 tx=0x7fab2400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e86727a1fa6369 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.687+0000 7fab197fa640 1 -- 172.21.15.64:0/103774124 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab24014070 con 0x5631afe5c410 2024-09-12T03:37:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.687+0000 7fab197fa640 1 -- 172.21.15.64:0/103774124 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab240047a0 con 0x5631afe5c410 2024-09-12T03:37:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.687+0000 7fab197fa640 1 -- 172.21.15.64:0/103774124 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab24010630 con 0x5631afe5c410 2024-09-12T03:37:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.688+0000 7fab3c075740 1 -- 172.21.15.64:0/103774124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631afe5c410 msgr2=0x5631afe66770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:37.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.688+0000 7fab3c075740 1 --2- 172.21.15.64:0/103774124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631afe5c410 0x5631afe66770 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fab24008be0 tx=0x7fab2400e720 comp rx=0 tx=0).stop 2024-09-12T03:37:37.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.688+0000 7fab3c075740 1 -- 172.21.15.64:0/103774124 shutdown_connections 2024-09-12T03:37:37.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.688+0000 7fab3c075740 1 --2- 172.21.15.64:0/103774124 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5631afe66cb0 0x5631afe690a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:37.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.688+0000 7fab3c075740 1 --2- 172.21.15.64:0/103774124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631afe5c410 0x5631afe66770 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:37.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.688+0000 7fab3c075740 1 --2- 172.21.15.64:0/103774124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631afe5baf0 0x5631afe5bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:37.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.688+0000 7fab3c075740 1 -- 172.21.15.64:0/103774124 >> 172.21.15.64:0/103774124 conn(0x5631afe58060 msgr2=0x5631afe58440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:37.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.688+0000 7fab3c075740 1 -- 172.21.15.64:0/103774124 shutdown_connections 2024-09-12T03:37:37.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.688+0000 7fab3c075740 1 -- 172.21.15.64:0/103774124 wait complete. 2024-09-12T03:37:37.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.688+0000 7fab3c075740 1 -- start start 2024-09-12T03:37:37.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.689+0000 7fab3c075740 1 Processor -- start 2024-09-12T03:37:37.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.689+0000 7fab3c075740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631afe5baf0 0x5631afe63b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:37.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.689+0000 7fab3c075740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631afe5c410 0x5631afe640d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:37.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.689+0000 7fab3c075740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5631afe66cb0 0x5631afe64610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.689+0000 7fab3c075740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5631afe69880 con 0x5631afe5c410 2024-09-12T03:37:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.689+0000 7fab3c075740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5631afd49290 con 0x5631afe5baf0 2024-09-12T03:37:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.689+0000 7fab3c075740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5631afe69a00 con 0x5631afe66cb0 2024-09-12T03:37:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.689+0000 7fab3c075740 10 client.0 _ll_register_callbacks cb 0x5631afd7a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:37:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.689+0000 7fab327fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631afe5baf0 0x5631afe63b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.689+0000 7fab32ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631afe5c410 0x5631afe640d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.689+0000 7fab327fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631afe5baf0 0x5631afe63b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.64:42972/0 (socket says 172.21.15.64:42972) 2024-09-12T03:37:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.689+0000 7fab327fc640 1 -- 172.21.15.64:0/1384959472 learned_addr learned my addr 172.21.15.64:0/1384959472 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:37:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.689+0000 7fab327fc640 1 --2- 172.21.15.64:0/1384959472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5631afe66cb0 0x5631afe64610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.690+0000 7fab32ffd640 1 -- 172.21.15.64:0/1384959472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5631afe66cb0 msgr2=0x5631afe64610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:37:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.690+0000 7fab32ffd640 1 --2- 172.21.15.64:0/1384959472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5631afe66cb0 0x5631afe64610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.690+0000 7fab32ffd640 1 -- 172.21.15.64:0/1384959472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631afe5baf0 msgr2=0x5631afe63b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:37:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.690+0000 7fab32ffd640 1 --2- 172.21.15.64:0/1384959472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631afe5baf0 0x5631afe63b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.690+0000 7fab32ffd640 1 -- 172.21.15.64:0/1384959472 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631afad3210 con 0x5631afe5c410 2024-09-12T03:37:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.690+0000 7fab327fc640 1 --2- 172.21.15.64:0/1384959472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631afe5baf0 0x5631afe63b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:37:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.690+0000 7fab32ffd640 1 --2- 172.21.15.64:0/1384959472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631afe5c410 0x5631afe640d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fab2400ec20 tx=0x7fab24005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9ccef7a93cbb4af server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.690+0000 7fab16ff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:37:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.690+0000 7fab16ff5640 1 -- 172.21.15.64:0/1384959472 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab24014030 con 0x5631afe5c410 2024-09-12T03:37:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.690+0000 7fab16ff5640 1 -- 172.21.15.64:0/1384959472 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9583+0+0 (secure 0 0 0) 0x7fab240055b0 con 0x5631afe5c410 2024-09-12T03:37:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.690+0000 7fab3c075740 1 -- 172.21.15.64:0/1384959472 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab240047a0 con 0x5631afe5c410 2024-09-12T03:37:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.690+0000 7fab16ff5640 1 -- 172.21.15.64:0/1384959472 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fab24004050 con 0x5631afe5c410 2024-09-12T03:37:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.690+0000 7fab3c075740 10 client.16026 fetch_fsmap learned FSMap version 25 2024-09-12T03:37:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.690+0000 7fab3c075740 1 -- 172.21.15.64:0/1384959472 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5631afb9f620 con 0x5631afe5c410 2024-09-12T03:37:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.690+0000 7fab16ff5640 1 -- 172.21.15.64:0/1384959472 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab240047a0 con 0x5631afe5c410 2024-09-12T03:37:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.691+0000 7fab16ff5640 1 -- 172.21.15.64:0/1384959472 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fab24003b30 con 0x5631afe5c410 2024-09-12T03:37:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.691+0000 7fab3c075740 10 client.16026 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:37:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.691+0000 7fab3c075740 10 client.16026 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:37:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.691+0000 7fab3c075740 20 client.16026 populate_metadata read hostname 'smithi064' 2024-09-12T03:37:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.691+0000 7fab3c075740 1 --2- 172.21.15.64:0/1384959472 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5631afe76bf0 0x5631afe78fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.691+0000 7fab3c075740 4 client.16026 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:37:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.691+0000 7fab3c075740 1 -- 172.21.15.64:0/1384959472 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631afdcda20 con 0x5631afe76bf0 2024-09-12T03:37:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.691+0000 7fab327fc640 1 --2- 172.21.15.64:0/1384959472 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5631afe76bf0 0x5631afe78fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.691+0000 7fab16ff5640 10 client.16026 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:37:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.691+0000 7fab327fc640 1 --2- 172.21.15.64:0/1384959472 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5631afe76bf0 0x5631afe78fe0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4179e4d54d566e54 server_cookie=5fd17bc9e4e78f46 in_seq=0 out_seq=0 2024-09-12T03:37:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.692+0000 7fab16ff5640 1 -- 172.21.15.64:0/1384959472 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5631afdcda20 con 0x5631afe76bf0 2024-09-12T03:37:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.692+0000 7fab16ff5640 10 client.16026 handle_command_reply: tid=0 2024-09-12T03:37:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.693+0000 7fab3c075740 1 client.16026 shutdown 2024-09-12T03:37:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.693+0000 7fab3c075740 1 -- 172.21.15.64:0/1384959472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631afe5c410 msgr2=0x5631afe640d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.693+0000 7fab3c075740 1 --2- 172.21.15.64:0/1384959472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631afe5c410 0x5631afe640d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fab2400ec20 tx=0x7fab24005230 comp rx=0 tx=0).stop 2024-09-12T03:37:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.693+0000 7fab3c075740 1 -- 172.21.15.64:0/1384959472 shutdown_connections 2024-09-12T03:37:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.693+0000 7fab3c075740 1 --2- 172.21.15.64:0/1384959472 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5631afe76bf0 0x5631afe78fe0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.693+0000 7fab3c075740 1 --2- 172.21.15.64:0/1384959472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5631afe66cb0 0x5631afe64610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.693+0000 7fab3c075740 1 --2- 172.21.15.64:0/1384959472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631afe5c410 0x5631afe640d0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.693+0000 7fab3c075740 1 --2- 172.21.15.64:0/1384959472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631afe5baf0 0x5631afe63b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.693+0000 7fab3c075740 1 -- 172.21.15.64:0/1384959472 >> 172.21.15.64:0/1384959472 conn(0x5631afe58060 msgr2=0x5631afe5ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.693+0000 7fab16ff5640 0 client.16026 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:37:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.693+0000 7fab3c075740 1 -- 172.21.15.64:0/1384959472 shutdown_connections 2024-09-12T03:37:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.693+0000 7fab3c075740 1 -- 172.21.15.64:0/1384959472 wait complete. 2024-09-12T03:37:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.693+0000 7fab3c075740 20 client.16026 trim_cache size 0 max 16384 2024-09-12T03:37:37.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.708+0000 7fab3c075740 1 Processor -- start 2024-09-12T03:37:37.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.709+0000 7fab3c075740 1 -- start start 2024-09-12T03:37:37.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.709+0000 7fab3c075740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5631afe5baf0 0x5631afe5e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:37.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.709+0000 7fab3c075740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631afe5c410 0x5631afe64440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:37.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.709+0000 7fab3c075740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631afe66cb0 0x5631afe570b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:37.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.709+0000 7fab3c075740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fab341b3b10 con 0x5631afe5c410 2024-09-12T03:37:37.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.709+0000 7fab3c075740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fab341b13e0 con 0x5631afe66cb0 2024-09-12T03:37:37.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.709+0000 7fab3c075740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fab341af250 con 0x5631afe5baf0 2024-09-12T03:37:37.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.709+0000 7fab32ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631afe5c410 0x5631afe64440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:37.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.709+0000 7fab32ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631afe5c410 0x5631afe64440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55694/0 (socket says 172.21.15.64:55694) 2024-09-12T03:37:37.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.709+0000 7fab32ffd640 1 -- 172.21.15.64:0/3294220955 learned_addr learned my addr 172.21.15.64:0/3294220955 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:37:37.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.709+0000 7fab327fc640 1 --2- 172.21.15.64:0/3294220955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5631afe5baf0 0x5631afe5e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:37.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.709+0000 7fab32ffd640 1 -- 172.21.15.64:0/3294220955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5631afe5baf0 msgr2=0x5631afe5e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:37:37.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.709+0000 7fab32ffd640 1 --2- 172.21.15.64:0/3294220955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5631afe5baf0 0x5631afe5e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:37.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.709+0000 7fab32ffd640 1 -- 172.21.15.64:0/3294220955 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631afe66cb0 msgr2=0x5631afe570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:37:37.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.709+0000 7fab32ffd640 1 --2- 172.21.15.64:0/3294220955 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631afe66cb0 0x5631afe570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:37.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.709+0000 7fab32ffd640 1 -- 172.21.15.64:0/3294220955 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631afe5b870 con 0x5631afe5c410 2024-09-12T03:37:37.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.709+0000 7fab32ffd640 1 --2- 172.21.15.64:0/3294220955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631afe5c410 0x5631afe64440 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fab24000c00 tx=0x7fab24004f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7b29b601f266714 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:37.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.709+0000 7fab18ff9640 1 -- 172.21.15.64:0/3294220955 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab24014070 con 0x5631afe5c410 2024-09-12T03:37:37.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.709+0000 7fab18ff9640 1 -- 172.21.15.64:0/3294220955 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab24004d40 con 0x5631afe5c410 2024-09-12T03:37:37.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.709+0000 7fab18ff9640 1 -- 172.21.15.64:0/3294220955 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab240104d0 con 0x5631afe5c410 2024-09-12T03:37:37.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.710+0000 7fab3c075740 1 -- 172.21.15.64:0/3294220955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631afe5c410 msgr2=0x5631afe64440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.710+0000 7fab3c075740 1 --2- 172.21.15.64:0/3294220955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631afe5c410 0x5631afe64440 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fab24000c00 tx=0x7fab24004f70 comp rx=0 tx=0).stop 2024-09-12T03:37:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.710+0000 7fab3c075740 1 -- 172.21.15.64:0/3294220955 shutdown_connections 2024-09-12T03:37:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.710+0000 7fab3c075740 1 --2- 172.21.15.64:0/3294220955 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631afe66cb0 0x5631afe570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.710+0000 7fab3c075740 1 --2- 172.21.15.64:0/3294220955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631afe5c410 0x5631afe64440 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.710+0000 7fab3c075740 1 --2- 172.21.15.64:0/3294220955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5631afe5baf0 0x5631afe5e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.710+0000 7fab3c075740 1 -- 172.21.15.64:0/3294220955 >> 172.21.15.64:0/3294220955 conn(0x5631afe58060 msgr2=0x5631afe569a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.710+0000 7fab3c075740 1 -- 172.21.15.64:0/3294220955 shutdown_connections 2024-09-12T03:37:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.710+0000 7fab3c075740 1 -- 172.21.15.64:0/3294220955 wait complete. 2024-09-12T03:37:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.711+0000 7fab3c075740 1 -- start start 2024-09-12T03:37:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.711+0000 7fab3c075740 1 Processor -- start 2024-09-12T03:37:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.711+0000 7fab3c075740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631afe5baf0 0x5631afe69c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.711+0000 7fab3c075740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5631afe5c410 0x5631afe6a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.711+0000 7fab3c075740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631afe66cb0 0x5631afe6a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.711+0000 7fab3c075740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fab341af8a0 con 0x5631afe66cb0 2024-09-12T03:37:37.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.711+0000 7fab3c075740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fab341b0d10 con 0x5631afe5baf0 2024-09-12T03:37:37.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.711+0000 7fab3c075740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fab341aff70 con 0x5631afe5c410 2024-09-12T03:37:37.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.711+0000 7fab3c075740 10 client.0 _ll_register_callbacks cb 0x7fab28009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:37:37.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.711+0000 7fab327fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631afe66cb0 0x5631afe6a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:37.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.711+0000 7fab327fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631afe66cb0 0x5631afe6a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55698/0 (socket says 172.21.15.64:55698) 2024-09-12T03:37:37.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.711+0000 7fab327fc640 1 -- 172.21.15.64:0/3806944154 learned_addr learned my addr 172.21.15.64:0/3806944154 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:37:37.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.711+0000 7fab327fc640 1 --2- 172.21.15.64:0/3806944154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631afe5baf0 0x5631afe69c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:37.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.711+0000 7fab32ffd640 1 --2- 172.21.15.64:0/3806944154 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5631afe5c410 0x5631afe6a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:37.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.711+0000 7fab327fc640 1 -- 172.21.15.64:0/3806944154 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5631afe5c410 msgr2=0x5631afe6a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:37:37.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.711+0000 7fab327fc640 1 --2- 172.21.15.64:0/3806944154 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5631afe5c410 0x5631afe6a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:37.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.711+0000 7fab327fc640 1 -- 172.21.15.64:0/3806944154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631afe5baf0 msgr2=0x5631afe69c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:37:37.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.711+0000 7fab327fc640 1 --2- 172.21.15.64:0/3806944154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631afe5baf0 0x5631afe69c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:37.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.711+0000 7fab327fc640 1 -- 172.21.15.64:0/3806944154 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631afedf810 con 0x5631afe66cb0 2024-09-12T03:37:37.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.712+0000 7fab327fc640 1 --2- 172.21.15.64:0/3806944154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631afe66cb0 0x5631afe6a6d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fab280057c0 tx=0x7fab28043040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a279a89638723135 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:37.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.712+0000 7fab16ff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:37:37.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.712+0000 7fab16ff5640 1 -- 172.21.15.64:0/3806944154 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab28051020 con 0x5631afe66cb0 2024-09-12T03:37:37.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.712+0000 7fab16ff5640 1 -- 172.21.15.64:0/3806944154 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9583+0+0 (secure 0 0 0) 0x7fab28004250 con 0x5631afe66cb0 2024-09-12T03:37:37.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.712+0000 7fab3c075740 1 -- 172.21.15.64:0/3806944154 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5631afdc7d70 con 0x5631afe66cb0 2024-09-12T03:37:37.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.712+0000 7fab16ff5640 1 -- 172.21.15.64:0/3806944154 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fab2805c050 con 0x5631afe66cb0 2024-09-12T03:37:37.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.712+0000 7fab3c075740 10 client.16032 fetch_fsmap learned FSMap version 25 2024-09-12T03:37:37.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.712+0000 7fab3c075740 1 -- 172.21.15.64:0/3806944154 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5631afee0ac0 con 0x5631afe66cb0 2024-09-12T03:37:37.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.712+0000 7fab16ff5640 1 -- 172.21.15.64:0/3806944154 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5631afdc7d70 con 0x5631afe66cb0 2024-09-12T03:37:37.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.712+0000 7fab16ff5640 1 -- 172.21.15.64:0/3806944154 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fab2804d330 con 0x5631afe66cb0 2024-09-12T03:37:37.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.712+0000 7fab3c075740 10 client.16032 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:37:37.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.712+0000 7fab3c075740 10 client.16032 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:37:37.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.712+0000 7fab3c075740 20 client.16032 populate_metadata read hostname 'smithi064' 2024-09-12T03:37:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.712+0000 7fab3c075740 1 --2- 172.21.15.64:0/3806944154 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5631afee1210 0x5631afee3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.712+0000 7fab3c075740 4 client.16032 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:37:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.712+0000 7fab3c075740 1 -- 172.21.15.64:0/3806944154 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5631afd44900 con 0x5631afee1210 2024-09-12T03:37:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.713+0000 7fab32ffd640 1 --2- 172.21.15.64:0/3806944154 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5631afee1210 0x5631afee3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.713+0000 7fab16ff5640 10 client.16032 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:37:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.713+0000 7fab32ffd640 1 --2- 172.21.15.64:0/3806944154 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5631afee1210 0x5631afee3630 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77f52c9622b1aa4c server_cookie=2e9970a5fd00b87 in_seq=0 out_seq=0 2024-09-12T03:37:37.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.725+0000 7fab16ff5640 1 -- 172.21.15.64:0/3806944154 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5631afd44900 con 0x5631afee1210 2024-09-12T03:37:37.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.725+0000 7fab16ff5640 10 client.16032 handle_command_reply: tid=0 2024-09-12T03:37:37.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.725+0000 7fab3c075740 1 client.16032 shutdown 2024-09-12T03:37:37.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.725+0000 7fab3c075740 1 -- 172.21.15.64:0/3806944154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631afe66cb0 msgr2=0x5631afe6a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:37.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.725+0000 7fab3c075740 1 --2- 172.21.15.64:0/3806944154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631afe66cb0 0x5631afe6a6d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fab280057c0 tx=0x7fab28043040 comp rx=0 tx=0).stop 2024-09-12T03:37:37.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.725+0000 7fab3c075740 1 -- 172.21.15.64:0/3806944154 shutdown_connections 2024-09-12T03:37:37.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.725+0000 7fab3c075740 1 --2- 172.21.15.64:0/3806944154 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5631afee1210 0x5631afee3630 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:37.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.725+0000 7fab3c075740 1 --2- 172.21.15.64:0/3806944154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631afe66cb0 0x5631afe6a6d0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:37.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.725+0000 7fab16ff5640 0 client.16032 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:37:37.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.725+0000 7fab3c075740 1 --2- 172.21.15.64:0/3806944154 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5631afe5c410 0x5631afe6a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:37.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.725+0000 7fab3c075740 1 --2- 172.21.15.64:0/3806944154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5631afe5baf0 0x5631afe69c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:37.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.725+0000 7fab3c075740 1 -- 172.21.15.64:0/3806944154 >> 172.21.15.64:0/3806944154 conn(0x5631afe58060 msgr2=0x5631afe653f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:37.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.725+0000 7fab3c075740 1 -- 172.21.15.64:0/3806944154 shutdown_connections 2024-09-12T03:37:37.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.726+0000 7fab3c075740 1 -- 172.21.15.64:0/3806944154 wait complete. 2024-09-12T03:37:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.726+0000 7fab3c075740 20 client.16032 trim_cache size 0 max 16384 2024-09-12T03:37:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:37:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:37:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:37:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:37:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.726+0000 7fab392dd640 1 -- 172.21.15.64:0/1588921079 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fab1c0a2da0 msgr2=0x7fab1c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.726+0000 7fab392dd640 1 --2- 172.21.15.64:0/1588921079 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fab1c0a2da0 0x7fab1c0a5260 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fab2c09bf40 tx=0x7fab2c091440 comp rx=0 tx=0).stop 2024-09-12T03:37:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.726+0000 7fab392dd640 1 -- 172.21.15.64:0/1588921079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab34107b80 msgr2=0x7fab34074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.726+0000 7fab392dd640 1 --2- 172.21.15.64:0/1588921079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab34107b80 0x7fab34074f70 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fab2c00eb10 tx=0x7fab2c00c2d0 comp rx=0 tx=0).stop 2024-09-12T03:37:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.726+0000 7fab392dd640 1 -- 172.21.15.64:0/1588921079 shutdown_connections 2024-09-12T03:37:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.726+0000 7fab392dd640 1 --2- 172.21.15.64:0/1588921079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab34107b80 0x7fab34074f70 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.726+0000 7fab392dd640 1 --2- 172.21.15.64:0/1588921079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab34102480 0x7fab34074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.726+0000 7fab392dd640 1 --2- 172.21.15.64:0/1588921079 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fab1c0a2da0 0x7fab1c0a5260 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.726+0000 7fab392dd640 1 --2- 172.21.15.64:0/1588921079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab34101ae0 0x7fab340744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.726+0000 7fab392dd640 1 -- 172.21.15.64:0/1588921079 >> 172.21.15.64:0/1588921079 conn(0x7fab340fd7e0 msgr2=0x7fab340fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.726+0000 7fab392dd640 1 -- 172.21.15.64:0/1588921079 shutdown_connections 2024-09-12T03:37:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:37.726+0000 7fab392dd640 1 -- 172.21.15.64:0/1588921079 wait complete. 2024-09-12T03:37:38.701 INFO:tasks.workunit.client.0.smithi064.stdout:375/600: open fd = 3 2024-09-12T03:37:38.701 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.701 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.701 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.701 INFO:tasks.workunit.client.0.smithi064.stdout:376/600: open fd = 3 2024-09-12T03:37:38.701 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.701 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.701 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.701 INFO:tasks.workunit.client.0.smithi064.stdout:377/600: open fd = 3 2024-09-12T03:37:38.701 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.701 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.702 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.702 INFO:tasks.workunit.client.0.smithi064.stdout:378/600: open fd = 3 2024-09-12T03:37:38.702 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.702 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.702 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.702 INFO:tasks.workunit.client.0.smithi064.stdout:379/600: open fd = 3 2024-09-12T03:37:38.702 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.702 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.702 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.702 INFO:tasks.workunit.client.0.smithi064.stdout:380/600: open fd = 3 2024-09-12T03:37:38.702 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.702 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.703 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.703 INFO:tasks.workunit.client.0.smithi064.stdout:381/600: open fd = 3 2024-09-12T03:37:38.703 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.703 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.703 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.703 INFO:tasks.workunit.client.0.smithi064.stdout:382/600: open fd = 3 2024-09-12T03:37:38.703 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.703 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.703 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.703 INFO:tasks.workunit.client.0.smithi064.stdout:383/600: open fd = 3 2024-09-12T03:37:38.703 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.703 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.703 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.703 INFO:tasks.workunit.client.0.smithi064.stdout:384/600: open fd = 3 2024-09-12T03:37:38.704 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.704 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.704 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.704 INFO:tasks.workunit.client.0.smithi064.stdout:385/600: open fd = 3 2024-09-12T03:37:38.704 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.704 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.704 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.704 INFO:tasks.workunit.client.0.smithi064.stdout:386/600: open fd = 3 2024-09-12T03:37:38.704 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.704 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.704 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.704 INFO:tasks.workunit.client.0.smithi064.stdout:387/600: open fd = 3 2024-09-12T03:37:38.704 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.705 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.705 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.705 INFO:tasks.workunit.client.0.smithi064.stdout:388/600: open fd = 3 2024-09-12T03:37:38.705 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.705 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.705 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.705 INFO:tasks.workunit.client.0.smithi064.stdout:389/600: open fd = 3 2024-09-12T03:37:38.705 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.705 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.705 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.705 INFO:tasks.workunit.client.0.smithi064.stdout:390/600: open fd = 3 2024-09-12T03:37:38.705 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.705 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.706 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.706 INFO:tasks.workunit.client.0.smithi064.stdout:391/600: open fd = 3 2024-09-12T03:37:38.706 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.706 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.706 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.706 INFO:tasks.workunit.client.0.smithi064.stdout:392/600: open fd = 3 2024-09-12T03:37:38.706 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.706 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.706 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.706 INFO:tasks.workunit.client.0.smithi064.stdout:393/600: open fd = 3 2024-09-12T03:37:38.706 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.706 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.706 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.706 INFO:tasks.workunit.client.0.smithi064.stdout:394/600: open fd = 3 2024-09-12T03:37:38.707 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.707 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.707 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.707 INFO:tasks.workunit.client.0.smithi064.stdout:395/600: open fd = 3 2024-09-12T03:37:38.707 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.707 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.707 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.707 INFO:tasks.workunit.client.0.smithi064.stdout:396/600: open fd = 3 2024-09-12T03:37:38.707 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.708 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.708 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.708 INFO:tasks.workunit.client.0.smithi064.stdout:397/600: open fd = 3 2024-09-12T03:37:38.708 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.708 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.708 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.708 INFO:tasks.workunit.client.0.smithi064.stdout:398/600: open fd = 3 2024-09-12T03:37:38.708 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.708 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.708 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.708 INFO:tasks.workunit.client.0.smithi064.stdout:399/600: open fd = 3 2024-09-12T03:37:38.708 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.708 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.709 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.709 INFO:tasks.workunit.client.0.smithi064.stdout:400/600: open fd = 3 2024-09-12T03:37:38.709 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.709 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.709 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.709 INFO:tasks.workunit.client.0.smithi064.stdout:401/600: open fd = 3 2024-09-12T03:37:38.709 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.709 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.709 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.709 INFO:tasks.workunit.client.0.smithi064.stdout:402/600: open fd = 3 2024-09-12T03:37:38.710 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.710 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.710 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.710 INFO:tasks.workunit.client.0.smithi064.stdout:403/600: open fd = 3 2024-09-12T03:37:38.710 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.710 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.710 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.710 INFO:tasks.workunit.client.0.smithi064.stdout:404/600: open fd = 3 2024-09-12T03:37:38.711 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.711 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.711 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.711 INFO:tasks.workunit.client.0.smithi064.stdout:405/600: open fd = 3 2024-09-12T03:37:38.711 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.711 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.711 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.711 INFO:tasks.workunit.client.0.smithi064.stdout:406/600: open fd = 3 2024-09-12T03:37:38.712 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.712 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.712 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.712 INFO:tasks.workunit.client.0.smithi064.stdout:407/600: open fd = 3 2024-09-12T03:37:38.712 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.712 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.712 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.712 INFO:tasks.workunit.client.0.smithi064.stdout:408/600: open fd = 3 2024-09-12T03:37:38.713 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.713 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.713 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.713 INFO:tasks.workunit.client.0.smithi064.stdout:409/600: open fd = 3 2024-09-12T03:37:38.713 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.713 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.713 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.713 INFO:tasks.workunit.client.0.smithi064.stdout:410/600: open fd = 3 2024-09-12T03:37:38.714 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.714 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.714 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.714 INFO:tasks.workunit.client.0.smithi064.stdout:411/600: open fd = 3 2024-09-12T03:37:38.714 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.714 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.714 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.714 INFO:tasks.workunit.client.0.smithi064.stdout:412/600: open fd = 3 2024-09-12T03:37:38.715 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.715 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.715 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.715 INFO:tasks.workunit.client.0.smithi064.stdout:413/600: open fd = 3 2024-09-12T03:37:38.715 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.715 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.715 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.715 INFO:tasks.workunit.client.0.smithi064.stdout:414/600: open fd = 3 2024-09-12T03:37:38.716 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.716 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.716 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.716 INFO:tasks.workunit.client.0.smithi064.stdout:415/600: open fd = 3 2024-09-12T03:37:38.716 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.716 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.716 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.716 INFO:tasks.workunit.client.0.smithi064.stdout:416/600: open fd = 3 2024-09-12T03:37:38.716 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.717 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.717 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.717 INFO:tasks.workunit.client.0.smithi064.stdout:417/600: open fd = 3 2024-09-12T03:37:38.717 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.717 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.717 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.718 INFO:tasks.workunit.client.0.smithi064.stdout:418/600: open fd = 3 2024-09-12T03:37:38.719 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.719 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.719 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.719 INFO:tasks.workunit.client.0.smithi064.stdout:419/600: open fd = 3 2024-09-12T03:37:38.719 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.720 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.720 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.720 INFO:tasks.workunit.client.0.smithi064.stdout:420/600: open fd = 3 2024-09-12T03:37:38.720 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.720 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.720 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.720 INFO:tasks.workunit.client.0.smithi064.stdout:421/600: open fd = 3 2024-09-12T03:37:38.720 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.721 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.721 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.721 INFO:tasks.workunit.client.0.smithi064.stdout:422/600: open fd = 3 2024-09-12T03:37:38.721 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.721 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.721 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.721 INFO:tasks.workunit.client.0.smithi064.stdout:423/600: open fd = 3 2024-09-12T03:37:38.721 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.722 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.722 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.722 INFO:tasks.workunit.client.0.smithi064.stdout:424/600: open fd = 3 2024-09-12T03:37:38.722 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.722 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.722 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.722 INFO:tasks.workunit.client.0.smithi064.stdout:425/600: open fd = 3 2024-09-12T03:37:38.722 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.723 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.723 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.723 INFO:tasks.workunit.client.0.smithi064.stdout:426/600: open fd = 3 2024-09-12T03:37:38.723 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.723 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.723 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.723 INFO:tasks.workunit.client.0.smithi064.stdout:427/600: open fd = 3 2024-09-12T03:37:38.723 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.723 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.723 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.723 INFO:tasks.workunit.client.0.smithi064.stdout:428/600: open fd = 3 2024-09-12T03:37:38.723 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.724 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.724 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.724 INFO:tasks.workunit.client.0.smithi064.stdout:429/600: open fd = 3 2024-09-12T03:37:38.724 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.724 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.724 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.724 INFO:tasks.workunit.client.0.smithi064.stdout:430/600: open fd = 3 2024-09-12T03:37:38.724 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.724 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.724 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.724 INFO:tasks.workunit.client.0.smithi064.stdout:431/600: open fd = 3 2024-09-12T03:37:38.724 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.724 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.725 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.725 INFO:tasks.workunit.client.0.smithi064.stdout:432/600: open fd = 3 2024-09-12T03:37:38.725 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.725 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.725 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.725 INFO:tasks.workunit.client.0.smithi064.stdout:433/600: open fd = 3 2024-09-12T03:37:38.725 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.725 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.725 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.725 INFO:tasks.workunit.client.0.smithi064.stdout:434/600: open fd = 3 2024-09-12T03:37:38.725 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.725 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.726 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.726 INFO:tasks.workunit.client.0.smithi064.stdout:435/600: open fd = 3 2024-09-12T03:37:38.726 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.726 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.726 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:38.726 INFO:tasks.workunit.client.0.smithi064.stdout:436/600: open fd = 3 2024-09-12T03:37:38.726 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.726 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:37:38.726 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:37:39.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:37:39 smithi115 ceph-mon[34801]: pgmap v258: 129 pgs: 129 active+clean; 2.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 55 KiB/s wr, 17 op/s 2024-09-12T03:37:39.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:37:39 smithi195 ceph-mon[34786]: pgmap v258: 129 pgs: 129 active+clean; 2.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 55 KiB/s wr, 17 op/s 2024-09-12T03:37:39.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:37:39 smithi064 ceph-mon[30952]: pgmap v258: 129 pgs: 129 active+clean; 2.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 55 KiB/s wr, 17 op/s 2024-09-12T03:37:41.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:37:41 smithi115 ceph-mon[34801]: pgmap v259: 129 pgs: 129 active+clean; 2.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 9 op/s 2024-09-12T03:37:41.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:37:41 smithi195 ceph-mon[34786]: pgmap v259: 129 pgs: 129 active+clean; 2.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 9 op/s 2024-09-12T03:37:41.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:37:41 smithi064 ceph-mon[30952]: pgmap v259: 129 pgs: 129 active+clean; 2.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 9 op/s 2024-09-12T03:37:43.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:37:43 smithi115 ceph-mon[34801]: pgmap v260: 129 pgs: 129 active+clean; 2.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 52 KiB/s wr, 17 op/s 2024-09-12T03:37:43.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:37:43 smithi195 ceph-mon[34786]: pgmap v260: 129 pgs: 129 active+clean; 2.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 52 KiB/s wr, 17 op/s 2024-09-12T03:37:43.521 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:37:43 smithi064 ceph-mon[30952]: pgmap v260: 129 pgs: 129 active+clean; 2.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 52 KiB/s wr, 17 op/s 2024-09-12T03:37:45.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:37:45 smithi115 ceph-mon[34801]: pgmap v261: 129 pgs: 129 active+clean; 2.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-12T03:37:45.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:37:45 smithi195 ceph-mon[34786]: pgmap v261: 129 pgs: 129 active+clean; 2.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-12T03:37:45.521 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:37:45 smithi064 ceph-mon[30952]: pgmap v261: 129 pgs: 129 active+clean; 2.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-12T03:37:47.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:37:47 smithi115 ceph-mon[34801]: pgmap v262: 129 pgs: 129 active+clean; 2.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-12T03:37:47.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:37:47 smithi195 ceph-mon[34786]: pgmap v262: 129 pgs: 129 active+clean; 2.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-12T03:37:47.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:37:47 smithi064 ceph-mon[30952]: pgmap v262: 129 pgs: 129 active+clean; 2.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-12T03:37:49.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:37:49 smithi195 ceph-mon[34786]: pgmap v263: 129 pgs: 129 active+clean; 2.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 54 KiB/s wr, 18 op/s 2024-09-12T03:37:49.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:37:49 smithi064 ceph-mon[30952]: pgmap v263: 129 pgs: 129 active+clean; 2.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 54 KiB/s wr, 18 op/s 2024-09-12T03:37:49.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:37:49 smithi115 ceph-mon[34801]: pgmap v263: 129 pgs: 129 active+clean; 2.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 54 KiB/s wr, 18 op/s 2024-09-12T03:37:50.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.827+0000 7f07c3331640 1 -- 172.21.15.64:0/2631325762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07bc101280 msgr2=0x7f07bc10deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:50.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.827+0000 7f07c3331640 1 --2- 172.21.15.64:0/2631325762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07bc101280 0x7f07bc10deb0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f07ac009900 tx=0x7f07ac02a410 comp rx=0 tx=0).stop 2024-09-12T03:37:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.827+0000 7f07c3331640 1 -- 172.21.15.64:0/2631325762 shutdown_connections 2024-09-12T03:37:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.827+0000 7f07c3331640 1 --2- 172.21.15.64:0/2631325762 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07bc10e480 0x7f07bc110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.827+0000 7f07c3331640 1 --2- 172.21.15.64:0/2631325762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07bc101280 0x7f07bc10deb0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.827+0000 7f07c3331640 1 --2- 172.21.15.64:0/2631325762 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07bc1008d0 0x7f07bc100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.827+0000 7f07c3331640 1 -- 172.21.15.64:0/2631325762 >> 172.21.15.64:0/2631325762 conn(0x7f07bc0fc6b0 msgr2=0x7f07bc0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.827+0000 7f07c3331640 1 -- 172.21.15.64:0/2631325762 shutdown_connections 2024-09-12T03:37:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.827+0000 7f07c3331640 1 -- 172.21.15.64:0/2631325762 wait complete. 2024-09-12T03:37:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.828+0000 7f07c3331640 1 Processor -- start 2024-09-12T03:37:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.828+0000 7f07c3331640 1 -- start start 2024-09-12T03:37:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.828+0000 7f07c3331640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07bc1008d0 0x7f07bc19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.828+0000 7f07c3331640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07bc101280 0x7f07bc19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.828+0000 7f07c3331640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07bc10e480 0x7f07bc19edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.828+0000 7f07c3331640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc114000 con 0x7f07bc101280 2024-09-12T03:37:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.828+0000 7f07c3331640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc113e80 con 0x7f07bc10e480 2024-09-12T03:37:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.828+0000 7f07c3331640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc114180 con 0x7f07bc1008d0 2024-09-12T03:37:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.828+0000 7f07c08cc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07bc101280 0x7f07bc19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.828+0000 7f07c08cc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07bc101280 0x7f07bc19aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54210/0 (socket says 172.21.15.64:54210) 2024-09-12T03:37:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.828+0000 7f07c08cc640 1 -- 172.21.15.64:0/1422642179 learned_addr learned my addr 172.21.15.64:0/1422642179 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:37:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.828+0000 7f07c10cd640 1 --2- 172.21.15.64:0/1422642179 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07bc1008d0 0x7f07bc19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.828+0000 7f07c18ce640 1 --2- 172.21.15.64:0/1422642179 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07bc10e480 0x7f07bc19edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.828+0000 7f07c08cc640 1 -- 172.21.15.64:0/1422642179 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07bc1008d0 msgr2=0x7f07bc19a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.829+0000 7f07c08cc640 1 --2- 172.21.15.64:0/1422642179 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07bc1008d0 0x7f07bc19a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.829+0000 7f07c08cc640 1 -- 172.21.15.64:0/1422642179 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07bc10e480 msgr2=0x7f07bc19edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.829+0000 7f07c08cc640 1 --2- 172.21.15.64:0/1422642179 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07bc10e480 0x7f07bc19edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.829+0000 7f07c08cc640 1 -- 172.21.15.64:0/1422642179 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07bc19f4e0 con 0x7f07bc101280 2024-09-12T03:37:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.829+0000 7f07c08cc640 1 --2- 172.21.15.64:0/1422642179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07bc101280 0x7f07bc19aa30 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f07ac005ab0 tx=0x7f07ac003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.829+0000 7f07b27fc640 1 -- 172.21.15.64:0/1422642179 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac03f070 con 0x7f07bc101280 2024-09-12T03:37:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.829+0000 7f07b27fc640 1 -- 172.21.15.64:0/1422642179 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07ac031d60 con 0x7f07bc101280 2024-09-12T03:37:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.829+0000 7f07c3331640 1 -- 172.21.15.64:0/1422642179 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07bc19f770 con 0x7f07bc101280 2024-09-12T03:37:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.829+0000 7f07b27fc640 1 -- 172.21.15.64:0/1422642179 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac035030 con 0x7f07bc101280 2024-09-12T03:37:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.829+0000 7f07c3331640 1 -- 172.21.15.64:0/1422642179 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07bc1a7060 con 0x7f07bc101280 2024-09-12T03:37:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.831+0000 7f07b27fc640 1 -- 172.21.15.64:0/1422642179 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f07ac042070 con 0x7f07bc101280 2024-09-12T03:37:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.832+0000 7f07c60cd740 1 Processor -- start 2024-09-12T03:37:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.832+0000 7f07c60cd740 1 -- start start 2024-09-12T03:37:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.832+0000 7f07c60cd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555778abcaf0 0x555778abced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.832+0000 7f07c60cd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555778abd410 0x555778ac7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.832+0000 7f07c60cd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555778ac7cb0 0x555778aca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.832+0000 7f07c60cd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555778a212f0 con 0x555778abd410 2024-09-12T03:37:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.832+0000 7f07c60cd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5557789c9270 con 0x555778ac7cb0 2024-09-12T03:37:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.832+0000 7f07c60cd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555778a1d2b0 con 0x555778abcaf0 2024-09-12T03:37:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.832+0000 7f07c10cd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555778abd410 0x555778ac7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.833+0000 7f07c10cd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555778abd410 0x555778ac7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54224/0 (socket says 172.21.15.64:54224) 2024-09-12T03:37:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.833+0000 7f07c10cd640 1 -- 172.21.15.64:0/1995288760 learned_addr learned my addr 172.21.15.64:0/1995288760 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:37:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.833+0000 7f07c18ce640 1 --2- 172.21.15.64:0/1995288760 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555778abcaf0 0x555778abced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.833+0000 7f07b27fc640 1 --2- 172.21.15.64:0/1422642179 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f07a80a2e20 0x7f07a80a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.833+0000 7f07c08cc640 1 --2- 172.21.15.64:0/1995288760 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555778ac7cb0 0x555778aca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.833+0000 7f07c10cd640 1 -- 172.21.15.64:0/1995288760 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555778abcaf0 msgr2=0x555778abced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:37:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.833+0000 7f07c10cd640 1 --2- 172.21.15.64:0/1995288760 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555778abcaf0 0x555778abced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.833+0000 7f07c10cd640 1 -- 172.21.15.64:0/1995288760 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555778ac7cb0 msgr2=0x555778aca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:37:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.833+0000 7f07c10cd640 1 --2- 172.21.15.64:0/1995288760 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555778ac7cb0 0x555778aca0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.833+0000 7f07c10cd640 1 -- 172.21.15.64:0/1995288760 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557789c8340 con 0x555778abd410 2024-09-12T03:37:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.833+0000 7f07c18ce640 1 --2- 172.21.15.64:0/1995288760 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555778abcaf0 0x555778abced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:37:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.833+0000 7f07c18ce640 1 --2- 172.21.15.64:0/1422642179 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f07a80a2e20 0x7f07a80a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.833+0000 7f07b27fc640 1 -- 172.21.15.64:0/1422642179 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9583+0+0 (secure 0 0 0) 0x7f07ac0efda0 con 0x7f07bc101280 2024-09-12T03:37:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.833+0000 7f07c10cd640 1 --2- 172.21.15.64:0/1995288760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555778abd410 0x555778ac7770 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f07b800b760 tx=0x7f07b800bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67da78ea51086469 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.833+0000 7f07a77fe640 1 -- 172.21.15.64:0/1995288760 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8003dd0 con 0x555778abd410 2024-09-12T03:37:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.833+0000 7f07a77fe640 1 -- 172.21.15.64:0/1995288760 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07b8015070 con 0x555778abd410 2024-09-12T03:37:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.833+0000 7f07a77fe640 1 -- 172.21.15.64:0/1995288760 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8004d90 con 0x555778abd410 2024-09-12T03:37:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.834+0000 7f07c60cd740 1 -- 172.21.15.64:0/1995288760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555778abd410 msgr2=0x555778ac7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.834+0000 7f07c18ce640 1 --2- 172.21.15.64:0/1422642179 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f07a80a2e20 0x7f07a80a52e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f07bc06baf0 tx=0x7f07b4004a40 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.834+0000 7f07c60cd740 1 --2- 172.21.15.64:0/1995288760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555778abd410 0x555778ac7770 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f07b800b760 tx=0x7f07b800bc20 comp rx=0 tx=0).stop 2024-09-12T03:37:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.834+0000 7f07c60cd740 1 -- 172.21.15.64:0/1995288760 shutdown_connections 2024-09-12T03:37:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.834+0000 7f07c60cd740 1 --2- 172.21.15.64:0/1995288760 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555778ac7cb0 0x555778aca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.834+0000 7f07c60cd740 1 --2- 172.21.15.64:0/1995288760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555778abd410 0x555778ac7770 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.834+0000 7f07c60cd740 1 --2- 172.21.15.64:0/1995288760 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555778abcaf0 0x555778abced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.834+0000 7f07c60cd740 1 -- 172.21.15.64:0/1995288760 >> 172.21.15.64:0/1995288760 conn(0x555778ab9060 msgr2=0x555778ab9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.834+0000 7f07c60cd740 1 -- 172.21.15.64:0/1995288760 shutdown_connections 2024-09-12T03:37:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.834+0000 7f07c60cd740 1 -- 172.21.15.64:0/1995288760 wait complete. 2024-09-12T03:37:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.834+0000 7f07c60cd740 1 -- start start 2024-09-12T03:37:50.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.835+0000 7f07c60cd740 1 Processor -- start 2024-09-12T03:37:50.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.835+0000 7f07c60cd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555778abcaf0 0x555778ac4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:50.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.835+0000 7f07c60cd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555778abd410 0x555778ac50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:50.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.835+0000 7f07c60cd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555778ac7cb0 0x555778ac5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:50.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.835+0000 7f07c60cd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555778aca880 con 0x555778ac7cb0 2024-09-12T03:37:50.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.835+0000 7f07c60cd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5557789aa290 con 0x555778abd410 2024-09-12T03:37:50.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.835+0000 7f07c60cd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555778acaa00 con 0x555778abcaf0 2024-09-12T03:37:50.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.835+0000 7f07c60cd740 10 client.0 _ll_register_callbacks cb 0x5557789db140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:37:50.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.835+0000 7f07c18ce640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555778ac7cb0 0x555778ac5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:50.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.835+0000 7f07c18ce640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555778ac7cb0 0x555778ac5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54240/0 (socket says 172.21.15.64:54240) 2024-09-12T03:37:50.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.835+0000 7f07c18ce640 1 -- 172.21.15.64:0/4234806608 learned_addr learned my addr 172.21.15.64:0/4234806608 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:37:50.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.835+0000 7f07c08cc640 1 --2- 172.21.15.64:0/4234806608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555778abd410 0x555778ac50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:50.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.835+0000 7f07c10cd640 1 --2- 172.21.15.64:0/4234806608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555778abcaf0 0x555778ac4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.836+0000 7f07c18ce640 1 -- 172.21.15.64:0/4234806608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555778abcaf0 msgr2=0x555778ac4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:37:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.836+0000 7f07c18ce640 1 --2- 172.21.15.64:0/4234806608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555778abcaf0 0x555778ac4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.836+0000 7f07c18ce640 1 -- 172.21.15.64:0/4234806608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555778abd410 msgr2=0x555778ac50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:37:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.836+0000 7f07c18ce640 1 --2- 172.21.15.64:0/4234806608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555778abd410 0x555778ac50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.836+0000 7f07c18ce640 1 -- 172.21.15.64:0/4234806608 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555778734210 con 0x555778ac7cb0 2024-09-12T03:37:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.836+0000 7f07c18ce640 1 --2- 172.21.15.64:0/4234806608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555778ac7cb0 0x555778ac5600 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f07b4012840 tx=0x7f07b4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd1e51186e79f94 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.836+0000 7f07a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:37:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.836+0000 7f07a4ff9640 1 -- 172.21.15.64:0/4234806608 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b401db60 con 0x555778ac7cb0 2024-09-12T03:37:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.836+0000 7f07c60cd740 1 -- 172.21.15.64:0/4234806608 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07b8015070 con 0x555778ac7cb0 2024-09-12T03:37:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.836+0000 7f07a4ff9640 1 -- 172.21.15.64:0/4234806608 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9583+0+0 (secure 0 0 0) 0x7f07b401dd00 con 0x555778ac7cb0 2024-09-12T03:37:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.836+0000 7f07a4ff9640 1 -- 172.21.15.64:0/4234806608 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f07b4027470 con 0x555778ac7cb0 2024-09-12T03:37:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.836+0000 7f07c60cd740 10 client.16044 fetch_fsmap learned FSMap version 25 2024-09-12T03:37:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.836+0000 7f07c60cd740 1 -- 172.21.15.64:0/4234806608 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555778800620 con 0x555778ac7cb0 2024-09-12T03:37:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.836+0000 7f07a4ff9640 1 -- 172.21.15.64:0/4234806608 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07b8015070 con 0x555778ac7cb0 2024-09-12T03:37:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.837+0000 7f07a4ff9640 1 -- 172.21.15.64:0/4234806608 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f07b400f040 con 0x555778ac7cb0 2024-09-12T03:37:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.837+0000 7f07c60cd740 10 client.16044 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:37:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.837+0000 7f07c60cd740 10 client.16044 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:37:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.837+0000 7f07c60cd740 20 client.16044 populate_metadata read hostname 'smithi064' 2024-09-12T03:37:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.837+0000 7f07c60cd740 1 --2- 172.21.15.64:0/4234806608 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x555778ad7c50 0x555778ada060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.837+0000 7f07c60cd740 4 client.16044 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:37:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.837+0000 7f07c60cd740 1 -- 172.21.15.64:0/4234806608 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555778a2ea20 con 0x555778ad7c50 2024-09-12T03:37:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.837+0000 7f07c10cd640 1 --2- 172.21.15.64:0/4234806608 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x555778ad7c50 0x555778ada060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.837+0000 7f07c10cd640 1 --2- 172.21.15.64:0/4234806608 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x555778ad7c50 0x555778ada060 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0f7c7ba92d781d6 server_cookie=5430f510638fa53a in_seq=0 out_seq=0 2024-09-12T03:37:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.837+0000 7f07a4ff9640 10 client.16044 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:37:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.838+0000 7f07a4ff9640 1 -- 172.21.15.64:0/4234806608 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555778a2ea20 con 0x555778ad7c50 2024-09-12T03:37:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.838+0000 7f07a4ff9640 10 client.16044 handle_command_reply: tid=0 2024-09-12T03:37:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.838+0000 7f07c60cd740 1 client.16044 shutdown 2024-09-12T03:37:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.838+0000 7f07c60cd740 1 -- 172.21.15.64:0/4234806608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555778ac7cb0 msgr2=0x555778ac5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.838+0000 7f07c60cd740 1 --2- 172.21.15.64:0/4234806608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555778ac7cb0 0x555778ac5600 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f07b4012840 tx=0x7f07b4004ab0 comp rx=0 tx=0).stop 2024-09-12T03:37:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.838+0000 7f07c60cd740 1 -- 172.21.15.64:0/4234806608 shutdown_connections 2024-09-12T03:37:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.838+0000 7f07c60cd740 1 --2- 172.21.15.64:0/4234806608 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x555778ad7c50 0x555778ada060 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.838+0000 7f07c60cd740 1 --2- 172.21.15.64:0/4234806608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555778ac7cb0 0x555778ac5600 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.838+0000 7f07c60cd740 1 --2- 172.21.15.64:0/4234806608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555778abd410 0x555778ac50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.838+0000 7f07a4ff9640 0 client.16044 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:37:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.838+0000 7f07c60cd740 1 --2- 172.21.15.64:0/4234806608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555778abcaf0 0x555778ac4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.838+0000 7f07c60cd740 1 -- 172.21.15.64:0/4234806608 >> 172.21.15.64:0/4234806608 conn(0x555778ab9060 msgr2=0x555778abed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.839+0000 7f07c60cd740 1 -- 172.21.15.64:0/4234806608 shutdown_connections 2024-09-12T03:37:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.839+0000 7f07c60cd740 1 -- 172.21.15.64:0/4234806608 wait complete. 2024-09-12T03:37:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.839+0000 7f07c60cd740 20 client.16044 trim_cache size 0 max 16384 2024-09-12T03:37:50.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.853+0000 7f07c60cd740 1 Processor -- start 2024-09-12T03:37:50.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.853+0000 7f07c60cd740 1 -- start start 2024-09-12T03:37:50.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.854+0000 7f07c60cd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555778abcaf0 0x555778abf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:50.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.854+0000 7f07c60cd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555778abd410 0x555778ac5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:50.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.854+0000 7f07c60cd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555778ac7cb0 0x555778adf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:50.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.854+0000 7f07c60cd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc1b3be0 con 0x555778abcaf0 2024-09-12T03:37:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.854+0000 7f07c60cd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc1b14b0 con 0x555778abd410 2024-09-12T03:37:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.854+0000 7f07c60cd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc1af320 con 0x555778ac7cb0 2024-09-12T03:37:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.854+0000 7f07c10cd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555778abcaf0 0x555778abf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.854+0000 7f07c10cd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555778abcaf0 0x555778abf560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54248/0 (socket says 172.21.15.64:54248) 2024-09-12T03:37:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.854+0000 7f07c10cd640 1 -- 172.21.15.64:0/3111863765 learned_addr learned my addr 172.21.15.64:0/3111863765 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:37:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.854+0000 7f07c10cd640 1 -- 172.21.15.64:0/3111863765 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555778ac7cb0 msgr2=0x555778adf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:37:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.854+0000 7f07c18ce640 1 --2- 172.21.15.64:0/3111863765 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555778ac7cb0 0x555778adf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.854+0000 7f07c10cd640 1 --2- 172.21.15.64:0/3111863765 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555778ac7cb0 0x555778adf970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.854+0000 7f07c08cc640 1 --2- 172.21.15.64:0/3111863765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555778abd410 0x555778ac5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.854+0000 7f07c10cd640 1 -- 172.21.15.64:0/3111863765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555778abd410 msgr2=0x555778ac5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:37:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.854+0000 7f07c10cd640 1 --2- 172.21.15.64:0/3111863765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555778abd410 0x555778ac5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.854+0000 7f07c10cd640 1 -- 172.21.15.64:0/3111863765 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555778ac30b0 con 0x555778abcaf0 2024-09-12T03:37:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.854+0000 7f07c10cd640 1 --2- 172.21.15.64:0/3111863765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555778abcaf0 0x555778abf560 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f07b800b760 tx=0x7f07b8004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c6aa9baabb1dafc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:50.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.854+0000 7f07a6ffd640 1 -- 172.21.15.64:0/3111863765 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8003dd0 con 0x555778abcaf0 2024-09-12T03:37:50.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.854+0000 7f07a6ffd640 1 -- 172.21.15.64:0/3111863765 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07b8015050 con 0x555778abcaf0 2024-09-12T03:37:50.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.854+0000 7f07a6ffd640 1 -- 172.21.15.64:0/3111863765 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8019680 con 0x555778abcaf0 2024-09-12T03:37:50.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.855+0000 7f07c60cd740 1 -- 172.21.15.64:0/3111863765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555778abcaf0 msgr2=0x555778abf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:50.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.855+0000 7f07c60cd740 1 --2- 172.21.15.64:0/3111863765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555778abcaf0 0x555778abf560 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f07b800b760 tx=0x7f07b8004970 comp rx=0 tx=0).stop 2024-09-12T03:37:50.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.855+0000 7f07c60cd740 1 -- 172.21.15.64:0/3111863765 shutdown_connections 2024-09-12T03:37:50.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.855+0000 7f07c60cd740 1 --2- 172.21.15.64:0/3111863765 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555778ac7cb0 0x555778adf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:50.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.855+0000 7f07c60cd740 1 --2- 172.21.15.64:0/3111863765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555778abd410 0x555778ac5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:50.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.855+0000 7f07c60cd740 1 --2- 172.21.15.64:0/3111863765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555778abcaf0 0x555778abf560 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:50.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.855+0000 7f07c60cd740 1 -- 172.21.15.64:0/3111863765 >> 172.21.15.64:0/3111863765 conn(0x555778ab9060 msgr2=0x555778ac6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:50.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.855+0000 7f07c60cd740 1 -- 172.21.15.64:0/3111863765 shutdown_connections 2024-09-12T03:37:50.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.855+0000 7f07c60cd740 1 -- 172.21.15.64:0/3111863765 wait complete. 2024-09-12T03:37:50.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.855+0000 7f07c60cd740 1 -- start start 2024-09-12T03:37:50.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.855+0000 7f07c60cd740 1 Processor -- start 2024-09-12T03:37:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.856+0000 7f07c60cd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555778abcaf0 0x555778ad0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.856+0000 7f07c60cd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555778abd410 0x555778acb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.856+0000 7f07c60cd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555778ac7cb0 0x555778acbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.856+0000 7f07c60cd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc1af970 con 0x555778ac7cb0 2024-09-12T03:37:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.856+0000 7f07c60cd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc1b0de0 con 0x555778abd410 2024-09-12T03:37:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.856+0000 7f07c60cd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc1b0040 con 0x555778abcaf0 2024-09-12T03:37:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.856+0000 7f07c60cd740 10 client.0 _ll_register_callbacks cb 0x7f07b4009cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:37:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.856+0000 7f07c18ce640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555778ac7cb0 0x555778acbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.856+0000 7f07c18ce640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555778ac7cb0 0x555778acbfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54260/0 (socket says 172.21.15.64:54260) 2024-09-12T03:37:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.856+0000 7f07c18ce640 1 -- 172.21.15.64:0/2724434169 learned_addr learned my addr 172.21.15.64:0/2724434169 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:37:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.856+0000 7f07c10cd640 1 --2- 172.21.15.64:0/2724434169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555778abcaf0 0x555778ad0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.856+0000 7f07c08cc640 1 --2- 172.21.15.64:0/2724434169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555778abd410 0x555778acb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.856+0000 7f07c18ce640 1 -- 172.21.15.64:0/2724434169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555778abcaf0 msgr2=0x555778ad0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:37:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.856+0000 7f07c18ce640 1 --2- 172.21.15.64:0/2724434169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555778abcaf0 0x555778ad0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.856+0000 7f07c18ce640 1 -- 172.21.15.64:0/2724434169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555778abd410 msgr2=0x555778acb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:37:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.856+0000 7f07c18ce640 1 --2- 172.21.15.64:0/2724434169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555778abd410 0x555778acb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.856+0000 7f07c18ce640 1 -- 172.21.15.64:0/2724434169 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555778b41940 con 0x555778ac7cb0 2024-09-12T03:37:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.856+0000 7f07c18ce640 1 --2- 172.21.15.64:0/2724434169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555778ac7cb0 0x555778acbfe0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f07b401e7c0 tx=0x7f07b401ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a05878fb9f9f719 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:37:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.856+0000 7f07a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:37:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.857+0000 7f07a4ff9640 1 -- 172.21.15.64:0/2724434169 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b40123f0 con 0x555778ac7cb0 2024-09-12T03:37:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.857+0000 7f07a4ff9640 1 -- 172.21.15.64:0/2724434169 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9583+0+0 (secure 0 0 0) 0x7f07b401d9c0 con 0x555778ac7cb0 2024-09-12T03:37:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.857+0000 7f07c60cd740 1 -- 172.21.15.64:0/2724434169 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07b8015050 con 0x555778ac7cb0 2024-09-12T03:37:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.857+0000 7f07a4ff9640 1 -- 172.21.15.64:0/2724434169 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f07b4011c30 con 0x555778ac7cb0 2024-09-12T03:37:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.857+0000 7f07c60cd740 10 client.16050 fetch_fsmap learned FSMap version 25 2024-09-12T03:37:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.857+0000 7f07c60cd740 1 -- 172.21.15.64:0/2724434169 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555778b42af0 con 0x555778ac7cb0 2024-09-12T03:37:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.857+0000 7f07a4ff9640 1 -- 172.21.15.64:0/2724434169 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07b8015050 con 0x555778ac7cb0 2024-09-12T03:37:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.857+0000 7f07a4ff9640 1 -- 172.21.15.64:0/2724434169 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f07b4034450 con 0x555778ac7cb0 2024-09-12T03:37:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.857+0000 7f07c60cd740 10 client.16050 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:37:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.857+0000 7f07c60cd740 10 client.16050 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:37:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.857+0000 7f07c60cd740 20 client.16050 populate_metadata read hostname 'smithi064' 2024-09-12T03:37:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.857+0000 7f07c60cd740 1 --2- 172.21.15.64:0/2724434169 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x555778b43230 0x555778b45650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:37:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.857+0000 7f07c60cd740 4 client.16050 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:37:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.857+0000 7f07c60cd740 1 -- 172.21.15.64:0/2724434169 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555778a28d70 con 0x555778b43230 2024-09-12T03:37:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.858+0000 7f07c10cd640 1 --2- 172.21.15.64:0/2724434169 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x555778b43230 0x555778b45650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:37:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.858+0000 7f07c10cd640 1 --2- 172.21.15.64:0/2724434169 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x555778b43230 0x555778b45650 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1610771ecdf46af server_cookie=e1ff3224ea70e600 in_seq=0 out_seq=0 2024-09-12T03:37:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.858+0000 7f07a4ff9640 10 client.16050 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:37:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.870+0000 7f07a4ff9640 1 -- 172.21.15.64:0/2724434169 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555778a28d70 con 0x555778b43230 2024-09-12T03:37:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.870+0000 7f07a4ff9640 10 client.16050 handle_command_reply: tid=0 2024-09-12T03:37:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.870+0000 7f07c60cd740 1 client.16050 shutdown 2024-09-12T03:37:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.870+0000 7f07c60cd740 1 -- 172.21.15.64:0/2724434169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555778ac7cb0 msgr2=0x555778acbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.870+0000 7f07c60cd740 1 --2- 172.21.15.64:0/2724434169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555778ac7cb0 0x555778acbfe0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f07b401e7c0 tx=0x7f07b401ece0 comp rx=0 tx=0).stop 2024-09-12T03:37:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.870+0000 7f07c60cd740 1 -- 172.21.15.64:0/2724434169 shutdown_connections 2024-09-12T03:37:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.870+0000 7f07c60cd740 1 --2- 172.21.15.64:0/2724434169 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x555778b43230 0x555778b45650 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.870+0000 7f07c60cd740 1 --2- 172.21.15.64:0/2724434169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555778ac7cb0 0x555778acbfe0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.870+0000 7f07c60cd740 1 --2- 172.21.15.64:0/2724434169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555778abd410 0x555778acb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.870+0000 7f07a4ff9640 0 client.16050 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:37:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.870+0000 7f07c60cd740 1 --2- 172.21.15.64:0/2724434169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555778abcaf0 0x555778ad0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.870+0000 7f07c60cd740 1 -- 172.21.15.64:0/2724434169 >> 172.21.15.64:0/2724434169 conn(0x555778ab9060 msgr2=0x555778ab79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.870+0000 7f07c60cd740 1 -- 172.21.15.64:0/2724434169 shutdown_connections 2024-09-12T03:37:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.871+0000 7f07c60cd740 1 -- 172.21.15.64:0/2724434169 wait complete. 2024-09-12T03:37:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.871+0000 7f07c60cd740 20 client.16050 trim_cache size 0 max 16384 2024-09-12T03:37:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:37:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:37:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:37:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:37:50.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.871+0000 7f07c3331640 1 -- 172.21.15.64:0/1422642179 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f07a80a2e20 msgr2=0x7f07a80a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:50.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.871+0000 7f07c3331640 1 --2- 172.21.15.64:0/1422642179 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f07a80a2e20 0x7f07a80a52e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f07bc06baf0 tx=0x7f07b4004a40 comp rx=0 tx=0).stop 2024-09-12T03:37:50.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.871+0000 7f07c3331640 1 -- 172.21.15.64:0/1422642179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07bc101280 msgr2=0x7f07bc19aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:37:50.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.871+0000 7f07c3331640 1 --2- 172.21.15.64:0/1422642179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07bc101280 0x7f07bc19aa30 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f07ac005ab0 tx=0x7f07ac003cd0 comp rx=0 tx=0).stop 2024-09-12T03:37:50.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.871+0000 7f07c3331640 1 -- 172.21.15.64:0/1422642179 shutdown_connections 2024-09-12T03:37:50.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.871+0000 7f07c3331640 1 --2- 172.21.15.64:0/1422642179 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f07a80a2e20 0x7f07a80a52e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:50.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.871+0000 7f07c3331640 1 --2- 172.21.15.64:0/1422642179 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07bc10e480 0x7f07bc19edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:50.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.871+0000 7f07c3331640 1 --2- 172.21.15.64:0/1422642179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07bc101280 0x7f07bc19aa30 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:50.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.871+0000 7f07c3331640 1 --2- 172.21.15.64:0/1422642179 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07bc1008d0 0x7f07bc19a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:37:50.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.871+0000 7f07c3331640 1 -- 172.21.15.64:0/1422642179 >> 172.21.15.64:0/1422642179 conn(0x7f07bc0fc6b0 msgr2=0x7f07bc0feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:37:50.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.871+0000 7f07c3331640 1 -- 172.21.15.64:0/1422642179 shutdown_connections 2024-09-12T03:37:50.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:37:50.871+0000 7f07c3331640 1 -- 172.21.15.64:0/1422642179 wait complete. 2024-09-12T03:37:51.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:37:51 smithi195 ceph-mon[34786]: pgmap v264: 129 pgs: 129 active+clean; 2.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 39 KiB/s wr, 13 op/s 2024-09-12T03:37:51.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:37:51 smithi064 ceph-mon[30952]: pgmap v264: 129 pgs: 129 active+clean; 2.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 39 KiB/s wr, 13 op/s 2024-09-12T03:37:51.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:37:51 smithi115 ceph-mon[34801]: pgmap v264: 129 pgs: 129 active+clean; 2.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 39 KiB/s wr, 13 op/s 2024-09-12T03:37:53.445 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:37:53 smithi115 ceph-mon[34801]: pgmap v265: 129 pgs: 129 active+clean; 3.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 61 KiB/s wr, 20 op/s 2024-09-12T03:37:53.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:37:53 smithi195 ceph-mon[34786]: pgmap v265: 129 pgs: 129 active+clean; 3.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 61 KiB/s wr, 20 op/s 2024-09-12T03:37:53.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:37:53 smithi064 ceph-mon[30952]: pgmap v265: 129 pgs: 129 active+clean; 3.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 61 KiB/s wr, 20 op/s 2024-09-12T03:37:55.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:37:55 smithi195 ceph-mon[34786]: pgmap v266: 129 pgs: 129 active+clean; 3.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:37:55.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:37:55 smithi064 ceph-mon[30952]: pgmap v266: 129 pgs: 129 active+clean; 3.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:37:55.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:37:55 smithi115 ceph-mon[34801]: pgmap v266: 129 pgs: 129 active+clean; 3.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:37:57.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:37:57 smithi195 ceph-mon[34786]: pgmap v267: 129 pgs: 129 active+clean; 3.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:37:57.521 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:37:57 smithi064 ceph-mon[30952]: pgmap v267: 129 pgs: 129 active+clean; 3.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:37:57.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:37:57 smithi115 ceph-mon[34801]: pgmap v267: 129 pgs: 129 active+clean; 3.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:37:59.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:37:59 smithi195 ceph-mon[34786]: pgmap v268: 129 pgs: 129 active+clean; 3.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:37:59.521 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:37:59 smithi064 ceph-mon[30952]: pgmap v268: 129 pgs: 129 active+clean; 3.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:37:59.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:37:59 smithi115 ceph-mon[34801]: pgmap v268: 129 pgs: 129 active+clean; 3.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:38:01.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:38:01 smithi195 ceph-mon[34786]: pgmap v269: 129 pgs: 129 active+clean; 3.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 12 op/s 2024-09-12T03:38:01.521 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:38:01 smithi064 ceph-mon[30952]: pgmap v269: 129 pgs: 129 active+clean; 3.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 12 op/s 2024-09-12T03:38:01.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:38:01 smithi115 ceph-mon[34801]: pgmap v269: 129 pgs: 129 active+clean; 3.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 12 op/s 2024-09-12T03:38:03.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:38:03 smithi195 ceph-mon[34786]: pgmap v270: 129 pgs: 129 active+clean; 3.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 100 KiB/s wr, 35 op/s 2024-09-12T03:38:03.521 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:38:03 smithi064 ceph-mon[30952]: pgmap v270: 129 pgs: 129 active+clean; 3.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 100 KiB/s wr, 35 op/s 2024-09-12T03:38:03.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:38:03 smithi115 ceph-mon[34801]: pgmap v270: 129 pgs: 129 active+clean; 3.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 100 KiB/s wr, 35 op/s 2024-09-12T03:38:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.974+0000 7f3bb6596640 1 -- 172.21.15.64:0/1929659084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bb01009b0 msgr2=0x7f3bb0100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.974+0000 7f3bb6596640 1 --2- 172.21.15.64:0/1929659084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bb01009b0 0x7f3bb0100e10 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3b98009930 tx=0x7f3b9802a420 comp rx=0 tx=0).stop 2024-09-12T03:38:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.974+0000 7f3bb6596640 1 -- 172.21.15.64:0/1929659084 shutdown_connections 2024-09-12T03:38:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.974+0000 7f3bb6596640 1 --2- 172.21.15.64:0/1929659084 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bb0101350 0x7f3bb010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.974+0000 7f3bb6596640 1 --2- 172.21.15.64:0/1929659084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bb01009b0 0x7f3bb0100e10 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.974+0000 7f3bb6596640 1 --2- 172.21.15.64:0/1929659084 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3bb01069c0 0x7f3bb0106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.974+0000 7f3bb6596640 1 -- 172.21.15.64:0/1929659084 >> 172.21.15.64:0/1929659084 conn(0x7f3bb0077990 msgr2=0x7f3bb00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:38:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.974+0000 7f3bb6596640 1 -- 172.21.15.64:0/1929659084 shutdown_connections 2024-09-12T03:38:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.974+0000 7f3bb6596640 1 -- 172.21.15.64:0/1929659084 wait complete. 2024-09-12T03:38:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.975+0000 7f3bb6596640 1 Processor -- start 2024-09-12T03:38:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.975+0000 7f3bb6596640 1 -- start start 2024-09-12T03:38:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.975+0000 7f3bb6596640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3bb01009b0 0x7f3bb01971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.975+0000 7f3bb6596640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bb0101350 0x7f3bb01976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.975+0000 7f3bb6596640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bb01069c0 0x7f3bb019ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.975+0000 7f3bb6596640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb0110cb0 con 0x7f3bb01069c0 2024-09-12T03:38:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.975+0000 7f3bb6596640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb0110b30 con 0x7f3bb0101350 2024-09-12T03:38:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.975+0000 7f3bb6596640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb0110e30 con 0x7f3bb01009b0 2024-09-12T03:38:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.975+0000 7f3bb4b33640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bb01069c0 0x7f3bb019ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.975+0000 7f3bb4b33640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bb01069c0 0x7f3bb019ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47592/0 (socket says 172.21.15.64:47592) 2024-09-12T03:38:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.975+0000 7f3bb4b33640 1 -- 172.21.15.64:0/4949127 learned_addr learned my addr 172.21.15.64:0/4949127 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:38:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.975+0000 7f3baffff640 1 --2- 172.21.15.64:0/4949127 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3bb01009b0 0x7f3bb01971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.975+0000 7f3ba7fff640 1 --2- 172.21.15.64:0/4949127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bb0101350 0x7f3bb01976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.975+0000 7f3bb4b33640 1 -- 172.21.15.64:0/4949127 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3bb01009b0 msgr2=0x7f3bb01971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.975+0000 7f3bb4b33640 1 --2- 172.21.15.64:0/4949127 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3bb01009b0 0x7f3bb01971a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.975+0000 7f3bb4b33640 1 -- 172.21.15.64:0/4949127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bb0101350 msgr2=0x7f3bb01976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.975+0000 7f3bb4b33640 1 --2- 172.21.15.64:0/4949127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bb0101350 0x7f3bb01976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.976+0000 7f3bb4b33640 1 -- 172.21.15.64:0/4949127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb019c190 con 0x7f3bb01069c0 2024-09-12T03:38:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.976+0000 7f3ba7fff640 1 --2- 172.21.15.64:0/4949127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bb0101350 0x7f3bb01976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:38:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.976+0000 7f3bb4b33640 1 --2- 172.21.15.64:0/4949127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bb01069c0 0x7f3bb019ba60 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3ba800eaa0 tx=0x7f3ba800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:38:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.976+0000 7f3badffb640 1 -- 172.21.15.64:0/4949127 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba800cdb0 con 0x7f3bb01069c0 2024-09-12T03:38:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.976+0000 7f3badffb640 1 -- 172.21.15.64:0/4949127 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ba8005020 con 0x7f3bb01069c0 2024-09-12T03:38:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.976+0000 7f3badffb640 1 -- 172.21.15.64:0/4949127 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba80116d0 con 0x7f3bb01069c0 2024-09-12T03:38:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.976+0000 7f3bb6596640 1 -- 172.21.15.64:0/4949127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bb019c480 con 0x7f3bb01069c0 2024-09-12T03:38:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.976+0000 7f3bb6596640 1 -- 172.21.15.64:0/4949127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bb01a3d10 con 0x7f3bb01069c0 2024-09-12T03:38:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.979+0000 7f3badffb640 1 -- 172.21.15.64:0/4949127 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f3ba80051d0 con 0x7f3bb01069c0 2024-09-12T03:38:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.979+0000 7f3bb9408740 1 Processor -- start 2024-09-12T03:38:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.979+0000 7f3bb9408740 1 -- start start 2024-09-12T03:38:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.979+0000 7f3bb9408740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f257af0 0x563c0f257ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.979+0000 7f3bb9408740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563c0f258410 0x563c0f262770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.979+0000 7f3bb9408740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563c0f262cb0 0x563c0f2650a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.979+0000 7f3bb9408740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563c0f1bc2f0 con 0x563c0f257af0 2024-09-12T03:38:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.979+0000 7f3bb9408740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563c0f164270 con 0x563c0f258410 2024-09-12T03:38:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.979+0000 7f3bb9408740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563c0f1b82b0 con 0x563c0f262cb0 2024-09-12T03:38:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.979+0000 7f3ba7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f257af0 0x563c0f257ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.979+0000 7f3ba7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f257af0 0x563c0f257ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47608/0 (socket says 172.21.15.64:47608) 2024-09-12T03:38:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.979+0000 7f3ba7fff640 1 -- 172.21.15.64:0/1782640944 learned_addr learned my addr 172.21.15.64:0/1782640944 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:38:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.979+0000 7f3baffff640 1 --2- 172.21.15.64:0/1782640944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563c0f258410 0x563c0f262770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.979+0000 7f3ba7fff640 1 --2- 172.21.15.64:0/1782640944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563c0f262cb0 0x563c0f2650a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.980+0000 7f3ba7fff640 1 -- 172.21.15.64:0/1782640944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563c0f262cb0 msgr2=0x563c0f2650a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:38:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.980+0000 7f3ba7fff640 1 --2- 172.21.15.64:0/1782640944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563c0f262cb0 0x563c0f2650a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.980+0000 7f3ba7fff640 1 -- 172.21.15.64:0/1782640944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563c0f258410 msgr2=0x563c0f262770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:38:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.980+0000 7f3ba7fff640 1 --2- 172.21.15.64:0/1782640944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563c0f258410 0x563c0f262770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.980+0000 7f3ba7fff640 1 -- 172.21.15.64:0/1782640944 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c0f163340 con 0x563c0f257af0 2024-09-12T03:38:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.980+0000 7f3baffff640 1 --2- 172.21.15.64:0/1782640944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563c0f258410 0x563c0f262770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:38:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.980+0000 7f3badffb640 1 --2- 172.21.15.64:0/4949127 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f3b9c0a2e20 0x7f3b9c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.980+0000 7f3ba7fff640 1 --2- 172.21.15.64:0/1782640944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f257af0 0x563c0f257ed0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3b98005ec0 tx=0x7f3b98005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96025bf291be1103 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:38:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.980+0000 7f3baffff640 1 --2- 172.21.15.64:0/4949127 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f3b9c0a2e20 0x7f3b9c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.980+0000 7f3badffb640 1 -- 172.21.15.64:0/4949127 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9671+0+0 (secure 0 0 0) 0x7f3ba80cfad0 con 0x7f3bb01069c0 2024-09-12T03:38:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.980+0000 7f3ba67fc640 1 -- 172.21.15.64:0/1782640944 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b98035070 con 0x563c0f257af0 2024-09-12T03:38:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.980+0000 7f3ba67fc640 1 -- 172.21.15.64:0/1782640944 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b9803c070 con 0x563c0f257af0 2024-09-12T03:38:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.980+0000 7f3ba67fc640 1 -- 172.21.15.64:0/1782640944 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b980046c0 con 0x563c0f257af0 2024-09-12T03:38:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.981+0000 7f3bb9408740 1 -- 172.21.15.64:0/1782640944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f257af0 msgr2=0x563c0f257ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.981+0000 7f3baffff640 1 --2- 172.21.15.64:0/4949127 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f3b9c0a2e20 0x7f3b9c0a52e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0005d70 tx=0x7f3ba0005d00 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:38:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.981+0000 7f3bb9408740 1 --2- 172.21.15.64:0/1782640944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f257af0 0x563c0f257ed0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3b98005ec0 tx=0x7f3b98005b50 comp rx=0 tx=0).stop 2024-09-12T03:38:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.981+0000 7f3bb9408740 1 -- 172.21.15.64:0/1782640944 shutdown_connections 2024-09-12T03:38:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.981+0000 7f3bb9408740 1 --2- 172.21.15.64:0/1782640944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563c0f262cb0 0x563c0f2650a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.981+0000 7f3bb9408740 1 --2- 172.21.15.64:0/1782640944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563c0f258410 0x563c0f262770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.981+0000 7f3bb9408740 1 --2- 172.21.15.64:0/1782640944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f257af0 0x563c0f257ed0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.981+0000 7f3bb9408740 1 -- 172.21.15.64:0/1782640944 >> 172.21.15.64:0/1782640944 conn(0x563c0f254060 msgr2=0x563c0f254440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:38:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.981+0000 7f3bb9408740 1 -- 172.21.15.64:0/1782640944 shutdown_connections 2024-09-12T03:38:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.981+0000 7f3bb9408740 1 -- 172.21.15.64:0/1782640944 wait complete. 2024-09-12T03:38:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.981+0000 7f3bb9408740 1 -- start start 2024-09-12T03:38:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.982+0000 7f3bb9408740 1 Processor -- start 2024-09-12T03:38:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.982+0000 7f3bb9408740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563c0f257af0 0x563c0f25fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.982+0000 7f3bb9408740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563c0f258410 0x563c0f2600b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.982+0000 7f3bb9408740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f262cb0 0x563c0f2605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.982+0000 7f3bb9408740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563c0f265880 con 0x563c0f262cb0 2024-09-12T03:38:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.982+0000 7f3bb9408740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563c0f145290 con 0x563c0f257af0 2024-09-12T03:38:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.982+0000 7f3bb9408740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563c0f265a00 con 0x563c0f258410 2024-09-12T03:38:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.982+0000 7f3bb9408740 10 client.0 _ll_register_callbacks cb 0x563c0f176140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:38:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.982+0000 7f3bb4b33640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f262cb0 0x563c0f2605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.982+0000 7f3bb4b33640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f262cb0 0x563c0f2605f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47620/0 (socket says 172.21.15.64:47620) 2024-09-12T03:38:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.982+0000 7f3bb4b33640 1 -- 172.21.15.64:0/3253648935 learned_addr learned my addr 172.21.15.64:0/3253648935 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:38:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.982+0000 7f3ba7fff640 1 --2- 172.21.15.64:0/3253648935 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563c0f258410 0x563c0f2600b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.982+0000 7f3baffff640 1 --2- 172.21.15.64:0/3253648935 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563c0f257af0 0x563c0f25fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.982+0000 7f3bb4b33640 1 -- 172.21.15.64:0/3253648935 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563c0f258410 msgr2=0x563c0f2600b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:38:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.982+0000 7f3bb4b33640 1 --2- 172.21.15.64:0/3253648935 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563c0f258410 0x563c0f2600b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.982+0000 7f3bb4b33640 1 -- 172.21.15.64:0/3253648935 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563c0f257af0 msgr2=0x563c0f25fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:38:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.982+0000 7f3bb4b33640 1 --2- 172.21.15.64:0/3253648935 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563c0f257af0 0x563c0f25fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.982+0000 7f3bb4b33640 1 -- 172.21.15.64:0/3253648935 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c0eecf210 con 0x563c0f262cb0 2024-09-12T03:38:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.982+0000 7f3baffff640 1 --2- 172.21.15.64:0/3253648935 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563c0f257af0 0x563c0f25fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:38:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.983+0000 7f3bb4b33640 1 --2- 172.21.15.64:0/3253648935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f262cb0 0x563c0f2605f0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3ba80cce00 tx=0x7f3ba809c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ec9324cbae70706 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:38:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.983+0000 7f3b8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:38:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.983+0000 7f3b8f7fe640 1 -- 172.21.15.64:0/3253648935 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba801f070 con 0x563c0f262cb0 2024-09-12T03:38:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.983+0000 7f3b8f7fe640 1 -- 172.21.15.64:0/3253648935 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9671+0+0 (secure 0 0 0) 0x7f3ba809afc0 con 0x563c0f262cb0 2024-09-12T03:38:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.983+0000 7f3bb9408740 1 -- 172.21.15.64:0/3253648935 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b9803c070 con 0x563c0f262cb0 2024-09-12T03:38:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.983+0000 7f3b8f7fe640 1 -- 172.21.15.64:0/3253648935 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3ba809b1c0 con 0x563c0f262cb0 2024-09-12T03:38:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.983+0000 7f3bb9408740 10 client.16062 fetch_fsmap learned FSMap version 25 2024-09-12T03:38:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.983+0000 7f3bb9408740 1 -- 172.21.15.64:0/3253648935 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563c0ef9b620 con 0x563c0f262cb0 2024-09-12T03:38:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.983+0000 7f3b8f7fe640 1 -- 172.21.15.64:0/3253648935 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b9803c070 con 0x563c0f262cb0 2024-09-12T03:38:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.983+0000 7f3b8f7fe640 1 -- 172.21.15.64:0/3253648935 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f3ba800f040 con 0x563c0f262cb0 2024-09-12T03:38:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.983+0000 7f3bb9408740 10 client.16062 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:38:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.983+0000 7f3bb9408740 10 client.16062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:38:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.983+0000 7f3bb9408740 20 client.16062 populate_metadata read hostname 'smithi064' 2024-09-12T03:38:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.983+0000 7f3bb9408740 1 --2- 172.21.15.64:0/3253648935 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x563c0f272cb0 0x563c0f2750c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.983+0000 7f3bb9408740 4 client.16062 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:38:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.983+0000 7f3bb9408740 1 -- 172.21.15.64:0/3253648935 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c0f1c9a20 con 0x563c0f272cb0 2024-09-12T03:38:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.984+0000 7f3baffff640 1 --2- 172.21.15.64:0/3253648935 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x563c0f272cb0 0x563c0f2750c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.984+0000 7f3baffff640 1 --2- 172.21.15.64:0/3253648935 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x563c0f272cb0 0x563c0f2750c0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15899e0825a61e0e server_cookie=eae4d9a5818a5d6b in_seq=0 out_seq=0 2024-09-12T03:38:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.984+0000 7f3b8f7fe640 10 client.16062 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:38:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.985+0000 7f3b8f7fe640 1 -- 172.21.15.64:0/3253648935 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563c0f1c9a20 con 0x563c0f272cb0 2024-09-12T03:38:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.985+0000 7f3b8f7fe640 10 client.16062 handle_command_reply: tid=0 2024-09-12T03:38:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.985+0000 7f3bb9408740 1 client.16062 shutdown 2024-09-12T03:38:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.985+0000 7f3bb9408740 1 -- 172.21.15.64:0/3253648935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f262cb0 msgr2=0x563c0f2605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.985+0000 7f3bb9408740 1 --2- 172.21.15.64:0/3253648935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f262cb0 0x563c0f2605f0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3ba80cce00 tx=0x7f3ba809c3b0 comp rx=0 tx=0).stop 2024-09-12T03:38:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.985+0000 7f3bb9408740 1 -- 172.21.15.64:0/3253648935 shutdown_connections 2024-09-12T03:38:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.985+0000 7f3bb9408740 1 --2- 172.21.15.64:0/3253648935 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x563c0f272cb0 0x563c0f2750c0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.985+0000 7f3bb9408740 1 --2- 172.21.15.64:0/3253648935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f262cb0 0x563c0f2605f0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.985+0000 7f3bb9408740 1 --2- 172.21.15.64:0/3253648935 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563c0f258410 0x563c0f2600b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.985+0000 7f3b8f7fe640 0 client.16062 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:38:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.985+0000 7f3bb9408740 1 --2- 172.21.15.64:0/3253648935 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563c0f257af0 0x563c0f25fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.985+0000 7f3bb9408740 1 -- 172.21.15.64:0/3253648935 >> 172.21.15.64:0/3253648935 conn(0x563c0f254060 msgr2=0x563c0f259dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:38:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.985+0000 7f3bb9408740 1 -- 172.21.15.64:0/3253648935 shutdown_connections 2024-09-12T03:38:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.985+0000 7f3bb9408740 1 -- 172.21.15.64:0/3253648935 wait complete. 2024-09-12T03:38:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:03.985+0000 7f3bb9408740 20 client.16062 trim_cache size 0 max 16384 2024-09-12T03:38:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.000+0000 7f3bb9408740 1 Processor -- start 2024-09-12T03:38:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.000+0000 7f3bb9408740 1 -- start start 2024-09-12T03:38:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.000+0000 7f3bb9408740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563c0f257af0 0x563c0f25a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.000+0000 7f3bb9408740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563c0f258410 0x563c0f260420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.000+0000 7f3bb9408740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f262cb0 0x563c0f27a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.000+0000 7f3bb9408740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb01af890 con 0x563c0f262cb0 2024-09-12T03:38:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.000+0000 7f3bb9408740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb01ad160 con 0x563c0f258410 2024-09-12T03:38:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.000+0000 7f3bb9408740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb01aafd0 con 0x563c0f257af0 2024-09-12T03:38:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.000+0000 7f3bb4b33640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f262cb0 0x563c0f27a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.001+0000 7f3bb4b33640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f262cb0 0x563c0f27a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47626/0 (socket says 172.21.15.64:47626) 2024-09-12T03:38:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.001+0000 7f3bb4b33640 1 -- 172.21.15.64:0/706502960 learned_addr learned my addr 172.21.15.64:0/706502960 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:38:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.001+0000 7f3ba7fff640 1 --2- 172.21.15.64:0/706502960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563c0f258410 0x563c0f260420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.001+0000 7f3baffff640 1 --2- 172.21.15.64:0/706502960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563c0f257af0 0x563c0f25a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.001+0000 7f3bb4b33640 1 -- 172.21.15.64:0/706502960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563c0f257af0 msgr2=0x563c0f25a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:38:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.001+0000 7f3bb4b33640 1 --2- 172.21.15.64:0/706502960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563c0f257af0 0x563c0f25a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.001+0000 7f3bb4b33640 1 -- 172.21.15.64:0/706502960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563c0f258410 msgr2=0x563c0f260420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:38:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.001+0000 7f3bb4b33640 1 --2- 172.21.15.64:0/706502960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563c0f258410 0x563c0f260420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.001+0000 7f3bb4b33640 1 -- 172.21.15.64:0/706502960 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c0f275740 con 0x563c0f262cb0 2024-09-12T03:38:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.001+0000 7f3baffff640 1 --2- 172.21.15.64:0/706502960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563c0f257af0 0x563c0f25a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:38:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.001+0000 7f3bb4b33640 1 --2- 172.21.15.64:0/706502960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f262cb0 0x563c0f27a9d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f3ba80cd3a0 tx=0x7f3ba8005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3195443fb7490ae server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:38:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.001+0000 7f3ba5ffb640 1 -- 172.21.15.64:0/706502960 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba801f040 con 0x563c0f262cb0 2024-09-12T03:38:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.001+0000 7f3ba5ffb640 1 -- 172.21.15.64:0/706502960 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ba80981e0 con 0x563c0f262cb0 2024-09-12T03:38:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.001+0000 7f3ba5ffb640 1 -- 172.21.15.64:0/706502960 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba809ac90 con 0x563c0f262cb0 2024-09-12T03:38:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.002+0000 7f3bb9408740 1 -- 172.21.15.64:0/706502960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f262cb0 msgr2=0x563c0f27a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.002+0000 7f3bb9408740 1 --2- 172.21.15.64:0/706502960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f262cb0 0x563c0f27a9d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f3ba80cd3a0 tx=0x7f3ba8005770 comp rx=0 tx=0).stop 2024-09-12T03:38:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.002+0000 7f3bb9408740 1 -- 172.21.15.64:0/706502960 shutdown_connections 2024-09-12T03:38:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.002+0000 7f3bb9408740 1 --2- 172.21.15.64:0/706502960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f262cb0 0x563c0f27a9d0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.002+0000 7f3bb9408740 1 --2- 172.21.15.64:0/706502960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563c0f258410 0x563c0f260420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.002+0000 7f3bb9408740 1 --2- 172.21.15.64:0/706502960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563c0f257af0 0x563c0f25a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.002+0000 7f3bb9408740 1 -- 172.21.15.64:0/706502960 >> 172.21.15.64:0/706502960 conn(0x563c0f254060 msgr2=0x563c0f261bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:38:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.002+0000 7f3bb9408740 1 -- 172.21.15.64:0/706502960 shutdown_connections 2024-09-12T03:38:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.002+0000 7f3bb9408740 1 -- 172.21.15.64:0/706502960 wait complete. 2024-09-12T03:38:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.002+0000 7f3bb9408740 1 -- start start 2024-09-12T03:38:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.002+0000 7f3bb9408740 1 Processor -- start 2024-09-12T03:38:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.003+0000 7f3bb9408740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563c0f257af0 0x563c0f26b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.003+0000 7f3bb9408740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563c0f258410 0x563c0f266810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.003+0000 7f3bb9408740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f262cb0 0x563c0f266d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.003+0000 7f3bb9408740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb01ab620 con 0x563c0f262cb0 2024-09-12T03:38:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.003+0000 7f3bb9408740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb01aca90 con 0x563c0f258410 2024-09-12T03:38:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.003+0000 7f3bb9408740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb01abcf0 con 0x563c0f257af0 2024-09-12T03:38:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.003+0000 7f3bb9408740 10 client.0 _ll_register_callbacks cb 0x7f3ba8097e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:38:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.003+0000 7f3bb4b33640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f262cb0 0x563c0f266d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.003+0000 7f3bb4b33640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f262cb0 0x563c0f266d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47628/0 (socket says 172.21.15.64:47628) 2024-09-12T03:38:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.003+0000 7f3bb4b33640 1 -- 172.21.15.64:0/2076486449 learned_addr learned my addr 172.21.15.64:0/2076486449 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:38:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.003+0000 7f3ba7fff640 1 --2- 172.21.15.64:0/2076486449 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563c0f258410 0x563c0f266810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.003+0000 7f3baffff640 1 --2- 172.21.15.64:0/2076486449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563c0f257af0 0x563c0f26b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.003+0000 7f3bb4b33640 1 -- 172.21.15.64:0/2076486449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563c0f257af0 msgr2=0x563c0f26b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:38:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.003+0000 7f3bb4b33640 1 --2- 172.21.15.64:0/2076486449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563c0f257af0 0x563c0f26b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.003+0000 7f3bb4b33640 1 -- 172.21.15.64:0/2076486449 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563c0f258410 msgr2=0x563c0f266810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:38:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.003+0000 7f3bb4b33640 1 --2- 172.21.15.64:0/2076486449 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563c0f258410 0x563c0f266810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.003+0000 7f3bb4b33640 1 -- 172.21.15.64:0/2076486449 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c0f267550 con 0x563c0f262cb0 2024-09-12T03:38:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.003+0000 7f3baffff640 1 --2- 172.21.15.64:0/2076486449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563c0f257af0 0x563c0f26b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:38:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.003+0000 7f3bb4b33640 1 --2- 172.21.15.64:0/2076486449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f262cb0 0x563c0f266d50 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8097fd0 tx=0x7f3ba809b530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=217edb7818fa7e58 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:38:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.003+0000 7f3b8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:38:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.003+0000 7f3b8f7fe640 1 -- 172.21.15.64:0/2076486449 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba801f070 con 0x563c0f262cb0 2024-09-12T03:38:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.004+0000 7f3b8f7fe640 1 -- 172.21.15.64:0/2076486449 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9671+0+0 (secure 0 0 0) 0x7f3ba80a12a0 con 0x563c0f262cb0 2024-09-12T03:38:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.004+0000 7f3bb9408740 1 -- 172.21.15.64:0/2076486449 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ba80981e0 con 0x563c0f262cb0 2024-09-12T03:38:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.004+0000 7f3b8f7fe640 1 -- 172.21.15.64:0/2076486449 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3ba809c270 con 0x563c0f262cb0 2024-09-12T03:38:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.004+0000 7f3bb9408740 10 client.16068 fetch_fsmap learned FSMap version 25 2024-09-12T03:38:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.004+0000 7f3bb9408740 1 -- 172.21.15.64:0/2076486449 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563c0f2dd950 con 0x563c0f262cb0 2024-09-12T03:38:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.004+0000 7f3b8f7fe640 1 -- 172.21.15.64:0/2076486449 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ba80981e0 con 0x563c0f262cb0 2024-09-12T03:38:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.004+0000 7f3b8f7fe640 1 -- 172.21.15.64:0/2076486449 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f3ba80a4c60 con 0x563c0f262cb0 2024-09-12T03:38:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.004+0000 7f3bb9408740 10 client.16068 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:38:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.004+0000 7f3bb9408740 10 client.16068 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:38:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.004+0000 7f3bb9408740 20 client.16068 populate_metadata read hostname 'smithi064' 2024-09-12T03:38:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.004+0000 7f3bb9408740 1 --2- 172.21.15.64:0/2076486449 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x563c0f2dddd0 0x563c0f2e01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.004+0000 7f3bb9408740 4 client.16068 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:38:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.004+0000 7f3bb9408740 1 -- 172.21.15.64:0/2076486449 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563c0f1c3d70 con 0x563c0f2dddd0 2024-09-12T03:38:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.004+0000 7f3baffff640 1 --2- 172.21.15.64:0/2076486449 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x563c0f2dddd0 0x563c0f2e01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.005+0000 7f3baffff640 1 --2- 172.21.15.64:0/2076486449 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x563c0f2dddd0 0x563c0f2e01c0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b02e498b1d062516 server_cookie=72af95d3806e4785 in_seq=0 out_seq=0 2024-09-12T03:38:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.005+0000 7f3b8f7fe640 10 client.16068 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:38:04.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.018+0000 7f3b8f7fe640 1 -- 172.21.15.64:0/2076486449 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563c0f1c3d70 con 0x563c0f2dddd0 2024-09-12T03:38:04.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.018+0000 7f3b8f7fe640 10 client.16068 handle_command_reply: tid=0 2024-09-12T03:38:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.018+0000 7f3bb9408740 1 client.16068 shutdown 2024-09-12T03:38:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.018+0000 7f3bb9408740 1 -- 172.21.15.64:0/2076486449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f262cb0 msgr2=0x563c0f266d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.018+0000 7f3bb9408740 1 --2- 172.21.15.64:0/2076486449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f262cb0 0x563c0f266d50 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8097fd0 tx=0x7f3ba809b530 comp rx=0 tx=0).stop 2024-09-12T03:38:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.018+0000 7f3bb9408740 1 -- 172.21.15.64:0/2076486449 shutdown_connections 2024-09-12T03:38:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.018+0000 7f3bb9408740 1 --2- 172.21.15.64:0/2076486449 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x563c0f2dddd0 0x563c0f2e01c0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.018+0000 7f3bb9408740 1 --2- 172.21.15.64:0/2076486449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c0f262cb0 0x563c0f266d50 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.018+0000 7f3bb9408740 1 --2- 172.21.15.64:0/2076486449 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563c0f258410 0x563c0f266810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.018+0000 7f3b8f7fe640 0 client.16068 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:38:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.018+0000 7f3bb9408740 1 --2- 172.21.15.64:0/2076486449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563c0f257af0 0x563c0f26b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.018+0000 7f3bb9408740 1 -- 172.21.15.64:0/2076486449 >> 172.21.15.64:0/2076486449 conn(0x563c0f254060 msgr2=0x563c0f2613d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:38:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.019+0000 7f3bb9408740 1 -- 172.21.15.64:0/2076486449 shutdown_connections 2024-09-12T03:38:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.019+0000 7f3bb9408740 1 -- 172.21.15.64:0/2076486449 wait complete. 2024-09-12T03:38:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.019+0000 7f3bb9408740 20 client.16068 trim_cache size 0 max 16384 2024-09-12T03:38:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:38:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:38:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:38:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:38:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.019+0000 7f3bb6596640 1 -- 172.21.15.64:0/4949127 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f3b9c0a2e20 msgr2=0x7f3b9c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.019+0000 7f3bb6596640 1 --2- 172.21.15.64:0/4949127 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f3b9c0a2e20 0x7f3b9c0a52e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0005d70 tx=0x7f3ba0005d00 comp rx=0 tx=0).stop 2024-09-12T03:38:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.019+0000 7f3bb6596640 1 -- 172.21.15.64:0/4949127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bb01069c0 msgr2=0x7f3bb019ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.019+0000 7f3bb6596640 1 --2- 172.21.15.64:0/4949127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bb01069c0 0x7f3bb019ba60 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3ba800eaa0 tx=0x7f3ba800ef60 comp rx=0 tx=0).stop 2024-09-12T03:38:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.019+0000 7f3bb6596640 1 -- 172.21.15.64:0/4949127 shutdown_connections 2024-09-12T03:38:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.019+0000 7f3bb6596640 1 --2- 172.21.15.64:0/4949127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bb01069c0 0x7f3bb019ba60 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.019+0000 7f3bb6596640 1 --2- 172.21.15.64:0/4949127 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f3b9c0a2e20 0x7f3b9c0a52e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.019+0000 7f3bb6596640 1 --2- 172.21.15.64:0/4949127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bb0101350 0x7f3bb01976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.019+0000 7f3bb6596640 1 --2- 172.21.15.64:0/4949127 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3bb01009b0 0x7f3bb01971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.019+0000 7f3bb6596640 1 -- 172.21.15.64:0/4949127 >> 172.21.15.64:0/4949127 conn(0x7f3bb0077990 msgr2=0x7f3bb0103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:38:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.019+0000 7f3bb6596640 1 -- 172.21.15.64:0/4949127 shutdown_connections 2024-09-12T03:38:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:04.019+0000 7f3bb6596640 1 -- 172.21.15.64:0/4949127 wait complete. 2024-09-12T03:38:04.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:38:04 smithi195 ceph-mon[34786]: osdmap e98: 12 total, 12 up, 12 in 2024-09-12T03:38:04.521 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:38:04 smithi064 ceph-mon[30952]: osdmap e98: 12 total, 12 up, 12 in 2024-09-12T03:38:04.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:38:04 smithi115 ceph-mon[34801]: osdmap e98: 12 total, 12 up, 12 in 2024-09-12T03:38:05.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:38:05 smithi195 ceph-mon[34786]: pgmap v272: 129 pgs: 129 active+clean; 3.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 93 KiB/s wr, 32 op/s 2024-09-12T03:38:05.521 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:38:05 smithi064 ceph-mon[30952]: pgmap v272: 129 pgs: 129 active+clean; 3.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 93 KiB/s wr, 32 op/s 2024-09-12T03:38:05.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:38:05 smithi115 ceph-mon[34801]: pgmap v272: 129 pgs: 129 active+clean; 3.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 93 KiB/s wr, 32 op/s 2024-09-12T03:38:07.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:38:07 smithi195 ceph-mon[34786]: pgmap v273: 129 pgs: 129 active+clean; 3.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 93 KiB/s wr, 32 op/s 2024-09-12T03:38:07.521 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:38:07 smithi064 ceph-mon[30952]: pgmap v273: 129 pgs: 129 active+clean; 3.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 93 KiB/s wr, 32 op/s 2024-09-12T03:38:07.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:38:07 smithi115 ceph-mon[34801]: pgmap v273: 129 pgs: 129 active+clean; 3.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 93 KiB/s wr, 32 op/s 2024-09-12T03:38:09.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:38:09 smithi195 ceph-mon[34786]: pgmap v274: 129 pgs: 129 active+clean; 3.8 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 105 KiB/s wr, 36 op/s 2024-09-12T03:38:09.521 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:38:09 smithi064 ceph-mon[30952]: pgmap v274: 129 pgs: 129 active+clean; 3.8 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 105 KiB/s wr, 36 op/s 2024-09-12T03:38:09.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:38:09 smithi115 ceph-mon[34801]: pgmap v274: 129 pgs: 129 active+clean; 3.8 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 105 KiB/s wr, 36 op/s 2024-09-12T03:38:11.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:38:11 smithi195 ceph-mon[34786]: pgmap v275: 129 pgs: 129 active+clean; 3.8 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 105 KiB/s wr, 36 op/s 2024-09-12T03:38:11.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:38:11 smithi195 ceph-mon[34786]: osdmap e99: 12 total, 12 up, 12 in 2024-09-12T03:38:11.521 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:38:11 smithi064 ceph-mon[30952]: pgmap v275: 129 pgs: 129 active+clean; 3.8 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 105 KiB/s wr, 36 op/s 2024-09-12T03:38:11.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:38:11 smithi064 ceph-mon[30952]: osdmap e99: 12 total, 12 up, 12 in 2024-09-12T03:38:11.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:38:11 smithi115 ceph-mon[34801]: pgmap v275: 129 pgs: 129 active+clean; 3.8 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 105 KiB/s wr, 36 op/s 2024-09-12T03:38:11.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:38:11 smithi115 ceph-mon[34801]: osdmap e99: 12 total, 12 up, 12 in 2024-09-12T03:38:13.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:38:13 smithi195 ceph-mon[34786]: pgmap v277: 129 pgs: 129 active+clean; 4.0 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-09-12T03:38:13.521 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:38:13 smithi064 ceph-mon[30952]: pgmap v277: 129 pgs: 129 active+clean; 4.0 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-09-12T03:38:13.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:38:13 smithi115 ceph-mon[34801]: pgmap v277: 129 pgs: 129 active+clean; 4.0 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-09-12T03:38:15.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:38:15 smithi195 ceph-mon[34786]: pgmap v278: 129 pgs: 129 active+clean; 4.0 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-09-12T03:38:15.521 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:38:15 smithi064 ceph-mon[30952]: pgmap v278: 129 pgs: 129 active+clean; 4.0 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-09-12T03:38:15.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:38:15 smithi115 ceph-mon[34801]: pgmap v278: 129 pgs: 129 active+clean; 4.0 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-09-12T03:38:17.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.120+0000 7ff1ba1d8640 1 -- 172.21.15.64:0/3643831943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1b410c110 msgr2=0x7ff1b410e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:17.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.120+0000 7ff1ba1d8640 1 --2- 172.21.15.64:0/3643831943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1b410c110 0x7ff1b410e590 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac009ca0 tx=0x7ff1ac02a950 comp rx=0 tx=0).stop 2024-09-12T03:38:17.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.120+0000 7ff1ba1d8640 1 -- 172.21.15.64:0/3643831943 shutdown_connections 2024-09-12T03:38:17.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.120+0000 7ff1ba1d8640 1 --2- 172.21.15.64:0/3643831943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1b410c110 0x7ff1b410e590 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:17.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.120+0000 7ff1ba1d8640 1 --2- 172.21.15.64:0/3643831943 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1b41035e0 0x7ff1b410bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:17.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.120+0000 7ff1ba1d8640 1 --2- 172.21.15.64:0/3643831943 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1b4102cc0 0x7ff1b41030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:17.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.120+0000 7ff1ba1d8640 1 -- 172.21.15.64:0/3643831943 >> 172.21.15.64:0/3643831943 conn(0x7ff1b40fc6b0 msgr2=0x7ff1b40fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:38:17.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.121+0000 7ff1ba1d8640 1 -- 172.21.15.64:0/3643831943 shutdown_connections 2024-09-12T03:38:17.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.121+0000 7ff1ba1d8640 1 -- 172.21.15.64:0/3643831943 wait complete. 2024-09-12T03:38:17.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.121+0000 7ff1ba1d8640 1 Processor -- start 2024-09-12T03:38:17.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.121+0000 7ff1ba1d8640 1 -- start start 2024-09-12T03:38:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.122+0000 7ff1ba1d8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1b4102cc0 0x7ff1b4198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.122+0000 7ff1ba1d8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1b41035e0 0x7ff1b4198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.122+0000 7ff1ba1d8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1b410c110 0x7ff1b419cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.122+0000 7ff1ba1d8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b4111db0 con 0x7ff1b4102cc0 2024-09-12T03:38:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.122+0000 7ff1ba1d8640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b4111c30 con 0x7ff1b410c110 2024-09-12T03:38:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.122+0000 7ff1ba1d8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b4111f30 con 0x7ff1b41035e0 2024-09-12T03:38:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.122+0000 7ff1b37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1b4102cc0 0x7ff1b4198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.122+0000 7ff1b37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1b4102cc0 0x7ff1b4198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39962/0 (socket says 172.21.15.64:39962) 2024-09-12T03:38:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.122+0000 7ff1b37fe640 1 -- 172.21.15.64:0/2631491670 learned_addr learned my addr 172.21.15.64:0/2631491670 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:38:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.122+0000 7ff1b3fff640 1 --2- 172.21.15.64:0/2631491670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1b410c110 0x7ff1b419cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.122+0000 7ff1b2ffd640 1 --2- 172.21.15.64:0/2631491670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1b41035e0 0x7ff1b4198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.122+0000 7ff1b37fe640 1 -- 172.21.15.64:0/2631491670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1b41035e0 msgr2=0x7ff1b4198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.122+0000 7ff1b37fe640 1 --2- 172.21.15.64:0/2631491670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1b41035e0 0x7ff1b4198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:17.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.122+0000 7ff1b37fe640 1 -- 172.21.15.64:0/2631491670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1b410c110 msgr2=0x7ff1b419cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:17.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.122+0000 7ff1b37fe640 1 --2- 172.21.15.64:0/2631491670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1b410c110 0x7ff1b419cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:17.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.122+0000 7ff1b37fe640 1 -- 172.21.15.64:0/2631491670 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b419d210 con 0x7ff1b4102cc0 2024-09-12T03:38:17.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.122+0000 7ff1b3fff640 1 --2- 172.21.15.64:0/2631491670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1b410c110 0x7ff1b419cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:38:17.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.122+0000 7ff1b37fe640 1 --2- 172.21.15.64:0/2631491670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1b4102cc0 0x7ff1b4198220 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff1a800d950 tx=0x7ff1a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:38:17.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.122+0000 7ff1b0ff9640 1 -- 172.21.15.64:0/2631491670 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8017030 con 0x7ff1b4102cc0 2024-09-12T03:38:17.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.123+0000 7ff1b0ff9640 1 -- 172.21.15.64:0/2631491670 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1a8004020 con 0x7ff1b4102cc0 2024-09-12T03:38:17.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.123+0000 7ff1ba1d8640 1 -- 172.21.15.64:0/2631491670 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1b419d500 con 0x7ff1b4102cc0 2024-09-12T03:38:17.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.123+0000 7ff1b0ff9640 1 -- 172.21.15.64:0/2631491670 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a80047a0 con 0x7ff1b4102cc0 2024-09-12T03:38:17.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.123+0000 7ff1ba1d8640 1 -- 172.21.15.64:0/2631491670 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1b41a4d90 con 0x7ff1b4102cc0 2024-09-12T03:38:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.125+0000 7ff1b0ff9640 1 -- 172.21.15.64:0/2631491670 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ff1a80041d0 con 0x7ff1b4102cc0 2024-09-12T03:38:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.125+0000 7ff1bd048740 1 Processor -- start 2024-09-12T03:38:17.132 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.125+0000 7ff1bd048740 1 -- start start 2024-09-12T03:38:17.132 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.125+0000 7ff1bd048740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7fb399af0 0x55a7fb399ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:17.132 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.125+0000 7ff1bd048740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb39a410 0x55a7fb3a4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:17.132 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.125+0000 7ff1bd048740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb3a4cb0 0x55a7fb3a70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:17.132 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.125+0000 7ff1bd048740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a7fb2fe2f0 con 0x55a7fb399af0 2024-09-12T03:38:17.132 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.125+0000 7ff1bd048740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a7fb2a6270 con 0x55a7fb3a4cb0 2024-09-12T03:38:17.132 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.126+0000 7ff1bd048740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a7fb2fa2b0 con 0x55a7fb39a410 2024-09-12T03:38:17.132 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.126+0000 7ff1b3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7fb399af0 0x55a7fb399ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:17.132 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.126+0000 7ff1b3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7fb399af0 0x55a7fb399ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39978/0 (socket says 172.21.15.64:39978) 2024-09-12T03:38:17.132 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.126+0000 7ff1b3fff640 1 -- 172.21.15.64:0/2747233342 learned_addr learned my addr 172.21.15.64:0/2747233342 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:38:17.132 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.126+0000 7ff1b0ff9640 1 --2- 172.21.15.64:0/2631491670 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7ff19c0a2e20 0x7ff19c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:17.133 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.126+0000 7ff1b37fe640 1 --2- 172.21.15.64:0/2747233342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb39a410 0x55a7fb3a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:17.133 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.126+0000 7ff1b2ffd640 1 --2- 172.21.15.64:0/2747233342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb3a4cb0 0x55a7fb3a70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:17.133 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.126+0000 7ff1b3fff640 1 --2- 172.21.15.64:0/2631491670 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7ff19c0a2e20 0x7ff19c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:17.133 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.126+0000 7ff1b3fff640 1 -- 172.21.15.64:0/2747233342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb39a410 msgr2=0x55a7fb3a4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:38:17.133 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.126+0000 7ff1b3fff640 1 --2- 172.21.15.64:0/2747233342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb39a410 0x55a7fb3a4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:17.133 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.126+0000 7ff1b3fff640 1 -- 172.21.15.64:0/2747233342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb3a4cb0 msgr2=0x55a7fb3a70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:38:17.133 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.126+0000 7ff1b3fff640 1 --2- 172.21.15.64:0/2747233342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb3a4cb0 0x55a7fb3a70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:17.133 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.126+0000 7ff1b3fff640 1 -- 172.21.15.64:0/2747233342 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7fb2a5340 con 0x55a7fb399af0 2024-09-12T03:38:17.133 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.126+0000 7ff1b0ff9640 1 -- 172.21.15.64:0/2631491670 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9583+0+0 (secure 0 0 0) 0x7ff1a80cf310 con 0x7ff1b4102cc0 2024-09-12T03:38:17.133 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.126+0000 7ff1b3fff640 1 --2- 172.21.15.64:0/2747233342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7fb399af0 0x55a7fb399ed0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac003da0 tx=0x7ff1ac003dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6879ff36a11436e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:38:17.133 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.127+0000 7ff1b3fff640 1 --2- 172.21.15.64:0/2631491670 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7ff19c0a2e20 0x7ff19c0a52e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac003e40 tx=0x7ff1ac049000 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:38:17.133 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.127+0000 7ff199ffb640 1 -- 172.21.15.64:0/2747233342 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac04f070 con 0x55a7fb399af0 2024-09-12T03:38:17.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.127+0000 7ff199ffb640 1 -- 172.21.15.64:0/2747233342 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1ac0046f0 con 0x55a7fb399af0 2024-09-12T03:38:17.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.127+0000 7ff199ffb640 1 -- 172.21.15.64:0/2747233342 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac034040 con 0x55a7fb399af0 2024-09-12T03:38:17.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.127+0000 7ff1bd048740 1 -- 172.21.15.64:0/2747233342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7fb399af0 msgr2=0x55a7fb399ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:17.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.127+0000 7ff1bd048740 1 --2- 172.21.15.64:0/2747233342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7fb399af0 0x55a7fb399ed0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac003da0 tx=0x7ff1ac003dd0 comp rx=0 tx=0).stop 2024-09-12T03:38:17.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.127+0000 7ff1bd048740 1 -- 172.21.15.64:0/2747233342 shutdown_connections 2024-09-12T03:38:17.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.127+0000 7ff1bd048740 1 --2- 172.21.15.64:0/2747233342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb3a4cb0 0x55a7fb3a70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:17.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.127+0000 7ff1bd048740 1 --2- 172.21.15.64:0/2747233342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb39a410 0x55a7fb3a4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:17.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.127+0000 7ff1bd048740 1 --2- 172.21.15.64:0/2747233342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7fb399af0 0x55a7fb399ed0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:17.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.127+0000 7ff1bd048740 1 -- 172.21.15.64:0/2747233342 >> 172.21.15.64:0/2747233342 conn(0x55a7fb396060 msgr2=0x55a7fb396440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:38:17.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.127+0000 7ff1bd048740 1 -- 172.21.15.64:0/2747233342 shutdown_connections 2024-09-12T03:38:17.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.127+0000 7ff1bd048740 1 -- 172.21.15.64:0/2747233342 wait complete. 2024-09-12T03:38:17.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.128+0000 7ff1bd048740 1 -- start start 2024-09-12T03:38:17.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.128+0000 7ff1bd048740 1 Processor -- start 2024-09-12T03:38:17.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.128+0000 7ff1bd048740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb399af0 0x55a7fb3a1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:17.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.128+0000 7ff1bd048740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb39a410 0x55a7fb3a20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:17.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.128+0000 7ff1bd048740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7fb3a4cb0 0x55a7fb3a25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:17.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.128+0000 7ff1bd048740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a7fb3a7880 con 0x55a7fb3a4cb0 2024-09-12T03:38:17.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.128+0000 7ff1bd048740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a7fb287290 con 0x55a7fb39a410 2024-09-12T03:38:17.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.128+0000 7ff1bd048740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a7fb3a7a00 con 0x55a7fb399af0 2024-09-12T03:38:17.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.128+0000 7ff1bd048740 10 client.0 _ll_register_callbacks cb 0x55a7fb2b8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:38:17.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.129+0000 7ff1b3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7fb3a4cb0 0x55a7fb3a25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:17.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.129+0000 7ff1b3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7fb3a4cb0 0x55a7fb3a25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39986/0 (socket says 172.21.15.64:39986) 2024-09-12T03:38:17.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.129+0000 7ff1b3fff640 1 -- 172.21.15.64:0/1083247922 learned_addr learned my addr 172.21.15.64:0/1083247922 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:38:17.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.129+0000 7ff1b37fe640 1 --2- 172.21.15.64:0/1083247922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb399af0 0x55a7fb3a1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:17.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.129+0000 7ff1b2ffd640 1 --2- 172.21.15.64:0/1083247922 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb39a410 0x55a7fb3a20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:17.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.129+0000 7ff1b3fff640 1 -- 172.21.15.64:0/1083247922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb399af0 msgr2=0x55a7fb3a1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:38:17.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.129+0000 7ff1b3fff640 1 --2- 172.21.15.64:0/1083247922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb399af0 0x55a7fb3a1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:17.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.129+0000 7ff1b3fff640 1 -- 172.21.15.64:0/1083247922 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb39a410 msgr2=0x55a7fb3a20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:38:17.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.129+0000 7ff1b3fff640 1 --2- 172.21.15.64:0/1083247922 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb39a410 0x55a7fb3a20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:17.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.129+0000 7ff1b3fff640 1 -- 172.21.15.64:0/1083247922 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7fb011210 con 0x55a7fb3a4cb0 2024-09-12T03:38:17.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.129+0000 7ff1b37fe640 1 --2- 172.21.15.64:0/1083247922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb399af0 0x55a7fb3a1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:38:17.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.129+0000 7ff1b3fff640 1 --2- 172.21.15.64:0/1083247922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7fb3a4cb0 0x55a7fb3a25e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac049f50 tx=0x7ff1ac049f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec81fb6be13eb5f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:38:17.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.129+0000 7ff192ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:38:17.136 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.129+0000 7ff192ffd640 1 -- 172.21.15.64:0/1083247922 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac056070 con 0x55a7fb3a4cb0 2024-09-12T03:38:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.129+0000 7ff192ffd640 1 -- 172.21.15.64:0/1083247922 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9583+0+0 (secure 0 0 0) 0x7ff1ac004380 con 0x55a7fb3a4cb0 2024-09-12T03:38:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.129+0000 7ff1bd048740 1 -- 172.21.15.64:0/1083247922 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1ac0046f0 con 0x55a7fb3a4cb0 2024-09-12T03:38:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.129+0000 7ff192ffd640 1 -- 172.21.15.64:0/1083247922 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff1ac062050 con 0x55a7fb3a4cb0 2024-09-12T03:38:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.129+0000 7ff1bd048740 10 client.16080 fetch_fsmap learned FSMap version 25 2024-09-12T03:38:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.129+0000 7ff1bd048740 1 -- 172.21.15.64:0/1083247922 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a7fb0dd620 con 0x55a7fb3a4cb0 2024-09-12T03:38:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.130+0000 7ff192ffd640 1 -- 172.21.15.64:0/1083247922 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1ac0046f0 con 0x55a7fb3a4cb0 2024-09-12T03:38:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.130+0000 7ff192ffd640 1 -- 172.21.15.64:0/1083247922 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff1ac039970 con 0x55a7fb3a4cb0 2024-09-12T03:38:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.130+0000 7ff1bd048740 10 client.16080 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:38:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.130+0000 7ff1bd048740 10 client.16080 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:38:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.130+0000 7ff1bd048740 20 client.16080 populate_metadata read hostname 'smithi064' 2024-09-12T03:38:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.130+0000 7ff1bd048740 1 --2- 172.21.15.64:0/1083247922 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55a7fb3b4a50 0x55a7fb3b6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.130+0000 7ff1bd048740 4 client.16080 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:38:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.130+0000 7ff1bd048740 1 -- 172.21.15.64:0/1083247922 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7fb30ba20 con 0x55a7fb3b4a50 2024-09-12T03:38:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.130+0000 7ff1b37fe640 1 --2- 172.21.15.64:0/1083247922 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55a7fb3b4a50 0x55a7fb3b6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.131+0000 7ff1b37fe640 1 --2- 172.21.15.64:0/1083247922 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55a7fb3b4a50 0x55a7fb3b6e40 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34ebdf72ff8729d3 server_cookie=e2240ca6d0e895ba in_seq=0 out_seq=0 2024-09-12T03:38:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.131+0000 7ff192ffd640 10 client.16080 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:38:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.131+0000 7ff192ffd640 1 -- 172.21.15.64:0/1083247922 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a7fb30ba20 con 0x55a7fb3b4a50 2024-09-12T03:38:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.131+0000 7ff192ffd640 10 client.16080 handle_command_reply: tid=0 2024-09-12T03:38:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.131+0000 7ff1bd048740 1 client.16080 shutdown 2024-09-12T03:38:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.132+0000 7ff1bd048740 1 -- 172.21.15.64:0/1083247922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7fb3a4cb0 msgr2=0x55a7fb3a25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.132+0000 7ff1bd048740 1 --2- 172.21.15.64:0/1083247922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7fb3a4cb0 0x55a7fb3a25e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac049f50 tx=0x7ff1ac049f80 comp rx=0 tx=0).stop 2024-09-12T03:38:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.132+0000 7ff1bd048740 1 -- 172.21.15.64:0/1083247922 shutdown_connections 2024-09-12T03:38:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.132+0000 7ff1bd048740 1 --2- 172.21.15.64:0/1083247922 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55a7fb3b4a50 0x55a7fb3b6e40 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.132+0000 7ff1bd048740 1 --2- 172.21.15.64:0/1083247922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7fb3a4cb0 0x55a7fb3a25e0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.132+0000 7ff1bd048740 1 --2- 172.21.15.64:0/1083247922 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb39a410 0x55a7fb3a20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.132+0000 7ff1bd048740 1 --2- 172.21.15.64:0/1083247922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb399af0 0x55a7fb3a1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.132+0000 7ff1bd048740 1 -- 172.21.15.64:0/1083247922 >> 172.21.15.64:0/1083247922 conn(0x55a7fb396060 msgr2=0x55a7fb39bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:38:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.132+0000 7ff192ffd640 0 client.16080 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:38:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.132+0000 7ff1bd048740 1 -- 172.21.15.64:0/1083247922 shutdown_connections 2024-09-12T03:38:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.132+0000 7ff1bd048740 1 -- 172.21.15.64:0/1083247922 wait complete. 2024-09-12T03:38:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.132+0000 7ff1bd048740 20 client.16080 trim_cache size 0 max 16384 2024-09-12T03:38:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.146+0000 7ff1bd048740 1 Processor -- start 2024-09-12T03:38:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.146+0000 7ff1bd048740 1 -- start start 2024-09-12T03:38:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.146+0000 7ff1bd048740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb399af0 0x55a7fb3a2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.146+0000 7ff1bd048740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7fb39a410 0x55a7fb3ad070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.146+0000 7ff1bd048740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb3a4cb0 0x55a7fb3ad5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.146+0000 7ff1bd048740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b41b19d0 con 0x55a7fb39a410 2024-09-12T03:38:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.146+0000 7ff1bd048740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b41af2a0 con 0x55a7fb3a4cb0 2024-09-12T03:38:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.147+0000 7ff1bd048740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b41ad110 con 0x55a7fb399af0 2024-09-12T03:38:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.147+0000 7ff1b2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7fb39a410 0x55a7fb3ad070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.147+0000 7ff1b2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7fb39a410 0x55a7fb3ad070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39990/0 (socket says 172.21.15.64:39990) 2024-09-12T03:38:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.147+0000 7ff1b2ffd640 1 -- 172.21.15.64:0/2881673126 learned_addr learned my addr 172.21.15.64:0/2881673126 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:38:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.147+0000 7ff1b3fff640 1 --2- 172.21.15.64:0/2881673126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb3a4cb0 0x55a7fb3ad5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.147+0000 7ff1b37fe640 1 --2- 172.21.15.64:0/2881673126 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb399af0 0x55a7fb3a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.147+0000 7ff1b2ffd640 1 -- 172.21.15.64:0/2881673126 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb399af0 msgr2=0x55a7fb3a2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:38:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.147+0000 7ff1b2ffd640 1 --2- 172.21.15.64:0/2881673126 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb399af0 0x55a7fb3a2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.147+0000 7ff1b2ffd640 1 -- 172.21.15.64:0/2881673126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb3a4cb0 msgr2=0x55a7fb3ad5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:38:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.147+0000 7ff1b2ffd640 1 --2- 172.21.15.64:0/2881673126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb3a4cb0 0x55a7fb3ad5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.147+0000 7ff1b2ffd640 1 -- 172.21.15.64:0/2881673126 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7fb3912d0 con 0x55a7fb39a410 2024-09-12T03:38:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.147+0000 7ff1b3fff640 1 --2- 172.21.15.64:0/2881673126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb3a4cb0 0x55a7fb3ad5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:38:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.147+0000 7ff1b37fe640 1 --2- 172.21.15.64:0/2881673126 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb399af0 0x55a7fb3a2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:38:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.147+0000 7ff1b2ffd640 1 --2- 172.21.15.64:0/2881673126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7fb39a410 0x55a7fb3ad070 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4008a40 tx=0x7ff1a4008f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=400adaa011045a8a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:38:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.147+0000 7ff1997fa640 1 -- 172.21.15.64:0/2881673126 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4011cb0 con 0x55a7fb39a410 2024-09-12T03:38:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.147+0000 7ff1997fa640 1 -- 172.21.15.64:0/2881673126 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1a4011e50 con 0x55a7fb39a410 2024-09-12T03:38:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.147+0000 7ff1997fa640 1 -- 172.21.15.64:0/2881673126 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a401a3c0 con 0x55a7fb39a410 2024-09-12T03:38:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.148+0000 7ff1bd048740 1 -- 172.21.15.64:0/2881673126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7fb39a410 msgr2=0x55a7fb3ad070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.148+0000 7ff1bd048740 1 --2- 172.21.15.64:0/2881673126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7fb39a410 0x55a7fb3ad070 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4008a40 tx=0x7ff1a4008f00 comp rx=0 tx=0).stop 2024-09-12T03:38:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.148+0000 7ff1bd048740 1 -- 172.21.15.64:0/2881673126 shutdown_connections 2024-09-12T03:38:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.148+0000 7ff1bd048740 1 --2- 172.21.15.64:0/2881673126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb3a4cb0 0x55a7fb3ad5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.148+0000 7ff1bd048740 1 --2- 172.21.15.64:0/2881673126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7fb39a410 0x55a7fb3ad070 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.148+0000 7ff1bd048740 1 --2- 172.21.15.64:0/2881673126 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb399af0 0x55a7fb3a2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.148+0000 7ff1bd048740 1 -- 172.21.15.64:0/2881673126 >> 172.21.15.64:0/2881673126 conn(0x55a7fb396060 msgr2=0x55a7fb3a3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:38:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.148+0000 7ff1bd048740 1 -- 172.21.15.64:0/2881673126 shutdown_connections 2024-09-12T03:38:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.148+0000 7ff1bd048740 1 -- 172.21.15.64:0/2881673126 wait complete. 2024-09-12T03:38:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.148+0000 7ff1bd048740 1 -- start start 2024-09-12T03:38:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.148+0000 7ff1bd048740 1 Processor -- start 2024-09-12T03:38:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.149+0000 7ff1bd048740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7fb399af0 0x55a7fb3bc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.149+0000 7ff1bd048740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb39a410 0x55a7fb3bc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.149+0000 7ff1bd048740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb3a4cb0 0x55a7fb41aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.149+0000 7ff1bd048740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b41ad760 con 0x55a7fb399af0 2024-09-12T03:38:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.149+0000 7ff1bd048740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b41aebd0 con 0x55a7fb39a410 2024-09-12T03:38:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.149+0000 7ff1bd048740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b41ade30 con 0x55a7fb3a4cb0 2024-09-12T03:38:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.149+0000 7ff1bd048740 10 client.0 _ll_register_callbacks cb 0x7ff1ac04d070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:38:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.149+0000 7ff1b37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7fb399af0 0x55a7fb3bc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.149+0000 7ff1b37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7fb399af0 0x55a7fb3bc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40002/0 (socket says 172.21.15.64:40002) 2024-09-12T03:38:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.149+0000 7ff1b37fe640 1 -- 172.21.15.64:0/1069272372 learned_addr learned my addr 172.21.15.64:0/1069272372 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:38:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.149+0000 7ff1b3fff640 1 --2- 172.21.15.64:0/1069272372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb3a4cb0 0x55a7fb41aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.149+0000 7ff1b2ffd640 1 --2- 172.21.15.64:0/1069272372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb39a410 0x55a7fb3bc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.149+0000 7ff1b37fe640 1 -- 172.21.15.64:0/1069272372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb3a4cb0 msgr2=0x55a7fb41aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:38:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.149+0000 7ff1b37fe640 1 --2- 172.21.15.64:0/1069272372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb3a4cb0 0x55a7fb41aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.149+0000 7ff1b37fe640 1 -- 172.21.15.64:0/1069272372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb39a410 msgr2=0x55a7fb3bc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:38:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.149+0000 7ff1b37fe640 1 --2- 172.21.15.64:0/1069272372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb39a410 0x55a7fb3bc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.149+0000 7ff1b37fe640 1 -- 172.21.15.64:0/1069272372 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7fb41b130 con 0x55a7fb399af0 2024-09-12T03:38:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.149+0000 7ff1b3fff640 1 --2- 172.21.15.64:0/1069272372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb3a4cb0 0x55a7fb41aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:38:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.149+0000 7ff1b2ffd640 1 --2- 172.21.15.64:0/1069272372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb39a410 0x55a7fb3bc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:38:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.149+0000 7ff1b37fe640 1 --2- 172.21.15.64:0/1069272372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7fb399af0 0x55a7fb3bc350 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8097de0 tx=0x7ff1a800df40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecddc8fcf5f9553c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:38:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.149+0000 7ff192ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:38:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.149+0000 7ff192ffd640 1 -- 172.21.15.64:0/1069272372 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a809ad70 con 0x55a7fb399af0 2024-09-12T03:38:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.150+0000 7ff192ffd640 1 -- 172.21.15.64:0/1069272372 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9583+0+0 (secure 0 0 0) 0x7ff1a80a3bf0 con 0x55a7fb399af0 2024-09-12T03:38:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.150+0000 7ff1bd048740 1 -- 172.21.15.64:0/1069272372 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1a4011e50 con 0x55a7fb399af0 2024-09-12T03:38:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.150+0000 7ff192ffd640 1 -- 172.21.15.64:0/1069272372 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff1a809bb10 con 0x55a7fb399af0 2024-09-12T03:38:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.150+0000 7ff1bd048740 10 client.16086 fetch_fsmap learned FSMap version 25 2024-09-12T03:38:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.150+0000 7ff1bd048740 1 -- 172.21.15.64:0/1069272372 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a7fb4236c0 con 0x55a7fb399af0 2024-09-12T03:38:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.150+0000 7ff192ffd640 1 -- 172.21.15.64:0/1069272372 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1a4011e50 con 0x55a7fb399af0 2024-09-12T03:38:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.150+0000 7ff192ffd640 1 -- 172.21.15.64:0/1069272372 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff1a8009a30 con 0x55a7fb399af0 2024-09-12T03:38:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.150+0000 7ff1bd048740 10 client.16086 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:38:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.150+0000 7ff1bd048740 10 client.16086 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:38:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.150+0000 7ff1bd048740 20 client.16086 populate_metadata read hostname 'smithi064' 2024-09-12T03:38:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.150+0000 7ff1bd048740 1 --2- 172.21.15.64:0/1069272372 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55a7fb423df0 0x55a7fb426230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.150+0000 7ff1bd048740 4 client.16086 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:38:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.150+0000 7ff1bd048740 1 -- 172.21.15.64:0/1069272372 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a7fb3a7520 con 0x55a7fb423df0 2024-09-12T03:38:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.151+0000 7ff1b2ffd640 1 --2- 172.21.15.64:0/1069272372 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55a7fb423df0 0x55a7fb426230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.151+0000 7ff192ffd640 10 client.16086 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:38:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.151+0000 7ff1b2ffd640 1 --2- 172.21.15.64:0/1069272372 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55a7fb423df0 0x55a7fb426230 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce4a35451ab91909 server_cookie=22e5d83734154bfe in_seq=0 out_seq=0 2024-09-12T03:38:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.165+0000 7ff192ffd640 1 -- 172.21.15.64:0/1069272372 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a7fb3a7520 con 0x55a7fb423df0 2024-09-12T03:38:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.165+0000 7ff192ffd640 10 client.16086 handle_command_reply: tid=0 2024-09-12T03:38:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.165+0000 7ff1bd048740 1 client.16086 shutdown 2024-09-12T03:38:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.165+0000 7ff1bd048740 1 -- 172.21.15.64:0/1069272372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7fb399af0 msgr2=0x55a7fb3bc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.165+0000 7ff1bd048740 1 --2- 172.21.15.64:0/1069272372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7fb399af0 0x55a7fb3bc350 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8097de0 tx=0x7ff1a800df40 comp rx=0 tx=0).stop 2024-09-12T03:38:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.165+0000 7ff1bd048740 1 -- 172.21.15.64:0/1069272372 shutdown_connections 2024-09-12T03:38:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.165+0000 7ff1bd048740 1 --2- 172.21.15.64:0/1069272372 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55a7fb423df0 0x55a7fb426230 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.165+0000 7ff1bd048740 1 --2- 172.21.15.64:0/1069272372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb3a4cb0 0x55a7fb41aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.165+0000 7ff192ffd640 0 client.16086 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:38:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.165+0000 7ff1bd048740 1 --2- 172.21.15.64:0/1069272372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb39a410 0x55a7fb3bc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.165+0000 7ff1bd048740 1 --2- 172.21.15.64:0/1069272372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7fb399af0 0x55a7fb3bc350 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.165+0000 7ff1bd048740 1 -- 172.21.15.64:0/1069272372 >> 172.21.15.64:0/1069272372 conn(0x55a7fb396060 msgr2=0x55a7fb3a3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:38:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.165+0000 7ff1bd048740 1 -- 172.21.15.64:0/1069272372 shutdown_connections 2024-09-12T03:38:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.165+0000 7ff1bd048740 1 -- 172.21.15.64:0/1069272372 wait complete. 2024-09-12T03:38:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.165+0000 7ff1bd048740 20 client.16086 trim_cache size 0 max 16384 2024-09-12T03:38:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:38:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:38:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:38:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:38:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.166+0000 7ff1ba1d8640 1 -- 172.21.15.64:0/2631491670 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7ff19c0a2e20 msgr2=0x7ff19c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.166+0000 7ff1ba1d8640 1 --2- 172.21.15.64:0/2631491670 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7ff19c0a2e20 0x7ff19c0a52e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac003e40 tx=0x7ff1ac049000 comp rx=0 tx=0).stop 2024-09-12T03:38:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.166+0000 7ff1ba1d8640 1 -- 172.21.15.64:0/2631491670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1b4102cc0 msgr2=0x7ff1b4198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.166+0000 7ff1ba1d8640 1 --2- 172.21.15.64:0/2631491670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1b4102cc0 0x7ff1b4198220 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff1a800d950 tx=0x7ff1a800de10 comp rx=0 tx=0).stop 2024-09-12T03:38:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.166+0000 7ff1ba1d8640 1 -- 172.21.15.64:0/2631491670 shutdown_connections 2024-09-12T03:38:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.166+0000 7ff1ba1d8640 1 --2- 172.21.15.64:0/2631491670 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7ff19c0a2e20 0x7ff19c0a52e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.166+0000 7ff1ba1d8640 1 --2- 172.21.15.64:0/2631491670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1b410c110 0x7ff1b419cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.166+0000 7ff1ba1d8640 1 --2- 172.21.15.64:0/2631491670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1b41035e0 0x7ff1b4198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.166+0000 7ff1ba1d8640 1 --2- 172.21.15.64:0/2631491670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1b4102cc0 0x7ff1b4198220 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.166+0000 7ff1ba1d8640 1 -- 172.21.15.64:0/2631491670 >> 172.21.15.64:0/2631491670 conn(0x7ff1b40fc6b0 msgr2=0x7ff1b40fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:38:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.166+0000 7ff1ba1d8640 1 -- 172.21.15.64:0/2631491670 shutdown_connections 2024-09-12T03:38:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:17.166+0000 7ff1ba1d8640 1 -- 172.21.15.64:0/2631491670 wait complete. 2024-09-12T03:38:17.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:38:17 smithi195 ceph-mon[34786]: pgmap v279: 129 pgs: 129 active+clean; 4.0 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-09-12T03:38:17.521 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:38:17 smithi064 ceph-mon[30952]: pgmap v279: 129 pgs: 129 active+clean; 4.0 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-09-12T03:38:17.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:38:17 smithi115 ceph-mon[34801]: pgmap v279: 129 pgs: 129 active+clean; 4.0 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-09-12T03:38:19.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:38:19 smithi195 ceph-mon[34786]: pgmap v280: 129 pgs: 129 active+clean; 4.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-12T03:38:19.521 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:38:19 smithi064 ceph-mon[30952]: pgmap v280: 129 pgs: 129 active+clean; 4.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-12T03:38:19.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:38:19 smithi115 ceph-mon[34801]: pgmap v280: 129 pgs: 129 active+clean; 4.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-12T03:38:21.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:38:21 smithi195 ceph-mon[34786]: pgmap v281: 129 pgs: 129 active+clean; 4.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-12T03:38:21.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:38:21 smithi064 ceph-mon[30952]: pgmap v281: 129 pgs: 129 active+clean; 4.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-12T03:38:21.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:38:21 smithi115 ceph-mon[34801]: pgmap v281: 129 pgs: 129 active+clean; 4.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-12T03:38:23.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:38:23 smithi195 ceph-mon[34786]: pgmap v282: 129 pgs: 129 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 63 KiB/s wr, 20 op/s 2024-09-12T03:38:23.521 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:38:23 smithi064 ceph-mon[30952]: pgmap v282: 129 pgs: 129 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 63 KiB/s wr, 20 op/s 2024-09-12T03:38:23.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:38:23 smithi115 ceph-mon[34801]: pgmap v282: 129 pgs: 129 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 63 KiB/s wr, 20 op/s 2024-09-12T03:38:25.521 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:38:25 smithi064 ceph-mon[30952]: pgmap v283: 129 pgs: 129 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:38:25.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:38:25 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:38:25.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:38:25 smithi115 ceph-mon[34801]: pgmap v283: 129 pgs: 129 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:38:25.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:38:25 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:38:25.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:38:25 smithi195 ceph-mon[34786]: pgmap v283: 129 pgs: 129 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:38:25.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:38:25 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:38:26.521 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:38:26 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:38:26.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:38:26 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:38:26.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:38:26 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:38:26.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:38:26 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:38:26.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:38:26 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:38:26.643 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:38:26 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:38:26.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:38:26 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:38:26.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:38:26 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:38:26.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:38:26 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:38:27.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:38:27 smithi195 ceph-mon[34786]: pgmap v284: 129 pgs: 129 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:38:27.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:38:27 smithi064 ceph-mon[30952]: pgmap v284: 129 pgs: 129 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:38:27.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:38:27 smithi115 ceph-mon[34801]: pgmap v284: 129 pgs: 129 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:38:29.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:38:29 smithi195 ceph-mon[34786]: pgmap v285: 129 pgs: 129 active+clean; 4.6 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 58 KiB/s wr, 18 op/s 2024-09-12T03:38:29.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:38:29 smithi064 ceph-mon[30952]: pgmap v285: 129 pgs: 129 active+clean; 4.6 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 58 KiB/s wr, 18 op/s 2024-09-12T03:38:29.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:38:29 smithi115 ceph-mon[34801]: pgmap v285: 129 pgs: 129 active+clean; 4.6 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 58 KiB/s wr, 18 op/s 2024-09-12T03:38:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.264+0000 7ff9dd186640 1 -- 172.21.15.64:0/3687864437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9d81069c0 msgr2=0x7ff9d8106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.264+0000 7ff9dd186640 1 --2- 172.21.15.64:0/3687864437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9d81069c0 0x7ff9d8106da0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8009930 tx=0x7ff9c802a420 comp rx=0 tx=0).stop 2024-09-12T03:38:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.264+0000 7ff9dd186640 1 -- 172.21.15.64:0/3687864437 shutdown_connections 2024-09-12T03:38:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.264+0000 7ff9dd186640 1 --2- 172.21.15.64:0/3687864437 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9d8101350 0x7ff9d810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.264+0000 7ff9dd186640 1 --2- 172.21.15.64:0/3687864437 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9d81009b0 0x7ff9d8100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.264+0000 7ff9dd186640 1 --2- 172.21.15.64:0/3687864437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9d81069c0 0x7ff9d8106da0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.264+0000 7ff9dd186640 1 -- 172.21.15.64:0/3687864437 >> 172.21.15.64:0/3687864437 conn(0x7ff9d8077990 msgr2=0x7ff9d80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:38:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.264+0000 7ff9dd186640 1 -- 172.21.15.64:0/3687864437 shutdown_connections 2024-09-12T03:38:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.264+0000 7ff9dd186640 1 -- 172.21.15.64:0/3687864437 wait complete. 2024-09-12T03:38:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.265+0000 7ff9dd186640 1 Processor -- start 2024-09-12T03:38:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.265+0000 7ff9dd186640 1 -- start start 2024-09-12T03:38:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.265+0000 7ff9dd186640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9d81009b0 0x7ff9d80fcde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.265+0000 7ff9dd186640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9d8101350 0x7ff9d80fd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.265+0000 7ff9dd186640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9d81069c0 0x7ff9d80fd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.265+0000 7ff9dd186640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d8110d00 con 0x7ff9d8101350 2024-09-12T03:38:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.265+0000 7ff9dd186640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d8110b80 con 0x7ff9d81009b0 2024-09-12T03:38:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.265+0000 7ff9dd186640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d8110e80 con 0x7ff9d81069c0 2024-09-12T03:38:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.266+0000 7ff9d659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9d8101350 0x7ff9d80fd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.266+0000 7ff9d659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9d8101350 0x7ff9d80fd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51734/0 (socket says 172.21.15.64:51734) 2024-09-12T03:38:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.266+0000 7ff9d659c640 1 -- 172.21.15.64:0/409385920 learned_addr learned my addr 172.21.15.64:0/409385920 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:38:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.266+0000 7ff9d759e640 1 --2- 172.21.15.64:0/409385920 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9d81069c0 0x7ff9d80fd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.266+0000 7ff9d6d9d640 1 --2- 172.21.15.64:0/409385920 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9d81009b0 0x7ff9d80fcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.266+0000 7ff9d659c640 1 -- 172.21.15.64:0/409385920 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9d81069c0 msgr2=0x7ff9d80fd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.266+0000 7ff9d659c640 1 --2- 172.21.15.64:0/409385920 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9d81069c0 0x7ff9d80fd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.266+0000 7ff9d659c640 1 -- 172.21.15.64:0/409385920 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9d81009b0 msgr2=0x7ff9d80fcde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.266+0000 7ff9d659c640 1 --2- 172.21.15.64:0/409385920 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9d81009b0 0x7ff9d80fcde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.266+0000 7ff9d659c640 1 -- 172.21.15.64:0/409385920 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d81a6780 con 0x7ff9d8101350 2024-09-12T03:38:30.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.266+0000 7ff9d6d9d640 1 --2- 172.21.15.64:0/409385920 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9d81009b0 0x7ff9d80fcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:38:30.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.266+0000 7ff9d659c640 1 --2- 172.21.15.64:0/409385920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9d8101350 0x7ff9d80fd320 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff9c000e9f0 tx=0x7ff9c000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:38:30.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.266+0000 7ff9cffff640 1 -- 172.21.15.64:0/409385920 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c000cd80 con 0x7ff9d8101350 2024-09-12T03:38:30.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.266+0000 7ff9dd186640 1 -- 172.21.15.64:0/409385920 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9d81a6a70 con 0x7ff9d8101350 2024-09-12T03:38:30.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.266+0000 7ff9cffff640 1 -- 172.21.15.64:0/409385920 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9c00108e0 con 0x7ff9d8101350 2024-09-12T03:38:30.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.266+0000 7ff9cffff640 1 -- 172.21.15.64:0/409385920 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c0018920 con 0x7ff9d8101350 2024-09-12T03:38:30.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.266+0000 7ff9dd186640 1 -- 172.21.15.64:0/409385920 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9d81a6fc0 con 0x7ff9d8101350 2024-09-12T03:38:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.269+0000 7ff9cffff640 1 -- 172.21.15.64:0/409385920 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ff9c0010a90 con 0x7ff9d8101350 2024-09-12T03:38:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.270+0000 7ff9dfff7740 1 Processor -- start 2024-09-12T03:38:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.270+0000 7ff9dfff7740 1 -- start start 2024-09-12T03:38:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.270+0000 7ff9dfff7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ff3b73daf0 0x55ff3b73ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.270+0000 7ff9dfff7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff3b73e410 0x55ff3b748770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.270+0000 7ff9dfff7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff3b748cb0 0x55ff3b74b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.270+0000 7ff9dfff7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ff3b6a22f0 con 0x55ff3b73daf0 2024-09-12T03:38:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.270+0000 7ff9dfff7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ff3b64a270 con 0x55ff3b73e410 2024-09-12T03:38:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.270+0000 7ff9dfff7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ff3b69e2b0 con 0x55ff3b748cb0 2024-09-12T03:38:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.270+0000 7ff9d759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ff3b73daf0 0x55ff3b73ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.270+0000 7ff9d759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ff3b73daf0 0x55ff3b73ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51748/0 (socket says 172.21.15.64:51748) 2024-09-12T03:38:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.270+0000 7ff9d759e640 1 -- 172.21.15.64:0/1921916355 learned_addr learned my addr 172.21.15.64:0/1921916355 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:38:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.270+0000 7ff9d659c640 1 --2- 172.21.15.64:0/1921916355 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff3b748cb0 0x55ff3b74b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.270+0000 7ff9d6d9d640 1 --2- 172.21.15.64:0/1921916355 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff3b73e410 0x55ff3b748770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.271+0000 7ff9d759e640 1 -- 172.21.15.64:0/1921916355 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff3b748cb0 msgr2=0x55ff3b74b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:38:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.271+0000 7ff9d759e640 1 --2- 172.21.15.64:0/1921916355 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff3b748cb0 0x55ff3b74b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.271+0000 7ff9d759e640 1 -- 172.21.15.64:0/1921916355 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff3b73e410 msgr2=0x55ff3b748770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:38:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.271+0000 7ff9d759e640 1 --2- 172.21.15.64:0/1921916355 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff3b73e410 0x55ff3b748770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.271+0000 7ff9d759e640 1 -- 172.21.15.64:0/1921916355 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff3b649340 con 0x55ff3b73daf0 2024-09-12T03:38:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.271+0000 7ff9cffff640 1 --2- 172.21.15.64:0/409385920 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7ff9c40a2da0 0x7ff9c40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.271+0000 7ff9d6d9d640 1 --2- 172.21.15.64:0/1921916355 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff3b73e410 0x55ff3b748770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:38:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.271+0000 7ff9d6d9d640 1 --2- 172.21.15.64:0/409385920 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7ff9c40a2da0 0x7ff9c40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.271+0000 7ff9d759e640 1 --2- 172.21.15.64:0/1921916355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ff3b73daf0 0x55ff3b73ded0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0009f70 tx=0x7ff9d0010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb00b026e85ab3e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:38:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.271+0000 7ff9cffff640 1 -- 172.21.15.64:0/409385920 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9583+0+0 (secure 0 0 0) 0x7ff9c00cfbf0 con 0x7ff9d8101350 2024-09-12T03:38:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.271+0000 7ff9cd7fa640 1 -- 172.21.15.64:0/1921916355 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d0016070 con 0x55ff3b73daf0 2024-09-12T03:38:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.271+0000 7ff9cd7fa640 1 -- 172.21.15.64:0/1921916355 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9d0004d20 con 0x55ff3b73daf0 2024-09-12T03:38:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.271+0000 7ff9cd7fa640 1 -- 172.21.15.64:0/1921916355 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d0012790 con 0x55ff3b73daf0 2024-09-12T03:38:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.272+0000 7ff9dfff7740 1 -- 172.21.15.64:0/1921916355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ff3b73daf0 msgr2=0x55ff3b73ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.272+0000 7ff9dfff7740 1 --2- 172.21.15.64:0/1921916355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ff3b73daf0 0x55ff3b73ded0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0009f70 tx=0x7ff9d0010800 comp rx=0 tx=0).stop 2024-09-12T03:38:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.272+0000 7ff9d6d9d640 1 --2- 172.21.15.64:0/409385920 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7ff9c40a2da0 0x7ff9c40a5260 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8006010 tx=0x7ff9c8005b50 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:38:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.272+0000 7ff9dfff7740 1 -- 172.21.15.64:0/1921916355 shutdown_connections 2024-09-12T03:38:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.272+0000 7ff9dfff7740 1 --2- 172.21.15.64:0/1921916355 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff3b748cb0 0x55ff3b74b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.272+0000 7ff9dfff7740 1 --2- 172.21.15.64:0/1921916355 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff3b73e410 0x55ff3b748770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.272+0000 7ff9dfff7740 1 --2- 172.21.15.64:0/1921916355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ff3b73daf0 0x55ff3b73ded0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.272+0000 7ff9dfff7740 1 -- 172.21.15.64:0/1921916355 >> 172.21.15.64:0/1921916355 conn(0x55ff3b73a060 msgr2=0x55ff3b73a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:38:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.272+0000 7ff9dfff7740 1 -- 172.21.15.64:0/1921916355 shutdown_connections 2024-09-12T03:38:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.272+0000 7ff9dfff7740 1 -- 172.21.15.64:0/1921916355 wait complete. 2024-09-12T03:38:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.272+0000 7ff9dfff7740 1 -- start start 2024-09-12T03:38:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.272+0000 7ff9dfff7740 1 Processor -- start 2024-09-12T03:38:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.273+0000 7ff9dfff7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff3b73daf0 0x55ff3b745bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.273+0000 7ff9dfff7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff3b73e410 0x55ff3b7460f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.273+0000 7ff9dfff7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ff3b748cb0 0x55ff3b746630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.273+0000 7ff9dfff7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ff3b74b880 con 0x55ff3b748cb0 2024-09-12T03:38:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.273+0000 7ff9dfff7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ff3b62b290 con 0x55ff3b73daf0 2024-09-12T03:38:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.273+0000 7ff9dfff7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ff3b74ba00 con 0x55ff3b73e410 2024-09-12T03:38:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.273+0000 7ff9dfff7740 10 client.0 _ll_register_callbacks cb 0x55ff3b65c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:38:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.273+0000 7ff9d759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ff3b748cb0 0x55ff3b746630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.273+0000 7ff9d6d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff3b73daf0 0x55ff3b745bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.273+0000 7ff9d659c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff3b73e410 0x55ff3b7460f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.273+0000 7ff9d6d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff3b73daf0 0x55ff3b745bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.64:54284/0 (socket says 172.21.15.64:54284) 2024-09-12T03:38:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.273+0000 7ff9d6d9d640 1 -- 172.21.15.64:0/2877671720 learned_addr learned my addr 172.21.15.64:0/2877671720 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:38:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.273+0000 7ff9d759e640 1 -- 172.21.15.64:0/2877671720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff3b73e410 msgr2=0x55ff3b7460f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:38:30.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.273+0000 7ff9d759e640 1 --2- 172.21.15.64:0/2877671720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff3b73e410 0x55ff3b7460f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:30.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.273+0000 7ff9d759e640 1 -- 172.21.15.64:0/2877671720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff3b73daf0 msgr2=0x55ff3b745bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:38:30.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.273+0000 7ff9d759e640 1 --2- 172.21.15.64:0/2877671720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff3b73daf0 0x55ff3b745bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:30.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.273+0000 7ff9d759e640 1 -- 172.21.15.64:0/2877671720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff3b3b5210 con 0x55ff3b748cb0 2024-09-12T03:38:30.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.273+0000 7ff9d759e640 1 --2- 172.21.15.64:0/2877671720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ff3b748cb0 0x55ff3b746630 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0011010 tx=0x7ff9d0012760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24901f5ad1b50876 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:38:30.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.273+0000 7ff9be7fc640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:38:30.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.274+0000 7ff9be7fc640 1 -- 172.21.15.64:0/2877671720 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d0029060 con 0x55ff3b748cb0 2024-09-12T03:38:30.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.274+0000 7ff9be7fc640 1 -- 172.21.15.64:0/2877671720 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9583+0+0 (secure 0 0 0) 0x7ff9d0012ae0 con 0x55ff3b748cb0 2024-09-12T03:38:30.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.274+0000 7ff9dfff7740 1 -- 172.21.15.64:0/2877671720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9d0004d20 con 0x55ff3b748cb0 2024-09-12T03:38:30.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.274+0000 7ff9be7fc640 1 -- 172.21.15.64:0/2877671720 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff9d0030050 con 0x55ff3b748cb0 2024-09-12T03:38:30.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.274+0000 7ff9dfff7740 10 client.16098 fetch_fsmap learned FSMap version 25 2024-09-12T03:38:30.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.274+0000 7ff9dfff7740 1 -- 172.21.15.64:0/2877671720 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ff3b481620 con 0x55ff3b748cb0 2024-09-12T03:38:30.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.274+0000 7ff9be7fc640 1 -- 172.21.15.64:0/2877671720 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9d0004d20 con 0x55ff3b748cb0 2024-09-12T03:38:30.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.274+0000 7ff9be7fc640 1 -- 172.21.15.64:0/2877671720 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff9d000f040 con 0x55ff3b748cb0 2024-09-12T03:38:30.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.274+0000 7ff9dfff7740 10 client.16098 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:38:30.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.274+0000 7ff9dfff7740 10 client.16098 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:38:30.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.274+0000 7ff9dfff7740 20 client.16098 populate_metadata read hostname 'smithi064' 2024-09-12T03:38:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.274+0000 7ff9dfff7740 1 --2- 172.21.15.64:0/2877671720 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55ff3b758b90 0x55ff3b75af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.274+0000 7ff9dfff7740 4 client.16098 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:38:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.274+0000 7ff9dfff7740 1 -- 172.21.15.64:0/2877671720 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff3b6afa20 con 0x55ff3b758b90 2024-09-12T03:38:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.275+0000 7ff9d6d9d640 1 --2- 172.21.15.64:0/2877671720 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55ff3b758b90 0x55ff3b75af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.275+0000 7ff9d6d9d640 1 --2- 172.21.15.64:0/2877671720 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55ff3b758b90 0x55ff3b75af80 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2287527c3a53f28f server_cookie=ec805d560ce1c196 in_seq=0 out_seq=0 2024-09-12T03:38:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.275+0000 7ff9be7fc640 10 client.16098 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:38:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.276+0000 7ff9be7fc640 1 -- 172.21.15.64:0/2877671720 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ff3b6afa20 con 0x55ff3b758b90 2024-09-12T03:38:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.276+0000 7ff9be7fc640 10 client.16098 handle_command_reply: tid=0 2024-09-12T03:38:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.276+0000 7ff9dfff7740 1 client.16098 shutdown 2024-09-12T03:38:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.276+0000 7ff9dfff7740 1 -- 172.21.15.64:0/2877671720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ff3b748cb0 msgr2=0x55ff3b746630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.276+0000 7ff9dfff7740 1 --2- 172.21.15.64:0/2877671720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ff3b748cb0 0x55ff3b746630 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0011010 tx=0x7ff9d0012760 comp rx=0 tx=0).stop 2024-09-12T03:38:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.276+0000 7ff9dfff7740 1 -- 172.21.15.64:0/2877671720 shutdown_connections 2024-09-12T03:38:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.276+0000 7ff9dfff7740 1 --2- 172.21.15.64:0/2877671720 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55ff3b758b90 0x55ff3b75af80 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.276+0000 7ff9dfff7740 1 --2- 172.21.15.64:0/2877671720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ff3b748cb0 0x55ff3b746630 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:30.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.276+0000 7ff9dfff7740 1 --2- 172.21.15.64:0/2877671720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff3b73e410 0x55ff3b7460f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:30.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.276+0000 7ff9be7fc640 0 client.16098 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:38:30.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.276+0000 7ff9dfff7740 1 --2- 172.21.15.64:0/2877671720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff3b73daf0 0x55ff3b745bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:30.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.276+0000 7ff9dfff7740 1 -- 172.21.15.64:0/2877671720 >> 172.21.15.64:0/2877671720 conn(0x55ff3b73a060 msgr2=0x55ff3b73fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:38:30.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.276+0000 7ff9dfff7740 1 -- 172.21.15.64:0/2877671720 shutdown_connections 2024-09-12T03:38:30.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.276+0000 7ff9dfff7740 1 -- 172.21.15.64:0/2877671720 wait complete. 2024-09-12T03:38:30.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.276+0000 7ff9dfff7740 20 client.16098 trim_cache size 0 max 16384 2024-09-12T03:38:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.291+0000 7ff9dfff7740 1 Processor -- start 2024-09-12T03:38:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.291+0000 7ff9dfff7740 1 -- start start 2024-09-12T03:38:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.291+0000 7ff9dfff7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff3b73daf0 0x55ff3b746460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.291+0000 7ff9dfff7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ff3b73e410 0x55ff3b7390b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.291+0000 7ff9dfff7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff3b748cb0 0x55ff3b742e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.291+0000 7ff9dfff7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d81b3bf0 con 0x55ff3b73e410 2024-09-12T03:38:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.291+0000 7ff9dfff7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d81b14c0 con 0x55ff3b73daf0 2024-09-12T03:38:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.291+0000 7ff9dfff7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d81af330 con 0x55ff3b748cb0 2024-09-12T03:38:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.291+0000 7ff9d659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ff3b73e410 0x55ff3b7390b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.291+0000 7ff9d659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ff3b73e410 0x55ff3b7390b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51776/0 (socket says 172.21.15.64:51776) 2024-09-12T03:38:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.291+0000 7ff9d659c640 1 -- 172.21.15.64:0/707312107 learned_addr learned my addr 172.21.15.64:0/707312107 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:38:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.291+0000 7ff9d759e640 1 --2- 172.21.15.64:0/707312107 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff3b748cb0 0x55ff3b742e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.291+0000 7ff9d6d9d640 1 --2- 172.21.15.64:0/707312107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff3b73daf0 0x55ff3b746460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.291+0000 7ff9d659c640 1 -- 172.21.15.64:0/707312107 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff3b748cb0 msgr2=0x55ff3b742e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:38:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.292+0000 7ff9d659c640 1 --2- 172.21.15.64:0/707312107 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff3b748cb0 0x55ff3b742e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.292+0000 7ff9d659c640 1 -- 172.21.15.64:0/707312107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff3b73daf0 msgr2=0x55ff3b746460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:38:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.292+0000 7ff9d659c640 1 --2- 172.21.15.64:0/707312107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff3b73daf0 0x55ff3b746460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.292+0000 7ff9d659c640 1 -- 172.21.15.64:0/707312107 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff3b73e240 con 0x55ff3b73e410 2024-09-12T03:38:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.292+0000 7ff9d6d9d640 1 --2- 172.21.15.64:0/707312107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff3b73daf0 0x55ff3b746460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:38:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.292+0000 7ff9d659c640 1 --2- 172.21.15.64:0/707312107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ff3b73e410 0x55ff3b7390b0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ff9c00ccf80 tx=0x7ff9c009bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ca07658431ca10 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:38:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.292+0000 7ff9ccff9640 1 -- 172.21.15.64:0/707312107 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c009c3f0 con 0x55ff3b73e410 2024-09-12T03:38:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.292+0000 7ff9ccff9640 1 -- 172.21.15.64:0/707312107 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9c009bcf0 con 0x55ff3b73e410 2024-09-12T03:38:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.292+0000 7ff9ccff9640 1 -- 172.21.15.64:0/707312107 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c009b140 con 0x55ff3b73e410 2024-09-12T03:38:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.292+0000 7ff9dfff7740 1 -- 172.21.15.64:0/707312107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ff3b73e410 msgr2=0x55ff3b7390b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.293+0000 7ff9dfff7740 1 --2- 172.21.15.64:0/707312107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ff3b73e410 0x55ff3b7390b0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ff9c00ccf80 tx=0x7ff9c009bb80 comp rx=0 tx=0).stop 2024-09-12T03:38:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.293+0000 7ff9dfff7740 1 -- 172.21.15.64:0/707312107 shutdown_connections 2024-09-12T03:38:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.293+0000 7ff9dfff7740 1 --2- 172.21.15.64:0/707312107 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff3b748cb0 0x55ff3b742e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.293+0000 7ff9dfff7740 1 --2- 172.21.15.64:0/707312107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ff3b73e410 0x55ff3b7390b0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.293+0000 7ff9dfff7740 1 --2- 172.21.15.64:0/707312107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff3b73daf0 0x55ff3b746460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.293+0000 7ff9dfff7740 1 -- 172.21.15.64:0/707312107 >> 172.21.15.64:0/707312107 conn(0x55ff3b73a060 msgr2=0x55ff3b7389a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:38:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.293+0000 7ff9dfff7740 1 -- 172.21.15.64:0/707312107 shutdown_connections 2024-09-12T03:38:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.293+0000 7ff9dfff7740 1 -- 172.21.15.64:0/707312107 wait complete. 2024-09-12T03:38:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.293+0000 7ff9dfff7740 1 -- start start 2024-09-12T03:38:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.293+0000 7ff9dfff7740 1 Processor -- start 2024-09-12T03:38:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.293+0000 7ff9dfff7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ff3b73daf0 0x55ff3b75e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.293+0000 7ff9dfff7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff3b73e410 0x55ff3b75e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.293+0000 7ff9dfff7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff3b748cb0 0x55ff3b75b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.293+0000 7ff9dfff7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d81af980 con 0x55ff3b73daf0 2024-09-12T03:38:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.293+0000 7ff9dfff7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d81b0df0 con 0x55ff3b73e410 2024-09-12T03:38:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.293+0000 7ff9dfff7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d81b0050 con 0x55ff3b748cb0 2024-09-12T03:38:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.293+0000 7ff9dfff7740 10 client.0 _ll_register_callbacks cb 0x7ff9d0009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:38:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.293+0000 7ff9d6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ff3b73daf0 0x55ff3b75e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.293+0000 7ff9d6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ff3b73daf0 0x55ff3b75e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51778/0 (socket says 172.21.15.64:51778) 2024-09-12T03:38:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.293+0000 7ff9d6d9d640 1 -- 172.21.15.64:0/2128096549 learned_addr learned my addr 172.21.15.64:0/2128096549 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:38:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.294+0000 7ff9d659c640 1 --2- 172.21.15.64:0/2128096549 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff3b73e410 0x55ff3b75e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.294+0000 7ff9d759e640 1 --2- 172.21.15.64:0/2128096549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff3b748cb0 0x55ff3b75b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.294+0000 7ff9d6d9d640 1 -- 172.21.15.64:0/2128096549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff3b748cb0 msgr2=0x55ff3b75b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:38:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.294+0000 7ff9d6d9d640 1 --2- 172.21.15.64:0/2128096549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff3b748cb0 0x55ff3b75b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.294+0000 7ff9d6d9d640 1 -- 172.21.15.64:0/2128096549 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff3b73e410 msgr2=0x55ff3b75e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:38:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.294+0000 7ff9d6d9d640 1 --2- 172.21.15.64:0/2128096549 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff3b73e410 0x55ff3b75e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.294+0000 7ff9d6d9d640 1 -- 172.21.15.64:0/2128096549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff3b75bd70 con 0x55ff3b73daf0 2024-09-12T03:38:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.294+0000 7ff9d6d9d640 1 --2- 172.21.15.64:0/2128096549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ff3b73daf0 0x55ff3b75e020 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8009230 tx=0x7ff9c8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5874570d1a2e3216 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:38:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.294+0000 7ff9be7fc640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:38:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.294+0000 7ff9be7fc640 1 -- 172.21.15.64:0/2128096549 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8035050 con 0x55ff3b73daf0 2024-09-12T03:38:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.294+0000 7ff9be7fc640 1 -- 172.21.15.64:0/2128096549 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9583+0+0 (secure 0 0 0) 0x7ff9c80055f0 con 0x55ff3b73daf0 2024-09-12T03:38:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.294+0000 7ff9dfff7740 1 -- 172.21.15.64:0/2128096549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9c009bcf0 con 0x55ff3b73daf0 2024-09-12T03:38:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.294+0000 7ff9be7fc640 1 -- 172.21.15.64:0/2128096549 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff9c8061050 con 0x55ff3b73daf0 2024-09-12T03:38:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.294+0000 7ff9dfff7740 10 client.16104 fetch_fsmap learned FSMap version 25 2024-09-12T03:38:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.294+0000 7ff9dfff7740 1 -- 172.21.15.64:0/2128096549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ff3b7c3020 con 0x55ff3b73daf0 2024-09-12T03:38:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.295+0000 7ff9be7fc640 1 -- 172.21.15.64:0/2128096549 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9c009bcf0 con 0x55ff3b73daf0 2024-09-12T03:38:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.295+0000 7ff9be7fc640 1 -- 172.21.15.64:0/2128096549 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff9c80324b0 con 0x55ff3b73daf0 2024-09-12T03:38:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.295+0000 7ff9dfff7740 10 client.16104 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:38:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.295+0000 7ff9dfff7740 10 client.16104 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:38:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.295+0000 7ff9dfff7740 20 client.16104 populate_metadata read hostname 'smithi064' 2024-09-12T03:38:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.295+0000 7ff9dfff7740 1 --2- 172.21.15.64:0/2128096549 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55ff3b7c36f0 0x55ff3b7c5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.295+0000 7ff9dfff7740 4 client.16104 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:38:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.295+0000 7ff9dfff7740 1 -- 172.21.15.64:0/2128096549 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ff3b6a9d70 con 0x55ff3b7c36f0 2024-09-12T03:38:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.295+0000 7ff9d659c640 1 --2- 172.21.15.64:0/2128096549 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55ff3b7c36f0 0x55ff3b7c5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.295+0000 7ff9be7fc640 10 client.16104 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:38:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.295+0000 7ff9d659c640 1 --2- 172.21.15.64:0/2128096549 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55ff3b7c36f0 0x55ff3b7c5ae0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2b3a627f5f733a5 server_cookie=b6a4c1731aa08cc7 in_seq=0 out_seq=0 2024-09-12T03:38:30.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.309+0000 7ff9be7fc640 1 -- 172.21.15.64:0/2128096549 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ff3b6a9d70 con 0x55ff3b7c36f0 2024-09-12T03:38:30.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.309+0000 7ff9be7fc640 10 client.16104 handle_command_reply: tid=0 2024-09-12T03:38:30.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.309+0000 7ff9dfff7740 1 client.16104 shutdown 2024-09-12T03:38:30.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.309+0000 7ff9dfff7740 1 -- 172.21.15.64:0/2128096549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ff3b73daf0 msgr2=0x55ff3b75e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:30.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.309+0000 7ff9dfff7740 1 --2- 172.21.15.64:0/2128096549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ff3b73daf0 0x55ff3b75e020 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8009230 tx=0x7ff9c8009260 comp rx=0 tx=0).stop 2024-09-12T03:38:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.309+0000 7ff9dfff7740 1 -- 172.21.15.64:0/2128096549 shutdown_connections 2024-09-12T03:38:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.309+0000 7ff9dfff7740 1 --2- 172.21.15.64:0/2128096549 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55ff3b7c36f0 0x55ff3b7c5ae0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.309+0000 7ff9dfff7740 1 --2- 172.21.15.64:0/2128096549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff3b748cb0 0x55ff3b75b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.309+0000 7ff9dfff7740 1 --2- 172.21.15.64:0/2128096549 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ff3b73e410 0x55ff3b75e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.309+0000 7ff9be7fc640 0 client.16104 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:38:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.309+0000 7ff9dfff7740 1 --2- 172.21.15.64:0/2128096549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ff3b73daf0 0x55ff3b75e020 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.309+0000 7ff9dfff7740 1 -- 172.21.15.64:0/2128096549 >> 172.21.15.64:0/2128096549 conn(0x55ff3b73a060 msgr2=0x55ff3b7391c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:38:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.309+0000 7ff9dfff7740 1 -- 172.21.15.64:0/2128096549 shutdown_connections 2024-09-12T03:38:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.309+0000 7ff9dfff7740 1 -- 172.21.15.64:0/2128096549 wait complete. 2024-09-12T03:38:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.309+0000 7ff9dfff7740 20 client.16104 trim_cache size 0 max 16384 2024-09-12T03:38:30.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:38:30.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:38:30.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:38:30.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:38:30.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.310+0000 7ff9dd186640 1 -- 172.21.15.64:0/409385920 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7ff9c40a2da0 msgr2=0x7ff9c40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:30.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.310+0000 7ff9dd186640 1 --2- 172.21.15.64:0/409385920 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7ff9c40a2da0 0x7ff9c40a5260 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8006010 tx=0x7ff9c8005b50 comp rx=0 tx=0).stop 2024-09-12T03:38:30.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.310+0000 7ff9dd186640 1 -- 172.21.15.64:0/409385920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9d8101350 msgr2=0x7ff9d80fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:30.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.310+0000 7ff9dd186640 1 --2- 172.21.15.64:0/409385920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9d8101350 0x7ff9d80fd320 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff9c000e9f0 tx=0x7ff9c000eeb0 comp rx=0 tx=0).stop 2024-09-12T03:38:30.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.310+0000 7ff9dd186640 1 -- 172.21.15.64:0/409385920 shutdown_connections 2024-09-12T03:38:30.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.310+0000 7ff9dd186640 1 --2- 172.21.15.64:0/409385920 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9d81069c0 0x7ff9d80fd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:30.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.310+0000 7ff9dd186640 1 --2- 172.21.15.64:0/409385920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9d8101350 0x7ff9d80fd320 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:30.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.310+0000 7ff9dd186640 1 --2- 172.21.15.64:0/409385920 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7ff9c40a2da0 0x7ff9c40a5260 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:30.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.310+0000 7ff9dd186640 1 --2- 172.21.15.64:0/409385920 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9d81009b0 0x7ff9d80fcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:30.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.310+0000 7ff9dd186640 1 -- 172.21.15.64:0/409385920 >> 172.21.15.64:0/409385920 conn(0x7ff9d8077990 msgr2=0x7ff9d80ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:38:30.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.310+0000 7ff9dd186640 1 -- 172.21.15.64:0/409385920 shutdown_connections 2024-09-12T03:38:30.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:30.310+0000 7ff9dd186640 1 -- 172.21.15.64:0/409385920 wait complete. 2024-09-12T03:38:31.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:38:31 smithi195 ceph-mon[34786]: pgmap v286: 129 pgs: 129 active+clean; 4.6 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-12T03:38:31.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:38:31 smithi064 ceph-mon[30952]: pgmap v286: 129 pgs: 129 active+clean; 4.6 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-12T03:38:31.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:38:31 smithi115 ceph-mon[34801]: pgmap v286: 129 pgs: 129 active+clean; 4.6 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-12T03:38:33.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:38:33 smithi195 ceph-mon[34786]: pgmap v287: 129 pgs: 129 active+clean; 4.8 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 64 KiB/s wr, 21 op/s 2024-09-12T03:38:33.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:38:33 smithi064 ceph-mon[30952]: pgmap v287: 129 pgs: 129 active+clean; 4.8 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 64 KiB/s wr, 21 op/s 2024-09-12T03:38:33.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:38:33 smithi115 ceph-mon[34801]: pgmap v287: 129 pgs: 129 active+clean; 4.8 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 64 KiB/s wr, 21 op/s 2024-09-12T03:38:35.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:38:35 smithi195 ceph-mon[34786]: pgmap v288: 129 pgs: 129 active+clean; 4.8 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-12T03:38:35.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:38:35 smithi064 ceph-mon[30952]: pgmap v288: 129 pgs: 129 active+clean; 4.8 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-12T03:38:35.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:38:35 smithi115 ceph-mon[34801]: pgmap v288: 129 pgs: 129 active+clean; 4.8 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-12T03:38:37.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:38:37 smithi195 ceph-mon[34786]: pgmap v289: 129 pgs: 129 active+clean; 4.8 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-12T03:38:37.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:38:37 smithi064 ceph-mon[30952]: pgmap v289: 129 pgs: 129 active+clean; 4.8 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-12T03:38:37.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:38:37 smithi115 ceph-mon[34801]: pgmap v289: 129 pgs: 129 active+clean; 4.8 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 13 op/s 2024-09-12T03:38:39.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:38:39 smithi195 ceph-mon[34786]: pgmap v290: 129 pgs: 129 active+clean; 5.0 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-09-12T03:38:39.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:38:39 smithi064 ceph-mon[30952]: pgmap v290: 129 pgs: 129 active+clean; 5.0 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-09-12T03:38:39.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:38:39 smithi115 ceph-mon[34801]: pgmap v290: 129 pgs: 129 active+clean; 5.0 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-09-12T03:38:41.256 INFO:tasks.workunit.client.0.smithi064.stdout:437/600: open fd = 3 2024-09-12T03:38:41.257 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.257 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.257 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.257 INFO:tasks.workunit.client.0.smithi064.stdout:438/600: open fd = 3 2024-09-12T03:38:41.257 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.257 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.257 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.257 INFO:tasks.workunit.client.0.smithi064.stdout:439/600: open fd = 3 2024-09-12T03:38:41.258 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.258 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.258 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.258 INFO:tasks.workunit.client.0.smithi064.stdout:440/600: open fd = 3 2024-09-12T03:38:41.258 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.258 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.258 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.258 INFO:tasks.workunit.client.0.smithi064.stdout:441/600: open fd = 3 2024-09-12T03:38:41.258 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.258 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.258 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.259 INFO:tasks.workunit.client.0.smithi064.stdout:442/600: open fd = 3 2024-09-12T03:38:41.259 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.259 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.259 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.259 INFO:tasks.workunit.client.0.smithi064.stdout:443/600: open fd = 3 2024-09-12T03:38:41.259 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.259 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.259 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.259 INFO:tasks.workunit.client.0.smithi064.stdout:444/600: open fd = 3 2024-09-12T03:38:41.259 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.259 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.259 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.259 INFO:tasks.workunit.client.0.smithi064.stdout:445/600: open fd = 3 2024-09-12T03:38:41.259 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.260 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.260 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.260 INFO:tasks.workunit.client.0.smithi064.stdout:446/600: open fd = 3 2024-09-12T03:38:41.260 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.260 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.260 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.260 INFO:tasks.workunit.client.0.smithi064.stdout:447/600: open fd = 3 2024-09-12T03:38:41.260 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.260 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.260 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.260 INFO:tasks.workunit.client.0.smithi064.stdout:448/600: open fd = 3 2024-09-12T03:38:41.260 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.260 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.260 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.261 INFO:tasks.workunit.client.0.smithi064.stdout:449/600: open fd = 3 2024-09-12T03:38:41.261 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.261 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.261 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.261 INFO:tasks.workunit.client.0.smithi064.stdout:450/600: open fd = 3 2024-09-12T03:38:41.261 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.261 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.261 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.261 INFO:tasks.workunit.client.0.smithi064.stdout:451/600: open fd = 3 2024-09-12T03:38:41.261 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.261 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.261 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.262 INFO:tasks.workunit.client.0.smithi064.stdout:452/600: open fd = 3 2024-09-12T03:38:41.262 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.262 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.262 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.262 INFO:tasks.workunit.client.0.smithi064.stdout:453/600: open fd = 3 2024-09-12T03:38:41.262 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.262 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.262 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.262 INFO:tasks.workunit.client.0.smithi064.stdout:454/600: open fd = 3 2024-09-12T03:38:41.262 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.262 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.262 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.262 INFO:tasks.workunit.client.0.smithi064.stdout:455/600: open fd = 3 2024-09-12T03:38:41.262 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.263 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.263 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.263 INFO:tasks.workunit.client.0.smithi064.stdout:456/600: open fd = 3 2024-09-12T03:38:41.263 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.263 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.263 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.263 INFO:tasks.workunit.client.0.smithi064.stdout:457/600: open fd = 3 2024-09-12T03:38:41.263 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.263 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.263 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.263 INFO:tasks.workunit.client.0.smithi064.stdout:458/600: open fd = 3 2024-09-12T03:38:41.263 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.263 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.263 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.264 INFO:tasks.workunit.client.0.smithi064.stdout:459/600: open fd = 3 2024-09-12T03:38:41.264 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.264 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.264 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.264 INFO:tasks.workunit.client.0.smithi064.stdout:460/600: open fd = 3 2024-09-12T03:38:41.264 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.264 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.264 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.264 INFO:tasks.workunit.client.0.smithi064.stdout:461/600: open fd = 3 2024-09-12T03:38:41.264 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.264 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.264 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.264 INFO:tasks.workunit.client.0.smithi064.stdout:462/600: open fd = 3 2024-09-12T03:38:41.264 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.265 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.265 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.265 INFO:tasks.workunit.client.0.smithi064.stdout:463/600: open fd = 3 2024-09-12T03:38:41.265 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.265 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.265 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.265 INFO:tasks.workunit.client.0.smithi064.stdout:464/600: open fd = 3 2024-09-12T03:38:41.265 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.265 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.265 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.265 INFO:tasks.workunit.client.0.smithi064.stdout:465/600: open fd = 3 2024-09-12T03:38:41.265 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.265 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.265 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.265 INFO:tasks.workunit.client.0.smithi064.stdout:466/600: open fd = 3 2024-09-12T03:38:41.266 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.266 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.266 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.266 INFO:tasks.workunit.client.0.smithi064.stdout:467/600: open fd = 3 2024-09-12T03:38:41.266 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.266 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.266 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.266 INFO:tasks.workunit.client.0.smithi064.stdout:468/600: open fd = 3 2024-09-12T03:38:41.266 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.266 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.266 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.266 INFO:tasks.workunit.client.0.smithi064.stdout:469/600: open fd = 3 2024-09-12T03:38:41.266 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.267 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.267 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.267 INFO:tasks.workunit.client.0.smithi064.stdout:470/600: open fd = 3 2024-09-12T03:38:41.267 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.267 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.267 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.267 INFO:tasks.workunit.client.0.smithi064.stdout:471/600: open fd = 3 2024-09-12T03:38:41.267 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.267 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.267 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.267 INFO:tasks.workunit.client.0.smithi064.stdout:472/600: open fd = 3 2024-09-12T03:38:41.267 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.267 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.267 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.268 INFO:tasks.workunit.client.0.smithi064.stdout:473/600: open fd = 3 2024-09-12T03:38:41.268 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.268 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.268 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.268 INFO:tasks.workunit.client.0.smithi064.stdout:474/600: open fd = 3 2024-09-12T03:38:41.268 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.268 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.268 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.269 INFO:tasks.workunit.client.0.smithi064.stdout:475/600: open fd = 3 2024-09-12T03:38:41.269 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.269 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.269 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.269 INFO:tasks.workunit.client.0.smithi064.stdout:476/600: open fd = 3 2024-09-12T03:38:41.269 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.270 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.270 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.270 INFO:tasks.workunit.client.0.smithi064.stdout:477/600: open fd = 3 2024-09-12T03:38:41.270 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.270 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.270 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.270 INFO:tasks.workunit.client.0.smithi064.stdout:478/600: open fd = 3 2024-09-12T03:38:41.270 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.270 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.270 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.270 INFO:tasks.workunit.client.0.smithi064.stdout:479/600: open fd = 3 2024-09-12T03:38:41.270 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.270 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.271 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.271 INFO:tasks.workunit.client.0.smithi064.stdout:480/600: open fd = 3 2024-09-12T03:38:41.271 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.271 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.271 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.271 INFO:tasks.workunit.client.0.smithi064.stdout:481/600: open fd = 3 2024-09-12T03:38:41.271 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.271 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.271 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.271 INFO:tasks.workunit.client.0.smithi064.stdout:482/600: open fd = 3 2024-09-12T03:38:41.271 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.271 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.271 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.271 INFO:tasks.workunit.client.0.smithi064.stdout:483/600: open fd = 3 2024-09-12T03:38:41.272 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.272 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.272 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.272 INFO:tasks.workunit.client.0.smithi064.stdout:484/600: open fd = 3 2024-09-12T03:38:41.272 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.272 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.272 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.272 INFO:tasks.workunit.client.0.smithi064.stdout:485/600: open fd = 3 2024-09-12T03:38:41.272 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.272 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.272 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.272 INFO:tasks.workunit.client.0.smithi064.stdout:486/600: open fd = 3 2024-09-12T03:38:41.272 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.272 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.273 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.273 INFO:tasks.workunit.client.0.smithi064.stdout:487/600: open fd = 3 2024-09-12T03:38:41.273 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.273 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.273 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.273 INFO:tasks.workunit.client.0.smithi064.stdout:488/600: open fd = 3 2024-09-12T03:38:41.273 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.273 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.273 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.273 INFO:tasks.workunit.client.0.smithi064.stdout:489/600: open fd = 3 2024-09-12T03:38:41.274 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.274 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.274 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.274 INFO:tasks.workunit.client.0.smithi064.stdout:490/600: open fd = 3 2024-09-12T03:38:41.274 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.274 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.274 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.274 INFO:tasks.workunit.client.0.smithi064.stdout:491/600: open fd = 3 2024-09-12T03:38:41.274 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.274 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.275 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.275 INFO:tasks.workunit.client.0.smithi064.stdout:492/600: open fd = 3 2024-09-12T03:38:41.275 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.275 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.275 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.275 INFO:tasks.workunit.client.0.smithi064.stdout:493/600: open fd = 3 2024-09-12T03:38:41.275 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.275 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.275 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.275 INFO:tasks.workunit.client.0.smithi064.stdout:494/600: open fd = 3 2024-09-12T03:38:41.276 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.276 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.276 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.276 INFO:tasks.workunit.client.0.smithi064.stdout:495/600: open fd = 3 2024-09-12T03:38:41.276 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.276 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.276 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.276 INFO:tasks.workunit.client.0.smithi064.stdout:496/600: open fd = 3 2024-09-12T03:38:41.276 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.276 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.276 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.276 INFO:tasks.workunit.client.0.smithi064.stdout:497/600: open fd = 3 2024-09-12T03:38:41.276 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.277 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.277 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.277 INFO:tasks.workunit.client.0.smithi064.stdout:498/600: open fd = 3 2024-09-12T03:38:41.277 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.277 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:38:41.277 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:38:41.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:38:41 smithi195 ceph-mon[34786]: pgmap v291: 129 pgs: 129 active+clean; 5.0 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-12T03:38:41.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:38:41 smithi064 ceph-mon[30952]: pgmap v291: 129 pgs: 129 active+clean; 5.0 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-12T03:38:41.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:38:41 smithi115 ceph-mon[34801]: pgmap v291: 129 pgs: 129 active+clean; 5.0 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-12T03:38:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.410+0000 7fd28f8eb640 1 -- 172.21.15.64:0/3860361364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2881080f0 msgr2=0x7fd28810ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.410+0000 7fd28f8eb640 1 --2- 172.21.15.64:0/3860361364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2881080f0 0x7fd28810ea10 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fd280009ca0 tx=0x7fd28002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:38:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.410+0000 7fd28f8eb640 1 -- 172.21.15.64:0/3860361364 shutdown_connections 2024-09-12T03:38:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.410+0000 7fd28f8eb640 1 --2- 172.21.15.64:0/3860361364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2881080f0 0x7fd28810ea10 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.410+0000 7fd28f8eb640 1 --2- 172.21.15.64:0/3860361364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd288101250 0x7fd288107990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.410+0000 7fd28f8eb640 1 --2- 172.21.15.64:0/3860361364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2881008a0 0x7fd288100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.410+0000 7fd28f8eb640 1 -- 172.21.15.64:0/3860361364 >> 172.21.15.64:0/3860361364 conn(0x7fd288075ef0 msgr2=0x7fd288076300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:38:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.411+0000 7fd28f8eb640 1 -- 172.21.15.64:0/3860361364 shutdown_connections 2024-09-12T03:38:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.411+0000 7fd28f8eb640 1 -- 172.21.15.64:0/3860361364 wait complete. 2024-09-12T03:38:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.411+0000 7fd28f8eb640 1 Processor -- start 2024-09-12T03:38:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.411+0000 7fd28f8eb640 1 -- start start 2024-09-12T03:38:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.411+0000 7fd28f8eb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2881008a0 0x7fd288198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.411+0000 7fd28f8eb640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2881080f0 0x7fd288198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.411+0000 7fd28f8eb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd28819cac0 0x7fd28819cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.411+0000 7fd28f8eb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd288111dd0 con 0x7fd28819cac0 2024-09-12T03:38:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.411+0000 7fd28f8eb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd288111c50 con 0x7fd2881008a0 2024-09-12T03:38:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.411+0000 7fd28f8eb640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd288111f50 con 0x7fd2881080f0 2024-09-12T03:38:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.412+0000 7fd28de88640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd28819cac0 0x7fd28819cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.412+0000 7fd28de88640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd28819cac0 0x7fd28819cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56496/0 (socket says 172.21.15.64:56496) 2024-09-12T03:38:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.412+0000 7fd28de88640 1 -- 172.21.15.64:0/967471005 learned_addr learned my addr 172.21.15.64:0/967471005 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:38:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.412+0000 7fd28ce86640 1 --2- 172.21.15.64:0/967471005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2881080f0 0x7fd288198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.412+0000 7fd28d687640 1 --2- 172.21.15.64:0/967471005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2881008a0 0x7fd288198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.412+0000 7fd28de88640 1 -- 172.21.15.64:0/967471005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2881080f0 msgr2=0x7fd288198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.412+0000 7fd28de88640 1 --2- 172.21.15.64:0/967471005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2881080f0 0x7fd288198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.412+0000 7fd28de88640 1 -- 172.21.15.64:0/967471005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2881008a0 msgr2=0x7fd288198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.412+0000 7fd28de88640 1 --2- 172.21.15.64:0/967471005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2881008a0 0x7fd288198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.412+0000 7fd28de88640 1 -- 172.21.15.64:0/967471005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd28819d5b0 con 0x7fd28819cac0 2024-09-12T03:38:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.412+0000 7fd28d687640 1 --2- 172.21.15.64:0/967471005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2881008a0 0x7fd288198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:38:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.412+0000 7fd28de88640 1 --2- 172.21.15.64:0/967471005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd28819cac0 0x7fd28819cf70 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fd280009dc0 tx=0x7fd280032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:38:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.412+0000 7fd27e7fc640 1 -- 172.21.15.64:0/967471005 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd280003cf0 con 0x7fd28819cac0 2024-09-12T03:38:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.412+0000 7fd27e7fc640 1 -- 172.21.15.64:0/967471005 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2800044f0 con 0x7fd28819cac0 2024-09-12T03:38:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.412+0000 7fd28f8eb640 1 -- 172.21.15.64:0/967471005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd28819d840 con 0x7fd28819cac0 2024-09-12T03:38:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.412+0000 7fd27e7fc640 1 -- 172.21.15.64:0/967471005 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd280030bf0 con 0x7fd28819cac0 2024-09-12T03:38:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.413+0000 7fd28f8eb640 1 -- 172.21.15.64:0/967471005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2881a5090 con 0x7fd28819cac0 2024-09-12T03:38:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.416+0000 7fd27e7fc640 1 -- 172.21.15.64:0/967471005 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fd280004040 con 0x7fd28819cac0 2024-09-12T03:38:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.416+0000 7fd29268a740 1 Processor -- start 2024-09-12T03:38:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.416+0000 7fd29268a740 1 -- start start 2024-09-12T03:38:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.416+0000 7fd29268a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8a34eeaf0 0x55e8a34eeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.416+0000 7fd29268a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e8a34ef410 0x55e8a34f9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.416+0000 7fd29268a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8a34f9cb0 0x55e8a34fc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.416+0000 7fd29268a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e8a34532f0 con 0x55e8a34ef410 2024-09-12T03:38:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.416+0000 7fd29268a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e8a33fb270 con 0x55e8a34f9cb0 2024-09-12T03:38:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.416+0000 7fd29268a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e8a344f2b0 con 0x55e8a34eeaf0 2024-09-12T03:38:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.416+0000 7fd28d687640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e8a34ef410 0x55e8a34f9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.416+0000 7fd28d687640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e8a34ef410 0x55e8a34f9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56498/0 (socket says 172.21.15.64:56498) 2024-09-12T03:38:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.416+0000 7fd28d687640 1 -- 172.21.15.64:0/4129518867 learned_addr learned my addr 172.21.15.64:0/4129518867 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:38:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.417+0000 7fd27e7fc640 1 --2- 172.21.15.64:0/967471005 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fd2740a2e20 0x7fd2740a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.417+0000 7fd28ce86640 1 --2- 172.21.15.64:0/4129518867 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8a34eeaf0 0x55e8a34eeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.417+0000 7fd28ce86640 1 --2- 172.21.15.64:0/4129518867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8a34f9cb0 0x55e8a34fc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.417+0000 7fd28d687640 1 -- 172.21.15.64:0/4129518867 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8a34eeaf0 msgr2=0x55e8a34eeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:38:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.417+0000 7fd28d687640 1 --2- 172.21.15.64:0/4129518867 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8a34eeaf0 0x55e8a34eeed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.417+0000 7fd28d687640 1 -- 172.21.15.64:0/4129518867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8a34f9cb0 msgr2=0x55e8a34fc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:38:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.417+0000 7fd28d687640 1 --2- 172.21.15.64:0/4129518867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8a34f9cb0 0x55e8a34fc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.417+0000 7fd28d687640 1 -- 172.21.15.64:0/4129518867 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8a33fa340 con 0x55e8a34ef410 2024-09-12T03:38:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.417+0000 7fd28ce86640 1 --2- 172.21.15.64:0/4129518867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8a34f9cb0 0x55e8a34fc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:38:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.417+0000 7fd28de88640 1 --2- 172.21.15.64:0/967471005 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fd2740a2e20 0x7fd2740a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.417+0000 7fd28d687640 1 --2- 172.21.15.64:0/4129518867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e8a34ef410 0x55e8a34f9770 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd28400bd40 tx=0x7fd28400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22cbdb4960246517 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:38:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.417+0000 7fd27e7fc640 1 -- 172.21.15.64:0/967471005 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9583+0+0 (secure 0 0 0) 0x7fd2800f0d00 con 0x7fd28819cac0 2024-09-12T03:38:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.417+0000 7fd2737fe640 1 -- 172.21.15.64:0/4129518867 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd284015070 con 0x55e8a34ef410 2024-09-12T03:38:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.417+0000 7fd2737fe640 1 -- 172.21.15.64:0/4129518867 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd28400be30 con 0x55e8a34ef410 2024-09-12T03:38:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.417+0000 7fd2737fe640 1 -- 172.21.15.64:0/4129518867 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd284011990 con 0x55e8a34ef410 2024-09-12T03:38:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.418+0000 7fd29268a740 1 -- 172.21.15.64:0/4129518867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e8a34ef410 msgr2=0x55e8a34f9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.418+0000 7fd28de88640 1 --2- 172.21.15.64:0/967471005 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fd2740a2e20 0x7fd2740a52e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd2800ee630 tx=0x7fd2800095c0 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:38:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.418+0000 7fd29268a740 1 --2- 172.21.15.64:0/4129518867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e8a34ef410 0x55e8a34f9770 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd28400bd40 tx=0x7fd28400f730 comp rx=0 tx=0).stop 2024-09-12T03:38:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.418+0000 7fd29268a740 1 -- 172.21.15.64:0/4129518867 shutdown_connections 2024-09-12T03:38:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.418+0000 7fd29268a740 1 --2- 172.21.15.64:0/4129518867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8a34f9cb0 0x55e8a34fc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.418+0000 7fd29268a740 1 --2- 172.21.15.64:0/4129518867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e8a34ef410 0x55e8a34f9770 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.418+0000 7fd29268a740 1 --2- 172.21.15.64:0/4129518867 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8a34eeaf0 0x55e8a34eeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.418+0000 7fd29268a740 1 -- 172.21.15.64:0/4129518867 >> 172.21.15.64:0/4129518867 conn(0x55e8a34eb060 msgr2=0x55e8a34eb440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:38:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.418+0000 7fd29268a740 1 -- 172.21.15.64:0/4129518867 shutdown_connections 2024-09-12T03:38:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.418+0000 7fd29268a740 1 -- 172.21.15.64:0/4129518867 wait complete. 2024-09-12T03:38:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.418+0000 7fd29268a740 1 -- start start 2024-09-12T03:38:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.419+0000 7fd29268a740 1 Processor -- start 2024-09-12T03:38:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.419+0000 7fd29268a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8a34eeaf0 0x55e8a34f6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.419+0000 7fd29268a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e8a34ef410 0x55e8a34f7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.419+0000 7fd29268a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8a34f9cb0 0x55e8a34f75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.419+0000 7fd29268a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e8a34fc880 con 0x55e8a34ef410 2024-09-12T03:38:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.419+0000 7fd29268a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e8a33dc290 con 0x55e8a34f9cb0 2024-09-12T03:38:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.419+0000 7fd29268a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e8a34fca00 con 0x55e8a34eeaf0 2024-09-12T03:38:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.419+0000 7fd29268a740 10 client.0 _ll_register_callbacks cb 0x55e8a340d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:38:43.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.419+0000 7fd28d687640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e8a34ef410 0x55e8a34f7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:43.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.419+0000 7fd28d687640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e8a34ef410 0x55e8a34f7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56500/0 (socket says 172.21.15.64:56500) 2024-09-12T03:38:43.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.419+0000 7fd28d687640 1 -- 172.21.15.64:0/1595983225 learned_addr learned my addr 172.21.15.64:0/1595983225 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:38:43.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.419+0000 7fd28ce86640 1 --2- 172.21.15.64:0/1595983225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8a34f9cb0 0x55e8a34f75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:43.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.420+0000 7fd28d687640 1 -- 172.21.15.64:0/1595983225 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8a34eeaf0 msgr2=0x55e8a34f6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:38:43.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.420+0000 7fd28d687640 1 --2- 172.21.15.64:0/1595983225 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8a34eeaf0 0x55e8a34f6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:43.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.420+0000 7fd28d687640 1 -- 172.21.15.64:0/1595983225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8a34f9cb0 msgr2=0x55e8a34f75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:38:43.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.420+0000 7fd28d687640 1 --2- 172.21.15.64:0/1595983225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8a34f9cb0 0x55e8a34f75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:43.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.420+0000 7fd28d687640 1 -- 172.21.15.64:0/1595983225 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8a3166210 con 0x55e8a34ef410 2024-09-12T03:38:43.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.420+0000 7fd28d687640 1 --2- 172.21.15.64:0/1595983225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e8a34ef410 0x55e8a34f7090 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd28400c010 tx=0x7fd284005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b5acacdbe071bcb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:38:43.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.420+0000 7fd270ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:38:43.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.420+0000 7fd270ff9640 1 -- 172.21.15.64:0/1595983225 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd284015070 con 0x55e8a34ef410 2024-09-12T03:38:43.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.420+0000 7fd270ff9640 1 -- 172.21.15.64:0/1595983225 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9583+0+0 (secure 0 0 0) 0x7fd284004020 con 0x55e8a34ef410 2024-09-12T03:38:43.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.420+0000 7fd29268a740 1 -- 172.21.15.64:0/1595983225 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd28400be30 con 0x55e8a34ef410 2024-09-12T03:38:43.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.420+0000 7fd270ff9640 1 -- 172.21.15.64:0/1595983225 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd28402e050 con 0x55e8a34ef410 2024-09-12T03:38:43.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.420+0000 7fd29268a740 10 client.16116 fetch_fsmap learned FSMap version 25 2024-09-12T03:38:43.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.420+0000 7fd29268a740 1 -- 172.21.15.64:0/1595983225 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e8a3232620 con 0x55e8a34ef410 2024-09-12T03:38:43.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.420+0000 7fd270ff9640 1 -- 172.21.15.64:0/1595983225 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd28400be30 con 0x55e8a34ef410 2024-09-12T03:38:43.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.421+0000 7fd270ff9640 1 -- 172.21.15.64:0/1595983225 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd2840046c0 con 0x55e8a34ef410 2024-09-12T03:38:43.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.421+0000 7fd29268a740 10 client.16116 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:38:43.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.421+0000 7fd29268a740 10 client.16116 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:38:43.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.421+0000 7fd29268a740 20 client.16116 populate_metadata read hostname 'smithi064' 2024-09-12T03:38:43.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.421+0000 7fd29268a740 1 --2- 172.21.15.64:0/1595983225 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55e8a3509b30 0x55e8a350bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:43.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.421+0000 7fd29268a740 4 client.16116 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:38:43.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.421+0000 7fd29268a740 1 -- 172.21.15.64:0/1595983225 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8a3460a20 con 0x55e8a3509b30 2024-09-12T03:38:43.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.421+0000 7fd28ce86640 1 --2- 172.21.15.64:0/1595983225 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55e8a3509b30 0x55e8a350bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:43.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.422+0000 7fd270ff9640 10 client.16116 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:38:43.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.422+0000 7fd28ce86640 1 --2- 172.21.15.64:0/1595983225 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55e8a3509b30 0x55e8a350bf20 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fef08e0e4a3e5355 server_cookie=39b699271f283a6b in_seq=0 out_seq=0 2024-09-12T03:38:43.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.422+0000 7fd270ff9640 1 -- 172.21.15.64:0/1595983225 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e8a3460a20 con 0x55e8a3509b30 2024-09-12T03:38:43.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.422+0000 7fd270ff9640 10 client.16116 handle_command_reply: tid=0 2024-09-12T03:38:43.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.423+0000 7fd29268a740 1 client.16116 shutdown 2024-09-12T03:38:43.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.423+0000 7fd29268a740 1 -- 172.21.15.64:0/1595983225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e8a34ef410 msgr2=0x55e8a34f7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:43.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.423+0000 7fd29268a740 1 --2- 172.21.15.64:0/1595983225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e8a34ef410 0x55e8a34f7090 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd28400c010 tx=0x7fd284005270 comp rx=0 tx=0).stop 2024-09-12T03:38:43.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.423+0000 7fd29268a740 1 -- 172.21.15.64:0/1595983225 shutdown_connections 2024-09-12T03:38:43.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.423+0000 7fd29268a740 1 --2- 172.21.15.64:0/1595983225 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55e8a3509b30 0x55e8a350bf20 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:43.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.423+0000 7fd29268a740 1 --2- 172.21.15.64:0/1595983225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8a34f9cb0 0x55e8a34f75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:43.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.423+0000 7fd29268a740 1 --2- 172.21.15.64:0/1595983225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e8a34ef410 0x55e8a34f7090 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:43.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.423+0000 7fd270ff9640 0 client.16116 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:38:43.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.423+0000 7fd29268a740 1 --2- 172.21.15.64:0/1595983225 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8a34eeaf0 0x55e8a34f6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:43.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.423+0000 7fd29268a740 1 -- 172.21.15.64:0/1595983225 >> 172.21.15.64:0/1595983225 conn(0x55e8a34eb060 msgr2=0x55e8a34f0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:38:43.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.423+0000 7fd29268a740 1 -- 172.21.15.64:0/1595983225 shutdown_connections 2024-09-12T03:38:43.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.423+0000 7fd29268a740 1 -- 172.21.15.64:0/1595983225 wait complete. 2024-09-12T03:38:43.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.423+0000 7fd29268a740 20 client.16116 trim_cache size 0 max 16384 2024-09-12T03:38:43.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.438+0000 7fd29268a740 1 Processor -- start 2024-09-12T03:38:43.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.438+0000 7fd29268a740 1 -- start start 2024-09-12T03:38:43.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.438+0000 7fd29268a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8a34eeaf0 0x55e8a34f7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:43.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.438+0000 7fd29268a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e8a34ef410 0x55e8a3511550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:43.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.438+0000 7fd29268a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8a34f9cb0 0x55e8a3511a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:43.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.438+0000 7fd29268a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd2881b1860 con 0x55e8a34ef410 2024-09-12T03:38:43.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.438+0000 7fd29268a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd2881af130 con 0x55e8a34f9cb0 2024-09-12T03:38:43.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.438+0000 7fd29268a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd2881acfa0 con 0x55e8a34eeaf0 2024-09-12T03:38:43.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.438+0000 7fd28d687640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e8a34ef410 0x55e8a3511550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:43.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.438+0000 7fd28d687640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e8a34ef410 0x55e8a3511550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56510/0 (socket says 172.21.15.64:56510) 2024-09-12T03:38:43.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.438+0000 7fd28d687640 1 -- 172.21.15.64:0/2980578435 learned_addr learned my addr 172.21.15.64:0/2980578435 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:38:43.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.438+0000 7fd28ce86640 1 --2- 172.21.15.64:0/2980578435 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8a34f9cb0 0x55e8a3511a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:43.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.439+0000 7fd28ce86640 1 --2- 172.21.15.64:0/2980578435 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8a34eeaf0 0x55e8a34f7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:43.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.439+0000 7fd28d687640 1 -- 172.21.15.64:0/2980578435 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8a34eeaf0 msgr2=0x55e8a34f7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:38:43.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.439+0000 7fd28d687640 1 --2- 172.21.15.64:0/2980578435 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8a34eeaf0 0x55e8a34f7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:43.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.439+0000 7fd28d687640 1 -- 172.21.15.64:0/2980578435 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8a34f9cb0 msgr2=0x55e8a3511a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:38:43.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.439+0000 7fd28d687640 1 --2- 172.21.15.64:0/2980578435 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8a34f9cb0 0x55e8a3511a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:43.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.439+0000 7fd28d687640 1 -- 172.21.15.64:0/2980578435 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8a34f96a0 con 0x55e8a34ef410 2024-09-12T03:38:43.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.439+0000 7fd28ce86640 1 --2- 172.21.15.64:0/2980578435 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8a34f9cb0 0x55e8a3511a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:38:43.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.439+0000 7fd28d687640 1 --2- 172.21.15.64:0/2980578435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e8a34ef410 0x55e8a3511550 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fd28401af00 tx=0x7fd28400b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33000a85c3665459 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:38:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.439+0000 7fd272ffd640 1 -- 172.21.15.64:0/2980578435 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd284015070 con 0x55e8a34ef410 2024-09-12T03:38:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.439+0000 7fd272ffd640 1 -- 172.21.15.64:0/2980578435 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd284004d00 con 0x55e8a34ef410 2024-09-12T03:38:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.439+0000 7fd272ffd640 1 -- 172.21.15.64:0/2980578435 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28401a4b0 con 0x55e8a34ef410 2024-09-12T03:38:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.440+0000 7fd29268a740 1 -- 172.21.15.64:0/2980578435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e8a34ef410 msgr2=0x55e8a3511550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.440+0000 7fd29268a740 1 --2- 172.21.15.64:0/2980578435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e8a34ef410 0x55e8a3511550 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fd28401af00 tx=0x7fd28400b9c0 comp rx=0 tx=0).stop 2024-09-12T03:38:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.440+0000 7fd29268a740 1 -- 172.21.15.64:0/2980578435 shutdown_connections 2024-09-12T03:38:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.440+0000 7fd29268a740 1 --2- 172.21.15.64:0/2980578435 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8a34f9cb0 0x55e8a3511a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.440+0000 7fd29268a740 1 --2- 172.21.15.64:0/2980578435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e8a34ef410 0x55e8a3511550 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.440+0000 7fd29268a740 1 --2- 172.21.15.64:0/2980578435 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8a34eeaf0 0x55e8a34f7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.440+0000 7fd29268a740 1 -- 172.21.15.64:0/2980578435 >> 172.21.15.64:0/2980578435 conn(0x55e8a34eb060 msgr2=0x55e8a34f8bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:38:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.440+0000 7fd29268a740 1 -- 172.21.15.64:0/2980578435 shutdown_connections 2024-09-12T03:38:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.440+0000 7fd29268a740 1 -- 172.21.15.64:0/2980578435 wait complete. 2024-09-12T03:38:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.440+0000 7fd29268a740 1 -- start start 2024-09-12T03:38:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.440+0000 7fd29268a740 1 Processor -- start 2024-09-12T03:38:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.440+0000 7fd29268a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e8a34eeaf0 0x55e8a35021e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.441+0000 7fd29268a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8a34ef410 0x55e8a34f35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.441+0000 7fd29268a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8a34f9cb0 0x55e8a34f3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.441+0000 7fd29268a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd2881ad5f0 con 0x55e8a34eeaf0 2024-09-12T03:38:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.441+0000 7fd29268a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd2881aea60 con 0x55e8a34ef410 2024-09-12T03:38:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.441+0000 7fd29268a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd2881adcc0 con 0x55e8a34f9cb0 2024-09-12T03:38:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.441+0000 7fd29268a740 10 client.0 _ll_register_callbacks cb 0x7fd28400bbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:38:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.441+0000 7fd28ce86640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e8a34eeaf0 0x55e8a35021e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.441+0000 7fd28ce86640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e8a34eeaf0 0x55e8a35021e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56524/0 (socket says 172.21.15.64:56524) 2024-09-12T03:38:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.441+0000 7fd28ce86640 1 -- 172.21.15.64:0/321456140 learned_addr learned my addr 172.21.15.64:0/321456140 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:38:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.441+0000 7fd28ce86640 1 --2- 172.21.15.64:0/321456140 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8a34f9cb0 0x55e8a34f3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.441+0000 7fd28d687640 1 --2- 172.21.15.64:0/321456140 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8a34ef410 0x55e8a34f35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.441+0000 7fd28ce86640 1 -- 172.21.15.64:0/321456140 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8a34f9cb0 msgr2=0x55e8a34f3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:38:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.441+0000 7fd28ce86640 1 --2- 172.21.15.64:0/321456140 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8a34f9cb0 0x55e8a34f3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.441+0000 7fd28ce86640 1 -- 172.21.15.64:0/321456140 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8a34ef410 msgr2=0x55e8a34f35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:38:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.441+0000 7fd28ce86640 1 --2- 172.21.15.64:0/321456140 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8a34ef410 0x55e8a34f35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.441+0000 7fd28ce86640 1 -- 172.21.15.64:0/321456140 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8a34f40e0 con 0x55e8a34eeaf0 2024-09-12T03:38:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.441+0000 7fd28ce86640 1 --2- 172.21.15.64:0/321456140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e8a34eeaf0 0x55e8a35021e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fd27800bd70 tx=0x7fd278004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a17990b7e3c4bc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:38:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.441+0000 7fd270ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:38:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.441+0000 7fd270ff9640 1 -- 172.21.15.64:0/321456140 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd278005020 con 0x55e8a34eeaf0 2024-09-12T03:38:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.442+0000 7fd270ff9640 1 -- 172.21.15.64:0/321456140 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9583+0+0 (secure 0 0 0) 0x7fd2780051c0 con 0x55e8a34eeaf0 2024-09-12T03:38:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.442+0000 7fd29268a740 1 -- 172.21.15.64:0/321456140 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd284004d00 con 0x55e8a34eeaf0 2024-09-12T03:38:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.442+0000 7fd270ff9640 1 -- 172.21.15.64:0/321456140 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd278032050 con 0x55e8a34eeaf0 2024-09-12T03:38:43.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.442+0000 7fd29268a740 10 client.16122 fetch_fsmap learned FSMap version 25 2024-09-12T03:38:43.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.442+0000 7fd29268a740 1 -- 172.21.15.64:0/321456140 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e8a3574320 con 0x55e8a34eeaf0 2024-09-12T03:38:43.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.442+0000 7fd270ff9640 1 -- 172.21.15.64:0/321456140 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd284004d00 con 0x55e8a34eeaf0 2024-09-12T03:38:43.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.442+0000 7fd270ff9640 1 -- 172.21.15.64:0/321456140 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd27801e6c0 con 0x55e8a34eeaf0 2024-09-12T03:38:43.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.442+0000 7fd29268a740 10 client.16122 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:38:43.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.442+0000 7fd29268a740 10 client.16122 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:38:43.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.442+0000 7fd29268a740 20 client.16122 populate_metadata read hostname 'smithi064' 2024-09-12T03:38:43.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.442+0000 7fd29268a740 1 --2- 172.21.15.64:0/321456140 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55e8a35749f0 0x55e8a3576e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:43.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.442+0000 7fd29268a740 4 client.16122 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:38:43.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.442+0000 7fd29268a740 1 -- 172.21.15.64:0/321456140 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e8a345ad70 con 0x55e8a35749f0 2024-09-12T03:38:43.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.442+0000 7fd28d687640 1 --2- 172.21.15.64:0/321456140 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55e8a35749f0 0x55e8a3576e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:43.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.443+0000 7fd28d687640 1 --2- 172.21.15.64:0/321456140 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55e8a35749f0 0x55e8a3576e10 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f25d741be170e9f server_cookie=323e4d2fe78296b6 in_seq=0 out_seq=0 2024-09-12T03:38:43.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.443+0000 7fd270ff9640 10 client.16122 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:38:43.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.456+0000 7fd270ff9640 1 -- 172.21.15.64:0/321456140 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e8a345ad70 con 0x55e8a35749f0 2024-09-12T03:38:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.456+0000 7fd270ff9640 10 client.16122 handle_command_reply: tid=0 2024-09-12T03:38:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.456+0000 7fd29268a740 1 client.16122 shutdown 2024-09-12T03:38:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.457+0000 7fd29268a740 1 -- 172.21.15.64:0/321456140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e8a34eeaf0 msgr2=0x55e8a35021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.457+0000 7fd29268a740 1 --2- 172.21.15.64:0/321456140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e8a34eeaf0 0x55e8a35021e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fd27800bd70 tx=0x7fd278004730 comp rx=0 tx=0).stop 2024-09-12T03:38:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.457+0000 7fd29268a740 1 -- 172.21.15.64:0/321456140 shutdown_connections 2024-09-12T03:38:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.457+0000 7fd29268a740 1 --2- 172.21.15.64:0/321456140 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55e8a35749f0 0x55e8a3576e10 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.457+0000 7fd29268a740 1 --2- 172.21.15.64:0/321456140 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8a34f9cb0 0x55e8a34f3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.457+0000 7fd29268a740 1 --2- 172.21.15.64:0/321456140 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8a34ef410 0x55e8a34f35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.457+0000 7fd270ff9640 0 client.16122 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:38:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.457+0000 7fd29268a740 1 --2- 172.21.15.64:0/321456140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e8a34eeaf0 0x55e8a35021e0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.457+0000 7fd29268a740 1 -- 172.21.15.64:0/321456140 >> 172.21.15.64:0/321456140 conn(0x55e8a34eb060 msgr2=0x55e8a34f04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:38:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.457+0000 7fd29268a740 1 -- 172.21.15.64:0/321456140 shutdown_connections 2024-09-12T03:38:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.457+0000 7fd29268a740 1 -- 172.21.15.64:0/321456140 wait complete. 2024-09-12T03:38:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.457+0000 7fd29268a740 20 client.16122 trim_cache size 0 max 16384 2024-09-12T03:38:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:38:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:38:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:38:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:38:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.457+0000 7fd28f8eb640 1 -- 172.21.15.64:0/967471005 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fd2740a2e20 msgr2=0x7fd2740a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.458+0000 7fd28f8eb640 1 --2- 172.21.15.64:0/967471005 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fd2740a2e20 0x7fd2740a52e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd2800ee630 tx=0x7fd2800095c0 comp rx=0 tx=0).stop 2024-09-12T03:38:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.458+0000 7fd28f8eb640 1 -- 172.21.15.64:0/967471005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd28819cac0 msgr2=0x7fd28819cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.458+0000 7fd28f8eb640 1 --2- 172.21.15.64:0/967471005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd28819cac0 0x7fd28819cf70 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fd280009dc0 tx=0x7fd280032ff0 comp rx=0 tx=0).stop 2024-09-12T03:38:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.458+0000 7fd28f8eb640 1 -- 172.21.15.64:0/967471005 shutdown_connections 2024-09-12T03:38:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.458+0000 7fd28f8eb640 1 --2- 172.21.15.64:0/967471005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd28819cac0 0x7fd28819cf70 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.458+0000 7fd28f8eb640 1 --2- 172.21.15.64:0/967471005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2881080f0 0x7fd288198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.458+0000 7fd28f8eb640 1 --2- 172.21.15.64:0/967471005 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fd2740a2e20 0x7fd2740a52e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.458+0000 7fd28f8eb640 1 --2- 172.21.15.64:0/967471005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2881008a0 0x7fd288198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.458+0000 7fd28f8eb640 1 -- 172.21.15.64:0/967471005 >> 172.21.15.64:0/967471005 conn(0x7fd288075ef0 msgr2=0x7fd2880ff5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:38:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.458+0000 7fd28f8eb640 1 -- 172.21.15.64:0/967471005 shutdown_connections 2024-09-12T03:38:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:43.458+0000 7fd28f8eb640 1 -- 172.21.15.64:0/967471005 wait complete. 2024-09-12T03:38:43.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:38:43 smithi195 ceph-mon[34786]: pgmap v292: 129 pgs: 129 active+clean; 5.2 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:38:43.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:38:43 smithi064 ceph-mon[30952]: pgmap v292: 129 pgs: 129 active+clean; 5.2 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:38:43.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:38:43 smithi115 ceph-mon[34801]: pgmap v292: 129 pgs: 129 active+clean; 5.2 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:38:45.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:38:45 smithi064 ceph-mon[30952]: pgmap v293: 129 pgs: 129 active+clean; 5.2 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-12T03:38:45.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:38:45 smithi115 ceph-mon[34801]: pgmap v293: 129 pgs: 129 active+clean; 5.2 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-12T03:38:45.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:38:45 smithi195 ceph-mon[34786]: pgmap v293: 129 pgs: 129 active+clean; 5.2 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-12T03:38:47.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:38:47 smithi064 ceph-mon[30952]: pgmap v294: 129 pgs: 129 active+clean; 5.2 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-12T03:38:47.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:38:47 smithi115 ceph-mon[34801]: pgmap v294: 129 pgs: 129 active+clean; 5.2 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-12T03:38:47.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:38:47 smithi195 ceph-mon[34786]: pgmap v294: 129 pgs: 129 active+clean; 5.2 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-12T03:38:49.758 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:38:49 smithi195 ceph-mon[34786]: pgmap v295: 129 pgs: 129 active+clean; 5.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:38:49.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:38:49 smithi064 ceph-mon[30952]: pgmap v295: 129 pgs: 129 active+clean; 5.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:38:49.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:38:49 smithi115 ceph-mon[34801]: pgmap v295: 129 pgs: 129 active+clean; 5.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:38:51.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:38:51 smithi064 ceph-mon[30952]: pgmap v296: 129 pgs: 129 active+clean; 5.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:38:51.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:38:51 smithi115 ceph-mon[34801]: pgmap v296: 129 pgs: 129 active+clean; 5.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:38:51.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:38:51 smithi195 ceph-mon[34786]: pgmap v296: 129 pgs: 129 active+clean; 5.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:38:53.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:38:53 smithi064 ceph-mon[30952]: pgmap v297: 129 pgs: 129 active+clean; 5.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:38:53.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:38:53 smithi115 ceph-mon[34801]: pgmap v297: 129 pgs: 129 active+clean; 5.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:38:53.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:38:53 smithi195 ceph-mon[34786]: pgmap v297: 129 pgs: 129 active+clean; 5.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:38:55.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:38:55 smithi064 ceph-mon[30952]: pgmap v298: 129 pgs: 129 active+clean; 5.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 13 op/s 2024-09-12T03:38:55.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:38:55 smithi115 ceph-mon[34801]: pgmap v298: 129 pgs: 129 active+clean; 5.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 13 op/s 2024-09-12T03:38:55.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:38:55 smithi195 ceph-mon[34786]: pgmap v298: 129 pgs: 129 active+clean; 5.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 13 op/s 2024-09-12T03:38:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.557+0000 7fcc1a85e640 1 -- 172.21.15.64:0/1140549125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc14107b80 msgr2=0x7fcc14107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.557+0000 7fcc1a85e640 1 --2- 172.21.15.64:0/1140549125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc14107b80 0x7fcc14107f60 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fcc04009930 tx=0x7fcc0402a420 comp rx=0 tx=0).stop 2024-09-12T03:38:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.557+0000 7fcc1a85e640 1 -- 172.21.15.64:0/1140549125 shutdown_connections 2024-09-12T03:38:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.557+0000 7fcc1a85e640 1 --2- 172.21.15.64:0/1140549125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc14102480 0x7fcc1410e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.557+0000 7fcc1a85e640 1 --2- 172.21.15.64:0/1140549125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc14101ae0 0x7fcc14101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.557+0000 7fcc1a85e640 1 --2- 172.21.15.64:0/1140549125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc14107b80 0x7fcc14107f60 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.557+0000 7fcc1a85e640 1 -- 172.21.15.64:0/1140549125 >> 172.21.15.64:0/1140549125 conn(0x7fcc140fd7e0 msgr2=0x7fcc140ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:38:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.557+0000 7fcc1a85e640 1 -- 172.21.15.64:0/1140549125 shutdown_connections 2024-09-12T03:38:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.557+0000 7fcc1a85e640 1 -- 172.21.15.64:0/1140549125 wait complete. 2024-09-12T03:38:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.558+0000 7fcc1a85e640 1 Processor -- start 2024-09-12T03:38:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.558+0000 7fcc1a85e640 1 -- start start 2024-09-12T03:38:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.558+0000 7fcc1a85e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc14101ae0 0x7fcc141983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.558+0000 7fcc1a85e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc14102480 0x7fcc14198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.558+0000 7fcc1a85e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc14107b80 0x7fcc1419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.558+0000 7fcc1a85e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcc14111de0 con 0x7fcc14101ae0 2024-09-12T03:38:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.558+0000 7fcc1a85e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcc14111c60 con 0x7fcc14102480 2024-09-12T03:38:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.558+0000 7fcc1a85e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcc14111f60 con 0x7fcc14107b80 2024-09-12T03:38:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.558+0000 7fcc13fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc14101ae0 0x7fcc141983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.558+0000 7fcc13fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc14101ae0 0x7fcc141983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45688/0 (socket says 172.21.15.64:45688) 2024-09-12T03:38:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.558+0000 7fcc13fff640 1 -- 172.21.15.64:0/3786506924 learned_addr learned my addr 172.21.15.64:0/3786506924 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:38:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.558+0000 7fcc18dfb640 1 --2- 172.21.15.64:0/3786506924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc14107b80 0x7fcc1419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.558+0000 7fcc137fe640 1 --2- 172.21.15.64:0/3786506924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc14102480 0x7fcc14198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.559+0000 7fcc13fff640 1 -- 172.21.15.64:0/3786506924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc14107b80 msgr2=0x7fcc1419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.559+0000 7fcc13fff640 1 --2- 172.21.15.64:0/3786506924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc14107b80 0x7fcc1419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.559+0000 7fcc13fff640 1 -- 172.21.15.64:0/3786506924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc14102480 msgr2=0x7fcc14198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.559+0000 7fcc13fff640 1 --2- 172.21.15.64:0/3786506924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc14102480 0x7fcc14198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.559+0000 7fcc13fff640 1 -- 172.21.15.64:0/3786506924 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc1419d3d0 con 0x7fcc14101ae0 2024-09-12T03:38:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.559+0000 7fcc13fff640 1 --2- 172.21.15.64:0/3786506924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc14101ae0 0x7fcc141983e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fcc04004770 tx=0x7fcc040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:38:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.559+0000 7fcc117fa640 1 -- 172.21.15.64:0/3786506924 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0403f070 con 0x7fcc14101ae0 2024-09-12T03:38:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.559+0000 7fcc117fa640 1 -- 172.21.15.64:0/3786506924 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc0403b460 con 0x7fcc14101ae0 2024-09-12T03:38:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.559+0000 7fcc117fa640 1 -- 172.21.15.64:0/3786506924 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc04035070 con 0x7fcc14101ae0 2024-09-12T03:38:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.559+0000 7fcc1a85e640 1 -- 172.21.15.64:0/3786506924 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc1419d660 con 0x7fcc14101ae0 2024-09-12T03:38:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.559+0000 7fcc1a85e640 1 -- 172.21.15.64:0/3786506924 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc141a4f50 con 0x7fcc14101ae0 2024-09-12T03:38:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.562+0000 7fcc117fa640 1 -- 172.21.15.64:0/3786506924 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fcc0403b610 con 0x7fcc14101ae0 2024-09-12T03:38:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.562+0000 7fcc1d5fb740 1 Processor -- start 2024-09-12T03:38:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.562+0000 7fcc1d5fb740 1 -- start start 2024-09-12T03:38:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.562+0000 7fcc1d5fb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6ec46c9b0 0x55b6ec46cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.563+0000 7fcc1d5fb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6ec46d2d0 0x55b6ec47bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.563+0000 7fcc1d5fb740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6ec47c0c0 0x55b6ec47e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.563+0000 7fcc1d5fb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b6ec3d12f0 con 0x55b6ec46c9b0 2024-09-12T03:38:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.563+0000 7fcc1d5fb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b6ec379270 con 0x55b6ec46d2d0 2024-09-12T03:38:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.563+0000 7fcc1d5fb740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b6ec3cd2b0 con 0x55b6ec47c0c0 2024-09-12T03:38:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.563+0000 7fcc18dfb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6ec46c9b0 0x55b6ec46cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.563+0000 7fcc18dfb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6ec46c9b0 0x55b6ec46cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45690/0 (socket says 172.21.15.64:45690) 2024-09-12T03:38:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.563+0000 7fcc18dfb640 1 -- 172.21.15.64:0/3510607532 learned_addr learned my addr 172.21.15.64:0/3510607532 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:38:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.563+0000 7fcc13fff640 1 --2- 172.21.15.64:0/3510607532 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6ec46d2d0 0x55b6ec47bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.563+0000 7fcc137fe640 1 --2- 172.21.15.64:0/3510607532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6ec47c0c0 0x55b6ec47e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.563+0000 7fcc18dfb640 1 -- 172.21.15.64:0/3510607532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6ec47c0c0 msgr2=0x55b6ec47e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:38:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.563+0000 7fcc18dfb640 1 --2- 172.21.15.64:0/3510607532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6ec47c0c0 0x55b6ec47e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.563+0000 7fcc18dfb640 1 -- 172.21.15.64:0/3510607532 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6ec46d2d0 msgr2=0x55b6ec47bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:38:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.563+0000 7fcc18dfb640 1 --2- 172.21.15.64:0/3510607532 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6ec46d2d0 0x55b6ec47bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.563+0000 7fcc18dfb640 1 -- 172.21.15.64:0/3510607532 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6ec378340 con 0x55b6ec46c9b0 2024-09-12T03:38:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.563+0000 7fcc137fe640 1 --2- 172.21.15.64:0/3510607532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6ec47c0c0 0x55b6ec47e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:38:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.563+0000 7fcc117fa640 1 --2- 172.21.15.64:0/3786506924 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fcbfc0a2da0 0x7fcbfc0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.563+0000 7fcc18dfb640 1 --2- 172.21.15.64:0/3510607532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6ec46c9b0 0x55b6ec46cd90 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c00a430 tx=0x7fcc0c009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d5eb40db384fee4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:38:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.563+0000 7fcc13fff640 1 --2- 172.21.15.64:0/3786506924 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fcbfc0a2da0 0x7fcbfc0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.563+0000 7fcc117fa640 1 -- 172.21.15.64:0/3786506924 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9583+0+0 (secure 0 0 0) 0x7fcc040efc00 con 0x7fcc14101ae0 2024-09-12T03:38:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.563+0000 7fcbfa7fc640 1 -- 172.21.15.64:0/3510607532 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c013790 con 0x55b6ec46c9b0 2024-09-12T03:38:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.563+0000 7fcbfa7fc640 1 -- 172.21.15.64:0/3510607532 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc0c019070 con 0x55b6ec46c9b0 2024-09-12T03:38:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.563+0000 7fcbfa7fc640 1 -- 172.21.15.64:0/3510607532 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c014ca0 con 0x55b6ec46c9b0 2024-09-12T03:38:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.564+0000 7fcc1d5fb740 1 -- 172.21.15.64:0/3510607532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6ec46c9b0 msgr2=0x55b6ec46cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.564+0000 7fcc13fff640 1 --2- 172.21.15.64:0/3786506924 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fcbfc0a2da0 0x7fcbfc0a5260 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcc040ecfe0 tx=0x7fcc0402a420 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:38:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.564+0000 7fcc1d5fb740 1 --2- 172.21.15.64:0/3510607532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6ec46c9b0 0x55b6ec46cd90 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c00a430 tx=0x7fcc0c009650 comp rx=0 tx=0).stop 2024-09-12T03:38:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.564+0000 7fcc1d5fb740 1 -- 172.21.15.64:0/3510607532 shutdown_connections 2024-09-12T03:38:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.564+0000 7fcc1d5fb740 1 --2- 172.21.15.64:0/3510607532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6ec47c0c0 0x55b6ec47e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.564+0000 7fcc1d5fb740 1 --2- 172.21.15.64:0/3510607532 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6ec46d2d0 0x55b6ec47bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.564+0000 7fcc1d5fb740 1 --2- 172.21.15.64:0/3510607532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6ec46c9b0 0x55b6ec46cd90 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.564+0000 7fcc1d5fb740 1 -- 172.21.15.64:0/3510607532 >> 172.21.15.64:0/3510607532 conn(0x55b6ec469060 msgr2=0x55b6ec469440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:38:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.564+0000 7fcc1d5fb740 1 -- 172.21.15.64:0/3510607532 shutdown_connections 2024-09-12T03:38:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.564+0000 7fcc1d5fb740 1 -- 172.21.15.64:0/3510607532 wait complete. 2024-09-12T03:38:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.565+0000 7fcc1d5fb740 1 -- start start 2024-09-12T03:38:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.565+0000 7fcc1d5fb740 1 Processor -- start 2024-09-12T03:38:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.565+0000 7fcc1d5fb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6ec46c9b0 0x55b6ec478ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.565+0000 7fcc1d5fb740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6ec46d2d0 0x55b6ec479530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.565+0000 7fcc1d5fb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6ec47c0c0 0x55b6ec479a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.565+0000 7fcc1d5fb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b6ec47ec90 con 0x55b6ec46c9b0 2024-09-12T03:38:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.565+0000 7fcc1d5fb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b6ec35a290 con 0x55b6ec47c0c0 2024-09-12T03:38:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.565+0000 7fcc1d5fb740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b6ec47ee10 con 0x55b6ec46d2d0 2024-09-12T03:38:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.565+0000 7fcc1d5fb740 10 client.0 _ll_register_callbacks cb 0x55b6ec38b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:38:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.565+0000 7fcc18dfb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6ec46c9b0 0x55b6ec478ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.565+0000 7fcc18dfb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6ec46c9b0 0x55b6ec478ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45700/0 (socket says 172.21.15.64:45700) 2024-09-12T03:38:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.565+0000 7fcc18dfb640 1 -- 172.21.15.64:0/59986025 learned_addr learned my addr 172.21.15.64:0/59986025 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:38:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.565+0000 7fcc18dfb640 1 --2- 172.21.15.64:0/59986025 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6ec47c0c0 0x55b6ec479a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.565+0000 7fcc18dfb640 1 -- 172.21.15.64:0/59986025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6ec46d2d0 msgr2=0x55b6ec479530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:38:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.566+0000 7fcc18dfb640 1 --2- 172.21.15.64:0/59986025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6ec46d2d0 0x55b6ec479530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.566+0000 7fcc18dfb640 1 -- 172.21.15.64:0/59986025 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6ec47c0c0 msgr2=0x55b6ec479a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:38:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.566+0000 7fcc18dfb640 1 --2- 172.21.15.64:0/59986025 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6ec47c0c0 0x55b6ec479a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.566+0000 7fcc18dfb640 1 -- 172.21.15.64:0/59986025 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6ec0e4210 con 0x55b6ec46c9b0 2024-09-12T03:38:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.566+0000 7fcc18dfb640 1 --2- 172.21.15.64:0/59986025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6ec46c9b0 0x55b6ec478ff0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c014c70 tx=0x7fcc0c007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aef0a8f1b8072af1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:38:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.566+0000 7fcbf37fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:38:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.566+0000 7fcbf37fe640 1 -- 172.21.15.64:0/59986025 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c013790 con 0x55b6ec46c9b0 2024-09-12T03:38:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.566+0000 7fcbf37fe640 1 -- 172.21.15.64:0/59986025 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9583+0+0 (secure 0 0 0) 0x7fcc0c013930 con 0x55b6ec46c9b0 2024-09-12T03:38:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.566+0000 7fcc1d5fb740 1 -- 172.21.15.64:0/59986025 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc0c019070 con 0x55b6ec46c9b0 2024-09-12T03:38:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.566+0000 7fcbf37fe640 1 -- 172.21.15.64:0/59986025 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcc0c033050 con 0x55b6ec46c9b0 2024-09-12T03:38:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.566+0000 7fcc1d5fb740 10 client.16134 fetch_fsmap learned FSMap version 25 2024-09-12T03:38:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.566+0000 7fcc1d5fb740 1 -- 172.21.15.64:0/59986025 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b6ec1b0620 con 0x55b6ec46c9b0 2024-09-12T03:38:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.566+0000 7fcbf37fe640 1 -- 172.21.15.64:0/59986025 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc0c019070 con 0x55b6ec46c9b0 2024-09-12T03:38:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.566+0000 7fcbf37fe640 1 -- 172.21.15.64:0/59986025 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fcc0c021c00 con 0x55b6ec46c9b0 2024-09-12T03:38:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.567+0000 7fcc1d5fb740 10 client.16134 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:38:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.567+0000 7fcc1d5fb740 10 client.16134 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:38:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.567+0000 7fcc1d5fb740 20 client.16134 populate_metadata read hostname 'smithi064' 2024-09-12T03:38:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.567+0000 7fcc1d5fb740 1 --2- 172.21.15.64:0/59986025 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55b6ec48bee0 0x55b6ec48e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.567+0000 7fcc1d5fb740 4 client.16134 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:38:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.567+0000 7fcc1d5fb740 1 -- 172.21.15.64:0/59986025 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6ec3dea20 con 0x55b6ec48bee0 2024-09-12T03:38:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.567+0000 7fcc137fe640 1 --2- 172.21.15.64:0/59986025 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55b6ec48bee0 0x55b6ec48e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.567+0000 7fcbf37fe640 10 client.16134 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:38:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.567+0000 7fcc137fe640 1 --2- 172.21.15.64:0/59986025 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55b6ec48bee0 0x55b6ec48e2d0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af9ee81b14b7aa23 server_cookie=26aa2d584265ce65 in_seq=0 out_seq=0 2024-09-12T03:38:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.568+0000 7fcbf37fe640 1 -- 172.21.15.64:0/59986025 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b6ec3dea20 con 0x55b6ec48bee0 2024-09-12T03:38:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.568+0000 7fcbf37fe640 10 client.16134 handle_command_reply: tid=0 2024-09-12T03:38:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.568+0000 7fcc1d5fb740 1 client.16134 shutdown 2024-09-12T03:38:56.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.568+0000 7fcc1d5fb740 1 -- 172.21.15.64:0/59986025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6ec46c9b0 msgr2=0x55b6ec478ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:56.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.568+0000 7fcc1d5fb740 1 --2- 172.21.15.64:0/59986025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6ec46c9b0 0x55b6ec478ff0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c014c70 tx=0x7fcc0c007000 comp rx=0 tx=0).stop 2024-09-12T03:38:56.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.569+0000 7fcc1d5fb740 1 -- 172.21.15.64:0/59986025 shutdown_connections 2024-09-12T03:38:56.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.569+0000 7fcc1d5fb740 1 --2- 172.21.15.64:0/59986025 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55b6ec48bee0 0x55b6ec48e2d0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:56.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.569+0000 7fcc1d5fb740 1 --2- 172.21.15.64:0/59986025 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6ec47c0c0 0x55b6ec479a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:56.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.569+0000 7fcc1d5fb740 1 --2- 172.21.15.64:0/59986025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6ec46d2d0 0x55b6ec479530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:56.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.569+0000 7fcbf37fe640 0 client.16134 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:38:56.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.569+0000 7fcc1d5fb740 1 --2- 172.21.15.64:0/59986025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6ec46c9b0 0x55b6ec478ff0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:56.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.569+0000 7fcc1d5fb740 1 -- 172.21.15.64:0/59986025 >> 172.21.15.64:0/59986025 conn(0x55b6ec469060 msgr2=0x55b6ec4681c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:38:56.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.569+0000 7fcc1d5fb740 1 -- 172.21.15.64:0/59986025 shutdown_connections 2024-09-12T03:38:56.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.569+0000 7fcc1d5fb740 1 -- 172.21.15.64:0/59986025 wait complete. 2024-09-12T03:38:56.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.569+0000 7fcc1d5fb740 20 client.16134 trim_cache size 0 max 16384 2024-09-12T03:38:56.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.583+0000 7fcc1d5fb740 1 Processor -- start 2024-09-12T03:38:56.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.583+0000 7fcc1d5fb740 1 -- start start 2024-09-12T03:38:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.584+0000 7fcc1d5fb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6ec46c9b0 0x55b6ec493930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.584+0000 7fcc1d5fb740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6ec46d2d0 0x55b6ec493e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.584+0000 7fcc1d5fb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6ec47c0c0 0x55b6ec476230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.584+0000 7fcc1d5fb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcc141af790 con 0x55b6ec46c9b0 2024-09-12T03:38:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.584+0000 7fcc1d5fb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcc141ad060 con 0x55b6ec47c0c0 2024-09-12T03:38:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.584+0000 7fcc1d5fb740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcc141aaed0 con 0x55b6ec46d2d0 2024-09-12T03:38:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.584+0000 7fcc18dfb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6ec46c9b0 0x55b6ec493930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.584+0000 7fcc18dfb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6ec46c9b0 0x55b6ec493930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45714/0 (socket says 172.21.15.64:45714) 2024-09-12T03:38:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.584+0000 7fcc18dfb640 1 -- 172.21.15.64:0/1062821755 learned_addr learned my addr 172.21.15.64:0/1062821755 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:38:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.584+0000 7fcc137fe640 1 --2- 172.21.15.64:0/1062821755 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6ec46d2d0 0x55b6ec493e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.584+0000 7fcc18dfb640 1 -- 172.21.15.64:0/1062821755 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6ec46d2d0 msgr2=0x55b6ec493e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:38:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.584+0000 7fcc18dfb640 1 --2- 172.21.15.64:0/1062821755 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6ec46d2d0 0x55b6ec493e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.584+0000 7fcc18dfb640 1 -- 172.21.15.64:0/1062821755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6ec47c0c0 msgr2=0x55b6ec476230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:38:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.584+0000 7fcc18dfb640 1 --2- 172.21.15.64:0/1062821755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6ec47c0c0 0x55b6ec476230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.584+0000 7fcc18dfb640 1 -- 172.21.15.64:0/1062821755 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6ec477520 con 0x55b6ec46c9b0 2024-09-12T03:38:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.584+0000 7fcc18dfb640 1 --2- 172.21.15.64:0/1062821755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6ec46c9b0 0x55b6ec493930 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c004a80 tx=0x7fcc0c009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f4120c95c9f12e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:38:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.585+0000 7fcbf9ffb640 1 -- 172.21.15.64:0/1062821755 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c01d5a0 con 0x55b6ec46c9b0 2024-09-12T03:38:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.585+0000 7fcbf9ffb640 1 -- 172.21.15.64:0/1062821755 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc0c025070 con 0x55b6ec46c9b0 2024-09-12T03:38:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.585+0000 7fcbf9ffb640 1 -- 172.21.15.64:0/1062821755 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c013bd0 con 0x55b6ec46c9b0 2024-09-12T03:38:56.591 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.585+0000 7fcc1d5fb740 1 -- 172.21.15.64:0/1062821755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6ec46c9b0 msgr2=0x55b6ec493930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:56.591 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.585+0000 7fcc1d5fb740 1 --2- 172.21.15.64:0/1062821755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6ec46c9b0 0x55b6ec493930 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c004a80 tx=0x7fcc0c009ad0 comp rx=0 tx=0).stop 2024-09-12T03:38:56.591 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.585+0000 7fcc1d5fb740 1 -- 172.21.15.64:0/1062821755 shutdown_connections 2024-09-12T03:38:56.591 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.585+0000 7fcc1d5fb740 1 --2- 172.21.15.64:0/1062821755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6ec47c0c0 0x55b6ec476230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:56.591 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.585+0000 7fcc1d5fb740 1 --2- 172.21.15.64:0/1062821755 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6ec46d2d0 0x55b6ec493e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:56.591 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.585+0000 7fcc1d5fb740 1 --2- 172.21.15.64:0/1062821755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6ec46c9b0 0x55b6ec493930 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:56.591 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.585+0000 7fcc1d5fb740 1 -- 172.21.15.64:0/1062821755 >> 172.21.15.64:0/1062821755 conn(0x55b6ec469060 msgr2=0x55b6ec47b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:38:56.591 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.585+0000 7fcc1d5fb740 1 -- 172.21.15.64:0/1062821755 shutdown_connections 2024-09-12T03:38:56.591 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.585+0000 7fcc1d5fb740 1 -- 172.21.15.64:0/1062821755 wait complete. 2024-09-12T03:38:56.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.586+0000 7fcc1d5fb740 1 -- start start 2024-09-12T03:38:56.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.586+0000 7fcc1d5fb740 1 Processor -- start 2024-09-12T03:38:56.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.586+0000 7fcc1d5fb740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6ec46c9b0 0x55b6ec47fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:56.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.586+0000 7fcc1d5fb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6ec46d2d0 0x55b6ec480380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:56.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.586+0000 7fcc1d5fb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6ec47c0c0 0x55b6ec4808c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:56.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.586+0000 7fcc1d5fb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcc141ab520 con 0x55b6ec46d2d0 2024-09-12T03:38:56.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.586+0000 7fcc1d5fb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcc141ac990 con 0x55b6ec47c0c0 2024-09-12T03:38:56.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.586+0000 7fcc1d5fb740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcc141abbf0 con 0x55b6ec46c9b0 2024-09-12T03:38:56.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.586+0000 7fcc1d5fb740 10 client.0 _ll_register_callbacks cb 0x7fcc0c009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:38:56.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.586+0000 7fcc137fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6ec46d2d0 0x55b6ec480380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:56.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.586+0000 7fcc137fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6ec46d2d0 0x55b6ec480380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45720/0 (socket says 172.21.15.64:45720) 2024-09-12T03:38:56.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.586+0000 7fcc137fe640 1 -- 172.21.15.64:0/2338682141 learned_addr learned my addr 172.21.15.64:0/2338682141 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:38:56.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.587+0000 7fcc18dfb640 1 --2- 172.21.15.64:0/2338682141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6ec47c0c0 0x55b6ec4808c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:56.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.587+0000 7fcc137fe640 1 -- 172.21.15.64:0/2338682141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6ec46c9b0 msgr2=0x55b6ec47fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:38:56.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.587+0000 7fcc18dfb640 1 --2- 172.21.15.64:0/2338682141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6ec46c9b0 0x55b6ec47fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:56.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.587+0000 7fcc137fe640 1 --2- 172.21.15.64:0/2338682141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6ec46c9b0 0x55b6ec47fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:56.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.587+0000 7fcc137fe640 1 -- 172.21.15.64:0/2338682141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6ec47c0c0 msgr2=0x55b6ec4808c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:38:56.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.587+0000 7fcc137fe640 1 --2- 172.21.15.64:0/2338682141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6ec47c0c0 0x55b6ec4808c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.587+0000 7fcc137fe640 1 -- 172.21.15.64:0/2338682141 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6ec4f5930 con 0x55b6ec46d2d0 2024-09-12T03:38:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.587+0000 7fcc18dfb640 1 --2- 172.21.15.64:0/2338682141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6ec46c9b0 0x55b6ec47fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:38:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.587+0000 7fcc137fe640 1 --2- 172.21.15.64:0/2338682141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6ec46d2d0 0x55b6ec480380 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fcc0800ba60 tx=0x7fcc08012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99eddd92b726aeee server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:38:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.587+0000 7fcbf37fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:38:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.587+0000 7fcbf37fe640 1 -- 172.21.15.64:0/2338682141 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc08005220 con 0x55b6ec46d2d0 2024-09-12T03:38:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.587+0000 7fcbf37fe640 1 -- 172.21.15.64:0/2338682141 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9583+0+0 (secure 0 0 0) 0x7fcc080053c0 con 0x55b6ec46d2d0 2024-09-12T03:38:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.587+0000 7fcc1d5fb740 1 -- 172.21.15.64:0/2338682141 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc0c025070 con 0x55b6ec46d2d0 2024-09-12T03:38:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.587+0000 7fcbf37fe640 1 -- 172.21.15.64:0/2338682141 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcc080055c0 con 0x55b6ec46d2d0 2024-09-12T03:38:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.587+0000 7fcc1d5fb740 10 client.16140 fetch_fsmap learned FSMap version 25 2024-09-12T03:38:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.587+0000 7fcc1d5fb740 1 -- 172.21.15.64:0/2338682141 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b6ec4f6c90 con 0x55b6ec46d2d0 2024-09-12T03:38:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.587+0000 7fcbf37fe640 1 -- 172.21.15.64:0/2338682141 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc0c025070 con 0x55b6ec46d2d0 2024-09-12T03:38:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.588+0000 7fcbf37fe640 1 -- 172.21.15.64:0/2338682141 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fcc08020070 con 0x55b6ec46d2d0 2024-09-12T03:38:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.588+0000 7fcc1d5fb740 10 client.16140 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:38:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.588+0000 7fcc1d5fb740 10 client.16140 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:38:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.588+0000 7fcc1d5fb740 20 client.16140 populate_metadata read hostname 'smithi064' 2024-09-12T03:38:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.588+0000 7fcc1d5fb740 1 --2- 172.21.15.64:0/2338682141 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55b6ec4f7440 0x55b6ec4f9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:38:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.588+0000 7fcc1d5fb740 4 client.16140 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:38:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.588+0000 7fcc1d5fb740 1 -- 172.21.15.64:0/2338682141 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b6ec3d8d70 con 0x55b6ec4f7440 2024-09-12T03:38:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.588+0000 7fcc18dfb640 1 --2- 172.21.15.64:0/2338682141 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55b6ec4f7440 0x55b6ec4f9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:38:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.588+0000 7fcbf37fe640 10 client.16140 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:38:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.588+0000 7fcc18dfb640 1 --2- 172.21.15.64:0/2338682141 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55b6ec4f7440 0x55b6ec4f9860 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88dfd9ba9d21b75a server_cookie=c48e59f0a92f6b20 in_seq=0 out_seq=0 2024-09-12T03:38:56.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.601+0000 7fcbf37fe640 1 -- 172.21.15.64:0/2338682141 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b6ec3d8d70 con 0x55b6ec4f7440 2024-09-12T03:38:56.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.602+0000 7fcbf37fe640 10 client.16140 handle_command_reply: tid=0 2024-09-12T03:38:56.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.602+0000 7fcc1d5fb740 1 client.16140 shutdown 2024-09-12T03:38:56.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.602+0000 7fcc1d5fb740 1 -- 172.21.15.64:0/2338682141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6ec46d2d0 msgr2=0x55b6ec480380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:56.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.602+0000 7fcc1d5fb740 1 --2- 172.21.15.64:0/2338682141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6ec46d2d0 0x55b6ec480380 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fcc0800ba60 tx=0x7fcc08012820 comp rx=0 tx=0).stop 2024-09-12T03:38:56.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.602+0000 7fcc1d5fb740 1 -- 172.21.15.64:0/2338682141 shutdown_connections 2024-09-12T03:38:56.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.602+0000 7fcc1d5fb740 1 --2- 172.21.15.64:0/2338682141 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55b6ec4f7440 0x55b6ec4f9860 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:56.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.602+0000 7fcc1d5fb740 1 --2- 172.21.15.64:0/2338682141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b6ec47c0c0 0x55b6ec4808c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:56.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.602+0000 7fcc1d5fb740 1 --2- 172.21.15.64:0/2338682141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6ec46d2d0 0x55b6ec480380 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:56.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.602+0000 7fcc1d5fb740 1 --2- 172.21.15.64:0/2338682141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b6ec46c9b0 0x55b6ec47fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:56.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.602+0000 7fcc1d5fb740 1 -- 172.21.15.64:0/2338682141 >> 172.21.15.64:0/2338682141 conn(0x55b6ec469060 msgr2=0x55b6ec478fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:38:56.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.602+0000 7fcbf37fe640 0 client.16140 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:38:56.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.602+0000 7fcc1d5fb740 1 -- 172.21.15.64:0/2338682141 shutdown_connections 2024-09-12T03:38:56.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.602+0000 7fcc1d5fb740 1 -- 172.21.15.64:0/2338682141 wait complete. 2024-09-12T03:38:56.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.602+0000 7fcc1d5fb740 20 client.16140 trim_cache size 0 max 16384 2024-09-12T03:38:56.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:38:56.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:38:56.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:38:56.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:38:56.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.603+0000 7fcc1a85e640 1 -- 172.21.15.64:0/3786506924 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fcbfc0a2da0 msgr2=0x7fcbfc0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:56.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.603+0000 7fcc1a85e640 1 --2- 172.21.15.64:0/3786506924 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fcbfc0a2da0 0x7fcbfc0a5260 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcc040ecfe0 tx=0x7fcc0402a420 comp rx=0 tx=0).stop 2024-09-12T03:38:56.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.603+0000 7fcc1a85e640 1 -- 172.21.15.64:0/3786506924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc14101ae0 msgr2=0x7fcc141983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:38:56.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.603+0000 7fcc1a85e640 1 --2- 172.21.15.64:0/3786506924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc14101ae0 0x7fcc141983e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fcc04004770 tx=0x7fcc040042f0 comp rx=0 tx=0).stop 2024-09-12T03:38:56.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.603+0000 7fcc1a85e640 1 -- 172.21.15.64:0/3786506924 shutdown_connections 2024-09-12T03:38:56.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.603+0000 7fcc1a85e640 1 --2- 172.21.15.64:0/3786506924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc14107b80 0x7fcc1419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:56.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.603+0000 7fcc1a85e640 1 --2- 172.21.15.64:0/3786506924 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fcbfc0a2da0 0x7fcbfc0a5260 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:56.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.603+0000 7fcc1a85e640 1 --2- 172.21.15.64:0/3786506924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc14102480 0x7fcc14198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:56.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.603+0000 7fcc1a85e640 1 --2- 172.21.15.64:0/3786506924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc14101ae0 0x7fcc141983e0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:38:56.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.603+0000 7fcc1a85e640 1 -- 172.21.15.64:0/3786506924 >> 172.21.15.64:0/3786506924 conn(0x7fcc140fd7e0 msgr2=0x7fcc140ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:38:56.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.603+0000 7fcc1a85e640 1 -- 172.21.15.64:0/3786506924 shutdown_connections 2024-09-12T03:38:56.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:38:56.603+0000 7fcc1a85e640 1 -- 172.21.15.64:0/3786506924 wait complete. 2024-09-12T03:38:57.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:38:57 smithi064 ceph-mon[30952]: pgmap v299: 129 pgs: 129 active+clean; 5.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 13 op/s 2024-09-12T03:38:57.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:38:57 smithi115 ceph-mon[34801]: pgmap v299: 129 pgs: 129 active+clean; 5.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 13 op/s 2024-09-12T03:38:57.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:38:57 smithi195 ceph-mon[34786]: pgmap v299: 129 pgs: 129 active+clean; 5.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 13 op/s 2024-09-12T03:38:59.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:38:59 smithi064 ceph-mon[30952]: pgmap v300: 129 pgs: 129 active+clean; 1.8 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 63 KiB/s wr, 21 op/s 2024-09-12T03:38:59.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:38:59 smithi115 ceph-mon[34801]: pgmap v300: 129 pgs: 129 active+clean; 1.8 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 63 KiB/s wr, 21 op/s 2024-09-12T03:38:59.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:38:59 smithi195 ceph-mon[34786]: pgmap v300: 129 pgs: 129 active+clean; 1.8 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 63 KiB/s wr, 21 op/s 2024-09-12T03:39:01.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:01 smithi064 ceph-mon[30952]: pgmap v301: 129 pgs: 129 active+clean; 1.8 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:39:01.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:01 smithi115 ceph-mon[34801]: pgmap v301: 129 pgs: 129 active+clean; 1.8 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:39:01.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:01 smithi195 ceph-mon[34786]: pgmap v301: 129 pgs: 129 active+clean; 1.8 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:39:03.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:03 smithi064 ceph-mon[30952]: pgmap v302: 129 pgs: 129 active+clean; 2.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 102 KiB/s wr, 35 op/s 2024-09-12T03:39:03.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:03 smithi115 ceph-mon[34801]: pgmap v302: 129 pgs: 129 active+clean; 2.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 102 KiB/s wr, 35 op/s 2024-09-12T03:39:03.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:03 smithi195 ceph-mon[34786]: pgmap v302: 129 pgs: 129 active+clean; 2.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 102 KiB/s wr, 35 op/s 2024-09-12T03:39:04.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:04 smithi064 ceph-mon[30952]: osdmap e100: 12 total, 12 up, 12 in 2024-09-12T03:39:04.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:04 smithi115 ceph-mon[34801]: osdmap e100: 12 total, 12 up, 12 in 2024-09-12T03:39:04.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:04 smithi195 ceph-mon[34786]: osdmap e100: 12 total, 12 up, 12 in 2024-09-12T03:39:05.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:05 smithi064 ceph-mon[30952]: pgmap v304: 129 pgs: 129 active+clean; 2.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 103 KiB/s wr, 36 op/s 2024-09-12T03:39:05.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:05 smithi115 ceph-mon[34801]: pgmap v304: 129 pgs: 129 active+clean; 2.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 103 KiB/s wr, 36 op/s 2024-09-12T03:39:05.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:05 smithi195 ceph-mon[34786]: pgmap v304: 129 pgs: 129 active+clean; 2.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 103 KiB/s wr, 36 op/s 2024-09-12T03:39:07.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:07 smithi064 ceph-mon[30952]: pgmap v305: 129 pgs: 129 active+clean; 2.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 103 KiB/s wr, 36 op/s 2024-09-12T03:39:07.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:07 smithi115 ceph-mon[34801]: pgmap v305: 129 pgs: 129 active+clean; 2.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 103 KiB/s wr, 36 op/s 2024-09-12T03:39:07.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:07 smithi195 ceph-mon[34786]: pgmap v305: 129 pgs: 129 active+clean; 2.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 103 KiB/s wr, 36 op/s 2024-09-12T03:39:09.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.703+0000 7fa2c514b640 1 -- 172.21.15.64:0/2893857033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2c010c110 msgr2=0x7fa2c010e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:39:09.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.703+0000 7fa2c514b640 1 --2- 172.21.15.64:0/2893857033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2c010c110 0x7fa2c010e560 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8009ca0 tx=0x7fa2b802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:39:09.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.703+0000 7fa2c514b640 1 -- 172.21.15.64:0/2893857033 shutdown_connections 2024-09-12T03:39:09.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.703+0000 7fa2c514b640 1 --2- 172.21.15.64:0/2893857033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2c010c110 0x7fa2c010e560 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.703+0000 7fa2c514b640 1 --2- 172.21.15.64:0/2893857033 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2c01035c0 0x7fa2c010baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.703+0000 7fa2c514b640 1 --2- 172.21.15.64:0/2893857033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2c0102c10 0x7fa2c0102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.703+0000 7fa2c514b640 1 -- 172.21.15.64:0/2893857033 >> 172.21.15.64:0/2893857033 conn(0x7fa2c00fc6b0 msgr2=0x7fa2c00fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:39:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.703+0000 7fa2c514b640 1 -- 172.21.15.64:0/2893857033 shutdown_connections 2024-09-12T03:39:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.703+0000 7fa2c514b640 1 -- 172.21.15.64:0/2893857033 wait complete. 2024-09-12T03:39:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.704+0000 7fa2c514b640 1 Processor -- start 2024-09-12T03:39:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.704+0000 7fa2c514b640 1 -- start start 2024-09-12T03:39:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.704+0000 7fa2c514b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2c0102c10 0x7fa2c01981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.704+0000 7fa2c514b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2c01035c0 0x7fa2c0198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.704+0000 7fa2c514b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2c010c110 0x7fa2c019caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.704+0000 7fa2c514b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c0111d70 con 0x7fa2c0102c10 2024-09-12T03:39:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.704+0000 7fa2c514b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c0111bf0 con 0x7fa2c010c110 2024-09-12T03:39:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.704+0000 7fa2c514b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c0111ef0 con 0x7fa2c01035c0 2024-09-12T03:39:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.704+0000 7fa2bed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2c0102c10 0x7fa2c01981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.704+0000 7fa2bed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2c0102c10 0x7fa2c01981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49490/0 (socket says 172.21.15.64:49490) 2024-09-12T03:39:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.704+0000 7fa2bed9d640 1 -- 172.21.15.64:0/3306076713 learned_addr learned my addr 172.21.15.64:0/3306076713 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:39:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.705+0000 7fa2be59c640 1 --2- 172.21.15.64:0/3306076713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2c01035c0 0x7fa2c0198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.705+0000 7fa2bf59e640 1 --2- 172.21.15.64:0/3306076713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2c010c110 0x7fa2c019caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.705+0000 7fa2bed9d640 1 -- 172.21.15.64:0/3306076713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2c01035c0 msgr2=0x7fa2c0198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:39:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.705+0000 7fa2bed9d640 1 --2- 172.21.15.64:0/3306076713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2c01035c0 0x7fa2c0198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.705+0000 7fa2bed9d640 1 -- 172.21.15.64:0/3306076713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2c010c110 msgr2=0x7fa2c019caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:39:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.705+0000 7fa2bed9d640 1 --2- 172.21.15.64:0/3306076713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2c010c110 0x7fa2c019caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.705+0000 7fa2bed9d640 1 -- 172.21.15.64:0/3306076713 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c019d1d0 con 0x7fa2c0102c10 2024-09-12T03:39:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.705+0000 7fa2bed9d640 1 --2- 172.21.15.64:0/3306076713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2c0102c10 0x7fa2c01981e0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fa2b000d950 tx=0x7fa2b000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:39:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.705+0000 7fa2a7fff640 1 -- 172.21.15.64:0/3306076713 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b0017030 con 0x7fa2c0102c10 2024-09-12T03:39:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.705+0000 7fa2a7fff640 1 -- 172.21.15.64:0/3306076713 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2b0004020 con 0x7fa2c0102c10 2024-09-12T03:39:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.705+0000 7fa2c514b640 1 -- 172.21.15.64:0/3306076713 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2c019d4c0 con 0x7fa2c0102c10 2024-09-12T03:39:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.705+0000 7fa2a7fff640 1 -- 172.21.15.64:0/3306076713 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b00047a0 con 0x7fa2c0102c10 2024-09-12T03:39:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.705+0000 7fa2c514b640 1 -- 172.21.15.64:0/3306076713 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2c01a4d50 con 0x7fa2c0102c10 2024-09-12T03:39:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.707+0000 7fa2a7fff640 1 -- 172.21.15.64:0/3306076713 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fa2b00041d0 con 0x7fa2c0102c10 2024-09-12T03:39:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.708+0000 7fa2c7fc2740 1 Processor -- start 2024-09-12T03:39:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.708+0000 7fa2c7fc2740 1 -- start start 2024-09-12T03:39:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.708+0000 7fa2c7fc2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624c6c9eaf0 0x5624c6c9eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.708+0000 7fa2c7fc2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5624c6c9f410 0x5624c6ca9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.708+0000 7fa2c7fc2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624c6ca9cb0 0x5624c6cac0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.708+0000 7fa2c7fc2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5624c6c032f0 con 0x5624c6ca9cb0 2024-09-12T03:39:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.708+0000 7fa2c7fc2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5624c6bab270 con 0x5624c6c9eaf0 2024-09-12T03:39:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.708+0000 7fa2c7fc2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5624c6bff2b0 con 0x5624c6c9f410 2024-09-12T03:39:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.708+0000 7fa2be59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624c6ca9cb0 0x5624c6cac0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.708+0000 7fa2be59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624c6ca9cb0 0x5624c6cac0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49498/0 (socket says 172.21.15.64:49498) 2024-09-12T03:39:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.708+0000 7fa2be59c640 1 -- 172.21.15.64:0/133373875 learned_addr learned my addr 172.21.15.64:0/133373875 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:39:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.708+0000 7fa2a7fff640 1 --2- 172.21.15.64:0/3306076713 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fa2a80a2da0 0x7fa2a80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.708+0000 7fa2bf59e640 1 --2- 172.21.15.64:0/133373875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624c6c9eaf0 0x5624c6c9eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.709+0000 7fa2bed9d640 1 --2- 172.21.15.64:0/133373875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5624c6c9f410 0x5624c6ca9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.709+0000 7fa2bf59e640 1 --2- 172.21.15.64:0/3306076713 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fa2a80a2da0 0x7fa2a80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.709+0000 7fa2be59c640 1 -- 172.21.15.64:0/133373875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5624c6c9f410 msgr2=0x5624c6ca9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:39:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.709+0000 7fa2be59c640 1 --2- 172.21.15.64:0/133373875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5624c6c9f410 0x5624c6ca9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.709+0000 7fa2be59c640 1 -- 172.21.15.64:0/133373875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624c6c9eaf0 msgr2=0x5624c6c9eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:39:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.709+0000 7fa2be59c640 1 --2- 172.21.15.64:0/133373875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624c6c9eaf0 0x5624c6c9eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.709+0000 7fa2be59c640 1 -- 172.21.15.64:0/133373875 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624c6baa340 con 0x5624c6ca9cb0 2024-09-12T03:39:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.709+0000 7fa2a7fff640 1 -- 172.21.15.64:0/3306076713 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9671+0+0 (secure 0 0 0) 0x7fa2b00cf400 con 0x7fa2c0102c10 2024-09-12T03:39:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.709+0000 7fa2bed9d640 1 --2- 172.21.15.64:0/133373875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5624c6c9f410 0x5624c6ca9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:39:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.709+0000 7fa2be59c640 1 --2- 172.21.15.64:0/133373875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624c6ca9cb0 0x5624c6cac0a0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fa2b400bc10 tx=0x7fa2b400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f352c665b449bb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:39:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.709+0000 7fa2a57fa640 1 -- 172.21.15.64:0/133373875 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4015070 con 0x5624c6ca9cb0 2024-09-12T03:39:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.709+0000 7fa2bf59e640 1 --2- 172.21.15.64:0/3306076713 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fa2a80a2da0 0x7fa2a80a5260 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8005e30 tx=0x7fa2b802a8c0 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:39:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.709+0000 7fa2a57fa640 1 -- 172.21.15.64:0/133373875 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2b40118f0 con 0x5624c6ca9cb0 2024-09-12T03:39:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.709+0000 7fa2a57fa640 1 -- 172.21.15.64:0/133373875 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4010730 con 0x5624c6ca9cb0 2024-09-12T03:39:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.710+0000 7fa2c7fc2740 1 -- 172.21.15.64:0/133373875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624c6ca9cb0 msgr2=0x5624c6cac0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:39:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.710+0000 7fa2c7fc2740 1 --2- 172.21.15.64:0/133373875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624c6ca9cb0 0x5624c6cac0a0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fa2b400bc10 tx=0x7fa2b400f730 comp rx=0 tx=0).stop 2024-09-12T03:39:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.710+0000 7fa2c7fc2740 1 -- 172.21.15.64:0/133373875 shutdown_connections 2024-09-12T03:39:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.710+0000 7fa2c7fc2740 1 --2- 172.21.15.64:0/133373875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624c6ca9cb0 0x5624c6cac0a0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.710+0000 7fa2c7fc2740 1 --2- 172.21.15.64:0/133373875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5624c6c9f410 0x5624c6ca9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.710+0000 7fa2c7fc2740 1 --2- 172.21.15.64:0/133373875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624c6c9eaf0 0x5624c6c9eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.710+0000 7fa2c7fc2740 1 -- 172.21.15.64:0/133373875 >> 172.21.15.64:0/133373875 conn(0x5624c6c9b060 msgr2=0x5624c6c9b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:39:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.710+0000 7fa2c7fc2740 1 -- 172.21.15.64:0/133373875 shutdown_connections 2024-09-12T03:39:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.710+0000 7fa2c7fc2740 1 -- 172.21.15.64:0/133373875 wait complete. 2024-09-12T03:39:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.710+0000 7fa2c7fc2740 1 -- start start 2024-09-12T03:39:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.711+0000 7fa2c7fc2740 1 Processor -- start 2024-09-12T03:39:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.711+0000 7fa2c7fc2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624c6c9eaf0 0x5624c6ca6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.711+0000 7fa2c7fc2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624c6c9f410 0x5624c6ca70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.711+0000 7fa2c7fc2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5624c6ca9cb0 0x5624c6ca7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.711+0000 7fa2c7fc2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5624c6cac880 con 0x5624c6c9f410 2024-09-12T03:39:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.711+0000 7fa2c7fc2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5624c6b8c290 con 0x5624c6c9eaf0 2024-09-12T03:39:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.711+0000 7fa2c7fc2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5624c6caca00 con 0x5624c6ca9cb0 2024-09-12T03:39:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.711+0000 7fa2c7fc2740 10 client.0 _ll_register_callbacks cb 0x5624c6bbd140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:39:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.711+0000 7fa2be59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624c6c9f410 0x5624c6ca70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.711+0000 7fa2be59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624c6c9f410 0x5624c6ca70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49506/0 (socket says 172.21.15.64:49506) 2024-09-12T03:39:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.711+0000 7fa2be59c640 1 -- 172.21.15.64:0/506208860 learned_addr learned my addr 172.21.15.64:0/506208860 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:39:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.711+0000 7fa2bed9d640 1 --2- 172.21.15.64:0/506208860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624c6c9eaf0 0x5624c6ca6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.711+0000 7fa2bf59e640 1 --2- 172.21.15.64:0/506208860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5624c6ca9cb0 0x5624c6ca7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.711+0000 7fa2be59c640 1 -- 172.21.15.64:0/506208860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5624c6ca9cb0 msgr2=0x5624c6ca7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:39:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.711+0000 7fa2be59c640 1 --2- 172.21.15.64:0/506208860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5624c6ca9cb0 0x5624c6ca7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.711+0000 7fa2be59c640 1 -- 172.21.15.64:0/506208860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624c6c9eaf0 msgr2=0x5624c6ca6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:39:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.711+0000 7fa2be59c640 1 --2- 172.21.15.64:0/506208860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624c6c9eaf0 0x5624c6ca6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.711+0000 7fa2be59c640 1 -- 172.21.15.64:0/506208860 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624c6916210 con 0x5624c6c9f410 2024-09-12T03:39:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.711+0000 7fa2bed9d640 1 --2- 172.21.15.64:0/506208860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624c6c9eaf0 0x5624c6ca6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:39:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.712+0000 7fa2be59c640 1 --2- 172.21.15.64:0/506208860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624c6c9f410 0x5624c6ca70d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa2b400b660 tx=0x7fa2b400be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1672567eff7792a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:39:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.712+0000 7fa2a2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:39:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.712+0000 7fa2a2ff5640 1 -- 172.21.15.64:0/506208860 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4028060 con 0x5624c6c9f410 2024-09-12T03:39:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.712+0000 7fa2a2ff5640 1 -- 172.21.15.64:0/506208860 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9671+0+0 (secure 0 0 0) 0x7fa2b4011cb0 con 0x5624c6c9f410 2024-09-12T03:39:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.712+0000 7fa2c7fc2740 1 -- 172.21.15.64:0/506208860 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2b40118f0 con 0x5624c6c9f410 2024-09-12T03:39:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.712+0000 7fa2a2ff5640 1 -- 172.21.15.64:0/506208860 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa2b402f050 con 0x5624c6c9f410 2024-09-12T03:39:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.712+0000 7fa2c7fc2740 10 client.16152 fetch_fsmap learned FSMap version 25 2024-09-12T03:39:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.712+0000 7fa2c7fc2740 1 -- 172.21.15.64:0/506208860 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5624c69e2620 con 0x5624c6c9f410 2024-09-12T03:39:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.712+0000 7fa2a2ff5640 1 -- 172.21.15.64:0/506208860 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2b40118f0 con 0x5624c6c9f410 2024-09-12T03:39:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.712+0000 7fa2a2ff5640 1 -- 172.21.15.64:0/506208860 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa2b4004710 con 0x5624c6c9f410 2024-09-12T03:39:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.712+0000 7fa2c7fc2740 10 client.16152 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:39:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.712+0000 7fa2c7fc2740 10 client.16152 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:39:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.712+0000 7fa2c7fc2740 20 client.16152 populate_metadata read hostname 'smithi064' 2024-09-12T03:39:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.712+0000 7fa2c7fc2740 1 --2- 172.21.15.64:0/506208860 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5624c6cb9ba0 0x5624c6cbbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.712+0000 7fa2c7fc2740 4 client.16152 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:39:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.712+0000 7fa2c7fc2740 1 -- 172.21.15.64:0/506208860 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624c6c10a20 con 0x5624c6cb9ba0 2024-09-12T03:39:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.713+0000 7fa2bed9d640 1 --2- 172.21.15.64:0/506208860 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5624c6cb9ba0 0x5624c6cbbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.713+0000 7fa2a2ff5640 10 client.16152 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:39:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.713+0000 7fa2bed9d640 1 --2- 172.21.15.64:0/506208860 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5624c6cb9ba0 0x5624c6cbbf90 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=224433cf4e3a8d48 server_cookie=f41e837cae1e8407 in_seq=0 out_seq=0 2024-09-12T03:39:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.714+0000 7fa2a2ff5640 1 -- 172.21.15.64:0/506208860 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5624c6c10a20 con 0x5624c6cb9ba0 2024-09-12T03:39:09.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.714+0000 7fa2a2ff5640 10 client.16152 handle_command_reply: tid=0 2024-09-12T03:39:09.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.714+0000 7fa2c7fc2740 1 client.16152 shutdown 2024-09-12T03:39:09.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.714+0000 7fa2c7fc2740 1 -- 172.21.15.64:0/506208860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624c6c9f410 msgr2=0x5624c6ca70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:39:09.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.714+0000 7fa2c7fc2740 1 --2- 172.21.15.64:0/506208860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624c6c9f410 0x5624c6ca70d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa2b400b660 tx=0x7fa2b400be40 comp rx=0 tx=0).stop 2024-09-12T03:39:09.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.714+0000 7fa2c7fc2740 1 -- 172.21.15.64:0/506208860 shutdown_connections 2024-09-12T03:39:09.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.714+0000 7fa2c7fc2740 1 --2- 172.21.15.64:0/506208860 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5624c6cb9ba0 0x5624c6cbbf90 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:09.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.714+0000 7fa2c7fc2740 1 --2- 172.21.15.64:0/506208860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5624c6ca9cb0 0x5624c6ca7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:09.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.714+0000 7fa2c7fc2740 1 --2- 172.21.15.64:0/506208860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624c6c9f410 0x5624c6ca70d0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:09.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.714+0000 7fa2a2ff5640 0 client.16152 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:39:09.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.714+0000 7fa2c7fc2740 1 --2- 172.21.15.64:0/506208860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624c6c9eaf0 0x5624c6ca6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:09.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.714+0000 7fa2c7fc2740 1 -- 172.21.15.64:0/506208860 >> 172.21.15.64:0/506208860 conn(0x5624c6c9b060 msgr2=0x5624c6ca0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:39:09.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.714+0000 7fa2c7fc2740 1 -- 172.21.15.64:0/506208860 shutdown_connections 2024-09-12T03:39:09.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.714+0000 7fa2c7fc2740 1 -- 172.21.15.64:0/506208860 wait complete. 2024-09-12T03:39:09.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.714+0000 7fa2c7fc2740 20 client.16152 trim_cache size 0 max 16384 2024-09-12T03:39:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.729+0000 7fa2c7fc2740 1 Processor -- start 2024-09-12T03:39:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.729+0000 7fa2c7fc2740 1 -- start start 2024-09-12T03:39:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.729+0000 7fa2c7fc2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5624c6c9eaf0 0x5624c6ca7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.729+0000 7fa2c7fc2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624c6c9f410 0x5624c6c9a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.729+0000 7fa2c7fc2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624c6ca9cb0 0x5624c6cc1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.729+0000 7fa2c7fc2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c01b1b00 con 0x5624c6c9f410 2024-09-12T03:39:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.729+0000 7fa2c7fc2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c01af3d0 con 0x5624c6ca9cb0 2024-09-12T03:39:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.729+0000 7fa2c7fc2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c01ad240 con 0x5624c6c9eaf0 2024-09-12T03:39:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.729+0000 7fa2be59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624c6c9f410 0x5624c6c9a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.729+0000 7fa2be59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624c6c9f410 0x5624c6c9a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49520/0 (socket says 172.21.15.64:49520) 2024-09-12T03:39:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.729+0000 7fa2be59c640 1 -- 172.21.15.64:0/2056929786 learned_addr learned my addr 172.21.15.64:0/2056929786 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:39:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.730+0000 7fa2be59c640 1 -- 172.21.15.64:0/2056929786 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5624c6c9eaf0 msgr2=0x5624c6ca7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:39:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.730+0000 7fa2be59c640 1 --2- 172.21.15.64:0/2056929786 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5624c6c9eaf0 0x5624c6ca7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.730+0000 7fa2be59c640 1 -- 172.21.15.64:0/2056929786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624c6ca9cb0 msgr2=0x5624c6cc1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:39:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.730+0000 7fa2be59c640 1 --2- 172.21.15.64:0/2056929786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624c6ca9cb0 0x5624c6cc1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.730+0000 7fa2be59c640 1 -- 172.21.15.64:0/2056929786 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624c6c9ae90 con 0x5624c6c9f410 2024-09-12T03:39:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.730+0000 7fa2be59c640 1 --2- 172.21.15.64:0/2056929786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624c6c9f410 0x5624c6c9a0b0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4004c80 tx=0x7fa2b4001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c863846a5989e2ad server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:39:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.730+0000 7fa2a4ff9640 1 -- 172.21.15.64:0/2056929786 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4038070 con 0x5624c6c9f410 2024-09-12T03:39:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.730+0000 7fa2a4ff9640 1 -- 172.21.15.64:0/2056929786 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2b4005b90 con 0x5624c6c9f410 2024-09-12T03:39:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.730+0000 7fa2a4ff9640 1 -- 172.21.15.64:0/2056929786 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4015070 con 0x5624c6c9f410 2024-09-12T03:39:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.730+0000 7fa2c7fc2740 1 -- 172.21.15.64:0/2056929786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624c6c9f410 msgr2=0x5624c6c9a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:39:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.730+0000 7fa2c7fc2740 1 --2- 172.21.15.64:0/2056929786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624c6c9f410 0x5624c6c9a0b0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4004c80 tx=0x7fa2b4001e50 comp rx=0 tx=0).stop 2024-09-12T03:39:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.730+0000 7fa2c7fc2740 1 -- 172.21.15.64:0/2056929786 shutdown_connections 2024-09-12T03:39:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.730+0000 7fa2c7fc2740 1 --2- 172.21.15.64:0/2056929786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624c6ca9cb0 0x5624c6cc1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.730+0000 7fa2c7fc2740 1 --2- 172.21.15.64:0/2056929786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624c6c9f410 0x5624c6c9a0b0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.730+0000 7fa2c7fc2740 1 --2- 172.21.15.64:0/2056929786 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5624c6c9eaf0 0x5624c6ca7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.730+0000 7fa2c7fc2740 1 -- 172.21.15.64:0/2056929786 >> 172.21.15.64:0/2056929786 conn(0x5624c6c9b060 msgr2=0x5624c6c999a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:39:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.730+0000 7fa2c7fc2740 1 -- 172.21.15.64:0/2056929786 shutdown_connections 2024-09-12T03:39:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.731+0000 7fa2c7fc2740 1 -- 172.21.15.64:0/2056929786 wait complete. 2024-09-12T03:39:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.731+0000 7fa2c7fc2740 1 -- start start 2024-09-12T03:39:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.731+0000 7fa2c7fc2740 1 Processor -- start 2024-09-12T03:39:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.731+0000 7fa2c7fc2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624c6c9eaf0 0x5624c6cadd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.731+0000 7fa2c7fc2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624c6c9f410 0x5624c6cae290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.731+0000 7fa2c7fc2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5624c6ca9cb0 0x5624c6cae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.731+0000 7fa2c7fc2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c01ad890 con 0x5624c6c9eaf0 2024-09-12T03:39:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.731+0000 7fa2c7fc2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c01aed00 con 0x5624c6c9f410 2024-09-12T03:39:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.731+0000 7fa2c7fc2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c01adf60 con 0x5624c6ca9cb0 2024-09-12T03:39:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.731+0000 7fa2c7fc2740 10 client.0 _ll_register_callbacks cb 0x7fa2b8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:39:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.731+0000 7fa2bed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624c6c9eaf0 0x5624c6cadd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.731+0000 7fa2bed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624c6c9eaf0 0x5624c6cadd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49532/0 (socket says 172.21.15.64:49532) 2024-09-12T03:39:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.731+0000 7fa2bed9d640 1 -- 172.21.15.64:0/4069712962 learned_addr learned my addr 172.21.15.64:0/4069712962 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:39:09.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.731+0000 7fa2be59c640 1 --2- 172.21.15.64:0/4069712962 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624c6c9f410 0x5624c6cae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:09.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.732+0000 7fa2bed9d640 1 -- 172.21.15.64:0/4069712962 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5624c6ca9cb0 msgr2=0x5624c6cae7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:39:09.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.732+0000 7fa2bed9d640 1 --2- 172.21.15.64:0/4069712962 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5624c6ca9cb0 0x5624c6cae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:09.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.732+0000 7fa2bed9d640 1 -- 172.21.15.64:0/4069712962 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624c6c9f410 msgr2=0x5624c6cae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:39:09.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.732+0000 7fa2bed9d640 1 --2- 172.21.15.64:0/4069712962 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624c6c9f410 0x5624c6cae290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:09.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.732+0000 7fa2bed9d640 1 -- 172.21.15.64:0/4069712962 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624c6d23800 con 0x5624c6c9eaf0 2024-09-12T03:39:09.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.732+0000 7fa2be59c640 1 --2- 172.21.15.64:0/4069712962 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624c6c9f410 0x5624c6cae290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:39:09.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.732+0000 7fa2bed9d640 1 --2- 172.21.15.64:0/4069712962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624c6c9eaf0 0x5624c6cadd50 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0009610 tx=0x7fa2b0066020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c85b94e6ca1f4e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:39:09.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.732+0000 7fa2a2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:39:09.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.732+0000 7fa2a2ff5640 1 -- 172.21.15.64:0/4069712962 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b0082020 con 0x5624c6c9eaf0 2024-09-12T03:39:09.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.732+0000 7fa2a2ff5640 1 -- 172.21.15.64:0/4069712962 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9671+0+0 (secure 0 0 0) 0x7fa2b0097b10 con 0x5624c6c9eaf0 2024-09-12T03:39:09.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.732+0000 7fa2c7fc2740 1 -- 172.21.15.64:0/4069712962 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2b4005b90 con 0x5624c6c9eaf0 2024-09-12T03:39:09.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.732+0000 7fa2a2ff5640 1 -- 172.21.15.64:0/4069712962 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa2b0066530 con 0x5624c6c9eaf0 2024-09-12T03:39:09.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.732+0000 7fa2c7fc2740 10 client.16158 fetch_fsmap learned FSMap version 25 2024-09-12T03:39:09.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.732+0000 7fa2c7fc2740 1 -- 172.21.15.64:0/4069712962 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5624c6d24b70 con 0x5624c6c9eaf0 2024-09-12T03:39:09.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.732+0000 7fa2a2ff5640 1 -- 172.21.15.64:0/4069712962 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2b4005b90 con 0x5624c6c9eaf0 2024-09-12T03:39:09.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.732+0000 7fa2a2ff5640 1 -- 172.21.15.64:0/4069712962 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa2b006a080 con 0x5624c6c9eaf0 2024-09-12T03:39:09.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.732+0000 7fa2c7fc2740 10 client.16158 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:39:09.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.732+0000 7fa2c7fc2740 10 client.16158 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:39:09.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.732+0000 7fa2c7fc2740 20 client.16158 populate_metadata read hostname 'smithi064' 2024-09-12T03:39:09.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.732+0000 7fa2c7fc2740 1 --2- 172.21.15.64:0/4069712962 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5624c6d251f0 0x5624c6d27630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:09.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.732+0000 7fa2c7fc2740 4 client.16158 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:39:09.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.732+0000 7fa2c7fc2740 1 -- 172.21.15.64:0/4069712962 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5624c6c0ad70 con 0x5624c6d251f0 2024-09-12T03:39:09.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.733+0000 7fa2be59c640 1 --2- 172.21.15.64:0/4069712962 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5624c6d251f0 0x5624c6d27630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:09.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.733+0000 7fa2a2ff5640 10 client.16158 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:39:09.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.733+0000 7fa2be59c640 1 --2- 172.21.15.64:0/4069712962 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5624c6d251f0 0x5624c6d27630 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc1906456d74d2d8 server_cookie=e89c011491a98e29 in_seq=0 out_seq=0 2024-09-12T03:39:09.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.746+0000 7fa2a2ff5640 1 -- 172.21.15.64:0/4069712962 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5624c6c0ad70 con 0x5624c6d251f0 2024-09-12T03:39:09.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.746+0000 7fa2a2ff5640 10 client.16158 handle_command_reply: tid=0 2024-09-12T03:39:09.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.746+0000 7fa2c7fc2740 1 client.16158 shutdown 2024-09-12T03:39:09.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.747+0000 7fa2c7fc2740 1 -- 172.21.15.64:0/4069712962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624c6c9eaf0 msgr2=0x5624c6cadd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:39:09.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.747+0000 7fa2c7fc2740 1 --2- 172.21.15.64:0/4069712962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624c6c9eaf0 0x5624c6cadd50 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0009610 tx=0x7fa2b0066020 comp rx=0 tx=0).stop 2024-09-12T03:39:09.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.747+0000 7fa2c7fc2740 1 -- 172.21.15.64:0/4069712962 shutdown_connections 2024-09-12T03:39:09.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.747+0000 7fa2c7fc2740 1 --2- 172.21.15.64:0/4069712962 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x5624c6d251f0 0x5624c6d27630 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:09.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.747+0000 7fa2c7fc2740 1 --2- 172.21.15.64:0/4069712962 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5624c6ca9cb0 0x5624c6cae7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:09.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.747+0000 7fa2a2ff5640 0 client.16158 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:39:09.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.747+0000 7fa2c7fc2740 1 --2- 172.21.15.64:0/4069712962 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624c6c9f410 0x5624c6cae290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:09.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.747+0000 7fa2c7fc2740 1 --2- 172.21.15.64:0/4069712962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624c6c9eaf0 0x5624c6cadd50 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.747+0000 7fa2c7fc2740 1 -- 172.21.15.64:0/4069712962 >> 172.21.15.64:0/4069712962 conn(0x5624c6c9b060 msgr2=0x5624c6c9a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:39:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.747+0000 7fa2c7fc2740 1 -- 172.21.15.64:0/4069712962 shutdown_connections 2024-09-12T03:39:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.747+0000 7fa2c7fc2740 1 -- 172.21.15.64:0/4069712962 wait complete. 2024-09-12T03:39:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.747+0000 7fa2c7fc2740 20 client.16158 trim_cache size 0 max 16384 2024-09-12T03:39:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:39:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:39:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:39:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:39:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.747+0000 7fa2c514b640 1 -- 172.21.15.64:0/3306076713 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fa2a80a2da0 msgr2=0x7fa2a80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:39:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.747+0000 7fa2c514b640 1 --2- 172.21.15.64:0/3306076713 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fa2a80a2da0 0x7fa2a80a5260 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8005e30 tx=0x7fa2b802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:39:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.747+0000 7fa2c514b640 1 -- 172.21.15.64:0/3306076713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2c0102c10 msgr2=0x7fa2c01981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:39:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.747+0000 7fa2c514b640 1 --2- 172.21.15.64:0/3306076713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2c0102c10 0x7fa2c01981e0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fa2b000d950 tx=0x7fa2b000de10 comp rx=0 tx=0).stop 2024-09-12T03:39:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.747+0000 7fa2c514b640 1 -- 172.21.15.64:0/3306076713 shutdown_connections 2024-09-12T03:39:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.747+0000 7fa2c514b640 1 --2- 172.21.15.64:0/3306076713 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fa2a80a2da0 0x7fa2a80a5260 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.747+0000 7fa2c514b640 1 --2- 172.21.15.64:0/3306076713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2c010c110 0x7fa2c019caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.747+0000 7fa2c514b640 1 --2- 172.21.15.64:0/3306076713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2c01035c0 0x7fa2c0198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.747+0000 7fa2c514b640 1 --2- 172.21.15.64:0/3306076713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2c0102c10 0x7fa2c01981e0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.747+0000 7fa2c514b640 1 -- 172.21.15.64:0/3306076713 >> 172.21.15.64:0/3306076713 conn(0x7fa2c00fc6b0 msgr2=0x7fa2c00fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:39:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.748+0000 7fa2c514b640 1 -- 172.21.15.64:0/3306076713 shutdown_connections 2024-09-12T03:39:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:09.748+0000 7fa2c514b640 1 -- 172.21.15.64:0/3306076713 wait complete. 2024-09-12T03:39:09.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:09 smithi064 ceph-mon[30952]: pgmap v306: 129 pgs: 129 active+clean; 2.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 98 KiB/s wr, 32 op/s 2024-09-12T03:39:09.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:09 smithi115 ceph-mon[34801]: pgmap v306: 129 pgs: 129 active+clean; 2.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 98 KiB/s wr, 32 op/s 2024-09-12T03:39:09.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:09 smithi195 ceph-mon[34786]: pgmap v306: 129 pgs: 129 active+clean; 2.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 98 KiB/s wr, 32 op/s 2024-09-12T03:39:11.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:11 smithi064 ceph-mon[30952]: pgmap v307: 129 pgs: 129 active+clean; 2.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 98 KiB/s wr, 32 op/s 2024-09-12T03:39:11.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:11 smithi064 ceph-mon[30952]: osdmap e101: 12 total, 12 up, 12 in 2024-09-12T03:39:11.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:11 smithi115 ceph-mon[34801]: pgmap v307: 129 pgs: 129 active+clean; 2.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 98 KiB/s wr, 32 op/s 2024-09-12T03:39:11.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:11 smithi115 ceph-mon[34801]: osdmap e101: 12 total, 12 up, 12 in 2024-09-12T03:39:11.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:11 smithi195 ceph-mon[34786]: pgmap v307: 129 pgs: 129 active+clean; 2.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 98 KiB/s wr, 32 op/s 2024-09-12T03:39:11.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:11 smithi195 ceph-mon[34786]: osdmap e101: 12 total, 12 up, 12 in 2024-09-12T03:39:13.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:13 smithi064 ceph-mon[30952]: pgmap v309: 129 pgs: 129 active+clean; 2.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 64 KiB/s wr, 19 op/s 2024-09-12T03:39:13.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:13 smithi115 ceph-mon[34801]: pgmap v309: 129 pgs: 129 active+clean; 2.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 64 KiB/s wr, 19 op/s 2024-09-12T03:39:13.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:13 smithi195 ceph-mon[34786]: pgmap v309: 129 pgs: 129 active+clean; 2.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 64 KiB/s wr, 19 op/s 2024-09-12T03:39:15.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:15 smithi064 ceph-mon[30952]: pgmap v310: 129 pgs: 129 active+clean; 2.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 15 op/s 2024-09-12T03:39:15.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:15 smithi115 ceph-mon[34801]: pgmap v310: 129 pgs: 129 active+clean; 2.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 15 op/s 2024-09-12T03:39:15.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:15 smithi195 ceph-mon[34786]: pgmap v310: 129 pgs: 129 active+clean; 2.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 15 op/s 2024-09-12T03:39:17.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:17 smithi064 ceph-mon[30952]: pgmap v311: 129 pgs: 129 active+clean; 2.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 15 op/s 2024-09-12T03:39:17.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:17 smithi115 ceph-mon[34801]: pgmap v311: 129 pgs: 129 active+clean; 2.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 15 op/s 2024-09-12T03:39:17.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:17 smithi195 ceph-mon[34786]: pgmap v311: 129 pgs: 129 active+clean; 2.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 15 op/s 2024-09-12T03:39:19.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:19 smithi115 ceph-mon[34801]: pgmap v312: 129 pgs: 129 active+clean; 2.6 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-09-12T03:39:19.965 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:19 smithi195 ceph-mon[34786]: pgmap v312: 129 pgs: 129 active+clean; 2.6 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-09-12T03:39:20.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:19 smithi064 ceph-mon[30952]: pgmap v312: 129 pgs: 129 active+clean; 2.6 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-09-12T03:39:21.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:21 smithi115 ceph-mon[34801]: pgmap v313: 129 pgs: 129 active+clean; 2.6 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-09-12T03:39:21.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:21 smithi195 ceph-mon[34786]: pgmap v313: 129 pgs: 129 active+clean; 2.6 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-09-12T03:39:22.021 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:21 smithi064 ceph-mon[30952]: pgmap v313: 129 pgs: 129 active+clean; 2.6 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-09-12T03:39:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.846+0000 7f6429ceb640 1 -- 172.21.15.64:0/2304220789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f642410c0f0 msgr2=0x7f642410e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:39:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.846+0000 7f6429ceb640 1 --2- 172.21.15.64:0/2304220789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f642410c0f0 0x7f642410e560 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f641c009ca0 tx=0x7f641c02a950 comp rx=0 tx=0).stop 2024-09-12T03:39:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.846+0000 7f6429ceb640 1 -- 172.21.15.64:0/2304220789 shutdown_connections 2024-09-12T03:39:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.846+0000 7f6429ceb640 1 --2- 172.21.15.64:0/2304220789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f642410c0f0 0x7f642410e560 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.846+0000 7f6429ceb640 1 --2- 172.21.15.64:0/2304220789 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6424103640 0x7f642410bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.846+0000 7f6429ceb640 1 --2- 172.21.15.64:0/2304220789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6424102d20 0x7f6424103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.846+0000 7f6429ceb640 1 -- 172.21.15.64:0/2304220789 >> 172.21.15.64:0/2304220789 conn(0x7f6424077990 msgr2=0x7f64240fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:39:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.846+0000 7f6429ceb640 1 -- 172.21.15.64:0/2304220789 shutdown_connections 2024-09-12T03:39:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.846+0000 7f6429ceb640 1 -- 172.21.15.64:0/2304220789 wait complete. 2024-09-12T03:39:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.847+0000 7f6429ceb640 1 Processor -- start 2024-09-12T03:39:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.847+0000 7f6429ceb640 1 -- start start 2024-09-12T03:39:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.847+0000 7f6429ceb640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6424102d20 0x7f6424198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.847+0000 7f6429ceb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6424103640 0x7f6424198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.847+0000 7f6429ceb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f642410c0f0 0x7f642419c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.847+0000 7f6429ceb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6424111d30 con 0x7f6424103640 2024-09-12T03:39:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.847+0000 7f6429ceb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6424111bb0 con 0x7f642410c0f0 2024-09-12T03:39:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.847+0000 7f6429ceb640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6424111eb0 con 0x7f6424102d20 2024-09-12T03:39:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.847+0000 7f6422ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6424103640 0x7f6424198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.847+0000 7f6422ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6424103640 0x7f6424198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40538/0 (socket says 172.21.15.64:40538) 2024-09-12T03:39:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.847+0000 7f6422ffd640 1 -- 172.21.15.64:0/267974722 learned_addr learned my addr 172.21.15.64:0/267974722 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:39:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.848+0000 7f64237fe640 1 --2- 172.21.15.64:0/267974722 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6424102d20 0x7f6424198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.848+0000 7f6423fff640 1 --2- 172.21.15.64:0/267974722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f642410c0f0 0x7f642419c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.848+0000 7f6422ffd640 1 -- 172.21.15.64:0/267974722 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6424102d20 msgr2=0x7f6424198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:39:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.848+0000 7f6422ffd640 1 --2- 172.21.15.64:0/267974722 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6424102d20 0x7f6424198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.848+0000 7f6422ffd640 1 -- 172.21.15.64:0/267974722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f642410c0f0 msgr2=0x7f642419c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:39:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.848+0000 7f6422ffd640 1 --2- 172.21.15.64:0/267974722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f642410c0f0 0x7f642419c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.848+0000 7f6422ffd640 1 -- 172.21.15.64:0/267974722 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f642419d0f0 con 0x7f6424103640 2024-09-12T03:39:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.848+0000 7f6423fff640 1 --2- 172.21.15.64:0/267974722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f642410c0f0 0x7f642419c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:39:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.848+0000 7f6422ffd640 1 --2- 172.21.15.64:0/267974722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6424103640 0x7f6424198640 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f641800c920 tx=0x7f641800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:39:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.848+0000 7f6420ff9640 1 -- 172.21.15.64:0/267974722 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6418007bb0 con 0x7f6424103640 2024-09-12T03:39:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.848+0000 7f6429ceb640 1 -- 172.21.15.64:0/267974722 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f642419d3e0 con 0x7f6424103640 2024-09-12T03:39:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.848+0000 7f6420ff9640 1 -- 172.21.15.64:0/267974722 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6418007d50 con 0x7f6424103640 2024-09-12T03:39:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.848+0000 7f6420ff9640 1 -- 172.21.15.64:0/267974722 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6418018730 con 0x7f6424103640 2024-09-12T03:39:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.848+0000 7f6429ceb640 1 -- 172.21.15.64:0/267974722 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64241a4c70 con 0x7f6424103640 2024-09-12T03:39:22.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.851+0000 7f6420ff9640 1 -- 172.21.15.64:0/267974722 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f641800f450 con 0x7f6424103640 2024-09-12T03:39:22.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.851+0000 7f642ca8a740 1 Processor -- start 2024-09-12T03:39:22.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.851+0000 7f642ca8a740 1 -- start start 2024-09-12T03:39:22.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.851+0000 7f642ca8a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55640e69c9d0 0x55640e69cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:22.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.851+0000 7f642ca8a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55640e69d2f0 0x55640e6abb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:22.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.851+0000 7f642ca8a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55640e6ac0c0 0x55640e6ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.851+0000 7f642ca8a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55640e6012f0 con 0x55640e69d2f0 2024-09-12T03:39:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.851+0000 7f642ca8a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55640e5a9270 con 0x55640e6ac0c0 2024-09-12T03:39:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.851+0000 7f642ca8a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55640e5fd2b0 con 0x55640e69c9d0 2024-09-12T03:39:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.852+0000 7f64237fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55640e69d2f0 0x55640e6abb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.852+0000 7f64237fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55640e69d2f0 0x55640e6abb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40548/0 (socket says 172.21.15.64:40548) 2024-09-12T03:39:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.852+0000 7f64237fe640 1 -- 172.21.15.64:0/764330286 learned_addr learned my addr 172.21.15.64:0/764330286 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:39:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.852+0000 7f6420ff9640 1 --2- 172.21.15.64:0/267974722 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f640c0a2da0 0x7f640c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.852+0000 7f6422ffd640 1 --2- 172.21.15.64:0/764330286 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55640e6ac0c0 0x55640e6ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.852+0000 7f6423fff640 1 --2- 172.21.15.64:0/764330286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55640e69c9d0 0x55640e69cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.852+0000 7f64237fe640 1 -- 172.21.15.64:0/764330286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55640e69c9d0 msgr2=0x55640e69cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:39:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.852+0000 7f64237fe640 1 --2- 172.21.15.64:0/764330286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55640e69c9d0 0x55640e69cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.852+0000 7f64237fe640 1 -- 172.21.15.64:0/764330286 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55640e6ac0c0 msgr2=0x55640e6ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:39:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.852+0000 7f64237fe640 1 --2- 172.21.15.64:0/764330286 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55640e6ac0c0 0x55640e6ae4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.852+0000 7f64237fe640 1 -- 172.21.15.64:0/764330286 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55640e5a8340 con 0x55640e69d2f0 2024-09-12T03:39:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.852+0000 7f6422ffd640 1 --2- 172.21.15.64:0/764330286 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55640e6ac0c0 0x55640e6ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:39:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.852+0000 7f6420ff9640 1 -- 172.21.15.64:0/267974722 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9583+0+0 (secure 0 0 0) 0x7f64180cf920 con 0x7f6424103640 2024-09-12T03:39:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.852+0000 7f6423fff640 1 --2- 172.21.15.64:0/267974722 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f640c0a2da0 0x7f640c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.852+0000 7f64237fe640 1 --2- 172.21.15.64:0/764330286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55640e69d2f0 0x55640e6abb80 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f641400ad30 tx=0x7f641400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed8f16ad63fb1c77 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:39:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.852+0000 7f6409ffb640 1 -- 172.21.15.64:0/764330286 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6414010840 con 0x55640e69d2f0 2024-09-12T03:39:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.853+0000 7f6409ffb640 1 -- 172.21.15.64:0/764330286 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6414014070 con 0x55640e69d2f0 2024-09-12T03:39:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.853+0000 7f6409ffb640 1 -- 172.21.15.64:0/764330286 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6414018ae0 con 0x55640e69d2f0 2024-09-12T03:39:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.853+0000 7f642ca8a740 1 -- 172.21.15.64:0/764330286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55640e69d2f0 msgr2=0x55640e6abb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:39:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.853+0000 7f642ca8a740 1 --2- 172.21.15.64:0/764330286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55640e69d2f0 0x55640e6abb80 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f641400ad30 tx=0x7f641400f790 comp rx=0 tx=0).stop 2024-09-12T03:39:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.853+0000 7f6423fff640 1 --2- 172.21.15.64:0/267974722 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f640c0a2da0 0x7f640c0a5260 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f641c009430 tx=0x7f641c03e040 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:39:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.853+0000 7f642ca8a740 1 -- 172.21.15.64:0/764330286 shutdown_connections 2024-09-12T03:39:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.853+0000 7f642ca8a740 1 --2- 172.21.15.64:0/764330286 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55640e6ac0c0 0x55640e6ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.853+0000 7f642ca8a740 1 --2- 172.21.15.64:0/764330286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55640e69d2f0 0x55640e6abb80 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.853+0000 7f642ca8a740 1 --2- 172.21.15.64:0/764330286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55640e69c9d0 0x55640e69cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.853+0000 7f642ca8a740 1 -- 172.21.15.64:0/764330286 >> 172.21.15.64:0/764330286 conn(0x55640e699060 msgr2=0x55640e699440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:39:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.853+0000 7f642ca8a740 1 -- 172.21.15.64:0/764330286 shutdown_connections 2024-09-12T03:39:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.853+0000 7f642ca8a740 1 -- 172.21.15.64:0/764330286 wait complete. 2024-09-12T03:39:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.854+0000 7f642ca8a740 1 -- start start 2024-09-12T03:39:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.854+0000 7f642ca8a740 1 Processor -- start 2024-09-12T03:39:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.854+0000 7f642ca8a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55640e69c9d0 0x55640e6a8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.854+0000 7f642ca8a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55640e69d2f0 0x55640e6a94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.854+0000 7f642ca8a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55640e6ac0c0 0x55640e6a9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.854+0000 7f642ca8a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55640e6aecf0 con 0x55640e69d2f0 2024-09-12T03:39:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.854+0000 7f642ca8a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55640e58a290 con 0x55640e69c9d0 2024-09-12T03:39:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.854+0000 7f642ca8a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55640e6aee70 con 0x55640e6ac0c0 2024-09-12T03:39:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.854+0000 7f642ca8a740 10 client.0 _ll_register_callbacks cb 0x55640e5bb140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:39:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.854+0000 7f6422ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55640e69d2f0 0x55640e6a94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.855+0000 7f6422ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55640e69d2f0 0x55640e6a94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40564/0 (socket says 172.21.15.64:40564) 2024-09-12T03:39:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.855+0000 7f6422ffd640 1 -- 172.21.15.64:0/3027750007 learned_addr learned my addr 172.21.15.64:0/3027750007 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:39:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.855+0000 7f6423fff640 1 --2- 172.21.15.64:0/3027750007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55640e6ac0c0 0x55640e6a9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.855+0000 7f6422ffd640 1 -- 172.21.15.64:0/3027750007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55640e6ac0c0 msgr2=0x55640e6a9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:39:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.855+0000 7f6422ffd640 1 --2- 172.21.15.64:0/3027750007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55640e6ac0c0 0x55640e6a9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.855+0000 7f6422ffd640 1 -- 172.21.15.64:0/3027750007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55640e69c9d0 msgr2=0x55640e6a8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:39:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.855+0000 7f6422ffd640 1 --2- 172.21.15.64:0/3027750007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55640e69c9d0 0x55640e6a8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.855+0000 7f6422ffd640 1 -- 172.21.15.64:0/3027750007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55640e314210 con 0x55640e69d2f0 2024-09-12T03:39:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.855+0000 7f6422ffd640 1 --2- 172.21.15.64:0/3027750007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55640e69d2f0 0x55640e6a94e0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f6418007590 tx=0x7f641809b620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20171ad919a5e9d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:39:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.855+0000 7f6402ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:39:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.855+0000 7f6402ffd640 1 -- 172.21.15.64:0/3027750007 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641809c120 con 0x55640e69d2f0 2024-09-12T03:39:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.855+0000 7f6402ffd640 1 -- 172.21.15.64:0/3027750007 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9583+0+0 (secure 0 0 0) 0x7f6418098090 con 0x55640e69d2f0 2024-09-12T03:39:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.855+0000 7f642ca8a740 1 -- 172.21.15.64:0/3027750007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6414014070 con 0x55640e69d2f0 2024-09-12T03:39:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.855+0000 7f6402ffd640 1 -- 172.21.15.64:0/3027750007 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f641801f300 con 0x55640e69d2f0 2024-09-12T03:39:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.855+0000 7f642ca8a740 10 client.16170 fetch_fsmap learned FSMap version 25 2024-09-12T03:39:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.855+0000 7f642ca8a740 1 -- 172.21.15.64:0/3027750007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55640e3e0620 con 0x55640e69d2f0 2024-09-12T03:39:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.855+0000 7f6402ffd640 1 -- 172.21.15.64:0/3027750007 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6414014070 con 0x55640e69d2f0 2024-09-12T03:39:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.856+0000 7f6402ffd640 1 -- 172.21.15.64:0/3027750007 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f64180a4200 con 0x55640e69d2f0 2024-09-12T03:39:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.856+0000 7f642ca8a740 10 client.16170 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:39:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.856+0000 7f642ca8a740 10 client.16170 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:39:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.856+0000 7f642ca8a740 20 client.16170 populate_metadata read hostname 'smithi064' 2024-09-12T03:39:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.856+0000 7f642ca8a740 1 --2- 172.21.15.64:0/3027750007 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55640e6bc100 0x55640e6be550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.856+0000 7f642ca8a740 4 client.16170 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:39:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.856+0000 7f642ca8a740 1 -- 172.21.15.64:0/3027750007 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55640e60ea20 con 0x55640e6bc100 2024-09-12T03:39:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.856+0000 7f64237fe640 1 --2- 172.21.15.64:0/3027750007 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55640e6bc100 0x55640e6be550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.856+0000 7f64237fe640 1 --2- 172.21.15.64:0/3027750007 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55640e6bc100 0x55640e6be550 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4effb61db01d14a8 server_cookie=7de95efad111aef4 in_seq=0 out_seq=0 2024-09-12T03:39:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.856+0000 7f6402ffd640 10 client.16170 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:39:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.857+0000 7f6402ffd640 1 -- 172.21.15.64:0/3027750007 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55640e60ea20 con 0x55640e6bc100 2024-09-12T03:39:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.857+0000 7f6402ffd640 10 client.16170 handle_command_reply: tid=0 2024-09-12T03:39:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.857+0000 7f642ca8a740 1 client.16170 shutdown 2024-09-12T03:39:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.857+0000 7f642ca8a740 1 -- 172.21.15.64:0/3027750007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55640e69d2f0 msgr2=0x55640e6a94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:39:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.857+0000 7f642ca8a740 1 --2- 172.21.15.64:0/3027750007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55640e69d2f0 0x55640e6a94e0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f6418007590 tx=0x7f641809b620 comp rx=0 tx=0).stop 2024-09-12T03:39:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.857+0000 7f642ca8a740 1 -- 172.21.15.64:0/3027750007 shutdown_connections 2024-09-12T03:39:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.857+0000 7f642ca8a740 1 --2- 172.21.15.64:0/3027750007 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55640e6bc100 0x55640e6be550 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.857+0000 7f642ca8a740 1 --2- 172.21.15.64:0/3027750007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55640e6ac0c0 0x55640e6a9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.857+0000 7f642ca8a740 1 --2- 172.21.15.64:0/3027750007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55640e69d2f0 0x55640e6a94e0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.857+0000 7f6402ffd640 0 client.16170 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:39:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.857+0000 7f642ca8a740 1 --2- 172.21.15.64:0/3027750007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55640e69c9d0 0x55640e6a8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.857+0000 7f642ca8a740 1 -- 172.21.15.64:0/3027750007 >> 172.21.15.64:0/3027750007 conn(0x55640e699060 msgr2=0x55640e69edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:39:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.858+0000 7f642ca8a740 1 -- 172.21.15.64:0/3027750007 shutdown_connections 2024-09-12T03:39:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.858+0000 7f642ca8a740 1 -- 172.21.15.64:0/3027750007 wait complete. 2024-09-12T03:39:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.858+0000 7f642ca8a740 20 client.16170 trim_cache size 0 max 16384 2024-09-12T03:39:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.872+0000 7f642ca8a740 1 Processor -- start 2024-09-12T03:39:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.872+0000 7f642ca8a740 1 -- start start 2024-09-12T03:39:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.872+0000 7f642ca8a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55640e69c9d0 0x55640e6c3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.872+0000 7f642ca8a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55640e69d2f0 0x55640e6c43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.872+0000 7f642ca8a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55640e6ac0c0 0x55640e6a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.872+0000 7f642ca8a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f64241af670 con 0x55640e69c9d0 2024-09-12T03:39:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.872+0000 7f642ca8a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f64241acf40 con 0x55640e69d2f0 2024-09-12T03:39:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.872+0000 7f642ca8a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f64241aadb0 con 0x55640e6ac0c0 2024-09-12T03:39:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.873+0000 7f64237fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55640e69c9d0 0x55640e6c3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.873+0000 7f64237fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55640e69c9d0 0x55640e6c3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40576/0 (socket says 172.21.15.64:40576) 2024-09-12T03:39:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.873+0000 7f64237fe640 1 -- 172.21.15.64:0/1117564527 learned_addr learned my addr 172.21.15.64:0/1117564527 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:39:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.873+0000 7f6422ffd640 1 --2- 172.21.15.64:0/1117564527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55640e69d2f0 0x55640e6c43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.873+0000 7f64237fe640 1 -- 172.21.15.64:0/1117564527 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55640e6ac0c0 msgr2=0x55640e6a61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:39:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.873+0000 7f6423fff640 1 --2- 172.21.15.64:0/1117564527 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55640e6ac0c0 0x55640e6a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.873+0000 7f64237fe640 1 --2- 172.21.15.64:0/1117564527 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55640e6ac0c0 0x55640e6a61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.873+0000 7f64237fe640 1 -- 172.21.15.64:0/1117564527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55640e69d2f0 msgr2=0x55640e6c43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:39:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.873+0000 7f64237fe640 1 --2- 172.21.15.64:0/1117564527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55640e69d2f0 0x55640e6c43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.873+0000 7f64237fe640 1 -- 172.21.15.64:0/1117564527 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55640e6a7f30 con 0x55640e69c9d0 2024-09-12T03:39:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.873+0000 7f6422ffd640 1 --2- 172.21.15.64:0/1117564527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55640e69d2f0 0x55640e6c43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:39:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.873+0000 7f6423fff640 1 --2- 172.21.15.64:0/1117564527 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55640e6ac0c0 0x55640e6a61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:39:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.873+0000 7f64237fe640 1 --2- 172.21.15.64:0/1117564527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55640e69c9d0 0x55640e6c3e60 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f641401a040 tx=0x7f641400a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e64674189f02b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:39:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.873+0000 7f64097fa640 1 -- 172.21.15.64:0/1117564527 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6414005240 con 0x55640e69c9d0 2024-09-12T03:39:22.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.873+0000 7f64097fa640 1 -- 172.21.15.64:0/1117564527 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6414014040 con 0x55640e69c9d0 2024-09-12T03:39:22.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.873+0000 7f64097fa640 1 -- 172.21.15.64:0/1117564527 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6414004660 con 0x55640e69c9d0 2024-09-12T03:39:22.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.874+0000 7f642ca8a740 1 -- 172.21.15.64:0/1117564527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55640e69c9d0 msgr2=0x55640e6c3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:39:22.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.874+0000 7f642ca8a740 1 --2- 172.21.15.64:0/1117564527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55640e69c9d0 0x55640e6c3e60 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f641401a040 tx=0x7f641400a2d0 comp rx=0 tx=0).stop 2024-09-12T03:39:22.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.874+0000 7f642ca8a740 1 -- 172.21.15.64:0/1117564527 shutdown_connections 2024-09-12T03:39:22.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.874+0000 7f642ca8a740 1 --2- 172.21.15.64:0/1117564527 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55640e6ac0c0 0x55640e6a61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:22.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.874+0000 7f642ca8a740 1 --2- 172.21.15.64:0/1117564527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55640e69d2f0 0x55640e6c43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:22.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.874+0000 7f642ca8a740 1 --2- 172.21.15.64:0/1117564527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55640e69c9d0 0x55640e6c3e60 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:22.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.874+0000 7f642ca8a740 1 -- 172.21.15.64:0/1117564527 >> 172.21.15.64:0/1117564527 conn(0x55640e699060 msgr2=0x55640e6ab020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:39:22.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.874+0000 7f642ca8a740 1 -- 172.21.15.64:0/1117564527 shutdown_connections 2024-09-12T03:39:22.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.874+0000 7f642ca8a740 1 -- 172.21.15.64:0/1117564527 wait complete. 2024-09-12T03:39:22.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.874+0000 7f642ca8a740 1 -- start start 2024-09-12T03:39:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.874+0000 7f642ca8a740 1 Processor -- start 2024-09-12T03:39:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.875+0000 7f642ca8a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55640e69c9d0 0x55640e6b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.875+0000 7f642ca8a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55640e69d2f0 0x55640e6afc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.875+0000 7f642ca8a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55640e6ac0c0 0x55640e6b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.875+0000 7f642ca8a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f64241ab400 con 0x55640e6ac0c0 2024-09-12T03:39:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.875+0000 7f642ca8a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f64241ac870 con 0x55640e69c9d0 2024-09-12T03:39:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.875+0000 7f642ca8a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f64241abad0 con 0x55640e69d2f0 2024-09-12T03:39:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.875+0000 7f642ca8a740 10 client.0 _ll_register_callbacks cb 0x7f641800a560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:39:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.875+0000 7f6423fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55640e6ac0c0 0x55640e6b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.875+0000 7f6423fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55640e6ac0c0 0x55640e6b01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40582/0 (socket says 172.21.15.64:40582) 2024-09-12T03:39:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.875+0000 7f6423fff640 1 -- 172.21.15.64:0/1759192818 learned_addr learned my addr 172.21.15.64:0/1759192818 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:39:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.875+0000 7f64237fe640 1 --2- 172.21.15.64:0/1759192818 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55640e69c9d0 0x55640e6b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.875+0000 7f6422ffd640 1 --2- 172.21.15.64:0/1759192818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55640e69d2f0 0x55640e6afc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.875+0000 7f6423fff640 1 -- 172.21.15.64:0/1759192818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55640e69d2f0 msgr2=0x55640e6afc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:39:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.875+0000 7f6423fff640 1 --2- 172.21.15.64:0/1759192818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55640e69d2f0 0x55640e6afc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.875+0000 7f6423fff640 1 -- 172.21.15.64:0/1759192818 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55640e69c9d0 msgr2=0x55640e6b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:39:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.875+0000 7f6423fff640 1 --2- 172.21.15.64:0/1759192818 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55640e69c9d0 0x55640e6b48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.875+0000 7f6423fff640 1 -- 172.21.15.64:0/1759192818 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55640e6b09e0 con 0x55640e6ac0c0 2024-09-12T03:39:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.875+0000 7f6423fff640 1 --2- 172.21.15.64:0/1759192818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55640e6ac0c0 0x55640e6b01b0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f641c02fc80 tx=0x7f641c009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e16ab24ec4914e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:39:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.875+0000 7f6402ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:39:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.876+0000 7f6402ffd640 1 -- 172.21.15.64:0/1759192818 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641c034040 con 0x55640e6ac0c0 2024-09-12T03:39:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.876+0000 7f6402ffd640 1 -- 172.21.15.64:0/1759192818 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9583+0+0 (secure 0 0 0) 0x7f641c045c20 con 0x55640e6ac0c0 2024-09-12T03:39:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.876+0000 7f642ca8a740 1 -- 172.21.15.64:0/1759192818 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6414014040 con 0x55640e6ac0c0 2024-09-12T03:39:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.876+0000 7f6402ffd640 1 -- 172.21.15.64:0/1759192818 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f641c045e20 con 0x55640e6ac0c0 2024-09-12T03:39:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.876+0000 7f642ca8a740 10 client.16176 fetch_fsmap learned FSMap version 25 2024-09-12T03:39:22.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.876+0000 7f642ca8a740 1 -- 172.21.15.64:0/1759192818 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55640e726e40 con 0x55640e6ac0c0 2024-09-12T03:39:22.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.876+0000 7f6402ffd640 1 -- 172.21.15.64:0/1759192818 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6414014040 con 0x55640e6ac0c0 2024-09-12T03:39:22.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.876+0000 7f6402ffd640 1 -- 172.21.15.64:0/1759192818 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f641c004680 con 0x55640e6ac0c0 2024-09-12T03:39:22.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.876+0000 7f642ca8a740 10 client.16176 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:39:22.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.876+0000 7f642ca8a740 10 client.16176 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:39:22.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.876+0000 7f642ca8a740 20 client.16176 populate_metadata read hostname 'smithi064' 2024-09-12T03:39:22.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.876+0000 7f642ca8a740 1 --2- 172.21.15.64:0/1759192818 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55640e7275c0 0x55640e7299e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:22.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.876+0000 7f642ca8a740 4 client.16176 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:39:22.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.876+0000 7f642ca8a740 1 -- 172.21.15.64:0/1759192818 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55640e608d70 con 0x55640e7275c0 2024-09-12T03:39:22.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.876+0000 7f64237fe640 1 --2- 172.21.15.64:0/1759192818 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55640e7275c0 0x55640e7299e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:22.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.877+0000 7f6402ffd640 10 client.16176 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:39:22.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.877+0000 7f64237fe640 1 --2- 172.21.15.64:0/1759192818 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55640e7275c0 0x55640e7299e0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28e16151a5f3834a server_cookie=cb03d585a56269d2 in_seq=0 out_seq=0 2024-09-12T03:39:22.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.890+0000 7f6402ffd640 1 -- 172.21.15.64:0/1759192818 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55640e608d70 con 0x55640e7275c0 2024-09-12T03:39:22.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.890+0000 7f6402ffd640 10 client.16176 handle_command_reply: tid=0 2024-09-12T03:39:22.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.890+0000 7f642ca8a740 1 client.16176 shutdown 2024-09-12T03:39:22.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.891+0000 7f642ca8a740 1 -- 172.21.15.64:0/1759192818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55640e6ac0c0 msgr2=0x55640e6b01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:39:22.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.891+0000 7f642ca8a740 1 --2- 172.21.15.64:0/1759192818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55640e6ac0c0 0x55640e6b01b0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f641c02fc80 tx=0x7f641c009360 comp rx=0 tx=0).stop 2024-09-12T03:39:22.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.891+0000 7f642ca8a740 1 -- 172.21.15.64:0/1759192818 shutdown_connections 2024-09-12T03:39:22.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.891+0000 7f642ca8a740 1 --2- 172.21.15.64:0/1759192818 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55640e7275c0 0x55640e7299e0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:22.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.891+0000 7f642ca8a740 1 --2- 172.21.15.64:0/1759192818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55640e6ac0c0 0x55640e6b01b0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.891+0000 7f642ca8a740 1 --2- 172.21.15.64:0/1759192818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55640e69d2f0 0x55640e6afc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.891+0000 7f6402ffd640 0 client.16176 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:39:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.891+0000 7f642ca8a740 1 --2- 172.21.15.64:0/1759192818 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55640e69c9d0 0x55640e6b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.891+0000 7f642ca8a740 1 -- 172.21.15.64:0/1759192818 >> 172.21.15.64:0/1759192818 conn(0x55640e699060 msgr2=0x55640e6979a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:39:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.891+0000 7f642ca8a740 1 -- 172.21.15.64:0/1759192818 shutdown_connections 2024-09-12T03:39:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.891+0000 7f642ca8a740 1 -- 172.21.15.64:0/1759192818 wait complete. 2024-09-12T03:39:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.891+0000 7f642ca8a740 20 client.16176 trim_cache size 0 max 16384 2024-09-12T03:39:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:39:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:39:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:39:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:39:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.891+0000 7f6429ceb640 1 -- 172.21.15.64:0/267974722 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f640c0a2da0 msgr2=0x7f640c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:39:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.891+0000 7f6429ceb640 1 --2- 172.21.15.64:0/267974722 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f640c0a2da0 0x7f640c0a5260 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f641c009430 tx=0x7f641c03e040 comp rx=0 tx=0).stop 2024-09-12T03:39:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.891+0000 7f6429ceb640 1 -- 172.21.15.64:0/267974722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6424103640 msgr2=0x7f6424198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:39:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.891+0000 7f6429ceb640 1 --2- 172.21.15.64:0/267974722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6424103640 0x7f6424198640 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f641800c920 tx=0x7f641800cd20 comp rx=0 tx=0).stop 2024-09-12T03:39:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.891+0000 7f6429ceb640 1 -- 172.21.15.64:0/267974722 shutdown_connections 2024-09-12T03:39:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.891+0000 7f6429ceb640 1 --2- 172.21.15.64:0/267974722 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f640c0a2da0 0x7f640c0a5260 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.891+0000 7f6429ceb640 1 --2- 172.21.15.64:0/267974722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f642410c0f0 0x7f642419c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.891+0000 7f6429ceb640 1 --2- 172.21.15.64:0/267974722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6424103640 0x7f6424198640 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.891+0000 7f6429ceb640 1 --2- 172.21.15.64:0/267974722 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6424102d20 0x7f6424198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.891+0000 7f6429ceb640 1 -- 172.21.15.64:0/267974722 >> 172.21.15.64:0/267974722 conn(0x7f6424077990 msgr2=0x7f64240fcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:39:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.892+0000 7f6429ceb640 1 -- 172.21.15.64:0/267974722 shutdown_connections 2024-09-12T03:39:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:22.892+0000 7f6429ceb640 1 -- 172.21.15.64:0/267974722 wait complete. 2024-09-12T03:39:23.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:23 smithi115 ceph-mon[34801]: pgmap v314: 129 pgs: 129 active+clean; 2.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 65 KiB/s wr, 20 op/s 2024-09-12T03:39:23.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:23 smithi195 ceph-mon[34786]: pgmap v314: 129 pgs: 129 active+clean; 2.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 65 KiB/s wr, 20 op/s 2024-09-12T03:39:24.021 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:23 smithi064 ceph-mon[30952]: pgmap v314: 129 pgs: 129 active+clean; 2.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 65 KiB/s wr, 20 op/s 2024-09-12T03:39:24.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:24 smithi115 ceph-mon[34801]: pgmap v315: 129 pgs: 129 active+clean; 2.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 10 op/s 2024-09-12T03:39:24.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:24 smithi195 ceph-mon[34786]: pgmap v315: 129 pgs: 129 active+clean; 2.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 10 op/s 2024-09-12T03:39:25.021 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:24 smithi064 ceph-mon[30952]: pgmap v315: 129 pgs: 129 active+clean; 2.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 10 op/s 2024-09-12T03:39:25.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:25 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:39:25.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:25 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:39:26.021 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:25 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:39:27.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:27 smithi115 ceph-mon[34801]: pgmap v316: 129 pgs: 129 active+clean; 2.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 10 op/s 2024-09-12T03:39:27.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:27 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:39:27.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:27 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:39:27.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:27 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:39:27.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:27 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:39:27.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:27 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:39:27.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:27 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:39:27.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:27 smithi195 ceph-mon[34786]: pgmap v316: 129 pgs: 129 active+clean; 2.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 10 op/s 2024-09-12T03:39:27.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:27 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:39:27.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:27 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:39:27.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:27 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:39:27.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:27 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:39:27.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:27 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:39:27.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:27 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:39:27.521 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:27 smithi064 ceph-mon[30952]: pgmap v316: 129 pgs: 129 active+clean; 2.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 10 op/s 2024-09-12T03:39:27.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:27 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:39:27.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:27 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:39:27.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:27 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:39:27.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:27 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:39:27.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:27 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:39:27.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:27 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:39:28.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:28 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:28 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:28 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:28 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:28 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:28 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:28 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:28 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:28 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.394 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:28 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.394 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:28 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.394 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:28 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.394 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:28 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.394 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:28 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.394 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:28 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.394 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:28 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.394 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:28 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.394 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:28 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.394 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:28 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.394 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:28 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.394 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:28 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.394 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:28 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.395 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:28 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.395 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:28 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.395 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:28 smithi115 ceph-mon[34801]: Adjusting osd_memory_target on smithi115 to 176.0M 2024-09-12T03:39:28.395 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:28 smithi115 ceph-mon[34801]: Adjusting osd_memory_target on smithi064 to 176.0M 2024-09-12T03:39:28.395 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:28 smithi115 ceph-mon[34801]: Adjusting osd_memory_target on smithi195 to 176.0M 2024-09-12T03:39:28.395 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:28 smithi115 ceph-mon[34801]: Unable to set osd_memory_target on smithi115 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-12T03:39:28.395 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:28 smithi115 ceph-mon[34801]: Unable to set osd_memory_target on smithi064 to 184586752: error parsing value: Value '184586752' is below minimum 939524096 2024-09-12T03:39:28.395 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:28 smithi115 ceph-mon[34801]: Unable to set osd_memory_target on smithi195 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-12T03:39:28.395 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:28 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:39:28.395 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:28 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:39:28.395 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:28 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:39:28.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:28 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:28 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:28 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.456 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:28 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:28 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:28 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:28 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:28 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:28 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:28 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:28 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:28 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:28 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:28 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:28 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:28 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.457 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:28 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:28 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:28 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:28 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:28 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:28 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:28 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:28 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:28 smithi195 ceph-mon[34786]: Adjusting osd_memory_target on smithi115 to 176.0M 2024-09-12T03:39:28.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:28 smithi195 ceph-mon[34786]: Adjusting osd_memory_target on smithi064 to 176.0M 2024-09-12T03:39:28.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:28 smithi195 ceph-mon[34786]: Adjusting osd_memory_target on smithi195 to 176.0M 2024-09-12T03:39:28.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:28 smithi195 ceph-mon[34786]: Unable to set osd_memory_target on smithi115 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-12T03:39:28.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:28 smithi195 ceph-mon[34786]: Unable to set osd_memory_target on smithi064 to 184586752: error parsing value: Value '184586752' is below minimum 939524096 2024-09-12T03:39:28.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:28 smithi195 ceph-mon[34786]: Unable to set osd_memory_target on smithi195 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-12T03:39:28.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:28 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:39:28.459 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:28 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:39:28.459 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:28 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:39:28.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:28 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:28 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.522 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:28 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:28 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:28 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:28 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:28 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:28 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:28 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:28 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:28 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:28 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:28 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:28 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:28 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.523 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:28 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:28 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:28 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:28 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:28 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:28 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:28 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:28 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:28 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:39:28.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:28 smithi064 ceph-mon[30952]: Adjusting osd_memory_target on smithi115 to 176.0M 2024-09-12T03:39:28.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:28 smithi064 ceph-mon[30952]: Adjusting osd_memory_target on smithi064 to 176.0M 2024-09-12T03:39:28.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:28 smithi064 ceph-mon[30952]: Adjusting osd_memory_target on smithi195 to 176.0M 2024-09-12T03:39:28.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:28 smithi064 ceph-mon[30952]: Unable to set osd_memory_target on smithi115 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-12T03:39:28.525 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:28 smithi064 ceph-mon[30952]: Unable to set osd_memory_target on smithi064 to 184586752: error parsing value: Value '184586752' is below minimum 939524096 2024-09-12T03:39:28.525 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:28 smithi064 ceph-mon[30952]: Unable to set osd_memory_target on smithi195 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-12T03:39:28.525 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:28 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:39:28.525 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:28 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:39:28.525 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:28 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:39:29.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:29 smithi195 ceph-mon[34786]: pgmap v317: 129 pgs: 129 active+clean; 3.0 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:39:29.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:29 smithi064 ceph-mon[30952]: pgmap v317: 129 pgs: 129 active+clean; 3.0 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:39:29.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:29 smithi115 ceph-mon[34801]: pgmap v317: 129 pgs: 129 active+clean; 3.0 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:39:31.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:31 smithi195 ceph-mon[34786]: pgmap v318: 129 pgs: 129 active+clean; 3.0 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:39:31.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:31 smithi064 ceph-mon[30952]: pgmap v318: 129 pgs: 129 active+clean; 3.0 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:39:31.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:31 smithi115 ceph-mon[34801]: pgmap v318: 129 pgs: 129 active+clean; 3.0 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:39:33.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:33 smithi195 ceph-mon[34786]: pgmap v319: 129 pgs: 129 active+clean; 3.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 54 KiB/s wr, 17 op/s 2024-09-12T03:39:33.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:33 smithi064 ceph-mon[30952]: pgmap v319: 129 pgs: 129 active+clean; 3.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 54 KiB/s wr, 17 op/s 2024-09-12T03:39:33.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:33 smithi115 ceph-mon[34801]: pgmap v319: 129 pgs: 129 active+clean; 3.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 54 KiB/s wr, 17 op/s 2024-09-12T03:39:35.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:35 smithi195 ceph-mon[34786]: pgmap v320: 129 pgs: 129 active+clean; 3.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-09-12T03:39:35.775 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:35 smithi064 ceph-mon[30952]: pgmap v320: 129 pgs: 129 active+clean; 3.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-09-12T03:39:35.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:35 smithi115 ceph-mon[34801]: pgmap v320: 129 pgs: 129 active+clean; 3.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-09-12T03:39:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.992+0000 7f92dad59640 1 -- 172.21.15.64:0/706615592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92d4101390 msgr2=0x7f92d410d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:39:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.992+0000 7f92dad59640 1 --2- 172.21.15.64:0/706615592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92d4101390 0x7f92d410d8c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f92cc009cd0 tx=0x7f92cc02a8e0 comp rx=0 tx=0).stop 2024-09-12T03:39:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.992+0000 7f92dad59640 1 -- 172.21.15.64:0/706615592 shutdown_connections 2024-09-12T03:39:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.992+0000 7f92dad59640 1 --2- 172.21.15.64:0/706615592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92d4101390 0x7f92d410d8c0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.992+0000 7f92dad59640 1 --2- 172.21.15.64:0/706615592 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92d41009f0 0x7f92d4100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.992+0000 7f92dad59640 1 --2- 172.21.15.64:0/706615592 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92d4106a00 0x7f92d4106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.992+0000 7f92dad59640 1 -- 172.21.15.64:0/706615592 >> 172.21.15.64:0/706615592 conn(0x7f92d40fc6b0 msgr2=0x7f92d40fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:39:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.993+0000 7f92dad59640 1 -- 172.21.15.64:0/706615592 shutdown_connections 2024-09-12T03:39:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.993+0000 7f92dad59640 1 -- 172.21.15.64:0/706615592 wait complete. 2024-09-12T03:39:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.993+0000 7f92dad59640 1 Processor -- start 2024-09-12T03:39:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.993+0000 7f92dad59640 1 -- start start 2024-09-12T03:39:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.993+0000 7f92dad59640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92d41009f0 0x7f92d419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.994+0000 7f92dad59640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92d4101390 0x7f92d419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.994+0000 7f92dad59640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92d4106a00 0x7f92d419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.994+0000 7f92dad59640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f92d4110cf0 con 0x7f92d4101390 2024-09-12T03:39:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.994+0000 7f92dad59640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f92d4110b70 con 0x7f92d4106a00 2024-09-12T03:39:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.994+0000 7f92dad59640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f92d4110e70 con 0x7f92d41009f0 2024-09-12T03:39:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.994+0000 7f92d3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92d4101390 0x7f92d419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.994+0000 7f92d3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92d4101390 0x7f92d419aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46190/0 (socket says 172.21.15.64:46190) 2024-09-12T03:39:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.994+0000 7f92d3fff640 1 -- 172.21.15.64:0/3295382662 learned_addr learned my addr 172.21.15.64:0/3295382662 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:39:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.994+0000 7f92d92f6640 1 --2- 172.21.15.64:0/3295382662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92d4106a00 0x7f92d419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.994+0000 7f92d8af5640 1 --2- 172.21.15.64:0/3295382662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92d41009f0 0x7f92d419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.994+0000 7f92d3fff640 1 -- 172.21.15.64:0/3295382662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92d41009f0 msgr2=0x7f92d419a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:39:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.994+0000 7f92d3fff640 1 --2- 172.21.15.64:0/3295382662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92d41009f0 0x7f92d419a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.994+0000 7f92d3fff640 1 -- 172.21.15.64:0/3295382662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92d4106a00 msgr2=0x7f92d419ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:39:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.994+0000 7f92d3fff640 1 --2- 172.21.15.64:0/3295382662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92d4106a00 0x7f92d419ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.994+0000 7f92d3fff640 1 -- 172.21.15.64:0/3295382662 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92d419f4b0 con 0x7f92d4101390 2024-09-12T03:39:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.994+0000 7f92d3fff640 1 --2- 172.21.15.64:0/3295382662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92d4101390 0x7f92d419aa00 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f92c000d950 tx=0x7f92c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:39:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.994+0000 7f92d1ffb640 1 -- 172.21.15.64:0/3295382662 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c0017030 con 0x7f92d4101390 2024-09-12T03:39:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.994+0000 7f92d1ffb640 1 -- 172.21.15.64:0/3295382662 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92c0004020 con 0x7f92d4101390 2024-09-12T03:39:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.994+0000 7f92dad59640 1 -- 172.21.15.64:0/3295382662 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92d419f7a0 con 0x7f92d4101390 2024-09-12T03:39:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.994+0000 7f92d1ffb640 1 -- 172.21.15.64:0/3295382662 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c00047a0 con 0x7f92d4101390 2024-09-12T03:39:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.995+0000 7f92dad59640 1 -- 172.21.15.64:0/3295382662 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92d41a7030 con 0x7f92d4101390 2024-09-12T03:39:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.997+0000 7f92d1ffb640 1 -- 172.21.15.64:0/3295382662 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f92c00041d0 con 0x7f92d4101390 2024-09-12T03:39:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.998+0000 7f92ddbd0740 1 Processor -- start 2024-09-12T03:39:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.998+0000 7f92ddbd0740 1 -- start start 2024-09-12T03:39:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.998+0000 7f92ddbd0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce83277af0 0x55ce83277ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.998+0000 7f92ddbd0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ce83278410 0x55ce83282770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.998+0000 7f92ddbd0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce83282cb0 0x55ce832850a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.998+0000 7f92ddbd0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ce831dc2f0 con 0x55ce83277af0 2024-09-12T03:39:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.998+0000 7f92ddbd0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ce83184270 con 0x55ce83282cb0 2024-09-12T03:39:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.998+0000 7f92ddbd0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ce831d82b0 con 0x55ce83278410 2024-09-12T03:39:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.998+0000 7f92d1ffb640 1 --2- 172.21.15.64:0/3295382662 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f92c40a2da0 0x7f92c40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:36.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.998+0000 7f92d92f6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce83277af0 0x55ce83277ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:36.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.998+0000 7f92d92f6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce83277af0 0x55ce83277ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46196/0 (socket says 172.21.15.64:46196) 2024-09-12T03:39:36.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.998+0000 7f92d92f6640 1 -- 172.21.15.64:0/1348508246 learned_addr learned my addr 172.21.15.64:0/1348508246 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:39:36.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.998+0000 7f92d3fff640 1 --2- 172.21.15.64:0/1348508246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce83282cb0 0x55ce832850a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:36.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.999+0000 7f92d92f6640 1 --2- 172.21.15.64:0/3295382662 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f92c40a2da0 0x7f92c40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:36.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.999+0000 7f92d8af5640 1 --2- 172.21.15.64:0/1348508246 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ce83278410 0x55ce83282770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:36.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.999+0000 7f92d1ffb640 1 -- 172.21.15.64:0/3295382662 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9583+0+0 (secure 0 0 0) 0x7f92c00cf370 con 0x7f92d4101390 2024-09-12T03:39:36.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.999+0000 7f92d92f6640 1 -- 172.21.15.64:0/1348508246 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ce83278410 msgr2=0x55ce83282770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:39:36.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.999+0000 7f92d92f6640 1 --2- 172.21.15.64:0/1348508246 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ce83278410 0x55ce83282770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:36.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.999+0000 7f92d92f6640 1 -- 172.21.15.64:0/1348508246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce83282cb0 msgr2=0x55ce832850a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:39:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.999+0000 7f92d92f6640 1 --2- 172.21.15.64:0/1348508246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce83282cb0 0x55ce832850a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.999+0000 7f92d92f6640 1 -- 172.21.15.64:0/1348508246 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce83183340 con 0x55ce83277af0 2024-09-12T03:39:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.999+0000 7f92d92f6640 1 --2- 172.21.15.64:0/1348508246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce83277af0 0x55ce83277ed0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f92cc004af0 tx=0x7f92cc031950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f932b236fd2552 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:39:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.999+0000 7f92beffd640 1 -- 172.21.15.64:0/1348508246 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc031c60 con 0x55ce83277af0 2024-09-12T03:39:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.999+0000 7f92beffd640 1 -- 172.21.15.64:0/1348508246 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92cc050070 con 0x55ce83277af0 2024-09-12T03:39:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.999+0000 7f92beffd640 1 -- 172.21.15.64:0/1348508246 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc030dd0 con 0x55ce83277af0 2024-09-12T03:39:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:35.999+0000 7f92d92f6640 1 --2- 172.21.15.64:0/3295382662 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f92c40a2da0 0x7f92c40a5260 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f92cc04f000 tx=0x7f92cc04d000 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:39:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.000+0000 7f92ddbd0740 1 -- 172.21.15.64:0/1348508246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce83277af0 msgr2=0x55ce83277ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:39:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.000+0000 7f92ddbd0740 1 --2- 172.21.15.64:0/1348508246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce83277af0 0x55ce83277ed0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f92cc004af0 tx=0x7f92cc031950 comp rx=0 tx=0).stop 2024-09-12T03:39:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.000+0000 7f92ddbd0740 1 -- 172.21.15.64:0/1348508246 shutdown_connections 2024-09-12T03:39:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.000+0000 7f92ddbd0740 1 --2- 172.21.15.64:0/1348508246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce83282cb0 0x55ce832850a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.000+0000 7f92ddbd0740 1 --2- 172.21.15.64:0/1348508246 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ce83278410 0x55ce83282770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.000+0000 7f92ddbd0740 1 --2- 172.21.15.64:0/1348508246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce83277af0 0x55ce83277ed0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.000+0000 7f92ddbd0740 1 -- 172.21.15.64:0/1348508246 >> 172.21.15.64:0/1348508246 conn(0x55ce83274060 msgr2=0x55ce83274440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:39:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.000+0000 7f92ddbd0740 1 -- 172.21.15.64:0/1348508246 shutdown_connections 2024-09-12T03:39:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.000+0000 7f92ddbd0740 1 -- 172.21.15.64:0/1348508246 wait complete. 2024-09-12T03:39:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.000+0000 7f92ddbd0740 1 -- start start 2024-09-12T03:39:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.000+0000 7f92ddbd0740 1 Processor -- start 2024-09-12T03:39:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.001+0000 7f92ddbd0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce83277af0 0x55ce8327fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.001+0000 7f92ddbd0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce83278410 0x55ce832800f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.001+0000 7f92ddbd0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ce83282cb0 0x55ce83280630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.001+0000 7f92ddbd0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ce83285880 con 0x55ce83278410 2024-09-12T03:39:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.001+0000 7f92ddbd0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ce83165290 con 0x55ce83277af0 2024-09-12T03:39:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.001+0000 7f92ddbd0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ce83285a00 con 0x55ce83282cb0 2024-09-12T03:39:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.001+0000 7f92ddbd0740 10 client.0 _ll_register_callbacks cb 0x55ce83196140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:39:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.001+0000 7f92d3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce83278410 0x55ce832800f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.001+0000 7f92d3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce83278410 0x55ce832800f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46204/0 (socket says 172.21.15.64:46204) 2024-09-12T03:39:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.001+0000 7f92d3fff640 1 -- 172.21.15.64:0/1841341244 learned_addr learned my addr 172.21.15.64:0/1841341244 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:39:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.001+0000 7f92d8af5640 1 --2- 172.21.15.64:0/1841341244 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce83277af0 0x55ce8327fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.001+0000 7f92d92f6640 1 --2- 172.21.15.64:0/1841341244 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ce83282cb0 0x55ce83280630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.001+0000 7f92d3fff640 1 -- 172.21.15.64:0/1841341244 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ce83282cb0 msgr2=0x55ce83280630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:39:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.001+0000 7f92d3fff640 1 --2- 172.21.15.64:0/1841341244 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ce83282cb0 0x55ce83280630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.001+0000 7f92d3fff640 1 -- 172.21.15.64:0/1841341244 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce83277af0 msgr2=0x55ce8327fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:39:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.001+0000 7f92d3fff640 1 --2- 172.21.15.64:0/1841341244 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce83277af0 0x55ce8327fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.001+0000 7f92d3fff640 1 -- 172.21.15.64:0/1841341244 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce82eef210 con 0x55ce83278410 2024-09-12T03:39:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.002+0000 7f92d3fff640 1 --2- 172.21.15.64:0/1841341244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce83278410 0x55ce832800f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f92c009b3a0 tx=0x7f92c00207f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f9afa86b131812 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:39:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.002+0000 7f92b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:39:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.002+0000 7f92b3fff640 1 -- 172.21.15.64:0/1841341244 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c00cf160 con 0x55ce83278410 2024-09-12T03:39:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.002+0000 7f92b3fff640 1 -- 172.21.15.64:0/1841341244 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9583+0+0 (secure 0 0 0) 0x7f92c00a3bf0 con 0x55ce83278410 2024-09-12T03:39:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.002+0000 7f92ddbd0740 1 -- 172.21.15.64:0/1841341244 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92cc050070 con 0x55ce83278410 2024-09-12T03:39:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.002+0000 7f92b3fff640 1 -- 172.21.15.64:0/1841341244 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f92c00ccac0 con 0x55ce83278410 2024-09-12T03:39:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.002+0000 7f92ddbd0740 10 client.16188 fetch_fsmap learned FSMap version 25 2024-09-12T03:39:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.002+0000 7f92ddbd0740 1 -- 172.21.15.64:0/1841341244 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ce82fbb620 con 0x55ce83278410 2024-09-12T03:39:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.002+0000 7f92b3fff640 1 -- 172.21.15.64:0/1841341244 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92cc050070 con 0x55ce83278410 2024-09-12T03:39:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.002+0000 7f92b3fff640 1 -- 172.21.15.64:0/1841341244 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f92c0017c10 con 0x55ce83278410 2024-09-12T03:39:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.002+0000 7f92ddbd0740 10 client.16188 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:39:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.002+0000 7f92ddbd0740 10 client.16188 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:39:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.002+0000 7f92ddbd0740 20 client.16188 populate_metadata read hostname 'smithi064' 2024-09-12T03:39:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.002+0000 7f92ddbd0740 1 --2- 172.21.15.64:0/1841341244 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55ce83292c20 0x55ce83295010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.002+0000 7f92ddbd0740 4 client.16188 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:39:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.002+0000 7f92ddbd0740 1 -- 172.21.15.64:0/1841341244 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce831e9a20 con 0x55ce83292c20 2024-09-12T03:39:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.003+0000 7f92d8af5640 1 --2- 172.21.15.64:0/1841341244 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55ce83292c20 0x55ce83295010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.003+0000 7f92b3fff640 10 client.16188 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:39:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.003+0000 7f92d8af5640 1 --2- 172.21.15.64:0/1841341244 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55ce83292c20 0x55ce83295010 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fea793f1dd29714d server_cookie=1920c22f86d6cba8 in_seq=0 out_seq=0 2024-09-12T03:39:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.004+0000 7f92b3fff640 1 -- 172.21.15.64:0/1841341244 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ce831e9a20 con 0x55ce83292c20 2024-09-12T03:39:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.004+0000 7f92b3fff640 10 client.16188 handle_command_reply: tid=0 2024-09-12T03:39:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.004+0000 7f92ddbd0740 1 client.16188 shutdown 2024-09-12T03:39:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.004+0000 7f92ddbd0740 1 -- 172.21.15.64:0/1841341244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce83278410 msgr2=0x55ce832800f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:39:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.004+0000 7f92ddbd0740 1 --2- 172.21.15.64:0/1841341244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce83278410 0x55ce832800f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f92c009b3a0 tx=0x7f92c00207f0 comp rx=0 tx=0).stop 2024-09-12T03:39:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.004+0000 7f92ddbd0740 1 -- 172.21.15.64:0/1841341244 shutdown_connections 2024-09-12T03:39:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.004+0000 7f92ddbd0740 1 --2- 172.21.15.64:0/1841341244 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55ce83292c20 0x55ce83295010 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.004+0000 7f92ddbd0740 1 --2- 172.21.15.64:0/1841341244 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ce83282cb0 0x55ce83280630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.004+0000 7f92ddbd0740 1 --2- 172.21.15.64:0/1841341244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce83278410 0x55ce832800f0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.004+0000 7f92b3fff640 0 client.16188 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:39:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.004+0000 7f92ddbd0740 1 --2- 172.21.15.64:0/1841341244 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce83277af0 0x55ce8327fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.004+0000 7f92ddbd0740 1 -- 172.21.15.64:0/1841341244 >> 172.21.15.64:0/1841341244 conn(0x55ce83274060 msgr2=0x55ce83279d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:39:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.004+0000 7f92ddbd0740 1 -- 172.21.15.64:0/1841341244 shutdown_connections 2024-09-12T03:39:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.004+0000 7f92ddbd0740 1 -- 172.21.15.64:0/1841341244 wait complete. 2024-09-12T03:39:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.004+0000 7f92ddbd0740 20 client.16188 trim_cache size 0 max 16384 2024-09-12T03:39:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.019+0000 7f92ddbd0740 1 Processor -- start 2024-09-12T03:39:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.019+0000 7f92ddbd0740 1 -- start start 2024-09-12T03:39:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.019+0000 7f92ddbd0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce83277af0 0x55ce8327a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.019+0000 7f92ddbd0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ce83278410 0x55ce83280460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.019+0000 7f92ddbd0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce83282cb0 0x55ce8329a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.019+0000 7f92ddbd0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f92d41b3d70 con 0x55ce83282cb0 2024-09-12T03:39:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.019+0000 7f92ddbd0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f92d41b1640 con 0x55ce83277af0 2024-09-12T03:39:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.019+0000 7f92ddbd0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f92d41af4b0 con 0x55ce83278410 2024-09-12T03:39:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.019+0000 7f92d92f6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce83282cb0 0x55ce8329a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.019+0000 7f92d92f6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce83282cb0 0x55ce8329a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46218/0 (socket says 172.21.15.64:46218) 2024-09-12T03:39:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.019+0000 7f92d92f6640 1 -- 172.21.15.64:0/3507769671 learned_addr learned my addr 172.21.15.64:0/3507769671 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:39:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.020+0000 7f92d8af5640 1 --2- 172.21.15.64:0/3507769671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce83277af0 0x55ce8327a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.020+0000 7f92d3fff640 1 --2- 172.21.15.64:0/3507769671 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ce83278410 0x55ce83280460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.020+0000 7f92d92f6640 1 -- 172.21.15.64:0/3507769671 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ce83278410 msgr2=0x55ce83280460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:39:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.020+0000 7f92d92f6640 1 --2- 172.21.15.64:0/3507769671 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ce83278410 0x55ce83280460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.020+0000 7f92d92f6640 1 -- 172.21.15.64:0/3507769671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce83277af0 msgr2=0x55ce8327a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:39:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.020+0000 7f92d92f6640 1 --2- 172.21.15.64:0/3507769671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce83277af0 0x55ce8327a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.020+0000 7f92d92f6640 1 -- 172.21.15.64:0/3507769671 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce83295640 con 0x55ce83282cb0 2024-09-12T03:39:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.020+0000 7f92d3fff640 1 --2- 172.21.15.64:0/3507769671 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ce83278410 0x55ce83280460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:39:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.020+0000 7f92d8af5640 1 --2- 172.21.15.64:0/3507769671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce83277af0 0x55ce8327a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:39:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.020+0000 7f92d92f6640 1 --2- 172.21.15.64:0/3507769671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce83282cb0 0x55ce8329a8d0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f92cc005d80 tx=0x7f92cc031430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=222336a263235724 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:39:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.020+0000 7f92be7fc640 1 -- 172.21.15.64:0/3507769671 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc003e00 con 0x55ce83282cb0 2024-09-12T03:39:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.020+0000 7f92be7fc640 1 -- 172.21.15.64:0/3507769671 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92cc05c070 con 0x55ce83282cb0 2024-09-12T03:39:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.020+0000 7f92be7fc640 1 -- 172.21.15.64:0/3507769671 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc030ce0 con 0x55ce83282cb0 2024-09-12T03:39:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.021+0000 7f92ddbd0740 1 -- 172.21.15.64:0/3507769671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce83282cb0 msgr2=0x55ce8329a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:39:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.021+0000 7f92ddbd0740 1 --2- 172.21.15.64:0/3507769671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce83282cb0 0x55ce8329a8d0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f92cc005d80 tx=0x7f92cc031430 comp rx=0 tx=0).stop 2024-09-12T03:39:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.021+0000 7f92ddbd0740 1 -- 172.21.15.64:0/3507769671 shutdown_connections 2024-09-12T03:39:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.021+0000 7f92ddbd0740 1 --2- 172.21.15.64:0/3507769671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce83282cb0 0x55ce8329a8d0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:36.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.021+0000 7f92ddbd0740 1 --2- 172.21.15.64:0/3507769671 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ce83278410 0x55ce83280460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:36.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.021+0000 7f92ddbd0740 1 --2- 172.21.15.64:0/3507769671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce83277af0 0x55ce8327a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:36.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.021+0000 7f92ddbd0740 1 -- 172.21.15.64:0/3507769671 >> 172.21.15.64:0/3507769671 conn(0x55ce83274060 msgr2=0x55ce83281c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:39:36.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.021+0000 7f92ddbd0740 1 -- 172.21.15.64:0/3507769671 shutdown_connections 2024-09-12T03:39:36.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.021+0000 7f92ddbd0740 1 -- 172.21.15.64:0/3507769671 wait complete. 2024-09-12T03:39:36.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.021+0000 7f92ddbd0740 1 -- start start 2024-09-12T03:39:36.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.021+0000 7f92ddbd0740 1 Processor -- start 2024-09-12T03:39:36.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.021+0000 7f92ddbd0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce83277af0 0x55ce8328b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:36.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.022+0000 7f92ddbd0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce83278410 0x55ce8327c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:36.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.022+0000 7f92ddbd0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ce83282cb0 0x55ce8327cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:36.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.022+0000 7f92ddbd0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f92d41afb00 con 0x55ce83277af0 2024-09-12T03:39:36.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.022+0000 7f92ddbd0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f92d41b0f70 con 0x55ce83278410 2024-09-12T03:39:36.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.022+0000 7f92ddbd0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f92d41b01d0 con 0x55ce83282cb0 2024-09-12T03:39:36.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.022+0000 7f92ddbd0740 10 client.0 _ll_register_callbacks cb 0x7f92c00046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:39:36.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.022+0000 7f92d8af5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce83277af0 0x55ce8328b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:36.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.022+0000 7f92d8af5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce83277af0 0x55ce8328b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46234/0 (socket says 172.21.15.64:46234) 2024-09-12T03:39:36.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.022+0000 7f92d8af5640 1 -- 172.21.15.64:0/1950535278 learned_addr learned my addr 172.21.15.64:0/1950535278 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:39:36.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.022+0000 7f92d92f6640 1 --2- 172.21.15.64:0/1950535278 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ce83282cb0 0x55ce8327cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:36.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.022+0000 7f92d3fff640 1 --2- 172.21.15.64:0/1950535278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce83278410 0x55ce8327c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:36.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.022+0000 7f92d8af5640 1 -- 172.21.15.64:0/1950535278 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ce83282cb0 msgr2=0x55ce8327cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:39:36.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.022+0000 7f92d8af5640 1 --2- 172.21.15.64:0/1950535278 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ce83282cb0 0x55ce8327cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:36.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.022+0000 7f92d8af5640 1 -- 172.21.15.64:0/1950535278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce83278410 msgr2=0x55ce8327c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:39:36.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.022+0000 7f92d8af5640 1 --2- 172.21.15.64:0/1950535278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce83278410 0x55ce8327c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:36.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.022+0000 7f92d8af5640 1 -- 172.21.15.64:0/1950535278 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce8327d290 con 0x55ce83277af0 2024-09-12T03:39:36.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.022+0000 7f92d8af5640 1 --2- 172.21.15.64:0/1950535278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce83277af0 0x55ce8328b1f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f92c8004700 tx=0x7f92c8012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cbe4628ece90b0d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:39:36.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.022+0000 7f92b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:39:36.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.022+0000 7f92b3fff640 1 -- 172.21.15.64:0/1950535278 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c801e790 con 0x55ce83277af0 2024-09-12T03:39:36.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.022+0000 7f92ddbd0740 1 -- 172.21.15.64:0/1950535278 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92cc05c070 con 0x55ce83277af0 2024-09-12T03:39:36.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.022+0000 7f92b3fff640 1 -- 172.21.15.64:0/1950535278 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9583+0+0 (secure 0 0 0) 0x7f92c801e930 con 0x55ce83277af0 2024-09-12T03:39:36.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.023+0000 7f92b3fff640 1 -- 172.21.15.64:0/1950535278 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f92c8030050 con 0x55ce83277af0 2024-09-12T03:39:36.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.023+0000 7f92ddbd0740 10 client.16194 fetch_fsmap learned FSMap version 25 2024-09-12T03:39:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.023+0000 7f92ddbd0740 1 -- 172.21.15.64:0/1950535278 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ce832fd720 con 0x55ce83277af0 2024-09-12T03:39:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.023+0000 7f92b3fff640 1 -- 172.21.15.64:0/1950535278 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92cc05c070 con 0x55ce83277af0 2024-09-12T03:39:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.023+0000 7f92b3fff640 1 -- 172.21.15.64:0/1950535278 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f92c8011440 con 0x55ce83277af0 2024-09-12T03:39:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.023+0000 7f92ddbd0740 10 client.16194 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:39:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.023+0000 7f92ddbd0740 10 client.16194 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:39:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.023+0000 7f92ddbd0740 20 client.16194 populate_metadata read hostname 'smithi064' 2024-09-12T03:39:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.023+0000 7f92ddbd0740 1 --2- 172.21.15.64:0/1950535278 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55ce832fde50 0x55ce83300270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.023+0000 7f92ddbd0740 4 client.16194 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:39:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.023+0000 7f92ddbd0740 1 -- 172.21.15.64:0/1950535278 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ce831e3d70 con 0x55ce832fde50 2024-09-12T03:39:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.023+0000 7f92d3fff640 1 --2- 172.21.15.64:0/1950535278 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55ce832fde50 0x55ce83300270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.024+0000 7f92b3fff640 10 client.16194 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:39:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.024+0000 7f92d3fff640 1 --2- 172.21.15.64:0/1950535278 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55ce832fde50 0x55ce83300270 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=696da10193160352 server_cookie=f2897b10f5bb10d5 in_seq=0 out_seq=0 2024-09-12T03:39:36.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.037+0000 7f92b3fff640 1 -- 172.21.15.64:0/1950535278 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ce831e3d70 con 0x55ce832fde50 2024-09-12T03:39:36.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.037+0000 7f92b3fff640 10 client.16194 handle_command_reply: tid=0 2024-09-12T03:39:36.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.037+0000 7f92ddbd0740 1 client.16194 shutdown 2024-09-12T03:39:36.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.037+0000 7f92ddbd0740 1 -- 172.21.15.64:0/1950535278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce83277af0 msgr2=0x55ce8328b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:39:36.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.037+0000 7f92ddbd0740 1 --2- 172.21.15.64:0/1950535278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce83277af0 0x55ce8328b1f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f92c8004700 tx=0x7f92c8012820 comp rx=0 tx=0).stop 2024-09-12T03:39:36.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.037+0000 7f92ddbd0740 1 -- 172.21.15.64:0/1950535278 shutdown_connections 2024-09-12T03:39:36.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.037+0000 7f92ddbd0740 1 --2- 172.21.15.64:0/1950535278 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55ce832fde50 0x55ce83300270 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:36.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.037+0000 7f92ddbd0740 1 --2- 172.21.15.64:0/1950535278 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ce83282cb0 0x55ce8327cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:36.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.037+0000 7f92ddbd0740 1 --2- 172.21.15.64:0/1950535278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ce83278410 0x55ce8327c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:36.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.037+0000 7f92ddbd0740 1 --2- 172.21.15.64:0/1950535278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce83277af0 0x55ce8328b1f0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:36.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.037+0000 7f92b3fff640 0 client.16194 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:39:36.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.037+0000 7f92ddbd0740 1 -- 172.21.15.64:0/1950535278 >> 172.21.15.64:0/1950535278 conn(0x55ce83274060 msgr2=0x55ce832729a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:39:36.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.037+0000 7f92ddbd0740 1 -- 172.21.15.64:0/1950535278 shutdown_connections 2024-09-12T03:39:36.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.037+0000 7f92ddbd0740 1 -- 172.21.15.64:0/1950535278 wait complete. 2024-09-12T03:39:36.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.037+0000 7f92ddbd0740 20 client.16194 trim_cache size 0 max 16384 2024-09-12T03:39:36.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:39:36.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:39:36.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:39:36.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:39:36.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.038+0000 7f92dad59640 1 -- 172.21.15.64:0/3295382662 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f92c40a2da0 msgr2=0x7f92c40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:39:36.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.038+0000 7f92dad59640 1 --2- 172.21.15.64:0/3295382662 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f92c40a2da0 0x7f92c40a5260 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f92cc04f000 tx=0x7f92cc04d000 comp rx=0 tx=0).stop 2024-09-12T03:39:36.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.038+0000 7f92dad59640 1 -- 172.21.15.64:0/3295382662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92d4101390 msgr2=0x7f92d419aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:39:36.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.038+0000 7f92dad59640 1 --2- 172.21.15.64:0/3295382662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92d4101390 0x7f92d419aa00 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f92c000d950 tx=0x7f92c000de10 comp rx=0 tx=0).stop 2024-09-12T03:39:36.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.038+0000 7f92dad59640 1 -- 172.21.15.64:0/3295382662 shutdown_connections 2024-09-12T03:39:36.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.038+0000 7f92dad59640 1 --2- 172.21.15.64:0/3295382662 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f92c40a2da0 0x7f92c40a5260 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:36.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.038+0000 7f92dad59640 1 --2- 172.21.15.64:0/3295382662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92d4106a00 0x7f92d419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:36.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.038+0000 7f92dad59640 1 --2- 172.21.15.64:0/3295382662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92d4101390 0x7f92d419aa00 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:36.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.038+0000 7f92dad59640 1 --2- 172.21.15.64:0/3295382662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92d41009f0 0x7f92d419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:36.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.038+0000 7f92dad59640 1 -- 172.21.15.64:0/3295382662 >> 172.21.15.64:0/3295382662 conn(0x7f92d40fc6b0 msgr2=0x7f92d406a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:39:36.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.038+0000 7f92dad59640 1 -- 172.21.15.64:0/3295382662 shutdown_connections 2024-09-12T03:39:36.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:36.038+0000 7f92dad59640 1 -- 172.21.15.64:0/3295382662 wait complete. 2024-09-12T03:39:37.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:37 smithi195 ceph-mon[34786]: pgmap v321: 129 pgs: 129 active+clean; 3.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-09-12T03:39:37.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:37 smithi064 ceph-mon[30952]: pgmap v321: 129 pgs: 129 active+clean; 3.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-09-12T03:39:37.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:37 smithi115 ceph-mon[34801]: pgmap v321: 129 pgs: 129 active+clean; 3.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-09-12T03:39:39.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:39 smithi195 ceph-mon[34786]: pgmap v322: 129 pgs: 129 active+clean; 3.4 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 20 op/s 2024-09-12T03:39:39.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:39 smithi064 ceph-mon[30952]: pgmap v322: 129 pgs: 129 active+clean; 3.4 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 20 op/s 2024-09-12T03:39:39.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:39 smithi115 ceph-mon[34801]: pgmap v322: 129 pgs: 129 active+clean; 3.4 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 20 op/s 2024-09-12T03:39:41.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:41 smithi064 ceph-mon[30952]: pgmap v323: 129 pgs: 129 active+clean; 3.4 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-12T03:39:41.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:41 smithi115 ceph-mon[34801]: pgmap v323: 129 pgs: 129 active+clean; 3.4 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-12T03:39:41.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:41 smithi195 ceph-mon[34786]: pgmap v323: 129 pgs: 129 active+clean; 3.4 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-12T03:39:43.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:43 smithi064 ceph-mon[30952]: pgmap v324: 129 pgs: 129 active+clean; 3.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 52 KiB/s wr, 17 op/s 2024-09-12T03:39:43.794 INFO:tasks.workunit.client.0.smithi064.stdout:499/600: open fd = 3 2024-09-12T03:39:43.794 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.794 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.794 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.794 INFO:tasks.workunit.client.0.smithi064.stdout:500/600: open fd = 3 2024-09-12T03:39:43.794 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.794 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.794 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.794 INFO:tasks.workunit.client.0.smithi064.stdout:501/600: open fd = 3 2024-09-12T03:39:43.795 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.795 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.795 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.795 INFO:tasks.workunit.client.0.smithi064.stdout:502/600: open fd = 3 2024-09-12T03:39:43.795 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.795 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.795 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.795 INFO:tasks.workunit.client.0.smithi064.stdout:503/600: open fd = 3 2024-09-12T03:39:43.795 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.795 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.795 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.796 INFO:tasks.workunit.client.0.smithi064.stdout:504/600: open fd = 3 2024-09-12T03:39:43.796 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.796 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.796 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.796 INFO:tasks.workunit.client.0.smithi064.stdout:505/600: open fd = 3 2024-09-12T03:39:43.796 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.796 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.796 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.796 INFO:tasks.workunit.client.0.smithi064.stdout:506/600: open fd = 3 2024-09-12T03:39:43.796 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.796 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.796 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.797 INFO:tasks.workunit.client.0.smithi064.stdout:507/600: open fd = 3 2024-09-12T03:39:43.797 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.797 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.797 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.797 INFO:tasks.workunit.client.0.smithi064.stdout:508/600: open fd = 3 2024-09-12T03:39:43.797 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.797 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.797 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.797 INFO:tasks.workunit.client.0.smithi064.stdout:509/600: open fd = 3 2024-09-12T03:39:43.797 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.798 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.798 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.798 INFO:tasks.workunit.client.0.smithi064.stdout:510/600: open fd = 3 2024-09-12T03:39:43.798 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.798 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.798 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.798 INFO:tasks.workunit.client.0.smithi064.stdout:511/600: open fd = 3 2024-09-12T03:39:43.798 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.799 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.799 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.799 INFO:tasks.workunit.client.0.smithi064.stdout:512/600: open fd = 3 2024-09-12T03:39:43.799 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.799 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.799 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.799 INFO:tasks.workunit.client.0.smithi064.stdout:513/600: open fd = 3 2024-09-12T03:39:43.799 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.799 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.799 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.799 INFO:tasks.workunit.client.0.smithi064.stdout:514/600: open fd = 3 2024-09-12T03:39:43.799 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.799 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.799 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.800 INFO:tasks.workunit.client.0.smithi064.stdout:515/600: open fd = 3 2024-09-12T03:39:43.800 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.800 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.800 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.800 INFO:tasks.workunit.client.0.smithi064.stdout:516/600: open fd = 3 2024-09-12T03:39:43.800 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.800 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.800 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.801 INFO:tasks.workunit.client.0.smithi064.stdout:517/600: open fd = 3 2024-09-12T03:39:43.801 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.801 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.801 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.801 INFO:tasks.workunit.client.0.smithi064.stdout:518/600: open fd = 3 2024-09-12T03:39:43.801 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.801 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.801 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.801 INFO:tasks.workunit.client.0.smithi064.stdout:519/600: open fd = 3 2024-09-12T03:39:43.801 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.801 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.801 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.801 INFO:tasks.workunit.client.0.smithi064.stdout:520/600: open fd = 3 2024-09-12T03:39:43.802 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.802 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.802 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.802 INFO:tasks.workunit.client.0.smithi064.stdout:521/600: open fd = 3 2024-09-12T03:39:43.802 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.802 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.802 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.802 INFO:tasks.workunit.client.0.smithi064.stdout:522/600: open fd = 3 2024-09-12T03:39:43.802 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.802 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.802 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.802 INFO:tasks.workunit.client.0.smithi064.stdout:523/600: open fd = 3 2024-09-12T03:39:43.802 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.802 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.803 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.803 INFO:tasks.workunit.client.0.smithi064.stdout:524/600: open fd = 3 2024-09-12T03:39:43.803 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.803 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.803 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.803 INFO:tasks.workunit.client.0.smithi064.stdout:525/600: open fd = 3 2024-09-12T03:39:43.803 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.803 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.803 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.803 INFO:tasks.workunit.client.0.smithi064.stdout:526/600: open fd = 3 2024-09-12T03:39:43.803 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.803 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.804 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.804 INFO:tasks.workunit.client.0.smithi064.stdout:527/600: open fd = 3 2024-09-12T03:39:43.804 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.804 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.804 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.804 INFO:tasks.workunit.client.0.smithi064.stdout:528/600: open fd = 3 2024-09-12T03:39:43.804 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.804 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.804 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.804 INFO:tasks.workunit.client.0.smithi064.stdout:529/600: open fd = 3 2024-09-12T03:39:43.804 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.804 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.805 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.805 INFO:tasks.workunit.client.0.smithi064.stdout:530/600: open fd = 3 2024-09-12T03:39:43.805 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.805 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.805 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.805 INFO:tasks.workunit.client.0.smithi064.stdout:531/600: open fd = 3 2024-09-12T03:39:43.805 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.805 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.805 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.805 INFO:tasks.workunit.client.0.smithi064.stdout:532/600: open fd = 3 2024-09-12T03:39:43.805 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.806 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.806 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.806 INFO:tasks.workunit.client.0.smithi064.stdout:533/600: open fd = 3 2024-09-12T03:39:43.806 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.806 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.806 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.806 INFO:tasks.workunit.client.0.smithi064.stdout:534/600: open fd = 3 2024-09-12T03:39:43.806 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.806 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.806 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.806 INFO:tasks.workunit.client.0.smithi064.stdout:535/600: open fd = 3 2024-09-12T03:39:43.806 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.806 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.806 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.807 INFO:tasks.workunit.client.0.smithi064.stdout:536/600: open fd = 3 2024-09-12T03:39:43.807 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.807 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.807 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.807 INFO:tasks.workunit.client.0.smithi064.stdout:537/600: open fd = 3 2024-09-12T03:39:43.807 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.807 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.807 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.807 INFO:tasks.workunit.client.0.smithi064.stdout:538/600: open fd = 3 2024-09-12T03:39:43.807 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.807 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.807 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.807 INFO:tasks.workunit.client.0.smithi064.stdout:539/600: open fd = 3 2024-09-12T03:39:43.808 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.808 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.808 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.808 INFO:tasks.workunit.client.0.smithi064.stdout:540/600: open fd = 3 2024-09-12T03:39:43.808 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.808 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.808 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.808 INFO:tasks.workunit.client.0.smithi064.stdout:541/600: open fd = 3 2024-09-12T03:39:43.808 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.808 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.808 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.809 INFO:tasks.workunit.client.0.smithi064.stdout:542/600: open fd = 3 2024-09-12T03:39:43.809 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.809 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.809 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.809 INFO:tasks.workunit.client.0.smithi064.stdout:543/600: open fd = 3 2024-09-12T03:39:43.809 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.810 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.810 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.810 INFO:tasks.workunit.client.0.smithi064.stdout:544/600: open fd = 3 2024-09-12T03:39:43.810 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.810 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.810 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.810 INFO:tasks.workunit.client.0.smithi064.stdout:545/600: open fd = 3 2024-09-12T03:39:43.810 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.810 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.810 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.810 INFO:tasks.workunit.client.0.smithi064.stdout:546/600: open fd = 3 2024-09-12T03:39:43.810 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.811 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.811 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.811 INFO:tasks.workunit.client.0.smithi064.stdout:547/600: open fd = 3 2024-09-12T03:39:43.811 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.811 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.811 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.811 INFO:tasks.workunit.client.0.smithi064.stdout:548/600: open fd = 3 2024-09-12T03:39:43.811 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.811 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.811 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.812 INFO:tasks.workunit.client.0.smithi064.stdout:549/600: open fd = 3 2024-09-12T03:39:43.812 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.812 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.812 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.812 INFO:tasks.workunit.client.0.smithi064.stdout:550/600: open fd = 3 2024-09-12T03:39:43.812 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.812 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.812 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.813 INFO:tasks.workunit.client.0.smithi064.stdout:551/600: open fd = 3 2024-09-12T03:39:43.813 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.813 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.813 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.813 INFO:tasks.workunit.client.0.smithi064.stdout:552/600: open fd = 3 2024-09-12T03:39:43.813 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.813 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.813 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.813 INFO:tasks.workunit.client.0.smithi064.stdout:553/600: open fd = 3 2024-09-12T03:39:43.814 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.814 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.814 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.814 INFO:tasks.workunit.client.0.smithi064.stdout:554/600: open fd = 3 2024-09-12T03:39:43.814 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.814 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.814 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.814 INFO:tasks.workunit.client.0.smithi064.stdout:555/600: open fd = 3 2024-09-12T03:39:43.814 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.814 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.814 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.815 INFO:tasks.workunit.client.0.smithi064.stdout:556/600: open fd = 3 2024-09-12T03:39:43.815 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.815 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.815 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.815 INFO:tasks.workunit.client.0.smithi064.stdout:557/600: open fd = 3 2024-09-12T03:39:43.815 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.815 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.816 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.816 INFO:tasks.workunit.client.0.smithi064.stdout:558/600: open fd = 3 2024-09-12T03:39:43.816 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.816 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.816 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.816 INFO:tasks.workunit.client.0.smithi064.stdout:559/600: open fd = 3 2024-09-12T03:39:43.816 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.816 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.816 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.817 INFO:tasks.workunit.client.0.smithi064.stdout:560/600: open fd = 3 2024-09-12T03:39:43.817 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.817 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:39:43.817 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:39:43.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:43 smithi115 ceph-mon[34801]: pgmap v324: 129 pgs: 129 active+clean; 3.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 52 KiB/s wr, 17 op/s 2024-09-12T03:39:43.958 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:43 smithi195 ceph-mon[34786]: pgmap v324: 129 pgs: 129 active+clean; 3.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 52 KiB/s wr, 17 op/s 2024-09-12T03:39:45.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:45 smithi064 ceph-mon[30952]: pgmap v325: 129 pgs: 129 active+clean; 3.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 12 op/s 2024-09-12T03:39:45.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:45 smithi115 ceph-mon[34801]: pgmap v325: 129 pgs: 129 active+clean; 3.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 12 op/s 2024-09-12T03:39:46.492 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:45 smithi195 ceph-mon[34786]: pgmap v325: 129 pgs: 129 active+clean; 3.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 12 op/s 2024-09-12T03:39:47.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:47 smithi064 ceph-mon[30952]: pgmap v326: 129 pgs: 129 active+clean; 3.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 12 op/s 2024-09-12T03:39:47.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:47 smithi115 ceph-mon[34801]: pgmap v326: 129 pgs: 129 active+clean; 3.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 12 op/s 2024-09-12T03:39:47.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:47 smithi195 ceph-mon[34786]: pgmap v326: 129 pgs: 129 active+clean; 3.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 12 op/s 2024-09-12T03:39:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.134+0000 7f82eb70a640 1 -- 172.21.15.64:0/1647146875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82e41069c0 msgr2=0x7f82e4106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:39:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.134+0000 7f82eb70a640 1 --2- 172.21.15.64:0/1647146875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82e41069c0 0x7f82e4106da0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f82e0009930 tx=0x7f82e002a420 comp rx=0 tx=0).stop 2024-09-12T03:39:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.134+0000 7f82eb70a640 1 -- 172.21.15.64:0/1647146875 shutdown_connections 2024-09-12T03:39:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.134+0000 7f82eb70a640 1 --2- 172.21.15.64:0/1647146875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82e4101350 0x7f82e410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.134+0000 7f82eb70a640 1 --2- 172.21.15.64:0/1647146875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82e41009b0 0x7f82e4100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.134+0000 7f82eb70a640 1 --2- 172.21.15.64:0/1647146875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82e41069c0 0x7f82e4106da0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.134+0000 7f82eb70a640 1 -- 172.21.15.64:0/1647146875 >> 172.21.15.64:0/1647146875 conn(0x7f82e4077990 msgr2=0x7f82e40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:39:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.134+0000 7f82eb70a640 1 -- 172.21.15.64:0/1647146875 shutdown_connections 2024-09-12T03:39:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.135+0000 7f82eb70a640 1 -- 172.21.15.64:0/1647146875 wait complete. 2024-09-12T03:39:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.135+0000 7f82eb70a640 1 Processor -- start 2024-09-12T03:39:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.135+0000 7f82eb70a640 1 -- start start 2024-09-12T03:39:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.135+0000 7f82eb70a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82e41009b0 0x7f82e4197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.135+0000 7f82eb70a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82e4101350 0x7f82e4197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.135+0000 7f82eb70a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82e41069c0 0x7f82e419ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.135+0000 7f82eb70a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f82e4110cb0 con 0x7f82e41009b0 2024-09-12T03:39:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.135+0000 7f82eb70a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f82e4110b30 con 0x7f82e41069c0 2024-09-12T03:39:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.135+0000 7f82eb70a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f82e4110e30 con 0x7f82e4101350 2024-09-12T03:39:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.135+0000 7f82e94a6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82e41009b0 0x7f82e4197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.135+0000 7f82e94a6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82e41009b0 0x7f82e4197140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50002/0 (socket says 172.21.15.64:50002) 2024-09-12T03:39:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.135+0000 7f82e94a6640 1 -- 172.21.15.64:0/4075236490 learned_addr learned my addr 172.21.15.64:0/4075236490 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:39:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.136+0000 7f82e9ca7640 1 --2- 172.21.15.64:0/4075236490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82e41069c0 0x7f82e419ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.136+0000 7f82e8ca5640 1 --2- 172.21.15.64:0/4075236490 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82e4101350 0x7f82e4197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.136+0000 7f82e94a6640 1 -- 172.21.15.64:0/4075236490 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82e4101350 msgr2=0x7f82e4197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:39:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.136+0000 7f82e94a6640 1 --2- 172.21.15.64:0/4075236490 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82e4101350 0x7f82e4197680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.136+0000 7f82e94a6640 1 -- 172.21.15.64:0/4075236490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82e41069c0 msgr2=0x7f82e419ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:39:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.136+0000 7f82e94a6640 1 --2- 172.21.15.64:0/4075236490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82e41069c0 0x7f82e419ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.136+0000 7f82e94a6640 1 -- 172.21.15.64:0/4075236490 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82e419c130 con 0x7f82e41009b0 2024-09-12T03:39:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.136+0000 7f82e94a6640 1 --2- 172.21.15.64:0/4075236490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82e41009b0 0x7f82e4197140 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f82e0009230 tx=0x7f82e00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:39:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.136+0000 7f82da7fc640 1 -- 172.21.15.64:0/4075236490 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e0035030 con 0x7f82e41009b0 2024-09-12T03:39:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.136+0000 7f82da7fc640 1 -- 172.21.15.64:0/4075236490 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82e00403e0 con 0x7f82e41009b0 2024-09-12T03:39:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.136+0000 7f82eb70a640 1 -- 172.21.15.64:0/4075236490 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82e419c3c0 con 0x7f82e41009b0 2024-09-12T03:39:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.136+0000 7f82da7fc640 1 -- 172.21.15.64:0/4075236490 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e00044e0 con 0x7f82e41009b0 2024-09-12T03:39:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.136+0000 7f82eb70a640 1 -- 172.21.15.64:0/4075236490 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82e40698a0 con 0x7f82e41009b0 2024-09-12T03:39:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.138+0000 7f82da7fc640 1 -- 172.21.15.64:0/4075236490 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f82e0040590 con 0x7f82e41009b0 2024-09-12T03:39:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.139+0000 7f82ee4a2740 1 Processor -- start 2024-09-12T03:39:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.139+0000 7f82ee4a2740 1 -- start start 2024-09-12T03:39:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.139+0000 7f82ee4a2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556514df4af0 0x556514df4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.139+0000 7f82ee4a2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556514df5410 0x556514dff770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.139+0000 7f82ee4a2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556514dffcb0 0x556514e020a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.139+0000 7f82ee4a2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556514d592f0 con 0x556514dffcb0 2024-09-12T03:39:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.139+0000 7f82ee4a2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556514d01270 con 0x556514df4af0 2024-09-12T03:39:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.139+0000 7f82ee4a2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556514d552b0 con 0x556514df5410 2024-09-12T03:39:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.140+0000 7f82da7fc640 1 --2- 172.21.15.64:0/4075236490 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f82d00a2da0 0x7f82d00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.140+0000 7f82e8ca5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556514dffcb0 0x556514e020a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.140+0000 7f82e8ca5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556514dffcb0 0x556514e020a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50018/0 (socket says 172.21.15.64:50018) 2024-09-12T03:39:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.140+0000 7f82e8ca5640 1 -- 172.21.15.64:0/2969421983 learned_addr learned my addr 172.21.15.64:0/2969421983 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:39:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.140+0000 7f82e9ca7640 1 --2- 172.21.15.64:0/4075236490 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f82d00a2da0 0x7f82d00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.140+0000 7f82e9ca7640 1 --2- 172.21.15.64:0/2969421983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556514df4af0 0x556514df4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.140+0000 7f82da7fc640 1 -- 172.21.15.64:0/4075236490 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9583+0+0 (secure 0 0 0) 0x7f82e00f0080 con 0x7f82e41009b0 2024-09-12T03:39:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.140+0000 7f82e94a6640 1 --2- 172.21.15.64:0/2969421983 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556514df5410 0x556514dff770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.140+0000 7f82e8ca5640 1 -- 172.21.15.64:0/2969421983 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556514df5410 msgr2=0x556514dff770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:39:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.140+0000 7f82e8ca5640 1 --2- 172.21.15.64:0/2969421983 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556514df5410 0x556514dff770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.140+0000 7f82e8ca5640 1 -- 172.21.15.64:0/2969421983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556514df4af0 msgr2=0x556514df4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:39:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.140+0000 7f82e8ca5640 1 --2- 172.21.15.64:0/2969421983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556514df4af0 0x556514df4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.140+0000 7f82e8ca5640 1 -- 172.21.15.64:0/2969421983 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556514d00340 con 0x556514dffcb0 2024-09-12T03:39:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.140+0000 7f82e9ca7640 1 --2- 172.21.15.64:0/2969421983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556514df4af0 0x556514df4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:39:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.140+0000 7f82e9ca7640 1 --2- 172.21.15.64:0/4075236490 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f82d00a2da0 0x7f82d00a5260 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f82dc010a90 tx=0x7f82dc010a20 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:39:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.140+0000 7f82e8ca5640 1 --2- 172.21.15.64:0/2969421983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556514dffcb0 0x556514e020a0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f82d400bcb0 tx=0x7f82d40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23d536ea76bc8757 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:39:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.140+0000 7f82cf7fe640 1 -- 172.21.15.64:0/2969421983 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d4004030 con 0x556514dffcb0 2024-09-12T03:39:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.140+0000 7f82cf7fe640 1 -- 172.21.15.64:0/2969421983 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82d40041d0 con 0x556514dffcb0 2024-09-12T03:39:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.140+0000 7f82cf7fe640 1 -- 172.21.15.64:0/2969421983 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d40116a0 con 0x556514dffcb0 2024-09-12T03:39:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.141+0000 7f82ee4a2740 1 -- 172.21.15.64:0/2969421983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556514dffcb0 msgr2=0x556514e020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:39:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.141+0000 7f82ee4a2740 1 --2- 172.21.15.64:0/2969421983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556514dffcb0 0x556514e020a0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f82d400bcb0 tx=0x7f82d40107a0 comp rx=0 tx=0).stop 2024-09-12T03:39:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.141+0000 7f82ee4a2740 1 -- 172.21.15.64:0/2969421983 shutdown_connections 2024-09-12T03:39:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.141+0000 7f82ee4a2740 1 --2- 172.21.15.64:0/2969421983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556514dffcb0 0x556514e020a0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.141+0000 7f82ee4a2740 1 --2- 172.21.15.64:0/2969421983 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556514df5410 0x556514dff770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.141+0000 7f82ee4a2740 1 --2- 172.21.15.64:0/2969421983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556514df4af0 0x556514df4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.141+0000 7f82ee4a2740 1 -- 172.21.15.64:0/2969421983 >> 172.21.15.64:0/2969421983 conn(0x556514df1060 msgr2=0x556514df1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:39:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.141+0000 7f82ee4a2740 1 -- 172.21.15.64:0/2969421983 shutdown_connections 2024-09-12T03:39:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.141+0000 7f82ee4a2740 1 -- 172.21.15.64:0/2969421983 wait complete. 2024-09-12T03:39:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.142+0000 7f82ee4a2740 1 -- start start 2024-09-12T03:39:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.142+0000 7f82ee4a2740 1 Processor -- start 2024-09-12T03:39:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.142+0000 7f82ee4a2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556514df4af0 0x556514dfcbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.142+0000 7f82ee4a2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556514df5410 0x556514dfd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.142+0000 7f82ee4a2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556514dffcb0 0x556514dfd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.142+0000 7f82ee4a2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556514e02880 con 0x556514df5410 2024-09-12T03:39:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.142+0000 7f82ee4a2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556514ce2290 con 0x556514df4af0 2024-09-12T03:39:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.142+0000 7f82ee4a2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556514e02a00 con 0x556514dffcb0 2024-09-12T03:39:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.142+0000 7f82ee4a2740 10 client.0 _ll_register_callbacks cb 0x556514d13140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:39:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.142+0000 7f82e8ca5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556514df5410 0x556514dfd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.142+0000 7f82e8ca5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556514df5410 0x556514dfd100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50020/0 (socket says 172.21.15.64:50020) 2024-09-12T03:39:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.142+0000 7f82e8ca5640 1 -- 172.21.15.64:0/3525472917 learned_addr learned my addr 172.21.15.64:0/3525472917 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:39:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.143+0000 7f82e9ca7640 1 --2- 172.21.15.64:0/3525472917 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556514dffcb0 0x556514dfd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.143+0000 7f82e94a6640 1 --2- 172.21.15.64:0/3525472917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556514df4af0 0x556514dfcbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.143+0000 7f82e8ca5640 1 -- 172.21.15.64:0/3525472917 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556514dffcb0 msgr2=0x556514dfd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:39:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.143+0000 7f82e8ca5640 1 --2- 172.21.15.64:0/3525472917 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556514dffcb0 0x556514dfd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.143+0000 7f82e8ca5640 1 -- 172.21.15.64:0/3525472917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556514df4af0 msgr2=0x556514dfcbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:39:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.143+0000 7f82e8ca5640 1 --2- 172.21.15.64:0/3525472917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556514df4af0 0x556514dfcbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.143+0000 7f82e8ca5640 1 -- 172.21.15.64:0/3525472917 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556514a6c210 con 0x556514df5410 2024-09-12T03:39:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.143+0000 7f82e8ca5640 1 --2- 172.21.15.64:0/3525472917 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556514df5410 0x556514dfd100 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f82d4011010 tx=0x7f82d400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b5023c2878003e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:39:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.143+0000 7f82ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:39:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.143+0000 7f82ccff9640 1 -- 172.21.15.64:0/3525472917 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d402a060 con 0x556514df5410 2024-09-12T03:39:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.143+0000 7f82ee4a2740 1 -- 172.21.15.64:0/3525472917 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82d40041d0 con 0x556514df5410 2024-09-12T03:39:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.143+0000 7f82ccff9640 1 -- 172.21.15.64:0/3525472917 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9583+0+0 (secure 0 0 0) 0x7f82d4004380 con 0x556514df5410 2024-09-12T03:39:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.143+0000 7f82ccff9640 1 -- 172.21.15.64:0/3525472917 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f82d4031050 con 0x556514df5410 2024-09-12T03:39:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.143+0000 7f82ee4a2740 10 client.16206 fetch_fsmap learned FSMap version 25 2024-09-12T03:39:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.143+0000 7f82ee4a2740 1 -- 172.21.15.64:0/3525472917 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556514b38620 con 0x556514df5410 2024-09-12T03:39:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.143+0000 7f82ccff9640 1 -- 172.21.15.64:0/3525472917 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82d40041d0 con 0x556514df5410 2024-09-12T03:39:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.144+0000 7f82ccff9640 1 -- 172.21.15.64:0/3525472917 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f82d40048f0 con 0x556514df5410 2024-09-12T03:39:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.144+0000 7f82ee4a2740 10 client.16206 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:39:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.144+0000 7f82ee4a2740 10 client.16206 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:39:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.144+0000 7f82ee4a2740 20 client.16206 populate_metadata read hostname 'smithi064' 2024-09-12T03:39:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.144+0000 7f82ee4a2740 1 --2- 172.21.15.64:0/3525472917 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x556514e0fbd0 0x556514e11fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.144+0000 7f82ee4a2740 4 client.16206 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:39:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.144+0000 7f82ee4a2740 1 -- 172.21.15.64:0/3525472917 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556514d66a20 con 0x556514e0fbd0 2024-09-12T03:39:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.144+0000 7f82e94a6640 1 --2- 172.21.15.64:0/3525472917 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x556514e0fbd0 0x556514e11fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.144+0000 7f82e94a6640 1 --2- 172.21.15.64:0/3525472917 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x556514e0fbd0 0x556514e11fc0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6934fe168758aa2a server_cookie=99951871b09dd009 in_seq=0 out_seq=0 2024-09-12T03:39:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.144+0000 7f82ccff9640 10 client.16206 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:39:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.145+0000 7f82ccff9640 1 -- 172.21.15.64:0/3525472917 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556514d66a20 con 0x556514e0fbd0 2024-09-12T03:39:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.145+0000 7f82ccff9640 10 client.16206 handle_command_reply: tid=0 2024-09-12T03:39:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.145+0000 7f82ee4a2740 1 client.16206 shutdown 2024-09-12T03:39:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.145+0000 7f82ee4a2740 1 -- 172.21.15.64:0/3525472917 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556514df5410 msgr2=0x556514dfd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:39:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.145+0000 7f82ee4a2740 1 --2- 172.21.15.64:0/3525472917 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556514df5410 0x556514dfd100 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f82d4011010 tx=0x7f82d400b8e0 comp rx=0 tx=0).stop 2024-09-12T03:39:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.145+0000 7f82ee4a2740 1 -- 172.21.15.64:0/3525472917 shutdown_connections 2024-09-12T03:39:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.145+0000 7f82ee4a2740 1 --2- 172.21.15.64:0/3525472917 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x556514e0fbd0 0x556514e11fc0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.145+0000 7f82ee4a2740 1 --2- 172.21.15.64:0/3525472917 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556514dffcb0 0x556514dfd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.145+0000 7f82ee4a2740 1 --2- 172.21.15.64:0/3525472917 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556514df5410 0x556514dfd100 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.145+0000 7f82ccff9640 0 client.16206 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:39:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.145+0000 7f82ee4a2740 1 --2- 172.21.15.64:0/3525472917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556514df4af0 0x556514dfcbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.145+0000 7f82ee4a2740 1 -- 172.21.15.64:0/3525472917 >> 172.21.15.64:0/3525472917 conn(0x556514df1060 msgr2=0x556514df6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:39:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.146+0000 7f82ee4a2740 1 -- 172.21.15.64:0/3525472917 shutdown_connections 2024-09-12T03:39:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.146+0000 7f82ee4a2740 1 -- 172.21.15.64:0/3525472917 wait complete. 2024-09-12T03:39:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.146+0000 7f82ee4a2740 20 client.16206 trim_cache size 0 max 16384 2024-09-12T03:39:49.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.160+0000 7f82ee4a2740 1 Processor -- start 2024-09-12T03:39:49.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.160+0000 7f82ee4a2740 1 -- start start 2024-09-12T03:39:49.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.160+0000 7f82ee4a2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556514df4af0 0x556514dfeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:49.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.161+0000 7f82ee4a2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556514df5410 0x556514dfd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:49.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.161+0000 7f82ee4a2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556514dffcb0 0x556514e17590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:49.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.161+0000 7f82ee4a2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f82e41af720 con 0x556514dffcb0 2024-09-12T03:39:49.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.161+0000 7f82ee4a2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f82e41acff0 con 0x556514df5410 2024-09-12T03:39:49.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.161+0000 7f82ee4a2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f82e41aae60 con 0x556514df4af0 2024-09-12T03:39:49.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.161+0000 7f82e9ca7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556514dffcb0 0x556514e17590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:49.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.161+0000 7f82e9ca7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556514dffcb0 0x556514e17590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50030/0 (socket says 172.21.15.64:50030) 2024-09-12T03:39:49.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.161+0000 7f82e9ca7640 1 -- 172.21.15.64:0/156406964 learned_addr learned my addr 172.21.15.64:0/156406964 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:39:49.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.161+0000 7f82e94a6640 1 --2- 172.21.15.64:0/156406964 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556514df4af0 0x556514dfeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:49.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.161+0000 7f82e8ca5640 1 --2- 172.21.15.64:0/156406964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556514df5410 0x556514dfd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:49.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.161+0000 7f82e9ca7640 1 -- 172.21.15.64:0/156406964 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556514df4af0 msgr2=0x556514dfeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:39:49.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.161+0000 7f82e9ca7640 1 --2- 172.21.15.64:0/156406964 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556514df4af0 0x556514dfeef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:49.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.161+0000 7f82e9ca7640 1 -- 172.21.15.64:0/156406964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556514df5410 msgr2=0x556514dfd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:39:49.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.161+0000 7f82e9ca7640 1 --2- 172.21.15.64:0/156406964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556514df5410 0x556514dfd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:49.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.161+0000 7f82e9ca7640 1 -- 172.21.15.64:0/156406964 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556514df5240 con 0x556514dffcb0 2024-09-12T03:39:49.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.161+0000 7f82e8ca5640 1 --2- 172.21.15.64:0/156406964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556514df5410 0x556514dfd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:39:49.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.161+0000 7f82e9ca7640 1 --2- 172.21.15.64:0/156406964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556514dffcb0 0x556514e17590 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f82dc0039c0 tx=0x7f82dc012070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b11e7848ca5ee2f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:39:49.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.161+0000 7f82ceffd640 1 -- 172.21.15.64:0/156406964 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc004530 con 0x556514dffcb0 2024-09-12T03:39:49.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.161+0000 7f82ceffd640 1 -- 172.21.15.64:0/156406964 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82dc0046d0 con 0x556514dffcb0 2024-09-12T03:39:49.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.161+0000 7f82ceffd640 1 -- 172.21.15.64:0/156406964 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc0227b0 con 0x556514dffcb0 2024-09-12T03:39:49.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.162+0000 7f82ee4a2740 1 -- 172.21.15.64:0/156406964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556514dffcb0 msgr2=0x556514e17590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:39:49.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.162+0000 7f82ee4a2740 1 --2- 172.21.15.64:0/156406964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556514dffcb0 0x556514e17590 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f82dc0039c0 tx=0x7f82dc012070 comp rx=0 tx=0).stop 2024-09-12T03:39:49.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.162+0000 7f82ee4a2740 1 -- 172.21.15.64:0/156406964 shutdown_connections 2024-09-12T03:39:49.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.162+0000 7f82ee4a2740 1 --2- 172.21.15.64:0/156406964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556514dffcb0 0x556514e17590 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.162+0000 7f82ee4a2740 1 --2- 172.21.15.64:0/156406964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556514df5410 0x556514dfd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.162+0000 7f82ee4a2740 1 --2- 172.21.15.64:0/156406964 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556514df4af0 0x556514dfeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.162+0000 7f82ee4a2740 1 -- 172.21.15.64:0/156406964 >> 172.21.15.64:0/156406964 conn(0x556514df1060 msgr2=0x556514df01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:39:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.162+0000 7f82ee4a2740 1 -- 172.21.15.64:0/156406964 shutdown_connections 2024-09-12T03:39:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.162+0000 7f82ee4a2740 1 -- 172.21.15.64:0/156406964 wait complete. 2024-09-12T03:39:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.162+0000 7f82ee4a2740 1 -- start start 2024-09-12T03:39:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.162+0000 7f82ee4a2740 1 Processor -- start 2024-09-12T03:39:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.163+0000 7f82ee4a2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556514df4af0 0x556514e08580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.163+0000 7f82ee4a2740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556514df5410 0x556514e038a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.163+0000 7f82ee4a2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556514dffcb0 0x556514e03fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.163+0000 7f82ee4a2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f82e41ab4b0 con 0x556514dffcb0 2024-09-12T03:39:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.163+0000 7f82ee4a2740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f82e41ac920 con 0x556514df5410 2024-09-12T03:39:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.163+0000 7f82ee4a2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f82e41abb80 con 0x556514df4af0 2024-09-12T03:39:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.163+0000 7f82ee4a2740 10 client.0 _ll_register_callbacks cb 0x7f82d400bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:39:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.163+0000 7f82e9ca7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556514dffcb0 0x556514e03fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.163+0000 7f82e9ca7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556514dffcb0 0x556514e03fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50040/0 (socket says 172.21.15.64:50040) 2024-09-12T03:39:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.163+0000 7f82e9ca7640 1 -- 172.21.15.64:0/1266021857 learned_addr learned my addr 172.21.15.64:0/1266021857 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:39:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.163+0000 7f82e94a6640 1 --2- 172.21.15.64:0/1266021857 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556514df4af0 0x556514e08580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.163+0000 7f82e8ca5640 1 --2- 172.21.15.64:0/1266021857 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556514df5410 0x556514e038a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.163+0000 7f82e9ca7640 1 -- 172.21.15.64:0/1266021857 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556514df4af0 msgr2=0x556514e08580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:39:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.163+0000 7f82e9ca7640 1 --2- 172.21.15.64:0/1266021857 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556514df4af0 0x556514e08580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.163+0000 7f82e9ca7640 1 -- 172.21.15.64:0/1266021857 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556514df5410 msgr2=0x556514e038a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:39:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.163+0000 7f82e9ca7640 1 --2- 172.21.15.64:0/1266021857 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556514df5410 0x556514e038a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.163+0000 7f82e9ca7640 1 -- 172.21.15.64:0/1266021857 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556514e79560 con 0x556514dffcb0 2024-09-12T03:39:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.163+0000 7f82e8ca5640 1 --2- 172.21.15.64:0/1266021857 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556514df5410 0x556514e038a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:39:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.164+0000 7f82e9ca7640 1 --2- 172.21.15.64:0/1266021857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556514dffcb0 0x556514e03fe0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f82dc006010 tx=0x7f82dc005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d5b49fb535d864 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:39:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.164+0000 7f82ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:39:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.164+0000 7f82ccff9640 1 -- 172.21.15.64:0/1266021857 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc0097f0 con 0x556514dffcb0 2024-09-12T03:39:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.164+0000 7f82ee4a2740 1 -- 172.21.15.64:0/1266021857 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82dc0046d0 con 0x556514dffcb0 2024-09-12T03:39:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.164+0000 7f82ccff9640 1 -- 172.21.15.64:0/1266021857 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9583+0+0 (secure 0 0 0) 0x7f82dc009990 con 0x556514dffcb0 2024-09-12T03:39:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.164+0000 7f82ccff9640 1 -- 172.21.15.64:0/1266021857 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f82dc009b90 con 0x556514dffcb0 2024-09-12T03:39:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.164+0000 7f82ee4a2740 10 client.16212 fetch_fsmap learned FSMap version 25 2024-09-12T03:39:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.164+0000 7f82ee4a2740 1 -- 172.21.15.64:0/1266021857 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556514e7a710 con 0x556514dffcb0 2024-09-12T03:39:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.164+0000 7f82ccff9640 1 -- 172.21.15.64:0/1266021857 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82dc0046d0 con 0x556514dffcb0 2024-09-12T03:39:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.164+0000 7f82ccff9640 1 -- 172.21.15.64:0/1266021857 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f82dc034070 con 0x556514dffcb0 2024-09-12T03:39:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.164+0000 7f82ee4a2740 10 client.16212 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:39:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.164+0000 7f82ee4a2740 10 client.16212 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:39:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.164+0000 7f82ee4a2740 20 client.16212 populate_metadata read hostname 'smithi064' 2024-09-12T03:39:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.164+0000 7f82ee4a2740 1 --2- 172.21.15.64:0/1266021857 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x556514e7ad20 0x556514e7d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:39:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.164+0000 7f82ee4a2740 4 client.16212 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:39:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.164+0000 7f82ee4a2740 1 -- 172.21.15.64:0/1266021857 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556514d60d70 con 0x556514e7ad20 2024-09-12T03:39:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.165+0000 7f82e94a6640 1 --2- 172.21.15.64:0/1266021857 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x556514e7ad20 0x556514e7d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:39:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.165+0000 7f82e94a6640 1 --2- 172.21.15.64:0/1266021857 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x556514e7ad20 0x556514e7d110 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=596707699094834c server_cookie=8882c667e156e9a2 in_seq=0 out_seq=0 2024-09-12T03:39:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.165+0000 7f82ccff9640 10 client.16212 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:39:49.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.178+0000 7f82ccff9640 1 -- 172.21.15.64:0/1266021857 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556514d60d70 con 0x556514e7ad20 2024-09-12T03:39:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.178+0000 7f82ccff9640 10 client.16212 handle_command_reply: tid=0 2024-09-12T03:39:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.178+0000 7f82ee4a2740 1 client.16212 shutdown 2024-09-12T03:39:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.178+0000 7f82ee4a2740 1 -- 172.21.15.64:0/1266021857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556514dffcb0 msgr2=0x556514e03fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:39:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.178+0000 7f82ee4a2740 1 --2- 172.21.15.64:0/1266021857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556514dffcb0 0x556514e03fe0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f82dc006010 tx=0x7f82dc005ae0 comp rx=0 tx=0).stop 2024-09-12T03:39:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.179+0000 7f82ee4a2740 1 -- 172.21.15.64:0/1266021857 shutdown_connections 2024-09-12T03:39:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.179+0000 7f82ee4a2740 1 --2- 172.21.15.64:0/1266021857 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x556514e7ad20 0x556514e7d110 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.179+0000 7f82ee4a2740 1 --2- 172.21.15.64:0/1266021857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556514dffcb0 0x556514e03fe0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.179+0000 7f82ee4a2740 1 --2- 172.21.15.64:0/1266021857 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556514df5410 0x556514e038a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.179+0000 7f82ccff9640 0 client.16212 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:39:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.179+0000 7f82ee4a2740 1 --2- 172.21.15.64:0/1266021857 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556514df4af0 0x556514e08580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.179+0000 7f82ee4a2740 1 -- 172.21.15.64:0/1266021857 >> 172.21.15.64:0/1266021857 conn(0x556514df1060 msgr2=0x556514e0ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:39:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.179+0000 7f82ee4a2740 1 -- 172.21.15.64:0/1266021857 shutdown_connections 2024-09-12T03:39:49.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.179+0000 7f82ee4a2740 1 -- 172.21.15.64:0/1266021857 wait complete. 2024-09-12T03:39:49.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.179+0000 7f82ee4a2740 20 client.16212 trim_cache size 0 max 16384 2024-09-12T03:39:49.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:39:49.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:39:49.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:39:49.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:39:49.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.179+0000 7f82eb70a640 1 -- 172.21.15.64:0/4075236490 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f82d00a2da0 msgr2=0x7f82d00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:39:49.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.179+0000 7f82eb70a640 1 --2- 172.21.15.64:0/4075236490 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f82d00a2da0 0x7f82d00a5260 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f82dc010a90 tx=0x7f82dc010a20 comp rx=0 tx=0).stop 2024-09-12T03:39:49.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.179+0000 7f82eb70a640 1 -- 172.21.15.64:0/4075236490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82e41009b0 msgr2=0x7f82e4197140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:39:49.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.179+0000 7f82eb70a640 1 --2- 172.21.15.64:0/4075236490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82e41009b0 0x7f82e4197140 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f82e0009230 tx=0x7f82e00042f0 comp rx=0 tx=0).stop 2024-09-12T03:39:49.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.179+0000 7f82eb70a640 1 -- 172.21.15.64:0/4075236490 shutdown_connections 2024-09-12T03:39:49.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.179+0000 7f82eb70a640 1 --2- 172.21.15.64:0/4075236490 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f82d00a2da0 0x7f82d00a5260 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:49.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.179+0000 7f82eb70a640 1 --2- 172.21.15.64:0/4075236490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82e41069c0 0x7f82e419ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:49.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.179+0000 7f82eb70a640 1 --2- 172.21.15.64:0/4075236490 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82e4101350 0x7f82e4197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.179+0000 7f82eb70a640 1 --2- 172.21.15.64:0/4075236490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82e41009b0 0x7f82e4197140 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:39:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.179+0000 7f82eb70a640 1 -- 172.21.15.64:0/4075236490 >> 172.21.15.64:0/4075236490 conn(0x7f82e4077990 msgr2=0x7f82e40ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:39:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.180+0000 7f82eb70a640 1 -- 172.21.15.64:0/4075236490 shutdown_connections 2024-09-12T03:39:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:39:49.180+0000 7f82eb70a640 1 -- 172.21.15.64:0/4075236490 wait complete. 2024-09-12T03:39:49.758 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:49 smithi195 ceph-mon[34786]: pgmap v327: 129 pgs: 129 active+clean; 3.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 55 KiB/s wr, 18 op/s 2024-09-12T03:39:49.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:49 smithi064 ceph-mon[30952]: pgmap v327: 129 pgs: 129 active+clean; 3.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 55 KiB/s wr, 18 op/s 2024-09-12T03:39:49.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:49 smithi115 ceph-mon[34801]: pgmap v327: 129 pgs: 129 active+clean; 3.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 55 KiB/s wr, 18 op/s 2024-09-12T03:39:51.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:51 smithi064 ceph-mon[30952]: pgmap v328: 129 pgs: 129 active+clean; 3.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-12T03:39:51.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:51 smithi115 ceph-mon[34801]: pgmap v328: 129 pgs: 129 active+clean; 3.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-12T03:39:51.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:51 smithi195 ceph-mon[34786]: pgmap v328: 129 pgs: 129 active+clean; 3.8 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-09-12T03:39:53.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:53 smithi064 ceph-mon[30952]: pgmap v329: 129 pgs: 129 active+clean; 4.0 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:39:53.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:53 smithi115 ceph-mon[34801]: pgmap v329: 129 pgs: 129 active+clean; 4.0 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:39:53.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:53 smithi195 ceph-mon[34786]: pgmap v329: 129 pgs: 129 active+clean; 4.0 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:39:55.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:55 smithi064 ceph-mon[30952]: pgmap v330: 129 pgs: 129 active+clean; 4.0 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:39:55.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:55 smithi115 ceph-mon[34801]: pgmap v330: 129 pgs: 129 active+clean; 4.0 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:39:55.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:55 smithi195 ceph-mon[34786]: pgmap v330: 129 pgs: 129 active+clean; 4.0 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:39:57.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:57 smithi064 ceph-mon[30952]: pgmap v331: 129 pgs: 129 active+clean; 4.0 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:39:57.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:57 smithi115 ceph-mon[34801]: pgmap v331: 129 pgs: 129 active+clean; 4.0 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:39:57.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:57 smithi195 ceph-mon[34786]: pgmap v331: 129 pgs: 129 active+clean; 4.0 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-09-12T03:39:59.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:39:59 smithi064 ceph-mon[30952]: pgmap v332: 129 pgs: 129 active+clean; 4.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:39:59.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:39:59 smithi115 ceph-mon[34801]: pgmap v332: 129 pgs: 129 active+clean; 4.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:39:59.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:39:59 smithi195 ceph-mon[34786]: pgmap v332: 129 pgs: 129 active+clean; 4.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 18 op/s 2024-09-12T03:40:00.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:00 smithi064 ceph-mon[30952]: overall HEALTH_OK 2024-09-12T03:40:00.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:00 smithi115 ceph-mon[34801]: overall HEALTH_OK 2024-09-12T03:40:00.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:00 smithi195 ceph-mon[34786]: overall HEALTH_OK 2024-09-12T03:40:01.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:01 smithi064 ceph-mon[30952]: pgmap v333: 129 pgs: 129 active+clean; 4.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-12T03:40:01.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:01 smithi115 ceph-mon[34801]: pgmap v333: 129 pgs: 129 active+clean; 4.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-12T03:40:01.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:01 smithi195 ceph-mon[34786]: pgmap v333: 129 pgs: 129 active+clean; 4.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-12T03:40:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.280+0000 7fc9fbbe4640 1 -- 172.21.15.64:0/607584046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9f4103680 msgr2=0x7fc9f410bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.280+0000 7fc9fbbe4640 1 --2- 172.21.15.64:0/607584046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9f4103680 0x7fc9f410bb60 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4009900 tx=0x7fc9e402a480 comp rx=0 tx=0).stop 2024-09-12T03:40:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.280+0000 7fc9fbbe4640 1 -- 172.21.15.64:0/607584046 shutdown_connections 2024-09-12T03:40:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.280+0000 7fc9fbbe4640 1 --2- 172.21.15.64:0/607584046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9f410c130 0x7fc9f410e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.280+0000 7fc9fbbe4640 1 --2- 172.21.15.64:0/607584046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9f4103680 0x7fc9f410bb60 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:02.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.280+0000 7fc9fbbe4640 1 --2- 172.21.15.64:0/607584046 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9f4102d60 0x7fc9f4103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:02.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.280+0000 7fc9fbbe4640 1 -- 172.21.15.64:0/607584046 >> 172.21.15.64:0/607584046 conn(0x7fc9f40fc6b0 msgr2=0x7fc9f40fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:40:02.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.280+0000 7fc9fbbe4640 1 -- 172.21.15.64:0/607584046 shutdown_connections 2024-09-12T03:40:02.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.280+0000 7fc9fbbe4640 1 -- 172.21.15.64:0/607584046 wait complete. 2024-09-12T03:40:02.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.280+0000 7fc9fbbe4640 1 Processor -- start 2024-09-12T03:40:02.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.280+0000 7fc9fbbe4640 1 -- start start 2024-09-12T03:40:02.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.281+0000 7fc9fbbe4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9f4102d60 0x7fc9f419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:02.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.281+0000 7fc9fbbe4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9f4103680 0x7fc9f419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:02.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.281+0000 7fc9fbbe4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9f410c130 0x7fc9f419ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:02.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.281+0000 7fc9fbbe4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f4111db0 con 0x7fc9f4102d60 2024-09-12T03:40:02.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.281+0000 7fc9fbbe4640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f4111c30 con 0x7fc9f4103680 2024-09-12T03:40:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.281+0000 7fc9fbbe4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f4111f30 con 0x7fc9f410c130 2024-09-12T03:40:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.281+0000 7fc9f9980640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9f4102d60 0x7fc9f419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.281+0000 7fc9f9980640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9f4102d60 0x7fc9f419a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57562/0 (socket says 172.21.15.64:57562) 2024-09-12T03:40:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.281+0000 7fc9f9980640 1 -- 172.21.15.64:0/81484111 learned_addr learned my addr 172.21.15.64:0/81484111 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:40:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.281+0000 7fc9f917f640 1 --2- 172.21.15.64:0/81484111 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9f4103680 0x7fc9f419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.281+0000 7fc9fa181640 1 --2- 172.21.15.64:0/81484111 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9f410c130 0x7fc9f419ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.281+0000 7fc9f9980640 1 -- 172.21.15.64:0/81484111 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9f410c130 msgr2=0x7fc9f419ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.281+0000 7fc9f9980640 1 --2- 172.21.15.64:0/81484111 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9f410c130 0x7fc9f419ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.281+0000 7fc9f9980640 1 -- 172.21.15.64:0/81484111 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9f4103680 msgr2=0x7fc9f419aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.281+0000 7fc9f9980640 1 --2- 172.21.15.64:0/81484111 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9f4103680 0x7fc9f419aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.281+0000 7fc9f9980640 1 -- 172.21.15.64:0/81484111 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9f419f4c0 con 0x7fc9f4102d60 2024-09-12T03:40:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.281+0000 7fc9fa181640 1 --2- 172.21.15.64:0/81484111 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9f410c130 0x7fc9f419ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:40:02.285 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.281+0000 7fc9f9980640 1 --2- 172.21.15.64:0/81484111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9f4102d60 0x7fc9f419a4d0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fc9f000e9f0 tx=0x7fc9f000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:40:02.285 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.282+0000 7fc9eaffd640 1 -- 172.21.15.64:0/81484111 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f000cd80 con 0x7fc9f4102d60 2024-09-12T03:40:02.285 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.282+0000 7fc9eaffd640 1 -- 172.21.15.64:0/81484111 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9f00054d0 con 0x7fc9f4102d60 2024-09-12T03:40:02.285 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.282+0000 7fc9fbbe4640 1 -- 172.21.15.64:0/81484111 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9f419f7b0 con 0x7fc9f4102d60 2024-09-12T03:40:02.285 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.282+0000 7fc9eaffd640 1 -- 172.21.15.64:0/81484111 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0010490 con 0x7fc9f4102d60 2024-09-12T03:40:02.285 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.282+0000 7fc9fbbe4640 1 -- 172.21.15.64:0/81484111 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9f41a7040 con 0x7fc9f4102d60 2024-09-12T03:40:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.284+0000 7fc9eaffd640 1 -- 172.21.15.64:0/81484111 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc9f0005020 con 0x7fc9f4102d60 2024-09-12T03:40:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.284+0000 7fc9fea5b740 1 Processor -- start 2024-09-12T03:40:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.285+0000 7fc9fea5b740 1 -- start start 2024-09-12T03:40:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.285+0000 7fc9fea5b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a03fed7af0 0x55a03fed7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.285+0000 7fc9fea5b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a03fed8410 0x55a03fee2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.285+0000 7fc9fea5b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a03fee2cb0 0x55a03fee50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.285+0000 7fc9fea5b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a03fe3c2f0 con 0x55a03fee2cb0 2024-09-12T03:40:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.285+0000 7fc9fea5b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a03fde4270 con 0x55a03fed8410 2024-09-12T03:40:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.285+0000 7fc9fea5b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a03fe382b0 con 0x55a03fed7af0 2024-09-12T03:40:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.285+0000 7fc9f917f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a03fee2cb0 0x55a03fee50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.285+0000 7fc9f917f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a03fee2cb0 0x55a03fee50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57578/0 (socket says 172.21.15.64:57578) 2024-09-12T03:40:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.285+0000 7fc9f917f640 1 -- 172.21.15.64:0/190096067 learned_addr learned my addr 172.21.15.64:0/190096067 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:40:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.285+0000 7fc9fa181640 1 --2- 172.21.15.64:0/190096067 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a03fed7af0 0x55a03fed7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.285+0000 7fc9f9980640 1 --2- 172.21.15.64:0/190096067 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a03fed8410 0x55a03fee2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.285+0000 7fc9eaffd640 1 --2- 172.21.15.64:0/81484111 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fc9e00a2e20 0x7fc9e00a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.285+0000 7fc9f917f640 1 -- 172.21.15.64:0/190096067 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a03fed7af0 msgr2=0x55a03fed7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:40:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.285+0000 7fc9f917f640 1 --2- 172.21.15.64:0/190096067 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a03fed7af0 0x55a03fed7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.285+0000 7fc9f917f640 1 -- 172.21.15.64:0/190096067 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a03fed8410 msgr2=0x55a03fee2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:40:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.285+0000 7fc9f917f640 1 --2- 172.21.15.64:0/190096067 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a03fed8410 0x55a03fee2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.285+0000 7fc9f917f640 1 -- 172.21.15.64:0/190096067 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a03fde3340 con 0x55a03fee2cb0 2024-09-12T03:40:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.285+0000 7fc9fa181640 1 --2- 172.21.15.64:0/190096067 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a03fed7af0 0x55a03fed7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:40:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.286+0000 7fc9fa181640 1 --2- 172.21.15.64:0/81484111 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fc9e00a2e20 0x7fc9e00a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.286+0000 7fc9eaffd640 1 -- 172.21.15.64:0/81484111 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9583+0+0 (secure 0 0 0) 0x7fc9f00cfc00 con 0x7fc9f4102d60 2024-09-12T03:40:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.286+0000 7fc9f917f640 1 --2- 172.21.15.64:0/190096067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a03fee2cb0 0x55a03fee50a0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4009900 tx=0x7fc9e4032ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1863386b528dac3e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:40:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.286+0000 7fc9dffff640 1 -- 172.21.15.64:0/190096067 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e4040070 con 0x55a03fee2cb0 2024-09-12T03:40:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.286+0000 7fc9fa181640 1 --2- 172.21.15.64:0/81484111 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fc9e00a2e20 0x7fc9e00a52e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec005fd0 tx=0x7fc9ec004a40 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:40:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.286+0000 7fc9dffff640 1 -- 172.21.15.64:0/190096067 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9e4032e30 con 0x55a03fee2cb0 2024-09-12T03:40:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.286+0000 7fc9dffff640 1 -- 172.21.15.64:0/190096067 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e4035040 con 0x55a03fee2cb0 2024-09-12T03:40:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.287+0000 7fc9fea5b740 1 -- 172.21.15.64:0/190096067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a03fee2cb0 msgr2=0x55a03fee50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.287+0000 7fc9fea5b740 1 --2- 172.21.15.64:0/190096067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a03fee2cb0 0x55a03fee50a0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4009900 tx=0x7fc9e4032ac0 comp rx=0 tx=0).stop 2024-09-12T03:40:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.287+0000 7fc9fea5b740 1 -- 172.21.15.64:0/190096067 shutdown_connections 2024-09-12T03:40:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.287+0000 7fc9fea5b740 1 --2- 172.21.15.64:0/190096067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a03fee2cb0 0x55a03fee50a0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.287+0000 7fc9fea5b740 1 --2- 172.21.15.64:0/190096067 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a03fed8410 0x55a03fee2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.287+0000 7fc9fea5b740 1 --2- 172.21.15.64:0/190096067 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a03fed7af0 0x55a03fed7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.287+0000 7fc9fea5b740 1 -- 172.21.15.64:0/190096067 >> 172.21.15.64:0/190096067 conn(0x55a03fed4060 msgr2=0x55a03fed4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:40:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.287+0000 7fc9fea5b740 1 -- 172.21.15.64:0/190096067 shutdown_connections 2024-09-12T03:40:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.287+0000 7fc9fea5b740 1 -- 172.21.15.64:0/190096067 wait complete. 2024-09-12T03:40:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.287+0000 7fc9fea5b740 1 -- start start 2024-09-12T03:40:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.287+0000 7fc9fea5b740 1 Processor -- start 2024-09-12T03:40:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.288+0000 7fc9fea5b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a03fed7af0 0x55a03fedfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.288+0000 7fc9fea5b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a03fed8410 0x55a03fee00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.288+0000 7fc9fea5b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a03fee2cb0 0x55a03fee0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.288+0000 7fc9fea5b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a03fee5880 con 0x55a03fed7af0 2024-09-12T03:40:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.288+0000 7fc9fea5b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a03fdc5290 con 0x55a03fed8410 2024-09-12T03:40:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.288+0000 7fc9fea5b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a03fee5a00 con 0x55a03fee2cb0 2024-09-12T03:40:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.288+0000 7fc9fea5b740 10 client.0 _ll_register_callbacks cb 0x55a03fdf6140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:40:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.288+0000 7fc9f9980640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a03fed7af0 0x55a03fedfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.288+0000 7fc9f9980640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a03fed7af0 0x55a03fedfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57592/0 (socket says 172.21.15.64:57592) 2024-09-12T03:40:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.288+0000 7fc9f9980640 1 -- 172.21.15.64:0/425069271 learned_addr learned my addr 172.21.15.64:0/425069271 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:40:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.288+0000 7fc9fa181640 1 --2- 172.21.15.64:0/425069271 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a03fee2cb0 0x55a03fee0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.288+0000 7fc9f9980640 1 -- 172.21.15.64:0/425069271 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a03fee2cb0 msgr2=0x55a03fee0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:40:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.288+0000 7fc9f917f640 1 --2- 172.21.15.64:0/425069271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a03fed8410 0x55a03fee00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.288+0000 7fc9f9980640 1 --2- 172.21.15.64:0/425069271 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a03fee2cb0 0x55a03fee0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.288+0000 7fc9f9980640 1 -- 172.21.15.64:0/425069271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a03fed8410 msgr2=0x55a03fee00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:40:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.288+0000 7fc9f9980640 1 --2- 172.21.15.64:0/425069271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a03fed8410 0x55a03fee00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.288+0000 7fc9f9980640 1 -- 172.21.15.64:0/425069271 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a03fb4f210 con 0x55a03fed7af0 2024-09-12T03:40:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.289+0000 7fc9f9980640 1 --2- 172.21.15.64:0/425069271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a03fed7af0 0x55a03fedfb80 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fc9f00ccf90 tx=0x7fc9f009b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d2b69e265bc4504 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:40:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.289+0000 7fc9dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:40:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.289+0000 7fc9dd7fa640 1 -- 172.21.15.64:0/425069271 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0005310 con 0x55a03fed7af0 2024-09-12T03:40:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.289+0000 7fc9dd7fa640 1 -- 172.21.15.64:0/425069271 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9583+0+0 (secure 0 0 0) 0x7fc9f0098370 con 0x55a03fed7af0 2024-09-12T03:40:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.289+0000 7fc9fea5b740 1 -- 172.21.15.64:0/425069271 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9e4032e30 con 0x55a03fed7af0 2024-09-12T03:40:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.289+0000 7fc9dd7fa640 1 -- 172.21.15.64:0/425069271 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc9f0005680 con 0x55a03fed7af0 2024-09-12T03:40:02.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.289+0000 7fc9fea5b740 10 client.16224 fetch_fsmap learned FSMap version 25 2024-09-12T03:40:02.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.289+0000 7fc9fea5b740 1 -- 172.21.15.64:0/425069271 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a03fc1b620 con 0x55a03fed7af0 2024-09-12T03:40:02.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.289+0000 7fc9dd7fa640 1 -- 172.21.15.64:0/425069271 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9e4032e30 con 0x55a03fed7af0 2024-09-12T03:40:02.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.289+0000 7fc9dd7fa640 1 -- 172.21.15.64:0/425069271 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc9f0097a50 con 0x55a03fed7af0 2024-09-12T03:40:02.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.289+0000 7fc9fea5b740 10 client.16224 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:40:02.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.289+0000 7fc9fea5b740 10 client.16224 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:40:02.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.289+0000 7fc9fea5b740 20 client.16224 populate_metadata read hostname 'smithi064' 2024-09-12T03:40:02.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.289+0000 7fc9fea5b740 1 --2- 172.21.15.64:0/425069271 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55a03fef2c50 0x55a03fef50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:02.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.289+0000 7fc9fea5b740 4 client.16224 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:40:02.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.289+0000 7fc9fea5b740 1 -- 172.21.15.64:0/425069271 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a03fe49a20 con 0x55a03fef2c50 2024-09-12T03:40:02.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.290+0000 7fc9f917f640 1 --2- 172.21.15.64:0/425069271 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55a03fef2c50 0x55a03fef50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:02.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.290+0000 7fc9f917f640 1 --2- 172.21.15.64:0/425069271 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55a03fef2c50 0x55a03fef50e0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bda4cca0a65c5942 server_cookie=2228e33ecc1b51e7 in_seq=0 out_seq=0 2024-09-12T03:40:02.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.290+0000 7fc9dd7fa640 10 client.16224 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:40:02.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.291+0000 7fc9dd7fa640 1 -- 172.21.15.64:0/425069271 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a03fe49a20 con 0x55a03fef2c50 2024-09-12T03:40:02.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.291+0000 7fc9dd7fa640 10 client.16224 handle_command_reply: tid=0 2024-09-12T03:40:02.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.291+0000 7fc9fea5b740 1 client.16224 shutdown 2024-09-12T03:40:02.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.291+0000 7fc9fea5b740 1 -- 172.21.15.64:0/425069271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a03fed7af0 msgr2=0x55a03fedfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:02.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.291+0000 7fc9fea5b740 1 --2- 172.21.15.64:0/425069271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a03fed7af0 0x55a03fedfb80 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fc9f00ccf90 tx=0x7fc9f009b900 comp rx=0 tx=0).stop 2024-09-12T03:40:02.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.291+0000 7fc9fea5b740 1 -- 172.21.15.64:0/425069271 shutdown_connections 2024-09-12T03:40:02.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.291+0000 7fc9fea5b740 1 --2- 172.21.15.64:0/425069271 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55a03fef2c50 0x55a03fef50e0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:02.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.291+0000 7fc9fea5b740 1 --2- 172.21.15.64:0/425069271 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a03fee2cb0 0x55a03fee0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:02.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.291+0000 7fc9fea5b740 1 --2- 172.21.15.64:0/425069271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a03fed8410 0x55a03fee00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:02.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.291+0000 7fc9dd7fa640 0 client.16224 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:40:02.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.291+0000 7fc9fea5b740 1 --2- 172.21.15.64:0/425069271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a03fed7af0 0x55a03fedfb80 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:02.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.291+0000 7fc9fea5b740 1 -- 172.21.15.64:0/425069271 >> 172.21.15.64:0/425069271 conn(0x55a03fed4060 msgr2=0x55a03fed9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:40:02.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.291+0000 7fc9fea5b740 1 -- 172.21.15.64:0/425069271 shutdown_connections 2024-09-12T03:40:02.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.291+0000 7fc9fea5b740 1 -- 172.21.15.64:0/425069271 wait complete. 2024-09-12T03:40:02.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.291+0000 7fc9fea5b740 20 client.16224 trim_cache size 0 max 16384 2024-09-12T03:40:02.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.306+0000 7fc9fea5b740 1 Processor -- start 2024-09-12T03:40:02.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.306+0000 7fc9fea5b740 1 -- start start 2024-09-12T03:40:02.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.306+0000 7fc9fea5b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a03fed7af0 0x55a03fee0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:02.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.306+0000 7fc9fea5b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a03fed8410 0x55a03fed9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:02.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.306+0000 7fc9fea5b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a03fee2cb0 0x55a03fefa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:02.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.306+0000 7fc9fea5b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f41b3ad0 con 0x55a03fed7af0 2024-09-12T03:40:02.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.306+0000 7fc9fea5b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f41b13a0 con 0x55a03fee2cb0 2024-09-12T03:40:02.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.306+0000 7fc9fea5b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f41af210 con 0x55a03fed8410 2024-09-12T03:40:02.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.306+0000 7fc9f9980640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a03fed7af0 0x55a03fee0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:02.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.306+0000 7fc9f9980640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a03fed7af0 0x55a03fee0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57598/0 (socket says 172.21.15.64:57598) 2024-09-12T03:40:02.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.306+0000 7fc9f9980640 1 -- 172.21.15.64:0/49115686 learned_addr learned my addr 172.21.15.64:0/49115686 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:40:02.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.307+0000 7fc9f917f640 1 --2- 172.21.15.64:0/49115686 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a03fed8410 0x55a03fed9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:02.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.307+0000 7fc9f9980640 1 -- 172.21.15.64:0/49115686 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a03fed8410 msgr2=0x55a03fed9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:40:02.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.307+0000 7fc9f9980640 1 --2- 172.21.15.64:0/49115686 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a03fed8410 0x55a03fed9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:02.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.307+0000 7fc9fa181640 1 --2- 172.21.15.64:0/49115686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a03fee2cb0 0x55a03fefa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:02.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.307+0000 7fc9f9980640 1 -- 172.21.15.64:0/49115686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a03fee2cb0 msgr2=0x55a03fefa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:40:02.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.307+0000 7fc9f9980640 1 --2- 172.21.15.64:0/49115686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a03fee2cb0 0x55a03fefa9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:02.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.307+0000 7fc9f9980640 1 -- 172.21.15.64:0/49115686 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a03fee26d0 con 0x55a03fed7af0 2024-09-12T03:40:02.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.307+0000 7fc9f917f640 1 --2- 172.21.15.64:0/49115686 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a03fed8410 0x55a03fed9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:40:02.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.307+0000 7fc9fa181640 1 --2- 172.21.15.64:0/49115686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a03fee2cb0 0x55a03fefa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:40:02.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.307+0000 7fc9f9980640 1 --2- 172.21.15.64:0/49115686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a03fed7af0 0x55a03fee0430 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0005820 tx=0x7fc9f009be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd03a75021e89de server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:40:02.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.307+0000 7fc9df7fe640 1 -- 172.21.15.64:0/49115686 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f00d3030 con 0x55a03fed7af0 2024-09-12T03:40:02.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.307+0000 7fc9df7fe640 1 -- 172.21.15.64:0/49115686 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9f0005310 con 0x55a03fed7af0 2024-09-12T03:40:02.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.307+0000 7fc9df7fe640 1 -- 172.21.15.64:0/49115686 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f009c400 con 0x55a03fed7af0 2024-09-12T03:40:02.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.308+0000 7fc9fea5b740 1 -- 172.21.15.64:0/49115686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a03fed7af0 msgr2=0x55a03fee0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:02.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.308+0000 7fc9fea5b740 1 --2- 172.21.15.64:0/49115686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a03fed7af0 0x55a03fee0430 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0005820 tx=0x7fc9f009be80 comp rx=0 tx=0).stop 2024-09-12T03:40:02.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.308+0000 7fc9fea5b740 1 -- 172.21.15.64:0/49115686 shutdown_connections 2024-09-12T03:40:02.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.308+0000 7fc9fea5b740 1 --2- 172.21.15.64:0/49115686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a03fee2cb0 0x55a03fefa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:02.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.308+0000 7fc9fea5b740 1 --2- 172.21.15.64:0/49115686 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a03fed8410 0x55a03fed9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:02.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.308+0000 7fc9fea5b740 1 --2- 172.21.15.64:0/49115686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a03fed7af0 0x55a03fee0430 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:02.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.308+0000 7fc9fea5b740 1 -- 172.21.15.64:0/49115686 >> 172.21.15.64:0/49115686 conn(0x55a03fed4060 msgr2=0x55a03fed9550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:40:02.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.308+0000 7fc9fea5b740 1 -- 172.21.15.64:0/49115686 shutdown_connections 2024-09-12T03:40:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.308+0000 7fc9fea5b740 1 -- 172.21.15.64:0/49115686 wait complete. 2024-09-12T03:40:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.308+0000 7fc9fea5b740 1 -- start start 2024-09-12T03:40:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.308+0000 7fc9fea5b740 1 Processor -- start 2024-09-12T03:40:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.309+0000 7fc9fea5b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a03fed7af0 0x55a03feeae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.309+0000 7fc9fea5b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a03fed8410 0x55a03feeb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.309+0000 7fc9fea5b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a03fee2cb0 0x55a03fee7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.309+0000 7fc9fea5b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f41af860 con 0x55a03fed8410 2024-09-12T03:40:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.309+0000 7fc9fea5b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f41b0cd0 con 0x55a03fed7af0 2024-09-12T03:40:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.309+0000 7fc9fea5b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f41aff30 con 0x55a03fee2cb0 2024-09-12T03:40:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.309+0000 7fc9fea5b740 10 client.0 _ll_register_callbacks cb 0x7fc9f000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:40:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.309+0000 7fc9f917f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a03fed8410 0x55a03feeb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.309+0000 7fc9f917f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a03fed8410 0x55a03feeb3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57600/0 (socket says 172.21.15.64:57600) 2024-09-12T03:40:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.309+0000 7fc9f917f640 1 -- 172.21.15.64:0/501758357 learned_addr learned my addr 172.21.15.64:0/501758357 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:40:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.309+0000 7fc9f9980640 1 --2- 172.21.15.64:0/501758357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a03fed7af0 0x55a03feeae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.309+0000 7fc9fa181640 1 --2- 172.21.15.64:0/501758357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a03fee2cb0 0x55a03fee7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.309+0000 7fc9f917f640 1 -- 172.21.15.64:0/501758357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a03fee2cb0 msgr2=0x55a03fee7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:40:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.309+0000 7fc9f917f640 1 --2- 172.21.15.64:0/501758357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a03fee2cb0 0x55a03fee7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.309+0000 7fc9f917f640 1 -- 172.21.15.64:0/501758357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a03fed7af0 msgr2=0x55a03feeae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:40:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.309+0000 7fc9f917f640 1 --2- 172.21.15.64:0/501758357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a03fed7af0 0x55a03feeae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.309+0000 7fc9f917f640 1 -- 172.21.15.64:0/501758357 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a03fee8320 con 0x55a03fed8410 2024-09-12T03:40:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.309+0000 7fc9f9980640 1 --2- 172.21.15.64:0/501758357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a03fed7af0 0x55a03feeae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:40:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.309+0000 7fc9f917f640 1 --2- 172.21.15.64:0/501758357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a03fed8410 0x55a03feeb3c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4009900 tx=0x7fc9e4031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68978aa3c7ef9b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:40:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.309+0000 7fc9dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:40:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.310+0000 7fc9dd7fa640 1 -- 172.21.15.64:0/501758357 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e4040070 con 0x55a03fed8410 2024-09-12T03:40:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.310+0000 7fc9dd7fa640 1 -- 172.21.15.64:0/501758357 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9583+0+0 (secure 0 0 0) 0x7fc9e40316b0 con 0x55a03fed8410 2024-09-12T03:40:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.310+0000 7fc9fea5b740 1 -- 172.21.15.64:0/501758357 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a03fe43d70 con 0x55a03fed8410 2024-09-12T03:40:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.310+0000 7fc9dd7fa640 1 -- 172.21.15.64:0/501758357 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc9e4057050 con 0x55a03fed8410 2024-09-12T03:40:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.310+0000 7fc9fea5b740 10 client.16230 fetch_fsmap learned FSMap version 25 2024-09-12T03:40:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.310+0000 7fc9fea5b740 1 -- 172.21.15.64:0/501758357 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a03ff5e3a0 con 0x55a03fed8410 2024-09-12T03:40:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.310+0000 7fc9dd7fa640 1 -- 172.21.15.64:0/501758357 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55a03fe43d70 con 0x55a03fed8410 2024-09-12T03:40:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.310+0000 7fc9dd7fa640 1 -- 172.21.15.64:0/501758357 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc9e40053a0 con 0x55a03fed8410 2024-09-12T03:40:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.310+0000 7fc9fea5b740 10 client.16230 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:40:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.310+0000 7fc9fea5b740 10 client.16230 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:40:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.310+0000 7fc9fea5b740 20 client.16230 populate_metadata read hostname 'smithi064' 2024-09-12T03:40:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.310+0000 7fc9fea5b740 1 --2- 172.21.15.64:0/501758357 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55a03ff5e7a0 0x55a03ff60b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.310+0000 7fc9fea5b740 4 client.16230 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:40:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.310+0000 7fc9fea5b740 1 -- 172.21.15.64:0/501758357 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a03fdc0900 con 0x55a03ff5e7a0 2024-09-12T03:40:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.311+0000 7fc9f9980640 1 --2- 172.21.15.64:0/501758357 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55a03ff5e7a0 0x55a03ff60b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.311+0000 7fc9dd7fa640 10 client.16230 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:40:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.311+0000 7fc9f9980640 1 --2- 172.21.15.64:0/501758357 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55a03ff5e7a0 0x55a03ff60b90 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=144d5d2ff50cd65a server_cookie=898b79d4a1ab4d8e in_seq=0 out_seq=0 2024-09-12T03:40:02.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.325+0000 7fc9dd7fa640 1 -- 172.21.15.64:0/501758357 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a03fdc0900 con 0x55a03ff5e7a0 2024-09-12T03:40:02.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.325+0000 7fc9dd7fa640 10 client.16230 handle_command_reply: tid=0 2024-09-12T03:40:02.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.325+0000 7fc9fea5b740 1 client.16230 shutdown 2024-09-12T03:40:02.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.325+0000 7fc9fea5b740 1 -- 172.21.15.64:0/501758357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a03fed8410 msgr2=0x55a03feeb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:02.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.325+0000 7fc9fea5b740 1 --2- 172.21.15.64:0/501758357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a03fed8410 0x55a03feeb3c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4009900 tx=0x7fc9e4031400 comp rx=0 tx=0).stop 2024-09-12T03:40:02.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.325+0000 7fc9fea5b740 1 -- 172.21.15.64:0/501758357 shutdown_connections 2024-09-12T03:40:02.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.325+0000 7fc9fea5b740 1 --2- 172.21.15.64:0/501758357 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x55a03ff5e7a0 0x55a03ff60b90 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:02.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.325+0000 7fc9fea5b740 1 --2- 172.21.15.64:0/501758357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a03fee2cb0 0x55a03fee7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:02.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.326+0000 7fc9fea5b740 1 --2- 172.21.15.64:0/501758357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a03fed8410 0x55a03feeb3c0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:02.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.326+0000 7fc9fea5b740 1 --2- 172.21.15.64:0/501758357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a03fed7af0 0x55a03feeae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:02.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.326+0000 7fc9dd7fa640 0 client.16230 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:40:02.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.326+0000 7fc9fea5b740 1 -- 172.21.15.64:0/501758357 >> 172.21.15.64:0/501758357 conn(0x55a03fed4060 msgr2=0x55a03fed31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:40:02.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.326+0000 7fc9fea5b740 1 -- 172.21.15.64:0/501758357 shutdown_connections 2024-09-12T03:40:02.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.326+0000 7fc9fea5b740 1 -- 172.21.15.64:0/501758357 wait complete. 2024-09-12T03:40:02.327 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.326+0000 7fc9fea5b740 20 client.16230 trim_cache size 0 max 16384 2024-09-12T03:40:02.327 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:40:02.327 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:40:02.327 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:40:02.327 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:40:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.326+0000 7fc9fbbe4640 1 -- 172.21.15.64:0/81484111 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fc9e00a2e20 msgr2=0x7fc9e00a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.326+0000 7fc9fbbe4640 1 --2- 172.21.15.64:0/81484111 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fc9e00a2e20 0x7fc9e00a52e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec005fd0 tx=0x7fc9ec004a40 comp rx=0 tx=0).stop 2024-09-12T03:40:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.326+0000 7fc9fbbe4640 1 -- 172.21.15.64:0/81484111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9f4102d60 msgr2=0x7fc9f419a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.326+0000 7fc9fbbe4640 1 --2- 172.21.15.64:0/81484111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9f4102d60 0x7fc9f419a4d0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fc9f000e9f0 tx=0x7fc9f000edf0 comp rx=0 tx=0).stop 2024-09-12T03:40:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.326+0000 7fc9fbbe4640 1 -- 172.21.15.64:0/81484111 shutdown_connections 2024-09-12T03:40:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.326+0000 7fc9fbbe4640 1 --2- 172.21.15.64:0/81484111 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9f410c130 0x7fc9f419ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.326+0000 7fc9fbbe4640 1 --2- 172.21.15.64:0/81484111 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fc9e00a2e20 0x7fc9e00a52e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:02.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.326+0000 7fc9fbbe4640 1 --2- 172.21.15.64:0/81484111 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9f4103680 0x7fc9f419aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:02.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.326+0000 7fc9fbbe4640 1 --2- 172.21.15.64:0/81484111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9f4102d60 0x7fc9f419a4d0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:02.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.326+0000 7fc9fbbe4640 1 -- 172.21.15.64:0/81484111 >> 172.21.15.64:0/81484111 conn(0x7fc9f40fc6b0 msgr2=0x7fc9f4100980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:40:02.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.326+0000 7fc9fbbe4640 1 -- 172.21.15.64:0/81484111 shutdown_connections 2024-09-12T03:40:02.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:02.326+0000 7fc9fbbe4640 1 -- 172.21.15.64:0/81484111 wait complete. 2024-09-12T03:40:03.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:03 smithi064 ceph-mon[30952]: pgmap v334: 129 pgs: 129 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 102 KiB/s wr, 35 op/s 2024-09-12T03:40:03.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:03 smithi115 ceph-mon[34801]: pgmap v334: 129 pgs: 129 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 102 KiB/s wr, 35 op/s 2024-09-12T03:40:03.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:03 smithi195 ceph-mon[34786]: pgmap v334: 129 pgs: 129 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 102 KiB/s wr, 35 op/s 2024-09-12T03:40:04.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:04 smithi064 ceph-mon[30952]: osdmap e102: 12 total, 12 up, 12 in 2024-09-12T03:40:04.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:04 smithi115 ceph-mon[34801]: osdmap e102: 12 total, 12 up, 12 in 2024-09-12T03:40:04.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:04 smithi195 ceph-mon[34786]: osdmap e102: 12 total, 12 up, 12 in 2024-09-12T03:40:05.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:05 smithi064 ceph-mon[30952]: pgmap v336: 129 pgs: 129 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 94 KiB/s wr, 32 op/s 2024-09-12T03:40:05.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:05 smithi115 ceph-mon[34801]: pgmap v336: 129 pgs: 129 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 94 KiB/s wr, 32 op/s 2024-09-12T03:40:05.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:05 smithi195 ceph-mon[34786]: pgmap v336: 129 pgs: 129 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 94 KiB/s wr, 32 op/s 2024-09-12T03:40:07.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:07 smithi064 ceph-mon[30952]: pgmap v337: 129 pgs: 129 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 94 KiB/s wr, 32 op/s 2024-09-12T03:40:07.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:07 smithi115 ceph-mon[34801]: pgmap v337: 129 pgs: 129 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 94 KiB/s wr, 32 op/s 2024-09-12T03:40:07.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:07 smithi195 ceph-mon[34786]: pgmap v337: 129 pgs: 129 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 94 KiB/s wr, 32 op/s 2024-09-12T03:40:09.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:09 smithi064 ceph-mon[30952]: pgmap v338: 129 pgs: 129 active+clean; 4.6 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 104 KiB/s wr, 36 op/s 2024-09-12T03:40:09.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:09 smithi115 ceph-mon[34801]: pgmap v338: 129 pgs: 129 active+clean; 4.6 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 104 KiB/s wr, 36 op/s 2024-09-12T03:40:09.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:09 smithi195 ceph-mon[34786]: pgmap v338: 129 pgs: 129 active+clean; 4.6 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 104 KiB/s wr, 36 op/s 2024-09-12T03:40:11.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:11 smithi064 ceph-mon[30952]: pgmap v339: 129 pgs: 129 active+clean; 4.6 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 104 KiB/s wr, 36 op/s 2024-09-12T03:40:11.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:11 smithi064 ceph-mon[30952]: osdmap e103: 12 total, 12 up, 12 in 2024-09-12T03:40:11.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:11 smithi115 ceph-mon[34801]: pgmap v339: 129 pgs: 129 active+clean; 4.6 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 104 KiB/s wr, 36 op/s 2024-09-12T03:40:11.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:11 smithi115 ceph-mon[34801]: osdmap e103: 12 total, 12 up, 12 in 2024-09-12T03:40:11.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:11 smithi195 ceph-mon[34786]: pgmap v339: 129 pgs: 129 active+clean; 4.6 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 104 KiB/s wr, 36 op/s 2024-09-12T03:40:11.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:11 smithi195 ceph-mon[34786]: osdmap e103: 12 total, 12 up, 12 in 2024-09-12T03:40:13.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:13 smithi064 ceph-mon[30952]: pgmap v341: 129 pgs: 129 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 59 KiB/s wr, 19 op/s 2024-09-12T03:40:13.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:13 smithi115 ceph-mon[34801]: pgmap v341: 129 pgs: 129 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 59 KiB/s wr, 19 op/s 2024-09-12T03:40:13.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:13 smithi195 ceph-mon[34786]: pgmap v341: 129 pgs: 129 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 59 KiB/s wr, 19 op/s 2024-09-12T03:40:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.426+0000 7f34793f6640 1 -- 172.21.15.64:0/1823025581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34741035e0 msgr2=0x7f347410bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.426+0000 7f34793f6640 1 --2- 172.21.15.64:0/1823025581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34741035e0 0x7f347410bac0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f345c009900 tx=0x7f345c02a480 comp rx=0 tx=0).stop 2024-09-12T03:40:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.426+0000 7f34793f6640 1 -- 172.21.15.64:0/1823025581 shutdown_connections 2024-09-12T03:40:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.426+0000 7f34793f6640 1 --2- 172.21.15.64:0/1823025581 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f347410c110 0x7f347410e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.426+0000 7f34793f6640 1 --2- 172.21.15.64:0/1823025581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34741035e0 0x7f347410bac0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.426+0000 7f34793f6640 1 --2- 172.21.15.64:0/1823025581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3474102cc0 0x7f34741030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.426+0000 7f34793f6640 1 -- 172.21.15.64:0/1823025581 >> 172.21.15.64:0/1823025581 conn(0x7f34740fc6b0 msgr2=0x7f34740fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:40:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.426+0000 7f34793f6640 1 -- 172.21.15.64:0/1823025581 shutdown_connections 2024-09-12T03:40:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.426+0000 7f34793f6640 1 -- 172.21.15.64:0/1823025581 wait complete. 2024-09-12T03:40:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.427+0000 7f34793f6640 1 Processor -- start 2024-09-12T03:40:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.427+0000 7f34793f6640 1 -- start start 2024-09-12T03:40:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.427+0000 7f34793f6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3474102cc0 0x7f347419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.427+0000 7f34793f6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34741035e0 0x7f347419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.427+0000 7f34793f6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f347410c110 0x7f347419ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.427+0000 7f34793f6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3474111de0 con 0x7f34741035e0 2024-09-12T03:40:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.427+0000 7f34793f6640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3474111c60 con 0x7f3474102cc0 2024-09-12T03:40:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.427+0000 7f34793f6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3474111f60 con 0x7f347410c110 2024-09-12T03:40:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.427+0000 7f346a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34741035e0 0x7f347419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.427+0000 7f346a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34741035e0 0x7f347419aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53238/0 (socket says 172.21.15.64:53238) 2024-09-12T03:40:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.427+0000 7f346a7fc640 1 -- 172.21.15.64:0/4225519207 learned_addr learned my addr 172.21.15.64:0/4225519207 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:40:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.428+0000 7f3472ffd640 1 --2- 172.21.15.64:0/4225519207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3474102cc0 0x7f347419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.428+0000 7f34737fe640 1 --2- 172.21.15.64:0/4225519207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f347410c110 0x7f347419ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.428+0000 7f346a7fc640 1 -- 172.21.15.64:0/4225519207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f347410c110 msgr2=0x7f347419ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.428+0000 7f346a7fc640 1 --2- 172.21.15.64:0/4225519207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f347410c110 0x7f347419ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.428+0000 7f346a7fc640 1 -- 172.21.15.64:0/4225519207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3474102cc0 msgr2=0x7f347419a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.428+0000 7f346a7fc640 1 --2- 172.21.15.64:0/4225519207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3474102cc0 0x7f347419a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.428+0000 7f346a7fc640 1 -- 172.21.15.64:0/4225519207 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f347419f530 con 0x7f34741035e0 2024-09-12T03:40:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.428+0000 7f34737fe640 1 --2- 172.21.15.64:0/4225519207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f347410c110 0x7f347419ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:40:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.428+0000 7f346a7fc640 1 --2- 172.21.15.64:0/4225519207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34741035e0 0x7f347419aa80 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f345c005940 tx=0x7f345c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:40:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.428+0000 7f3470ff9640 1 -- 172.21.15.64:0/4225519207 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c03f030 con 0x7f34741035e0 2024-09-12T03:40:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.428+0000 7f3470ff9640 1 -- 172.21.15.64:0/4225519207 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f345c003cb0 con 0x7f34741035e0 2024-09-12T03:40:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.428+0000 7f34793f6640 1 -- 172.21.15.64:0/4225519207 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f347419f7c0 con 0x7f34741035e0 2024-09-12T03:40:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.428+0000 7f3470ff9640 1 -- 172.21.15.64:0/4225519207 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c035070 con 0x7f34741035e0 2024-09-12T03:40:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.428+0000 7f34793f6640 1 -- 172.21.15.64:0/4225519207 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34741a70b0 con 0x7f34741035e0 2024-09-12T03:40:15.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.431+0000 7f3470ff9640 1 -- 172.21.15.64:0/4225519207 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f345c042030 con 0x7f34741035e0 2024-09-12T03:40:15.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.431+0000 7f347c269740 1 Processor -- start 2024-09-12T03:40:15.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.431+0000 7f347c269740 1 -- start start 2024-09-12T03:40:15.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.432+0000 7f347c269740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c093dfaf0 0x558c093dfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:15.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.432+0000 7f347c269740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c093e0410 0x558c093ea770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:15.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.432+0000 7f347c269740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c093eacb0 0x558c093ed0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:15.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.432+0000 7f347c269740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558c093442f0 con 0x558c093dfaf0 2024-09-12T03:40:15.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.432+0000 7f347c269740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558c092ec270 con 0x558c093e0410 2024-09-12T03:40:15.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.432+0000 7f347c269740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558c093402b0 con 0x558c093eacb0 2024-09-12T03:40:15.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.432+0000 7f34737fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c093dfaf0 0x558c093dfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.432+0000 7f34737fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c093dfaf0 0x558c093dfed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53248/0 (socket says 172.21.15.64:53248) 2024-09-12T03:40:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.432+0000 7f34737fe640 1 -- 172.21.15.64:0/3459084931 learned_addr learned my addr 172.21.15.64:0/3459084931 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:40:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.432+0000 7f3472ffd640 1 --2- 172.21.15.64:0/3459084931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c093e0410 0x558c093ea770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.432+0000 7f346a7fc640 1 --2- 172.21.15.64:0/3459084931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c093eacb0 0x558c093ed0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.432+0000 7f34737fe640 1 -- 172.21.15.64:0/3459084931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c093eacb0 msgr2=0x558c093ed0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:40:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.432+0000 7f34737fe640 1 --2- 172.21.15.64:0/3459084931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c093eacb0 0x558c093ed0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.432+0000 7f34737fe640 1 -- 172.21.15.64:0/3459084931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c093e0410 msgr2=0x558c093ea770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:40:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.432+0000 7f34737fe640 1 --2- 172.21.15.64:0/3459084931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c093e0410 0x558c093ea770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.432+0000 7f3470ff9640 1 --2- 172.21.15.64:0/4225519207 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f34600a2e20 0x7f34600a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.432+0000 7f34737fe640 1 -- 172.21.15.64:0/3459084931 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c092eb340 con 0x558c093dfaf0 2024-09-12T03:40:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.432+0000 7f3470ff9640 1 -- 172.21.15.64:0/4225519207 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9583+0+0 (secure 0 0 0) 0x7f345c0effc0 con 0x7f34741035e0 2024-09-12T03:40:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.432+0000 7f34737fe640 1 --2- 172.21.15.64:0/3459084931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c093dfaf0 0x558c093dfed0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f346c009f10 tx=0x7f346c00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d75bf0c64f593e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:40:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.432+0000 7f34737fe640 1 --2- 172.21.15.64:0/4225519207 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f34600a2e20 0x7f34600a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.433+0000 7f34697fa640 1 -- 172.21.15.64:0/3459084931 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c016070 con 0x558c093dfaf0 2024-09-12T03:40:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.433+0000 7f34697fa640 1 -- 172.21.15.64:0/3459084931 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f346c01b900 con 0x558c093dfaf0 2024-09-12T03:40:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.433+0000 7f34697fa640 1 -- 172.21.15.64:0/3459084931 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c010ba0 con 0x558c093dfaf0 2024-09-12T03:40:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.433+0000 7f347c269740 1 -- 172.21.15.64:0/3459084931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c093dfaf0 msgr2=0x558c093dfed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.433+0000 7f347c269740 1 --2- 172.21.15.64:0/3459084931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c093dfaf0 0x558c093dfed0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f346c009f10 tx=0x7f346c00e6c0 comp rx=0 tx=0).stop 2024-09-12T03:40:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.433+0000 7f34737fe640 1 --2- 172.21.15.64:0/4225519207 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f34600a2e20 0x7f34600a52e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f346c018000 tx=0x7f346c019730 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:40:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.434+0000 7f347c269740 1 -- 172.21.15.64:0/3459084931 shutdown_connections 2024-09-12T03:40:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.434+0000 7f347c269740 1 --2- 172.21.15.64:0/3459084931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c093eacb0 0x558c093ed0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.434+0000 7f347c269740 1 --2- 172.21.15.64:0/3459084931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c093e0410 0x558c093ea770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.434+0000 7f347c269740 1 --2- 172.21.15.64:0/3459084931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c093dfaf0 0x558c093dfed0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.434+0000 7f347c269740 1 -- 172.21.15.64:0/3459084931 >> 172.21.15.64:0/3459084931 conn(0x558c093dc060 msgr2=0x558c093dc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:40:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.434+0000 7f347c269740 1 -- 172.21.15.64:0/3459084931 shutdown_connections 2024-09-12T03:40:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.434+0000 7f347c269740 1 -- 172.21.15.64:0/3459084931 wait complete. 2024-09-12T03:40:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.434+0000 7f347c269740 1 -- start start 2024-09-12T03:40:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.434+0000 7f347c269740 1 Processor -- start 2024-09-12T03:40:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.434+0000 7f347c269740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c093dfaf0 0x558c093e7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.434+0000 7f347c269740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c093e0410 0x558c093e8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.434+0000 7f347c269740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c093eacb0 0x558c093e8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.434+0000 7f347c269740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558c093ed880 con 0x558c093dfaf0 2024-09-12T03:40:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.434+0000 7f347c269740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558c092cd290 con 0x558c093e0410 2024-09-12T03:40:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.434+0000 7f347c269740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558c093eda00 con 0x558c093eacb0 2024-09-12T03:40:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.434+0000 7f347c269740 10 client.0 _ll_register_callbacks cb 0x558c092fe140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:40:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.434+0000 7f3472ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c093dfaf0 0x558c093e7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.435+0000 7f3472ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c093dfaf0 0x558c093e7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53262/0 (socket says 172.21.15.64:53262) 2024-09-12T03:40:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.435+0000 7f3472ffd640 1 -- 172.21.15.64:0/1119748039 learned_addr learned my addr 172.21.15.64:0/1119748039 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:40:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.435+0000 7f346a7fc640 1 --2- 172.21.15.64:0/1119748039 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c093e0410 0x558c093e8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.435+0000 7f34737fe640 1 --2- 172.21.15.64:0/1119748039 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c093eacb0 0x558c093e8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.435+0000 7f3472ffd640 1 -- 172.21.15.64:0/1119748039 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c093eacb0 msgr2=0x558c093e8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:40:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.435+0000 7f3472ffd640 1 --2- 172.21.15.64:0/1119748039 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c093eacb0 0x558c093e8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.435+0000 7f3472ffd640 1 -- 172.21.15.64:0/1119748039 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c093e0410 msgr2=0x558c093e8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:40:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.435+0000 7f3472ffd640 1 --2- 172.21.15.64:0/1119748039 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c093e0410 0x558c093e8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.435+0000 7f3472ffd640 1 -- 172.21.15.64:0/1119748039 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c09057210 con 0x558c093dfaf0 2024-09-12T03:40:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.435+0000 7f34737fe640 1 --2- 172.21.15.64:0/1119748039 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c093eacb0 0x558c093e8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:40:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.435+0000 7f3472ffd640 1 --2- 172.21.15.64:0/1119748039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c093dfaf0 0x558c093e7b00 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f3464011570 tx=0x7f3464011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1407ba7d3eaa76fa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:40:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.435+0000 7f345a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:40:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.435+0000 7f345a7fc640 1 -- 172.21.15.64:0/1119748039 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3464013670 con 0x558c093dfaf0 2024-09-12T03:40:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.435+0000 7f345a7fc640 1 -- 172.21.15.64:0/1119748039 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9583+0+0 (secure 0 0 0) 0x7f3464013810 con 0x558c093dfaf0 2024-09-12T03:40:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.435+0000 7f347c269740 1 -- 172.21.15.64:0/1119748039 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f346c01b900 con 0x558c093dfaf0 2024-09-12T03:40:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.435+0000 7f345a7fc640 1 -- 172.21.15.64:0/1119748039 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3464026050 con 0x558c093dfaf0 2024-09-12T03:40:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.435+0000 7f347c269740 10 client.16242 fetch_fsmap learned FSMap version 25 2024-09-12T03:40:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.435+0000 7f347c269740 1 -- 172.21.15.64:0/1119748039 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558c09123620 con 0x558c093dfaf0 2024-09-12T03:40:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.436+0000 7f345a7fc640 1 -- 172.21.15.64:0/1119748039 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f346c01b900 con 0x558c093dfaf0 2024-09-12T03:40:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.436+0000 7f345a7fc640 1 -- 172.21.15.64:0/1119748039 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f3464014bd0 con 0x558c093dfaf0 2024-09-12T03:40:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.436+0000 7f347c269740 10 client.16242 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:40:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.436+0000 7f347c269740 10 client.16242 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:40:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.436+0000 7f347c269740 20 client.16242 populate_metadata read hostname 'smithi064' 2024-09-12T03:40:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.436+0000 7f347c269740 1 --2- 172.21.15.64:0/1119748039 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x558c093fac20 0x558c093fd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.436+0000 7f347c269740 4 client.16242 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:40:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.436+0000 7f347c269740 1 -- 172.21.15.64:0/1119748039 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c09351a20 con 0x558c093fac20 2024-09-12T03:40:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.436+0000 7f346a7fc640 1 --2- 172.21.15.64:0/1119748039 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x558c093fac20 0x558c093fd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.437+0000 7f345a7fc640 10 client.16242 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:40:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.437+0000 7f346a7fc640 1 --2- 172.21.15.64:0/1119748039 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x558c093fac20 0x558c093fd030 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e61cf1c75432d35 server_cookie=e6d629f25162f69 in_seq=0 out_seq=0 2024-09-12T03:40:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.437+0000 7f345a7fc640 1 -- 172.21.15.64:0/1119748039 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558c09351a20 con 0x558c093fac20 2024-09-12T03:40:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.437+0000 7f345a7fc640 10 client.16242 handle_command_reply: tid=0 2024-09-12T03:40:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.437+0000 7f347c269740 1 client.16242 shutdown 2024-09-12T03:40:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.438+0000 7f347c269740 1 -- 172.21.15.64:0/1119748039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c093dfaf0 msgr2=0x558c093e7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.438+0000 7f347c269740 1 --2- 172.21.15.64:0/1119748039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c093dfaf0 0x558c093e7b00 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f3464011570 tx=0x7f3464011970 comp rx=0 tx=0).stop 2024-09-12T03:40:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.438+0000 7f347c269740 1 -- 172.21.15.64:0/1119748039 shutdown_connections 2024-09-12T03:40:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.438+0000 7f347c269740 1 --2- 172.21.15.64:0/1119748039 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x558c093fac20 0x558c093fd030 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.438+0000 7f347c269740 1 --2- 172.21.15.64:0/1119748039 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c093eacb0 0x558c093e8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.438+0000 7f347c269740 1 --2- 172.21.15.64:0/1119748039 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c093e0410 0x558c093e8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.438+0000 7f347c269740 1 --2- 172.21.15.64:0/1119748039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c093dfaf0 0x558c093e7b00 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.438+0000 7f345a7fc640 0 client.16242 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:40:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.438+0000 7f347c269740 1 -- 172.21.15.64:0/1119748039 >> 172.21.15.64:0/1119748039 conn(0x558c093dc060 msgr2=0x558c093db1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:40:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.438+0000 7f347c269740 1 -- 172.21.15.64:0/1119748039 shutdown_connections 2024-09-12T03:40:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.438+0000 7f347c269740 1 -- 172.21.15.64:0/1119748039 wait complete. 2024-09-12T03:40:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.438+0000 7f347c269740 20 client.16242 trim_cache size 0 max 16384 2024-09-12T03:40:15.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.453+0000 7f347c269740 1 Processor -- start 2024-09-12T03:40:15.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.453+0000 7f347c269740 1 -- start start 2024-09-12T03:40:15.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.453+0000 7f347c269740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c093dfaf0 0x558c093e2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:15.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.453+0000 7f347c269740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c093e0410 0x558c093e83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:15.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.453+0000 7f347c269740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c093eacb0 0x558c09402940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:15.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.453+0000 7f347c269740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f34741b3bb0 con 0x558c093e0410 2024-09-12T03:40:15.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.453+0000 7f347c269740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f34741b1480 con 0x558c093eacb0 2024-09-12T03:40:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.453+0000 7f347c269740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f34741af2f0 con 0x558c093dfaf0 2024-09-12T03:40:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.453+0000 7f346a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c093e0410 0x558c093e83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.453+0000 7f346a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c093e0410 0x558c093e83b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53264/0 (socket says 172.21.15.64:53264) 2024-09-12T03:40:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.453+0000 7f346a7fc640 1 -- 172.21.15.64:0/2737270686 learned_addr learned my addr 172.21.15.64:0/2737270686 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:40:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.453+0000 7f3472ffd640 1 --2- 172.21.15.64:0/2737270686 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c093dfaf0 0x558c093e2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.453+0000 7f34737fe640 1 --2- 172.21.15.64:0/2737270686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c093eacb0 0x558c09402940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.453+0000 7f346a7fc640 1 -- 172.21.15.64:0/2737270686 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c093dfaf0 msgr2=0x558c093e2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:40:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.453+0000 7f346a7fc640 1 --2- 172.21.15.64:0/2737270686 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c093dfaf0 0x558c093e2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.453+0000 7f346a7fc640 1 -- 172.21.15.64:0/2737270686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c093eacb0 msgr2=0x558c09402940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:40:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.453+0000 7f346a7fc640 1 --2- 172.21.15.64:0/2737270686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c093eacb0 0x558c09402940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.453+0000 7f346a7fc640 1 -- 172.21.15.64:0/2737270686 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c093dbe90 con 0x558c093e0410 2024-09-12T03:40:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.453+0000 7f3472ffd640 1 --2- 172.21.15.64:0/2737270686 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c093dfaf0 0x558c093e2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:40:15.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.453+0000 7f34737fe640 1 --2- 172.21.15.64:0/2737270686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c093eacb0 0x558c09402940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:40:15.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.454+0000 7f346a7fc640 1 --2- 172.21.15.64:0/2737270686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c093e0410 0x558c093e83b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f345c0bc050 tx=0x7f345c0dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0fdd27e8c195d66 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:40:15.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.454+0000 7f3468ff9640 1 -- 172.21.15.64:0/2737270686 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c049040 con 0x558c093e0410 2024-09-12T03:40:15.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.454+0000 7f3468ff9640 1 -- 172.21.15.64:0/2737270686 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f345c0bc7c0 con 0x558c093e0410 2024-09-12T03:40:15.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.454+0000 7f3468ff9640 1 -- 172.21.15.64:0/2737270686 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c0dc0b0 con 0x558c093e0410 2024-09-12T03:40:15.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.454+0000 7f347c269740 1 -- 172.21.15.64:0/2737270686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c093e0410 msgr2=0x558c093e83b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:15.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.454+0000 7f347c269740 1 --2- 172.21.15.64:0/2737270686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c093e0410 0x558c093e83b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f345c0bc050 tx=0x7f345c0dc040 comp rx=0 tx=0).stop 2024-09-12T03:40:15.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.455+0000 7f347c269740 1 -- 172.21.15.64:0/2737270686 shutdown_connections 2024-09-12T03:40:15.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.455+0000 7f347c269740 1 --2- 172.21.15.64:0/2737270686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c093eacb0 0x558c09402940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:15.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.455+0000 7f347c269740 1 --2- 172.21.15.64:0/2737270686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c093e0410 0x558c093e83b0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:15.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.455+0000 7f347c269740 1 --2- 172.21.15.64:0/2737270686 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c093dfaf0 0x558c093e2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:15.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.455+0000 7f347c269740 1 -- 172.21.15.64:0/2737270686 >> 172.21.15.64:0/2737270686 conn(0x558c093dc060 msgr2=0x558c093e9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:40:15.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.455+0000 7f347c269740 1 -- 172.21.15.64:0/2737270686 shutdown_connections 2024-09-12T03:40:15.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.455+0000 7f347c269740 1 -- 172.21.15.64:0/2737270686 wait complete. 2024-09-12T03:40:15.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.455+0000 7f347c269740 1 -- start start 2024-09-12T03:40:15.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.455+0000 7f347c269740 1 Processor -- start 2024-09-12T03:40:15.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.455+0000 7f347c269740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c093dfaf0 0x558c093e49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:15.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.455+0000 7f347c269740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c093e0410 0x558c093e4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:15.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.455+0000 7f347c269740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c093eacb0 0x558c093e5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:15.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.455+0000 7f347c269740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f34741af940 con 0x558c093dfaf0 2024-09-12T03:40:15.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.455+0000 7f347c269740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f34741b0db0 con 0x558c093e0410 2024-09-12T03:40:15.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.455+0000 7f347c269740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f34741b0010 con 0x558c093eacb0 2024-09-12T03:40:15.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.455+0000 7f347c269740 10 client.0 _ll_register_callbacks cb 0x7f3464009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:40:15.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.455+0000 7f3472ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c093dfaf0 0x558c093e49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:15.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.455+0000 7f3472ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c093dfaf0 0x558c093e49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53268/0 (socket says 172.21.15.64:53268) 2024-09-12T03:40:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.455+0000 7f3472ffd640 1 -- 172.21.15.64:0/158906076 learned_addr learned my addr 172.21.15.64:0/158906076 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:40:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.456+0000 7f34737fe640 1 --2- 172.21.15.64:0/158906076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c093eacb0 0x558c093e5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.456+0000 7f346a7fc640 1 --2- 172.21.15.64:0/158906076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c093e0410 0x558c093e4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.456+0000 7f3472ffd640 1 -- 172.21.15.64:0/158906076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c093eacb0 msgr2=0x558c093e5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:40:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.456+0000 7f3472ffd640 1 --2- 172.21.15.64:0/158906076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c093eacb0 0x558c093e5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.456+0000 7f3472ffd640 1 -- 172.21.15.64:0/158906076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c093e0410 msgr2=0x558c093e4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:40:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.456+0000 7f3472ffd640 1 --2- 172.21.15.64:0/158906076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c093e0410 0x558c093e4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.456+0000 7f3472ffd640 1 -- 172.21.15.64:0/158906076 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c093dd1f0 con 0x558c093dfaf0 2024-09-12T03:40:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.456+0000 7f3472ffd640 1 --2- 172.21.15.64:0/158906076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c093dfaf0 0x558c093e49f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f346401d480 tx=0x7f346401d8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e62fc1cd79fdec27 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:40:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.456+0000 7f345a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-12T03:40:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.456+0000 7f345a7fc640 1 -- 172.21.15.64:0/158906076 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346402f060 con 0x558c093dfaf0 2024-09-12T03:40:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.456+0000 7f345a7fc640 1 -- 172.21.15.64:0/158906076 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9583+0+0 (secure 0 0 0) 0x7f3464013670 con 0x558c093dfaf0 2024-09-12T03:40:15.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.456+0000 7f347c269740 1 -- 172.21.15.64:0/158906076 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f345c0bc7c0 con 0x558c093dfaf0 2024-09-12T03:40:15.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.456+0000 7f345a7fc640 1 -- 172.21.15.64:0/158906076 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3464036050 con 0x558c093dfaf0 2024-09-12T03:40:15.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.456+0000 7f347c269740 10 client.16248 fetch_fsmap learned FSMap version 25 2024-09-12T03:40:15.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.456+0000 7f347c269740 1 -- 172.21.15.64:0/158906076 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558c09465910 con 0x558c093dfaf0 2024-09-12T03:40:15.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.456+0000 7f345a7fc640 1 -- 172.21.15.64:0/158906076 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f345c0bc7c0 con 0x558c093dfaf0 2024-09-12T03:40:15.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.457+0000 7f345a7fc640 1 -- 172.21.15.64:0/158906076 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f3464014700 con 0x558c093dfaf0 2024-09-12T03:40:15.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.457+0000 7f347c269740 10 client.16248 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:40:15.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.457+0000 7f347c269740 10 client.16248 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:40:15.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.457+0000 7f347c269740 20 client.16248 populate_metadata read hostname 'smithi064' 2024-09-12T03:40:15.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.457+0000 7f347c269740 1 --2- 172.21.15.64:0/158906076 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x558c09466060 0x558c09468480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:15.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.457+0000 7f347c269740 4 client.16248 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:40:15.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.457+0000 7f347c269740 1 -- 172.21.15.64:0/158906076 --> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558c0934bd70 con 0x558c09466060 2024-09-12T03:40:15.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.457+0000 7f346a7fc640 1 --2- 172.21.15.64:0/158906076 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x558c09466060 0x558c09468480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:15.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.457+0000 7f345a7fc640 10 client.16248 ms_handle_connect on v2:172.21.15.115:6838/2238031796 2024-09-12T03:40:15.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.457+0000 7f346a7fc640 1 --2- 172.21.15.64:0/158906076 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x558c09466060 0x558c09468480 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae09b58446b3c3dd server_cookie=13e70108f64e3dab in_seq=0 out_seq=0 2024-09-12T03:40:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.472+0000 7f345a7fc640 1 -- 172.21.15.64:0/158906076 <== mds.0 v2:172.21.15.115:6838/2238031796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558c0934bd70 con 0x558c09466060 2024-09-12T03:40:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.472+0000 7f345a7fc640 10 client.16248 handle_command_reply: tid=0 2024-09-12T03:40:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.472+0000 7f347c269740 1 client.16248 shutdown 2024-09-12T03:40:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.472+0000 7f347c269740 1 -- 172.21.15.64:0/158906076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c093dfaf0 msgr2=0x558c093e49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.472+0000 7f347c269740 1 --2- 172.21.15.64:0/158906076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c093dfaf0 0x558c093e49f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f346401d480 tx=0x7f346401d8a0 comp rx=0 tx=0).stop 2024-09-12T03:40:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.472+0000 7f347c269740 1 -- 172.21.15.64:0/158906076 shutdown_connections 2024-09-12T03:40:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.472+0000 7f347c269740 1 --2- 172.21.15.64:0/158906076 >> [v2:172.21.15.115:6838/2238031796,v1:172.21.15.115:6839/2238031796] conn(0x558c09466060 0x558c09468480 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.472+0000 7f347c269740 1 --2- 172.21.15.64:0/158906076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c093eacb0 0x558c093e5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.472+0000 7f347c269740 1 --2- 172.21.15.64:0/158906076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558c093e0410 0x558c093e4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.472+0000 7f347c269740 1 --2- 172.21.15.64:0/158906076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558c093dfaf0 0x558c093e49f0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.472+0000 7f345a7fc640 0 client.16248 ms_handle_reset on v2:172.21.15.115:6838/2238031796 2024-09-12T03:40:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.472+0000 7f347c269740 1 -- 172.21.15.64:0/158906076 >> 172.21.15.64:0/158906076 conn(0x558c093dc060 msgr2=0x558c093da9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:40:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.472+0000 7f347c269740 1 -- 172.21.15.64:0/158906076 shutdown_connections 2024-09-12T03:40:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.472+0000 7f347c269740 1 -- 172.21.15.64:0/158906076 wait complete. 2024-09-12T03:40:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.472+0000 7f347c269740 20 client.16248 trim_cache size 0 max 16384 2024-09-12T03:40:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-12T03:40:15.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-12T03:40:15.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-12T03:40:15.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-12T03:40:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.473+0000 7f34793f6640 1 -- 172.21.15.64:0/4225519207 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f34600a2e20 msgr2=0x7f34600a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.473+0000 7f34793f6640 1 --2- 172.21.15.64:0/4225519207 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f34600a2e20 0x7f34600a52e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f346c018000 tx=0x7f346c019730 comp rx=0 tx=0).stop 2024-09-12T03:40:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.473+0000 7f34793f6640 1 -- 172.21.15.64:0/4225519207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34741035e0 msgr2=0x7f347419aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.473+0000 7f34793f6640 1 --2- 172.21.15.64:0/4225519207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34741035e0 0x7f347419aa80 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f345c005940 tx=0x7f345c0042f0 comp rx=0 tx=0).stop 2024-09-12T03:40:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.473+0000 7f34793f6640 1 -- 172.21.15.64:0/4225519207 shutdown_connections 2024-09-12T03:40:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.473+0000 7f34793f6640 1 --2- 172.21.15.64:0/4225519207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f347410c110 0x7f347419ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.473+0000 7f34793f6640 1 --2- 172.21.15.64:0/4225519207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34741035e0 0x7f347419aa80 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.473+0000 7f34793f6640 1 --2- 172.21.15.64:0/4225519207 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f34600a2e20 0x7f34600a52e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.473+0000 7f34793f6640 1 --2- 172.21.15.64:0/4225519207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3474102cc0 0x7f347419a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.473+0000 7f34793f6640 1 -- 172.21.15.64:0/4225519207 >> 172.21.15.64:0/4225519207 conn(0x7f34740fc6b0 msgr2=0x7f34740fca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:40:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.473+0000 7f34793f6640 1 -- 172.21.15.64:0/4225519207 shutdown_connections 2024-09-12T03:40:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-12T03:40:15.473+0000 7f34793f6640 1 -- 172.21.15.64:0/4225519207 wait complete. 2024-09-12T03:40:15.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:15 smithi064 ceph-mon[30952]: pgmap v342: 129 pgs: 129 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-09-12T03:40:15.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:15 smithi115 ceph-mon[34801]: pgmap v342: 129 pgs: 129 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-09-12T03:40:15.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:15 smithi195 ceph-mon[34786]: pgmap v342: 129 pgs: 129 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-09-12T03:40:17.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:17 smithi115 ceph-mon[34801]: pgmap v343: 129 pgs: 129 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-09-12T03:40:17.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:17 smithi195 ceph-mon[34786]: pgmap v343: 129 pgs: 129 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-09-12T03:40:18.021 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:17 smithi064 ceph-mon[30952]: pgmap v343: 129 pgs: 129 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-09-12T03:40:19.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:19 smithi115 ceph-mon[34801]: pgmap v344: 129 pgs: 129 active+clean; 5.0 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-09-12T03:40:19.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:19 smithi195 ceph-mon[34786]: pgmap v344: 129 pgs: 129 active+clean; 5.0 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-09-12T03:40:20.021 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:19 smithi064 ceph-mon[30952]: pgmap v344: 129 pgs: 129 active+clean; 5.0 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-09-12T03:40:21.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:21 smithi115 ceph-mon[34801]: pgmap v345: 129 pgs: 129 active+clean; 5.0 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-09-12T03:40:21.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:21 smithi195 ceph-mon[34786]: pgmap v345: 129 pgs: 129 active+clean; 5.0 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-09-12T03:40:22.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:21 smithi064 ceph-mon[30952]: pgmap v345: 129 pgs: 129 active+clean; 5.0 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-09-12T03:40:23.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:23 smithi115 ceph-mon[34801]: pgmap v346: 129 pgs: 129 active+clean; 5.2 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 63 KiB/s wr, 20 op/s 2024-09-12T03:40:23.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:23 smithi195 ceph-mon[34786]: pgmap v346: 129 pgs: 129 active+clean; 5.2 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 63 KiB/s wr, 20 op/s 2024-09-12T03:40:24.021 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:23 smithi064 ceph-mon[30952]: pgmap v346: 129 pgs: 129 active+clean; 5.2 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 63 KiB/s wr, 20 op/s 2024-09-12T03:40:24.139 INFO:tasks.workunit.client.0.smithi064.stdout:561/600: open fd = 3 2024-09-12T03:40:24.140 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.140 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.140 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:40:24.140 INFO:tasks.workunit.client.0.smithi064.stdout:562/600: open fd = 3 2024-09-12T03:40:24.140 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.140 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.140 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:40:24.140 INFO:tasks.workunit.client.0.smithi064.stdout:563/600: open fd = 3 2024-09-12T03:40:24.140 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.140 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.140 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:40:24.140 INFO:tasks.workunit.client.0.smithi064.stdout:564/600: open fd = 3 2024-09-12T03:40:24.141 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.141 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.141 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:40:24.141 INFO:tasks.workunit.client.0.smithi064.stdout:565/600: open fd = 3 2024-09-12T03:40:24.141 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.141 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.141 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:40:24.141 INFO:tasks.workunit.client.0.smithi064.stdout:566/600: open fd = 3 2024-09-12T03:40:24.141 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.141 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.141 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:40:24.141 INFO:tasks.workunit.client.0.smithi064.stdout:567/600: open fd = 3 2024-09-12T03:40:24.141 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.141 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.141 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:40:24.142 INFO:tasks.workunit.client.0.smithi064.stdout:568/600: open fd = 3 2024-09-12T03:40:24.142 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.142 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.142 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:40:24.142 INFO:tasks.workunit.client.0.smithi064.stdout:569/600: open fd = 3 2024-09-12T03:40:24.142 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.142 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.142 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:40:24.142 INFO:tasks.workunit.client.0.smithi064.stdout:570/600: open fd = 3 2024-09-12T03:40:24.142 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.142 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.142 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:40:24.142 INFO:tasks.workunit.client.0.smithi064.stdout:571/600: open fd = 3 2024-09-12T03:40:24.142 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.142 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.143 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:40:24.143 INFO:tasks.workunit.client.0.smithi064.stdout:572/600: open fd = 3 2024-09-12T03:40:24.143 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.143 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.143 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:40:24.143 INFO:tasks.workunit.client.0.smithi064.stdout:573/600: open fd = 3 2024-09-12T03:40:24.143 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.143 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.143 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:40:24.143 INFO:tasks.workunit.client.0.smithi064.stdout:574/600: open fd = 3 2024-09-12T03:40:24.143 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.143 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.143 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:40:24.143 INFO:tasks.workunit.client.0.smithi064.stdout:575/600: open fd = 3 2024-09-12T03:40:24.144 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.144 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.144 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:40:24.144 INFO:tasks.workunit.client.0.smithi064.stdout:576/600: open fd = 3 2024-09-12T03:40:24.144 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.144 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.144 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:40:24.144 INFO:tasks.workunit.client.0.smithi064.stdout:577/600: open fd = 3 2024-09-12T03:40:24.144 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.144 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.144 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:40:24.144 INFO:tasks.workunit.client.0.smithi064.stdout:578/600: open fd = 3 2024-09-12T03:40:24.144 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.145 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.145 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:40:24.145 INFO:tasks.workunit.client.0.smithi064.stdout:579/600: open fd = 3 2024-09-12T03:40:24.145 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.145 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.145 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:40:24.145 INFO:tasks.workunit.client.0.smithi064.stdout:580/600: open fd = 3 2024-09-12T03:40:24.145 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.145 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.145 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:40:24.145 INFO:tasks.workunit.client.0.smithi064.stdout:581/600: open fd = 3 2024-09-12T03:40:24.145 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.145 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.145 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:40:24.145 INFO:tasks.workunit.client.0.smithi064.stdout:582/600: open fd = 3 2024-09-12T03:40:24.146 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.146 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.146 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:40:24.146 INFO:tasks.workunit.client.0.smithi064.stdout:583/600: open fd = 3 2024-09-12T03:40:24.146 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.146 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.146 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:40:24.146 INFO:tasks.workunit.client.0.smithi064.stdout:584/600: open fd = 3 2024-09-12T03:40:24.146 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.146 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.146 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:40:24.146 INFO:tasks.workunit.client.0.smithi064.stdout:585/600: open fd = 3 2024-09-12T03:40:24.146 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.146 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.147 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:40:24.147 INFO:tasks.workunit.client.0.smithi064.stdout:586/600: open fd = 3 2024-09-12T03:40:24.147 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.147 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.147 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:40:24.147 INFO:tasks.workunit.client.0.smithi064.stdout:587/600: open fd = 3 2024-09-12T03:40:24.147 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.147 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.147 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:40:24.147 INFO:tasks.workunit.client.0.smithi064.stdout:588/600: open fd = 3 2024-09-12T03:40:24.147 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.147 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.147 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:40:24.147 INFO:tasks.workunit.client.0.smithi064.stdout:589/600: open fd = 3 2024-09-12T03:40:24.148 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.148 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.148 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:40:24.148 INFO:tasks.workunit.client.0.smithi064.stdout:590/600: open fd = 3 2024-09-12T03:40:24.148 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.148 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.148 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:40:24.148 INFO:tasks.workunit.client.0.smithi064.stdout:591/600: open fd = 3 2024-09-12T03:40:24.148 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.148 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.148 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:40:24.148 INFO:tasks.workunit.client.0.smithi064.stdout:592/600: open fd = 3 2024-09-12T03:40:24.148 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.148 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.149 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:40:24.149 INFO:tasks.workunit.client.0.smithi064.stdout:593/600: open fd = 3 2024-09-12T03:40:24.149 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.149 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.149 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:40:24.149 INFO:tasks.workunit.client.0.smithi064.stdout:594/600: open fd = 3 2024-09-12T03:40:24.149 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.149 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.149 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:40:24.149 INFO:tasks.workunit.client.0.smithi064.stdout:595/600: open fd = 3 2024-09-12T03:40:24.149 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.149 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.149 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:40:24.149 INFO:tasks.workunit.client.0.smithi064.stdout:596/600: open fd = 3 2024-09-12T03:40:24.149 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.150 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.150 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:40:24.150 INFO:tasks.workunit.client.0.smithi064.stdout:597/600: open fd = 3 2024-09-12T03:40:24.150 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.150 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.150 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:40:24.150 INFO:tasks.workunit.client.0.smithi064.stdout:598/600: open fd = 3 2024-09-12T03:40:24.150 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.150 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.150 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:40:24.150 INFO:tasks.workunit.client.0.smithi064.stdout:599/600: open fd = 3 2024-09-12T03:40:24.150 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.150 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.150 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:40:24.151 INFO:tasks.workunit.client.0.smithi064.stdout:600/600: open fd = 3 2024-09-12T03:40:24.151 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.151 INFO:tasks.workunit.client.0.smithi064.stdout:write ret = 32 2024-09-12T03:40:24.151 INFO:tasks.workunit.client.0.smithi064.stdout:pread ret = 64 2024-09-12T03:40:24.151 INFO:tasks.workunit.client.0.smithi064.stderr:+ echo OK 2024-09-12T03:40:24.152 INFO:tasks.workunit.client.0.smithi064.stdout:OK 2024-09-12T03:40:24.152 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-12T03:40:24.153 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-12T03:40:24.243 INFO:tasks.workunit:Stopping ['fs/test_o_trunc.sh'] on client.0... 2024-09-12T03:40:24.244 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-12T03:40:24.556 DEBUG:teuthology.parallel:result is None 2024-09-12T03:40:24.557 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-12T03:40:24.593 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-12T03:40:24.593 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-12T03:40:24.607 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi064.front.sepia.ceph.com 2024-09-12T03:40:24.642 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-12T03:40:24.658 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-12T03:40:24.658 INFO:teuthology.task.exec:Executing custom commands... 2024-09-12T03:40:24.658 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi064.front.sepia.ceph.com 2024-09-12T03:40:24.658 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-12T03:40:24.710 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-12T03:40:24.777 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-12T03:40:24.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:24.921+0000 7fbef4a42640 1 -- 172.21.15.64:0/3018452601 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbef0100fd0 msgr2=0x7fbef010d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:24.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:24.921+0000 7fbef4a42640 1 --2- 172.21.15.64:0/3018452601 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbef0100fd0 0x7fbef010d030 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fbee8009ca0 tx=0x7fbee802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:40:24.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:24.922+0000 7fbef4a42640 1 -- 172.21.15.64:0/3018452601 shutdown_connections 2024-09-12T03:40:24.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:24.922+0000 7fbef4a42640 1 --2- 172.21.15.64:0/3018452601 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbef0100fd0 0x7fbef010d030 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:24.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:24.922+0000 7fbef4a42640 1 --2- 172.21.15.64:0/3018452601 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbef0100630 0x7fbef0100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:24.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:24.922+0000 7fbef4a42640 1 --2- 172.21.15.64:0/3018452601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbef0106220 0x7fbef0106600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:24.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:24.922+0000 7fbef4a42640 1 -- 172.21.15.64:0/3018452601 >> 172.21.15.64:0/3018452601 conn(0x7fbef0075ef0 msgr2=0x7fbef0076300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:40:24.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:24.922+0000 7fbef4a42640 1 -- 172.21.15.64:0/3018452601 shutdown_connections 2024-09-12T03:40:24.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:24.922+0000 7fbef4a42640 1 -- 172.21.15.64:0/3018452601 wait complete. 2024-09-12T03:40:24.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:24.922+0000 7fbef4a42640 1 Processor -- start 2024-09-12T03:40:24.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:24.922+0000 7fbef4a42640 1 -- start start 2024-09-12T03:40:24.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:24.923+0000 7fbef4a42640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbef0100630 0x7fbef0198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:24.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:24.923+0000 7fbef4a42640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbef0100fd0 0x7fbef01988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:24.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:24.923+0000 7fbef4a42640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbef0106220 0x7fbef019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:24.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:24.923+0000 7fbef4a42640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbef0111df0 con 0x7fbef0100630 2024-09-12T03:40:24.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:24.923+0000 7fbef4a42640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbef0111c70 con 0x7fbef0100fd0 2024-09-12T03:40:24.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:24.923+0000 7fbef4a42640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbef0111f70 con 0x7fbef0106220 2024-09-12T03:40:24.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:24.923+0000 7fbeee59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbef0100630 0x7fbef0198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:24.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:24.923+0000 7fbeee59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbef0100630 0x7fbef0198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59438/0 (socket says 172.21.15.64:59438) 2024-09-12T03:40:24.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:24.923+0000 7fbeee59c640 1 -- 172.21.15.64:0/1808664660 learned_addr learned my addr 172.21.15.64:0/1808664660 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:40:24.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:24.923+0000 7fbeedd9b640 1 --2- 172.21.15.64:0/1808664660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbef0100fd0 0x7fbef01988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:24.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:24.923+0000 7fbeeed9d640 1 --2- 172.21.15.64:0/1808664660 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbef0106220 0x7fbef019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:24.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:24.923+0000 7fbeee59c640 1 -- 172.21.15.64:0/1808664660 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbef0106220 msgr2=0x7fbef019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:24.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:24.923+0000 7fbeee59c640 1 --2- 172.21.15.64:0/1808664660 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbef0106220 0x7fbef019cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:24.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:24.923+0000 7fbeee59c640 1 -- 172.21.15.64:0/1808664660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbef0100fd0 msgr2=0x7fbef01988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:24.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:24.923+0000 7fbeee59c640 1 --2- 172.21.15.64:0/1808664660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbef0100fd0 0x7fbef01988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:24.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:24.923+0000 7fbeee59c640 1 -- 172.21.15.64:0/1808664660 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef019d380 con 0x7fbef0100630 2024-09-12T03:40:24.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:24.923+0000 7fbeedd9b640 1 --2- 172.21.15.64:0/1808664660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbef0100fd0 0x7fbef01988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:40:24.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:24.923+0000 7fbeee59c640 1 --2- 172.21.15.64:0/1808664660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbef0100630 0x7fbef0198390 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fbee000d950 tx=0x7fbee000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:40:24.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:24.923+0000 7fbedf7fe640 1 -- 172.21.15.64:0/1808664660 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee0017070 con 0x7fbef0100630 2024-09-12T03:40:24.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:24.923+0000 7fbedf7fe640 1 -- 172.21.15.64:0/1808664660 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbee0009c30 con 0x7fbef0100630 2024-09-12T03:40:24.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:24.923+0000 7fbef4a42640 1 -- 172.21.15.64:0/1808664660 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbef019d670 con 0x7fbef0100630 2024-09-12T03:40:24.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:24.923+0000 7fbedf7fe640 1 -- 172.21.15.64:0/1808664660 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee000f620 con 0x7fbef0100630 2024-09-12T03:40:24.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:24.924+0000 7fbef4a42640 1 -- 172.21.15.64:0/1808664660 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbef01a4f00 con 0x7fbef0100630 2024-09-12T03:40:24.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:24.925+0000 7fbef4a42640 1 -- 172.21.15.64:0/1808664660 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbef0100a90 con 0x7fbef0100630 2024-09-12T03:40:24.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:24.926+0000 7fbedf7fe640 1 -- 172.21.15.64:0/1808664660 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fbee00109a0 con 0x7fbef0100630 2024-09-12T03:40:24.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:24.927+0000 7fbedf7fe640 1 --2- 172.21.15.64:0/1808664660 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fbed80a2e20 0x7fbed80a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:24.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:24.927+0000 7fbeedd9b640 1 --2- 172.21.15.64:0/1808664660 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fbed80a2e20 0x7fbed80a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:24.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:24.927+0000 7fbedf7fe640 1 -- 172.21.15.64:0/1808664660 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9583+0+0 (secure 0 0 0) 0x7fbee00cf4e0 con 0x7fbef0100630 2024-09-12T03:40:24.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:24.927+0000 7fbeedd9b640 1 --2- 172.21.15.64:0/1808664660 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fbed80a2e20 0x7fbed80a52e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbef01004d0 tx=0x7fbee4008040 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:40:24.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:24.931+0000 7fbedf7fe640 1 -- 172.21.15.64:0/1808664660 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbee0009de0 con 0x7fbef0100630 2024-09-12T03:40:25.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.128+0000 7fbef4a42640 1 -- 172.21.15.64:0/1808664660 --> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fbef0062630 con 0x7fbed80a2e20 2024-09-12T03:40:25.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.136+0000 7fbedf7fe640 1 -- 172.21.15.64:0/1808664660 <== mgr.15243 v2:172.21.15.64:6800/3960824462 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7fbef0062630 con 0x7fbed80a2e20 2024-09-12T03:40:25.136 INFO:teuthology.orchestra.run.smithi064.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-12T00:00:00", "created": "2024-09-12T03:29:16", "first": "2024-09-12T03:30:00", "last": "2024-09-12T03:40:00", "last_pruned": "2024-09-12T03:40:00", "created_count": 11, "pruned_count": 5, "active": true} 2024-09-12T03:40:25.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.138+0000 7fbef4a42640 1 -- 172.21.15.64:0/1808664660 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fbed80a2e20 msgr2=0x7fbed80a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:25.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.138+0000 7fbef4a42640 1 --2- 172.21.15.64:0/1808664660 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fbed80a2e20 0x7fbed80a52e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbef01004d0 tx=0x7fbee4008040 comp rx=0 tx=0).stop 2024-09-12T03:40:25.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.138+0000 7fbef4a42640 1 -- 172.21.15.64:0/1808664660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbef0100630 msgr2=0x7fbef0198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:25.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.138+0000 7fbef4a42640 1 --2- 172.21.15.64:0/1808664660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbef0100630 0x7fbef0198390 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fbee000d950 tx=0x7fbee000de10 comp rx=0 tx=0).stop 2024-09-12T03:40:25.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.138+0000 7fbef4a42640 1 -- 172.21.15.64:0/1808664660 shutdown_connections 2024-09-12T03:40:25.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.138+0000 7fbef4a42640 1 --2- 172.21.15.64:0/1808664660 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbef0106220 0x7fbef019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:25.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.138+0000 7fbef4a42640 1 --2- 172.21.15.64:0/1808664660 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fbed80a2e20 0x7fbed80a52e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:25.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.138+0000 7fbef4a42640 1 --2- 172.21.15.64:0/1808664660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbef0100fd0 0x7fbef01988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:25.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.138+0000 7fbef4a42640 1 --2- 172.21.15.64:0/1808664660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbef0100630 0x7fbef0198390 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:25.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.138+0000 7fbef4a42640 1 -- 172.21.15.64:0/1808664660 >> 172.21.15.64:0/1808664660 conn(0x7fbef0075ef0 msgr2=0x7fbef00ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:40:25.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.138+0000 7fbef4a42640 1 -- 172.21.15.64:0/1808664660 shutdown_connections 2024-09-12T03:40:25.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.138+0000 7fbef4a42640 1 -- 172.21.15.64:0/1808664660 wait complete. 2024-09-12T03:40:25.149 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-12T03:40:25.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.254+0000 7f277244c640 1 -- 172.21.15.64:0/779245512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f276c1069c0 msgr2=0x7f276c106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:25.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.254+0000 7f277244c640 1 --2- 172.21.15.64:0/779245512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f276c1069c0 0x7f276c106da0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f275c009930 tx=0x7f275c02a420 comp rx=0 tx=0).stop 2024-09-12T03:40:25.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.254+0000 7f277244c640 1 -- 172.21.15.64:0/779245512 shutdown_connections 2024-09-12T03:40:25.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.254+0000 7f277244c640 1 --2- 172.21.15.64:0/779245512 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f276c101350 0x7f276c10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:25.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.254+0000 7f277244c640 1 --2- 172.21.15.64:0/779245512 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f276c1009b0 0x7f276c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:25.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.254+0000 7f277244c640 1 --2- 172.21.15.64:0/779245512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f276c1069c0 0x7f276c106da0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:25.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.254+0000 7f277244c640 1 -- 172.21.15.64:0/779245512 >> 172.21.15.64:0/779245512 conn(0x7f276c077990 msgr2=0x7f276c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:40:25.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.254+0000 7f277244c640 1 -- 172.21.15.64:0/779245512 shutdown_connections 2024-09-12T03:40:25.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.254+0000 7f277244c640 1 -- 172.21.15.64:0/779245512 wait complete. 2024-09-12T03:40:25.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.254+0000 7f277244c640 1 Processor -- start 2024-09-12T03:40:25.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.254+0000 7f277244c640 1 -- start start 2024-09-12T03:40:25.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.255+0000 7f277244c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f276c1009b0 0x7f276c1982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:25.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.255+0000 7f277244c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f276c101350 0x7f276c198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:25.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.255+0000 7f277244c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f276c1069c0 0x7f276c19cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:25.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.255+0000 7f277244c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f276c111db0 con 0x7f276c1069c0 2024-09-12T03:40:25.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.255+0000 7f277244c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f276c111c30 con 0x7f276c1009b0 2024-09-12T03:40:25.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.255+0000 7f277244c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f276c111f30 con 0x7f276c101350 2024-09-12T03:40:25.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.255+0000 7f27709e9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f276c1069c0 0x7f276c19cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:25.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.255+0000 7f27709e9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f276c1069c0 0x7f276c19cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59446/0 (socket says 172.21.15.64:59446) 2024-09-12T03:40:25.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.255+0000 7f27709e9640 1 -- 172.21.15.64:0/505367408 learned_addr learned my addr 172.21.15.64:0/505367408 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:40:25.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.255+0000 7f276b7fe640 1 --2- 172.21.15.64:0/505367408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f276c101350 0x7f276c198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:25.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.255+0000 7f276bfff640 1 --2- 172.21.15.64:0/505367408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f276c1009b0 0x7f276c1982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:25.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.255+0000 7f27709e9640 1 -- 172.21.15.64:0/505367408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f276c101350 msgr2=0x7f276c198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:25.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.255+0000 7f27709e9640 1 --2- 172.21.15.64:0/505367408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f276c101350 0x7f276c198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:25.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.255+0000 7f27709e9640 1 -- 172.21.15.64:0/505367408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f276c1009b0 msgr2=0x7f276c1982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:25.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.255+0000 7f27709e9640 1 --2- 172.21.15.64:0/505367408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f276c1009b0 0x7f276c1982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:25.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.255+0000 7f27709e9640 1 -- 172.21.15.64:0/505367408 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f276c19d2b0 con 0x7f276c1069c0 2024-09-12T03:40:25.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.256+0000 7f27709e9640 1 --2- 172.21.15.64:0/505367408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f276c1069c0 0x7f276c19cb80 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f2764009fd0 tx=0x7f276400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:40:25.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.256+0000 7f27697fa640 1 -- 172.21.15.64:0/505367408 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2764017070 con 0x7f276c1069c0 2024-09-12T03:40:25.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.256+0000 7f27697fa640 1 -- 172.21.15.64:0/505367408 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2764003e20 con 0x7f276c1069c0 2024-09-12T03:40:25.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.256+0000 7f27697fa640 1 -- 172.21.15.64:0/505367408 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2764010460 con 0x7f276c1069c0 2024-09-12T03:40:25.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.256+0000 7f277244c640 1 -- 172.21.15.64:0/505367408 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f276c19d5a0 con 0x7f276c1069c0 2024-09-12T03:40:25.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.256+0000 7f277244c640 1 -- 172.21.15.64:0/505367408 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f276c1a4e30 con 0x7f276c1069c0 2024-09-12T03:40:25.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.257+0000 7f277244c640 1 -- 172.21.15.64:0/505367408 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f276c100e10 con 0x7f276c1069c0 2024-09-12T03:40:25.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.259+0000 7f27697fa640 1 -- 172.21.15.64:0/505367408 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f2764003970 con 0x7f276c1069c0 2024-09-12T03:40:25.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.259+0000 7f27697fa640 1 --2- 172.21.15.64:0/505367408 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f27540a2da0 0x7f27540a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:25.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.260+0000 7f276bfff640 1 --2- 172.21.15.64:0/505367408 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f27540a2da0 0x7f27540a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:25.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.260+0000 7f27697fa640 1 -- 172.21.15.64:0/505367408 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9583+0+0 (secure 0 0 0) 0x7f27640cf550 con 0x7f276c1069c0 2024-09-12T03:40:25.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.260+0000 7f276bfff640 1 --2- 172.21.15.64:0/505367408 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f27540a2da0 0x7f27540a5260 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f275c031440 tx=0x7f275c005a20 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:40:25.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.263+0000 7f27697fa640 1 -- 172.21.15.64:0/505367408 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f276409b700 con 0x7f276c1069c0 2024-09-12T03:40:25.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.459+0000 7f277244c640 1 -- 172.21.15.64:0/505367408 --> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f276c062630 con 0x7f27540a2da0 2024-09-12T03:40:25.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.466+0000 7f27697fa640 1 -- 172.21.15.64:0/505367408 <== mgr.15243 v2:172.21.15.64:6800/3960824462 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f276c062630 con 0x7f27540a2da0 2024-09-12T03:40:25.467 INFO:teuthology.orchestra.run.smithi064.stdout:/ 1m 6m3h 2024-09-12T03:40:25.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.468+0000 7f277244c640 1 -- 172.21.15.64:0/505367408 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f27540a2da0 msgr2=0x7f27540a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:25.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.468+0000 7f277244c640 1 --2- 172.21.15.64:0/505367408 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f27540a2da0 0x7f27540a5260 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f275c031440 tx=0x7f275c005a20 comp rx=0 tx=0).stop 2024-09-12T03:40:25.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.468+0000 7f277244c640 1 -- 172.21.15.64:0/505367408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f276c1069c0 msgr2=0x7f276c19cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:25.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.468+0000 7f277244c640 1 --2- 172.21.15.64:0/505367408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f276c1069c0 0x7f276c19cb80 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f2764009fd0 tx=0x7f276400b260 comp rx=0 tx=0).stop 2024-09-12T03:40:25.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.468+0000 7f277244c640 1 -- 172.21.15.64:0/505367408 shutdown_connections 2024-09-12T03:40:25.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.468+0000 7f277244c640 1 --2- 172.21.15.64:0/505367408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f276c1069c0 0x7f276c19cb80 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:25.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.468+0000 7f277244c640 1 --2- 172.21.15.64:0/505367408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f276c101350 0x7f276c198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:25.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.468+0000 7f277244c640 1 --2- 172.21.15.64:0/505367408 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f27540a2da0 0x7f27540a5260 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:25.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.468+0000 7f277244c640 1 --2- 172.21.15.64:0/505367408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f276c1009b0 0x7f276c1982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:25.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.468+0000 7f277244c640 1 -- 172.21.15.64:0/505367408 >> 172.21.15.64:0/505367408 conn(0x7f276c077990 msgr2=0x7f276c103960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:40:25.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.468+0000 7f277244c640 1 -- 172.21.15.64:0/505367408 shutdown_connections 2024-09-12T03:40:25.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.468+0000 7f277244c640 1 -- 172.21.15.64:0/505367408 wait complete. 2024-09-12T03:40:25.480 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-12T03:40:25.492 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:40:25.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.596+0000 7f41a1349640 1 -- 172.21.15.64:0/1154777846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f419c101320 msgr2=0x7f419c10d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:25.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.596+0000 7f41a1349640 1 --2- 172.21.15.64:0/1154777846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f419c101320 0x7f419c10d850 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f4194009cd0 tx=0x7f419402a8e0 comp rx=0 tx=0).stop 2024-09-12T03:40:25.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.596+0000 7f41a1349640 1 -- 172.21.15.64:0/1154777846 shutdown_connections 2024-09-12T03:40:25.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.596+0000 7f41a1349640 1 --2- 172.21.15.64:0/1154777846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f419c101320 0x7f419c10d850 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:25.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.596+0000 7f41a1349640 1 --2- 172.21.15.64:0/1154777846 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f419c100980 0x7f419c100de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:25.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.596+0000 7f41a1349640 1 --2- 172.21.15.64:0/1154777846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f419c106990 0x7f419c106d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:25.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.596+0000 7f41a1349640 1 -- 172.21.15.64:0/1154777846 >> 172.21.15.64:0/1154777846 conn(0x7f419c077a10 msgr2=0x7f419c0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:40:25.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.596+0000 7f41a1349640 1 -- 172.21.15.64:0/1154777846 shutdown_connections 2024-09-12T03:40:25.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.596+0000 7f41a1349640 1 -- 172.21.15.64:0/1154777846 wait complete. 2024-09-12T03:40:25.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.596+0000 7f41a1349640 1 Processor -- start 2024-09-12T03:40:25.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.597+0000 7f41a1349640 1 -- start start 2024-09-12T03:40:25.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.597+0000 7f41a1349640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f419c100980 0x7f419c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:25.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.597+0000 7f41a1349640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f419c101320 0x7f419c198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:25.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.597+0000 7f41a1349640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f419c106990 0x7f419c19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:25.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.597+0000 7f41a1349640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f419c110c80 con 0x7f419c106990 2024-09-12T03:40:25.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.597+0000 7f41a1349640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f419c110b00 con 0x7f419c100980 2024-09-12T03:40:25.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.597+0000 7f41a1349640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f419c110e00 con 0x7f419c101320 2024-09-12T03:40:25.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.597+0000 7f419b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f419c106990 0x7f419c19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:25.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.597+0000 7f419b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f419c106990 0x7f419c19cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59460/0 (socket says 172.21.15.64:59460) 2024-09-12T03:40:25.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.597+0000 7f419b7fe640 1 -- 172.21.15.64:0/121196740 learned_addr learned my addr 172.21.15.64:0/121196740 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:40:25.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.597+0000 7f419a7fc640 1 --2- 172.21.15.64:0/121196740 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f419c101320 0x7f419c198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:25.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.597+0000 7f419affd640 1 --2- 172.21.15.64:0/121196740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f419c100980 0x7f419c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:25.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.597+0000 7f419b7fe640 1 -- 172.21.15.64:0/121196740 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f419c101320 msgr2=0x7f419c198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:25.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.597+0000 7f419b7fe640 1 --2- 172.21.15.64:0/121196740 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f419c101320 0x7f419c198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:25.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.597+0000 7f419b7fe640 1 -- 172.21.15.64:0/121196740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f419c100980 msgr2=0x7f419c1983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:25.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.597+0000 7f419b7fe640 1 --2- 172.21.15.64:0/121196740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f419c100980 0x7f419c1983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:25.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.597+0000 7f419b7fe640 1 -- 172.21.15.64:0/121196740 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f419c19d3b0 con 0x7f419c106990 2024-09-12T03:40:25.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.598+0000 7f419b7fe640 1 --2- 172.21.15.64:0/121196740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f419c106990 0x7f419c19cc80 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f4194005cc0 tx=0x7f4194003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:40:25.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.598+0000 7f4193fff640 1 -- 172.21.15.64:0/121196740 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4194004450 con 0x7f419c106990 2024-09-12T03:40:25.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.598+0000 7f4193fff640 1 -- 172.21.15.64:0/121196740 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4194031c90 con 0x7f419c106990 2024-09-12T03:40:25.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.598+0000 7f41a1349640 1 -- 172.21.15.64:0/121196740 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f419c19d640 con 0x7f419c106990 2024-09-12T03:40:25.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.598+0000 7f4193fff640 1 -- 172.21.15.64:0/121196740 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4194030e70 con 0x7f419c106990 2024-09-12T03:40:25.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.598+0000 7f41a1349640 1 -- 172.21.15.64:0/121196740 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f419c1a4f30 con 0x7f419c106990 2024-09-12T03:40:25.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.599+0000 7f41a1349640 1 -- 172.21.15.64:0/121196740 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f419c100de0 con 0x7f419c106990 2024-09-12T03:40:25.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.600+0000 7f4193fff640 1 -- 172.21.15.64:0/121196740 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f41940045f0 con 0x7f419c106990 2024-09-12T03:40:25.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.601+0000 7f4193fff640 1 --2- 172.21.15.64:0/121196740 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f41880a2d00 0x7f41880a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:25.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.601+0000 7f419affd640 1 --2- 172.21.15.64:0/121196740 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f41880a2d00 0x7f41880a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:25.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.601+0000 7f4193fff640 1 -- 172.21.15.64:0/121196740 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9583+0+0 (secure 0 0 0) 0x7f419402e030 con 0x7f419c106990 2024-09-12T03:40:25.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.601+0000 7f419affd640 1 --2- 172.21.15.64:0/121196740 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f41880a2d00 0x7f41880a51c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f418400bbe0 tx=0x7f418400b390 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:40:25.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.605+0000 7f4193fff640 1 -- 172.21.15.64:0/121196740 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f419404b060 con 0x7f419c106990 2024-09-12T03:40:25.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:25 smithi064 ceph-mon[30952]: pgmap v347: 129 pgs: 129 active+clean; 5.2 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-12T03:40:25.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.782+0000 7f41a1349640 1 -- 172.21.15.64:0/121196740 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f419c109f20 con 0x7f419c106990 2024-09-12T03:40:25.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.784+0000 7f4193fff640 1 -- 172.21.15.64:0/121196740 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+25533 (secure 0 0 0) 0x7f41940098b0 con 0x7f419c106990 2024-09-12T03:40:25.785 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:40:25.785 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":103,"fsid":"b0812460-70b5-11ef-bcea-c7b262605968","created":"2024-09-12T03:19:15.900322+0000","modified":"2024-09-12T03:40:11.148161+0000","last_up_change":"2024-09-12T03:26:06.122971+0000","last_in_change":"2024-09-12T03:25:43.130689+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:22:21.915870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:26:55.826348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:26:57.984319+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T03:27:04.460292+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"326271c9-1f28-4ce6-b3be-fcc57479d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6803","nonce":1307832590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6805","nonce":1307832590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6809","nonce":1307832590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6807","nonce":1307832590}]},"public_addr":"172.21.15.64:6803/1307832590","cluster_addr":"172.21.15.64:6805/1307832590","heartbeat_back_addr":"172.21.15.64:6809/1307832590","heartbeat_front_addr":"172.21.15.64:6807/1307832590","state":["exists","up"]},{"osd":1,"uuid":"7dca0c28-c234-423c-936e-2351d63f918e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6801","nonce":3123728622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6803","nonce":3123728622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6807","nonce":3123728622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6805","nonce":3123728622}]},"public_addr":"172.21.15.115:6801/3123728622","cluster_addr":"172.21.15.115:6803/3123728622","heartbeat_back_addr":"172.21.15.115:6807/3123728622","heartbeat_front_addr":"172.21.15.115:6805/3123728622","state":["exists","up"]},{"osd":2,"uuid":"4b3a3c7a-aec6-455a-9a9a-1ac9cc900f24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6801","nonce":1664125451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6803","nonce":1664125451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6807","nonce":1664125451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6805","nonce":1664125451}]},"public_addr":"172.21.15.195:6801/1664125451","cluster_addr":"172.21.15.195:6803/1664125451","heartbeat_back_addr":"172.21.15.195:6807/1664125451","heartbeat_front_addr":"172.21.15.195:6805/1664125451","state":["exists","up"]},{"osd":3,"uuid":"8768c339-2c1e-4294-8b78-3c2adab6750f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6811","nonce":3935517185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6813","nonce":3935517185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6817","nonce":3935517185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6815","nonce":3935517185}]},"public_addr":"172.21.15.64:6811/3935517185","cluster_addr":"172.21.15.64:6813/3935517185","heartbeat_back_addr":"172.21.15.64:6817/3935517185","heartbeat_front_addr":"172.21.15.64:6815/3935517185","state":["exists","up"]},{"osd":4,"uuid":"123aac47-a8ae-47c7-93a7-0dd293356322","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6809","nonce":446665227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6811","nonce":446665227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6815","nonce":446665227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6813","nonce":446665227}]},"public_addr":"172.21.15.115:6809/446665227","cluster_addr":"172.21.15.115:6811/446665227","heartbeat_back_addr":"172.21.15.115:6815/446665227","heartbeat_front_addr":"172.21.15.115:6813/446665227","state":["exists","up"]},{"osd":5,"uuid":"c90bcaed-e7a8-4852-8677-b01dc73ea44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6809","nonce":1187293668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6811","nonce":1187293668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6815","nonce":1187293668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6813","nonce":1187293668}]},"public_addr":"172.21.15.195:6809/1187293668","cluster_addr":"172.21.15.195:6811/1187293668","heartbeat_back_addr":"172.21.15.195:6815/1187293668","heartbeat_front_addr":"172.21.15.195:6813/1187293668","state":["exists","up"]},{"osd":6,"uuid":"fd292547-6695-4889-8867-e74318bbb00f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6819","nonce":560343274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6821","nonce":560343274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6825","nonce":560343274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6823","nonce":560343274}]},"public_addr":"172.21.15.64:6819/560343274","cluster_addr":"172.21.15.64:6821/560343274","heartbeat_back_addr":"172.21.15.64:6825/560343274","heartbeat_front_addr":"172.21.15.64:6823/560343274","state":["exists","up"]},{"osd":7,"uuid":"ad7c4151-cd98-4113-879b-ca2b7c83c7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6817","nonce":3075734174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6819","nonce":3075734174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6823","nonce":3075734174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6821","nonce":3075734174}]},"public_addr":"172.21.15.115:6817/3075734174","cluster_addr":"172.21.15.115:6819/3075734174","heartbeat_back_addr":"172.21.15.115:6823/3075734174","heartbeat_front_addr":"172.21.15.115:6821/3075734174","state":["exists","up"]},{"osd":8,"uuid":"6de50049-c28f-4f21-b123-05e55bf59127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6817","nonce":2184742494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6819","nonce":2184742494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6823","nonce":2184742494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6821","nonce":2184742494}]},"public_addr":"172.21.15.195:6817/2184742494","cluster_addr":"172.21.15.195:6819/2184742494","heartbeat_back_addr":"172.21.15.195:6823/2184742494","heartbeat_front_addr":"172.21.15.195:6821/2184742494","state":["exists","up"]},{"osd":9,"uuid":"36ab792c-58f2-4beb-9024-7f3d1e3785dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6827","nonce":561728112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6829","nonce":561728112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6833","nonce":561728112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6831","nonce":561728112}]},"public_addr":"172.21.15.64:6827/561728112","cluster_addr":"172.21.15.64:6829/561728112","heartbeat_back_addr":"172.21.15.64:6833/561728112","heartbeat_front_addr":"172.21.15.64:6831/561728112","state":["exists","up"]},{"osd":10,"uuid":"d1f7881c-ec3f-466e-9cea-1d5cb441d970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6825","nonce":2977887627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6827","nonce":2977887627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6831","nonce":2977887627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6829","nonce":2977887627}]},"public_addr":"172.21.15.115:6825/2977887627","cluster_addr":"172.21.15.115:6827/2977887627","heartbeat_back_addr":"172.21.15.115:6831/2977887627","heartbeat_front_addr":"172.21.15.115:6829/2977887627","state":["exists","up"]},{"osd":11,"uuid":"7c46c028-22c7-450f-9927-2144a7a09e9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6825","nonce":591599352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6827","nonce":591599352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6831","nonce":591599352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6829","nonce":591599352}]},"public_addr":"172.21.15.195:6825/591599352","cluster_addr":"172.21.15.195:6827/591599352","heartbeat_back_addr":"172.21.15.195:6831/591599352","heartbeat_front_addr":"172.21.15.195:6829/591599352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:29.151454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:54.308909+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:18.680995+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:44.687548+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:10.166131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:35.026132+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:24:00.242966+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:24:24.600848+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:24:49.563060+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:25:15.025730+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:25:39.830995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:26:04.394276+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.14","mappings":[{"from":1,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1537191801":"2024-09-13T03:29:15.597759+0000","172.21.15.64:0/686137382":"2024-09-13T03:19:36.839431+0000","172.21.15.64:6800/3186640984":"2024-09-13T03:19:36.839431+0000","172.21.15.64:0/3334629354":"2024-09-13T03:20:07.411102+0000","172.21.15.64:6801/759931848":"2024-09-13T03:29:15.597759+0000","172.21.15.64:0/3079679964":"2024-09-13T03:19:36.839431+0000","172.21.15.64:0/1217136221":"2024-09-13T03:20:07.411102+0000","172.21.15.64:6800/68603379":"2024-09-13T03:20:07.411102+0000","172.21.15.64:6801/3186640984":"2024-09-13T03:19:36.839431+0000","172.21.15.64:0/3827202979":"2024-09-13T03:20:07.411102+0000","172.21.15.64:6800/759931848":"2024-09-13T03:29:15.597759+0000","172.21.15.64:0/3564418115":"2024-09-13T03:19:36.839431+0000","172.21.15.64:6801/68603379":"2024-09-13T03:20:07.411102+0000","172.21.15.64:0/3764771090":"2024-09-13T03:29:15.597759+0000","172.21.15.64:0/1757703005":"2024-09-13T03:29:15.597759+0000","172.21.15.64:0/3775732799":"2024-09-13T03:29:15.597759+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:40:25.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.786+0000 7f41a1349640 1 -- 172.21.15.64:0/121196740 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f41880a2d00 msgr2=0x7f41880a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:25.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.786+0000 7f41a1349640 1 --2- 172.21.15.64:0/121196740 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f41880a2d00 0x7f41880a51c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f418400bbe0 tx=0x7f418400b390 comp rx=0 tx=0).stop 2024-09-12T03:40:25.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.786+0000 7f41a1349640 1 -- 172.21.15.64:0/121196740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f419c106990 msgr2=0x7f419c19cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:25.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.786+0000 7f41a1349640 1 --2- 172.21.15.64:0/121196740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f419c106990 0x7f419c19cc80 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f4194005cc0 tx=0x7f4194003c20 comp rx=0 tx=0).stop 2024-09-12T03:40:25.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.786+0000 7f41a1349640 1 -- 172.21.15.64:0/121196740 shutdown_connections 2024-09-12T03:40:25.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.786+0000 7f41a1349640 1 --2- 172.21.15.64:0/121196740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f419c106990 0x7f419c19cc80 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:25.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.786+0000 7f41a1349640 1 --2- 172.21.15.64:0/121196740 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f419c101320 0x7f419c198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:25.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.786+0000 7f41a1349640 1 --2- 172.21.15.64:0/121196740 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f41880a2d00 0x7f41880a51c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:25.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.786+0000 7f41a1349640 1 --2- 172.21.15.64:0/121196740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f419c100980 0x7f419c1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:25.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.786+0000 7f41a1349640 1 -- 172.21.15.64:0/121196740 >> 172.21.15.64:0/121196740 conn(0x7f419c077a10 msgr2=0x7f419c069730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:40:25.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.786+0000 7f41a1349640 1 -- 172.21.15.64:0/121196740 shutdown_connections 2024-09-12T03:40:25.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.786+0000 7f41a1349640 1 -- 172.21.15.64:0/121196740 wait complete. 2024-09-12T03:40:25.799 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T03:22:21.915870+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-12T03:26:55.826348+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-12T03:26:57.984319+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 102, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-12T03:27:04.460292+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 102, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-12T03:40:25.799 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-12T03:40:25.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:25 smithi115 ceph-mon[34801]: pgmap v347: 129 pgs: 129 active+clean; 5.2 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-12T03:40:25.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.906+0000 7f16ed4ac640 1 -- 172.21.15.64:0/3181472367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16e81009b0 msgr2=0x7f16e8100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:25.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.906+0000 7f16ed4ac640 1 --2- 172.21.15.64:0/3181472367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16e81009b0 0x7f16e8100e10 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f16d0009930 tx=0x7f16d002a420 comp rx=0 tx=0).stop 2024-09-12T03:40:25.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.906+0000 7f16ed4ac640 1 -- 172.21.15.64:0/3181472367 shutdown_connections 2024-09-12T03:40:25.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.906+0000 7f16ed4ac640 1 --2- 172.21.15.64:0/3181472367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16e8101350 0x7f16e810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:25.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.906+0000 7f16ed4ac640 1 --2- 172.21.15.64:0/3181472367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16e81009b0 0x7f16e8100e10 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:25.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.906+0000 7f16ed4ac640 1 --2- 172.21.15.64:0/3181472367 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f16e81069c0 0x7f16e8106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:25.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.906+0000 7f16ed4ac640 1 -- 172.21.15.64:0/3181472367 >> 172.21.15.64:0/3181472367 conn(0x7f16e80fc690 msgr2=0x7f16e80feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:40:25.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.906+0000 7f16ed4ac640 1 -- 172.21.15.64:0/3181472367 shutdown_connections 2024-09-12T03:40:25.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.906+0000 7f16ed4ac640 1 -- 172.21.15.64:0/3181472367 wait complete. 2024-09-12T03:40:25.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.906+0000 7f16ed4ac640 1 Processor -- start 2024-09-12T03:40:25.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.907+0000 7f16ed4ac640 1 -- start start 2024-09-12T03:40:25.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.907+0000 7f16ed4ac640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16e81009b0 0x7f16e80fcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:25.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.907+0000 7f16ed4ac640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f16e8101350 0x7f16e80fd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:25.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.907+0000 7f16ed4ac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16e81069c0 0x7f16e80fd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:25.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.907+0000 7f16ed4ac640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f16e8110cb0 con 0x7f16e81069c0 2024-09-12T03:40:25.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.907+0000 7f16ed4ac640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f16e8110b30 con 0x7f16e81009b0 2024-09-12T03:40:25.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.907+0000 7f16ed4ac640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f16e8110e30 con 0x7f16e8101350 2024-09-12T03:40:25.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.907+0000 7f16e77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16e81069c0 0x7f16e80fd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:25.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.907+0000 7f16e77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16e81069c0 0x7f16e80fd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59468/0 (socket says 172.21.15.64:59468) 2024-09-12T03:40:25.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.907+0000 7f16e77fe640 1 -- 172.21.15.64:0/1931036010 learned_addr learned my addr 172.21.15.64:0/1931036010 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:40:25.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.907+0000 7f16e6ffd640 1 --2- 172.21.15.64:0/1931036010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16e81009b0 0x7f16e80fcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:25.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.907+0000 7f16e67fc640 1 --2- 172.21.15.64:0/1931036010 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f16e8101350 0x7f16e80fd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:25.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.907+0000 7f16e77fe640 1 -- 172.21.15.64:0/1931036010 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f16e8101350 msgr2=0x7f16e80fd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:25.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.907+0000 7f16e77fe640 1 --2- 172.21.15.64:0/1931036010 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f16e8101350 0x7f16e80fd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:25.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.907+0000 7f16e77fe640 1 -- 172.21.15.64:0/1931036010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16e81009b0 msgr2=0x7f16e80fcdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:25.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.907+0000 7f16e77fe640 1 --2- 172.21.15.64:0/1931036010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16e81009b0 0x7f16e80fcdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:25.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.907+0000 7f16e77fe640 1 -- 172.21.15.64:0/1931036010 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e81a6950 con 0x7f16e81069c0 2024-09-12T03:40:25.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.907+0000 7f16e67fc640 1 --2- 172.21.15.64:0/1931036010 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f16e8101350 0x7f16e80fd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:40:25.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.907+0000 7f16e77fe640 1 --2- 172.21.15.64:0/1931036010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16e81069c0 0x7f16e80fd870 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f16d8009fd0 tx=0x7f16d800def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:40:25.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.908+0000 7f16dffff640 1 -- 172.21.15.64:0/1931036010 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d8017070 con 0x7f16e81069c0 2024-09-12T03:40:25.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.908+0000 7f16ed4ac640 1 -- 172.21.15.64:0/1931036010 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16e81a6c40 con 0x7f16e81069c0 2024-09-12T03:40:25.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.908+0000 7f16dffff640 1 -- 172.21.15.64:0/1931036010 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16d8003e20 con 0x7f16e81069c0 2024-09-12T03:40:25.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.908+0000 7f16dffff640 1 -- 172.21.15.64:0/1931036010 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d8010460 con 0x7f16e81069c0 2024-09-12T03:40:25.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.908+0000 7f16ed4ac640 1 -- 172.21.15.64:0/1931036010 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16e81a7140 con 0x7f16e81069c0 2024-09-12T03:40:25.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.909+0000 7f16ed4ac640 1 -- 172.21.15.64:0/1931036010 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16e8100e10 con 0x7f16e81069c0 2024-09-12T03:40:25.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.911+0000 7f16dffff640 1 -- 172.21.15.64:0/1931036010 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f16d8003970 con 0x7f16e81069c0 2024-09-12T03:40:25.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.911+0000 7f16dffff640 1 --2- 172.21.15.64:0/1931036010 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f16d40a2d00 0x7f16d40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:25.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.911+0000 7f16dffff640 1 -- 172.21.15.64:0/1931036010 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9583+0+0 (secure 0 0 0) 0x7f16d80cf4e0 con 0x7f16e81069c0 2024-09-12T03:40:25.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.911+0000 7f16e6ffd640 1 --2- 172.21.15.64:0/1931036010 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f16d40a2d00 0x7f16d40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:25.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.912+0000 7f16e6ffd640 1 --2- 172.21.15.64:0/1931036010 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f16d40a2d00 0x7f16d40a51c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f16e0009990 tx=0x7f16e0009110 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:40:25.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:25.914+0000 7f16dffff640 1 -- 172.21.15.64:0/1931036010 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f16d809bce0 con 0x7f16e81069c0 2024-09-12T03:40:25.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:25 smithi195 ceph-mon[34786]: pgmap v347: 129 pgs: 129 active+clean; 5.2 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-12T03:40:26.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.092+0000 7f16ed4ac640 1 -- 172.21.15.64:0/1931036010 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f16e8109f50 con 0x7f16e81069c0 2024-09-12T03:40:26.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.092+0000 7f16dffff640 1 -- 172.21.15.64:0/1931036010 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) ==== 93+0+10 (secure 0 0 0) 0x7f16d809b5e0 con 0x7f16e81069c0 2024-09-12T03:40:26.093 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-09-12T03:40:26.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.094+0000 7f16ed4ac640 1 -- 172.21.15.64:0/1931036010 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f16d40a2d00 msgr2=0x7f16d40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:26.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.094+0000 7f16ed4ac640 1 --2- 172.21.15.64:0/1931036010 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f16d40a2d00 0x7f16d40a51c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f16e0009990 tx=0x7f16e0009110 comp rx=0 tx=0).stop 2024-09-12T03:40:26.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.094+0000 7f16ed4ac640 1 -- 172.21.15.64:0/1931036010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16e81069c0 msgr2=0x7f16e80fd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:26.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.094+0000 7f16ed4ac640 1 --2- 172.21.15.64:0/1931036010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16e81069c0 0x7f16e80fd870 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f16d8009fd0 tx=0x7f16d800def0 comp rx=0 tx=0).stop 2024-09-12T03:40:26.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.094+0000 7f16ed4ac640 1 -- 172.21.15.64:0/1931036010 shutdown_connections 2024-09-12T03:40:26.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.094+0000 7f16ed4ac640 1 --2- 172.21.15.64:0/1931036010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16e81069c0 0x7f16e80fd870 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:26.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.094+0000 7f16ed4ac640 1 --2- 172.21.15.64:0/1931036010 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f16e8101350 0x7f16e80fd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:26.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.094+0000 7f16ed4ac640 1 --2- 172.21.15.64:0/1931036010 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f16d40a2d00 0x7f16d40a51c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:26.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.094+0000 7f16ed4ac640 1 --2- 172.21.15.64:0/1931036010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16e81009b0 0x7f16e80fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:26.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.094+0000 7f16ed4ac640 1 -- 172.21.15.64:0/1931036010 >> 172.21.15.64:0/1931036010 conn(0x7f16e80fc690 msgr2=0x7f16e806a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:40:26.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.094+0000 7f16ed4ac640 1 -- 172.21.15.64:0/1931036010 shutdown_connections 2024-09-12T03:40:26.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.094+0000 7f16ed4ac640 1 -- 172.21.15.64:0/1931036010 wait complete. 2024-09-12T03:40:26.106 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-12T03:40:26.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.212+0000 7fc395bcd640 1 -- 172.21.15.64:0/3245181305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc390103600 msgr2=0x7fc39010bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:26.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.212+0000 7fc395bcd640 1 --2- 172.21.15.64:0/3245181305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc390103600 0x7fc39010bae0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc384009900 tx=0x7fc38402a480 comp rx=0 tx=0).stop 2024-09-12T03:40:26.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.212+0000 7fc395bcd640 1 -- 172.21.15.64:0/3245181305 shutdown_connections 2024-09-12T03:40:26.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.212+0000 7fc395bcd640 1 --2- 172.21.15.64:0/3245181305 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc39010c0b0 0x7fc39010e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:26.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.212+0000 7fc395bcd640 1 --2- 172.21.15.64:0/3245181305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc390103600 0x7fc39010bae0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:26.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.212+0000 7fc395bcd640 1 --2- 172.21.15.64:0/3245181305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc390102ce0 0x7fc3901030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:26.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.212+0000 7fc395bcd640 1 -- 172.21.15.64:0/3245181305 >> 172.21.15.64:0/3245181305 conn(0x7fc3900fc690 msgr2=0x7fc3900feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:40:26.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.212+0000 7fc395bcd640 1 -- 172.21.15.64:0/3245181305 shutdown_connections 2024-09-12T03:40:26.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.212+0000 7fc395bcd640 1 -- 172.21.15.64:0/3245181305 wait complete. 2024-09-12T03:40:26.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.213+0000 7fc395bcd640 1 Processor -- start 2024-09-12T03:40:26.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.213+0000 7fc395bcd640 1 -- start start 2024-09-12T03:40:26.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.213+0000 7fc395bcd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc390102ce0 0x7fc3901982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:26.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.213+0000 7fc395bcd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc390103600 0x7fc390198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:26.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.213+0000 7fc395bcd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc39010c0b0 0x7fc39019cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:26.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.213+0000 7fc395bcd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc390111d60 con 0x7fc39010c0b0 2024-09-12T03:40:26.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.213+0000 7fc395bcd640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc390111be0 con 0x7fc390103600 2024-09-12T03:40:26.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.213+0000 7fc395bcd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc390111ee0 con 0x7fc390102ce0 2024-09-12T03:40:26.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.214+0000 7fc38ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc39010c0b0 0x7fc39019cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:26.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.214+0000 7fc38ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc39010c0b0 0x7fc39019cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59486/0 (socket says 172.21.15.64:59486) 2024-09-12T03:40:26.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.214+0000 7fc38ffff640 1 -- 172.21.15.64:0/2949056155 learned_addr learned my addr 172.21.15.64:0/2949056155 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:40:26.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.214+0000 7fc38effd640 1 --2- 172.21.15.64:0/2949056155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc390103600 0x7fc390198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:26.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.214+0000 7fc38f7fe640 1 --2- 172.21.15.64:0/2949056155 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc390102ce0 0x7fc3901982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:26.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.214+0000 7fc38ffff640 1 -- 172.21.15.64:0/2949056155 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc390102ce0 msgr2=0x7fc3901982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:26.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.214+0000 7fc38ffff640 1 --2- 172.21.15.64:0/2949056155 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc390102ce0 0x7fc3901982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:26.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.214+0000 7fc38ffff640 1 -- 172.21.15.64:0/2949056155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc390103600 msgr2=0x7fc390198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:26.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.214+0000 7fc38ffff640 1 --2- 172.21.15.64:0/2949056155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc390103600 0x7fc390198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:26.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.214+0000 7fc38ffff640 1 -- 172.21.15.64:0/2949056155 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc39019d2c0 con 0x7fc39010c0b0 2024-09-12T03:40:26.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.214+0000 7fc38ffff640 1 --2- 172.21.15.64:0/2949056155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc39010c0b0 0x7fc39019cb90 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc38800eaa0 tx=0x7fc38800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:40:26.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.214+0000 7fc38cff9640 1 -- 172.21.15.64:0/2949056155 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc38800cdb0 con 0x7fc39010c0b0 2024-09-12T03:40:26.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.214+0000 7fc38cff9640 1 -- 172.21.15.64:0/2949056155 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc388005020 con 0x7fc39010c0b0 2024-09-12T03:40:26.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.214+0000 7fc395bcd640 1 -- 172.21.15.64:0/2949056155 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc39019d5b0 con 0x7fc39010c0b0 2024-09-12T03:40:26.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.214+0000 7fc38cff9640 1 -- 172.21.15.64:0/2949056155 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3880116d0 con 0x7fc39010c0b0 2024-09-12T03:40:26.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.214+0000 7fc395bcd640 1 -- 172.21.15.64:0/2949056155 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3901a4e40 con 0x7fc39010c0b0 2024-09-12T03:40:26.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.215+0000 7fc395bcd640 1 -- 172.21.15.64:0/2949056155 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3901030c0 con 0x7fc39010c0b0 2024-09-12T03:40:26.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.217+0000 7fc38cff9640 1 -- 172.21.15.64:0/2949056155 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc3880051d0 con 0x7fc39010c0b0 2024-09-12T03:40:26.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.218+0000 7fc38cff9640 1 --2- 172.21.15.64:0/2949056155 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fc3780a2d00 0x7fc3780a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:26.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.218+0000 7fc38cff9640 1 -- 172.21.15.64:0/2949056155 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9583+0+0 (secure 0 0 0) 0x7fc3880cfe50 con 0x7fc39010c0b0 2024-09-12T03:40:26.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.218+0000 7fc38f7fe640 1 --2- 172.21.15.64:0/2949056155 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fc3780a2d00 0x7fc3780a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:26.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.219+0000 7fc38f7fe640 1 --2- 172.21.15.64:0/2949056155 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fc3780a2d00 0x7fc3780a51c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc380005e10 tx=0x7fc380005da0 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:40:26.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.221+0000 7fc38cff9640 1 -- 172.21.15.64:0/2949056155 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc388016030 con 0x7fc39010c0b0 2024-09-12T03:40:26.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.399+0000 7fc395bcd640 1 -- 172.21.15.64:0/2949056155 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc39002d7c0 con 0x7fc39010c0b0 2024-09-12T03:40:26.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.399+0000 7fc38cff9640 1 -- 172.21.15.64:0/2949056155 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v103) ==== 104+0+11 (secure 0 0 0) 0x7fc38809c650 con 0x7fc39010c0b0 2024-09-12T03:40:26.400 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-09-12T03:40:26.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.401+0000 7fc395bcd640 1 -- 172.21.15.64:0/2949056155 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fc3780a2d00 msgr2=0x7fc3780a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:26.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.401+0000 7fc395bcd640 1 --2- 172.21.15.64:0/2949056155 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fc3780a2d00 0x7fc3780a51c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc380005e10 tx=0x7fc380005da0 comp rx=0 tx=0).stop 2024-09-12T03:40:26.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.401+0000 7fc395bcd640 1 -- 172.21.15.64:0/2949056155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc39010c0b0 msgr2=0x7fc39019cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:26.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.401+0000 7fc395bcd640 1 --2- 172.21.15.64:0/2949056155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc39010c0b0 0x7fc39019cb90 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc38800eaa0 tx=0x7fc38800ef60 comp rx=0 tx=0).stop 2024-09-12T03:40:26.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.401+0000 7fc395bcd640 1 -- 172.21.15.64:0/2949056155 shutdown_connections 2024-09-12T03:40:26.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.401+0000 7fc395bcd640 1 --2- 172.21.15.64:0/2949056155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc39010c0b0 0x7fc39019cb90 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:26.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.401+0000 7fc395bcd640 1 --2- 172.21.15.64:0/2949056155 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fc3780a2d00 0x7fc3780a51c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:26.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.401+0000 7fc395bcd640 1 --2- 172.21.15.64:0/2949056155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc390103600 0x7fc390198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:26.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.401+0000 7fc395bcd640 1 --2- 172.21.15.64:0/2949056155 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc390102ce0 0x7fc3901982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:26.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.401+0000 7fc395bcd640 1 -- 172.21.15.64:0/2949056155 >> 172.21.15.64:0/2949056155 conn(0x7fc3900fc690 msgr2=0x7fc3900fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:40:26.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.402+0000 7fc395bcd640 1 -- 172.21.15.64:0/2949056155 shutdown_connections 2024-09-12T03:40:26.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.402+0000 7fc395bcd640 1 -- 172.21.15.64:0/2949056155 wait complete. 2024-09-12T03:40:26.413 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-12T03:40:26.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.520+0000 7f2b1541e640 1 -- 172.21.15.64:0/2613647017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b10101350 msgr2=0x7f2b1010d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:26.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.520+0000 7f2b1541e640 1 --2- 172.21.15.64:0/2613647017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b10101350 0x7f2b1010d880 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f2b08009cd0 tx=0x7f2b0802a8e0 comp rx=0 tx=0).stop 2024-09-12T03:40:26.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.520+0000 7f2b1541e640 1 -- 172.21.15.64:0/2613647017 shutdown_connections 2024-09-12T03:40:26.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.520+0000 7f2b1541e640 1 --2- 172.21.15.64:0/2613647017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b10101350 0x7f2b1010d880 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:26.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.520+0000 7f2b1541e640 1 --2- 172.21.15.64:0/2613647017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b101009b0 0x7f2b10100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:26.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.520+0000 7f2b1541e640 1 --2- 172.21.15.64:0/2613647017 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b101069c0 0x7f2b10106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:26.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.520+0000 7f2b1541e640 1 -- 172.21.15.64:0/2613647017 >> 172.21.15.64:0/2613647017 conn(0x7f2b100fc690 msgr2=0x7f2b100feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:40:26.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.520+0000 7f2b1541e640 1 -- 172.21.15.64:0/2613647017 shutdown_connections 2024-09-12T03:40:26.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.520+0000 7f2b1541e640 1 -- 172.21.15.64:0/2613647017 wait complete. 2024-09-12T03:40:26.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.520+0000 7f2b1541e640 1 Processor -- start 2024-09-12T03:40:26.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.521+0000 7f2b1541e640 1 -- start start 2024-09-12T03:40:26.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.521+0000 7f2b1541e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b101009b0 0x7f2b10196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:26.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.521+0000 7f2b1541e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b10101350 0x7f2b101974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:26.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.521+0000 7f2b1541e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b101069c0 0x7f2b101979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:26.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.521+0000 7f2b1541e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2b10110cb0 con 0x7f2b10101350 2024-09-12T03:40:26.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.521+0000 7f2b1541e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2b10110b30 con 0x7f2b101009b0 2024-09-12T03:40:26.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.521+0000 7f2b1541e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2b10110e30 con 0x7f2b101069c0 2024-09-12T03:40:26.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.521+0000 7f2b0e7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b10101350 0x7f2b101974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:26.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.521+0000 7f2b0e7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b10101350 0x7f2b101974b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59492/0 (socket says 172.21.15.64:59492) 2024-09-12T03:40:26.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.521+0000 7f2b0e7fc640 1 -- 172.21.15.64:0/770743821 learned_addr learned my addr 172.21.15.64:0/770743821 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:40:26.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.521+0000 7f2b0f7fe640 1 --2- 172.21.15.64:0/770743821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b101069c0 0x7f2b101979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:26.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.521+0000 7f2b0effd640 1 --2- 172.21.15.64:0/770743821 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b101009b0 0x7f2b10196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:26.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.521+0000 7f2b0e7fc640 1 -- 172.21.15.64:0/770743821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b101069c0 msgr2=0x7f2b101979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:26.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.521+0000 7f2b0e7fc640 1 --2- 172.21.15.64:0/770743821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b101069c0 0x7f2b101979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:26.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.521+0000 7f2b0e7fc640 1 -- 172.21.15.64:0/770743821 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b101009b0 msgr2=0x7f2b10196f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:26.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.521+0000 7f2b0e7fc640 1 --2- 172.21.15.64:0/770743821 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b101009b0 0x7f2b10196f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:26.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.521+0000 7f2b0e7fc640 1 -- 172.21.15.64:0/770743821 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b101a67e0 con 0x7f2b10101350 2024-09-12T03:40:26.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.521+0000 7f2b0effd640 1 --2- 172.21.15.64:0/770743821 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b101009b0 0x7f2b10196f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:40:26.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.521+0000 7f2b0f7fe640 1 --2- 172.21.15.64:0/770743821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b101069c0 0x7f2b101979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:40:26.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.521+0000 7f2b0e7fc640 1 --2- 172.21.15.64:0/770743821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b10101350 0x7f2b101974b0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f2b0400d950 tx=0x7f2b0400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:40:26.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.522+0000 7f2af7fff640 1 -- 172.21.15.64:0/770743821 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b04017070 con 0x7f2b10101350 2024-09-12T03:40:26.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.522+0000 7f2b1541e640 1 -- 172.21.15.64:0/770743821 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b101a6ad0 con 0x7f2b10101350 2024-09-12T03:40:26.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.522+0000 7f2af7fff640 1 -- 172.21.15.64:0/770743821 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b04009c30 con 0x7f2b10101350 2024-09-12T03:40:26.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.522+0000 7f2af7fff640 1 -- 172.21.15.64:0/770743821 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0400f620 con 0x7f2b10101350 2024-09-12T03:40:26.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.522+0000 7f2b1541e640 1 -- 172.21.15.64:0/770743821 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b101a7020 con 0x7f2b10101350 2024-09-12T03:40:26.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.523+0000 7f2b1541e640 1 -- 172.21.15.64:0/770743821 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b10100e10 con 0x7f2b10101350 2024-09-12T03:40:26.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.524+0000 7f2af7fff640 1 -- 172.21.15.64:0/770743821 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f2b040109a0 con 0x7f2b10101350 2024-09-12T03:40:26.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.525+0000 7f2af7fff640 1 --2- 172.21.15.64:0/770743821 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f2af80a2d00 0x7f2af80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:26.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.525+0000 7f2b0effd640 1 --2- 172.21.15.64:0/770743821 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f2af80a2d00 0x7f2af80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:26.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.525+0000 7f2af7fff640 1 -- 172.21.15.64:0/770743821 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9583+0+0 (secure 0 0 0) 0x7f2b040cf550 con 0x7f2b10101350 2024-09-12T03:40:26.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.525+0000 7f2b0effd640 1 --2- 172.21.15.64:0/770743821 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f2af80a2d00 0x7f2af80a51c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2b00005fd0 tx=0x7f2b000094d0 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:40:26.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.529+0000 7f2af7fff640 1 -- 172.21.15.64:0/770743821 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2b04009de0 con 0x7f2b10101350 2024-09-12T03:40:26.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.706+0000 7f2b1541e640 1 -- 172.21.15.64:0/770743821 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2b1002d7c0 con 0x7f2b10101350 2024-09-12T03:40:26.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.707+0000 7f2af7fff640 1 -- 172.21.15.64:0/770743821 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v103) ==== 100+0+11 (secure 0 0 0) 0x7f2b0409bd50 con 0x7f2b10101350 2024-09-12T03:40:26.708 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-09-12T03:40:26.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.709+0000 7f2b1541e640 1 -- 172.21.15.64:0/770743821 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f2af80a2d00 msgr2=0x7f2af80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:26.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.709+0000 7f2b1541e640 1 --2- 172.21.15.64:0/770743821 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f2af80a2d00 0x7f2af80a51c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2b00005fd0 tx=0x7f2b000094d0 comp rx=0 tx=0).stop 2024-09-12T03:40:26.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.709+0000 7f2b1541e640 1 -- 172.21.15.64:0/770743821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b10101350 msgr2=0x7f2b101974b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:26.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.709+0000 7f2b1541e640 1 --2- 172.21.15.64:0/770743821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b10101350 0x7f2b101974b0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f2b0400d950 tx=0x7f2b0400de10 comp rx=0 tx=0).stop 2024-09-12T03:40:26.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.709+0000 7f2b1541e640 1 -- 172.21.15.64:0/770743821 shutdown_connections 2024-09-12T03:40:26.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.709+0000 7f2b1541e640 1 --2- 172.21.15.64:0/770743821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b101069c0 0x7f2b101979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:26.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.709+0000 7f2b1541e640 1 --2- 172.21.15.64:0/770743821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b10101350 0x7f2b101974b0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:26.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.709+0000 7f2b1541e640 1 --2- 172.21.15.64:0/770743821 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f2af80a2d00 0x7f2af80a51c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:26.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.709+0000 7f2b1541e640 1 --2- 172.21.15.64:0/770743821 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b101009b0 0x7f2b10196f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:26.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.709+0000 7f2b1541e640 1 -- 172.21.15.64:0/770743821 >> 172.21.15.64:0/770743821 conn(0x7f2b100fc690 msgr2=0x7f2b100fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:40:26.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.709+0000 7f2b1541e640 1 -- 172.21.15.64:0/770743821 shutdown_connections 2024-09-12T03:40:26.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.709+0000 7f2b1541e640 1 -- 172.21.15.64:0/770743821 wait complete. 2024-09-12T03:40:26.720 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:26 smithi064 ceph-mon[30952]: from='client.16254 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:40:26.720 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:26 smithi064 ceph-mon[30952]: from='client.16260 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:40:26.720 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:26 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/121196740' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:40:26.720 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:26 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/1931036010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:40:26.720 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:26 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/2949056155' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T03:40:26.722 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-12T03:40:26.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.827+0000 7f8607fff640 1 -- 172.21.15.64:0/4246831294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8608101220 msgr2=0x7f860810d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:26.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.827+0000 7f8607fff640 1 --2- 172.21.15.64:0/4246831294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8608101220 0x7f860810d750 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f8600009cd0 tx=0x7f860002a8e0 comp rx=0 tx=0).stop 2024-09-12T03:40:26.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.828+0000 7f8607fff640 1 -- 172.21.15.64:0/4246831294 shutdown_connections 2024-09-12T03:40:26.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.828+0000 7f8607fff640 1 --2- 172.21.15.64:0/4246831294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8608101220 0x7f860810d750 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:26.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.828+0000 7f8607fff640 1 --2- 172.21.15.64:0/4246831294 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8608100880 0x7f8608100ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:26.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.828+0000 7f8607fff640 1 --2- 172.21.15.64:0/4246831294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8608106890 0x7f8608106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:26.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.828+0000 7f8607fff640 1 -- 172.21.15.64:0/4246831294 >> 172.21.15.64:0/4246831294 conn(0x7f86080fc540 msgr2=0x7f86080fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:40:26.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.828+0000 7f8607fff640 1 -- 172.21.15.64:0/4246831294 shutdown_connections 2024-09-12T03:40:26.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.828+0000 7f8607fff640 1 -- 172.21.15.64:0/4246831294 wait complete. 2024-09-12T03:40:26.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.828+0000 7f8607fff640 1 Processor -- start 2024-09-12T03:40:26.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.828+0000 7f8607fff640 1 -- start start 2024-09-12T03:40:26.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.829+0000 7f8607fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8608100880 0x7f86080fcc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:26.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.829+0000 7f8607fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8608101220 0x7f86080fd170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:26.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.829+0000 7f8607fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8608106890 0x7f86080fd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:26.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.829+0000 7f8607fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8608110b80 con 0x7f8608101220 2024-09-12T03:40:26.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.829+0000 7f8607fff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8608110a00 con 0x7f8608106890 2024-09-12T03:40:26.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.829+0000 7f8607fff640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8608110d00 con 0x7f8608100880 2024-09-12T03:40:26.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.829+0000 7f86067fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8608101220 0x7f86080fd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:26.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.829+0000 7f86067fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8608101220 0x7f86080fd170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59508/0 (socket says 172.21.15.64:59508) 2024-09-12T03:40:26.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.829+0000 7f86067fc640 1 -- 172.21.15.64:0/1979083624 learned_addr learned my addr 172.21.15.64:0/1979083624 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:40:26.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.829+0000 7f8606ffd640 1 --2- 172.21.15.64:0/1979083624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8608100880 0x7f86080fcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:26.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.829+0000 7f86077fe640 1 --2- 172.21.15.64:0/1979083624 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8608106890 0x7f86080fd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:26.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.829+0000 7f86067fc640 1 -- 172.21.15.64:0/1979083624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8608100880 msgr2=0x7f86080fcc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:26.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.829+0000 7f86067fc640 1 --2- 172.21.15.64:0/1979083624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8608100880 0x7f86080fcc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:26.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.829+0000 7f86067fc640 1 -- 172.21.15.64:0/1979083624 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8608106890 msgr2=0x7f86080fd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:26.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.829+0000 7f86067fc640 1 --2- 172.21.15.64:0/1979083624 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8608106890 0x7f86080fd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:26.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.829+0000 7f86067fc640 1 -- 172.21.15.64:0/1979083624 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86081a6690 con 0x7f8608101220 2024-09-12T03:40:26.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.829+0000 7f86067fc640 1 --2- 172.21.15.64:0/1979083624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8608101220 0x7f86080fd170 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f85fc00c970 tx=0x7f85fc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:40:26.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.830+0000 7f85effff640 1 -- 172.21.15.64:0/1979083624 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85fc007bb0 con 0x7f8608101220 2024-09-12T03:40:26.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.830+0000 7f85effff640 1 -- 172.21.15.64:0/1979083624 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85fc007d50 con 0x7f8608101220 2024-09-12T03:40:26.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.830+0000 7f85effff640 1 -- 172.21.15.64:0/1979083624 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85fc00f450 con 0x7f8608101220 2024-09-12T03:40:26.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.830+0000 7f8607fff640 1 -- 172.21.15.64:0/1979083624 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86081a6980 con 0x7f8608101220 2024-09-12T03:40:26.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.830+0000 7f8607fff640 1 -- 172.21.15.64:0/1979083624 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86081a6e80 con 0x7f8608101220 2024-09-12T03:40:26.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.831+0000 7f8607fff640 1 -- 172.21.15.64:0/1979083624 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8608100ce0 con 0x7f8608101220 2024-09-12T03:40:26.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.832+0000 7f85effff640 1 -- 172.21.15.64:0/1979083624 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f85fc004f20 con 0x7f8608101220 2024-09-12T03:40:26.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.832+0000 7f85effff640 1 --2- 172.21.15.64:0/1979083624 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f85f00a2d00 0x7f85f00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:26.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.833+0000 7f8606ffd640 1 --2- 172.21.15.64:0/1979083624 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f85f00a2d00 0x7f85f00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:26.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.833+0000 7f85effff640 1 -- 172.21.15.64:0/1979083624 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9583+0+0 (secure 0 0 0) 0x7f85fc0cf540 con 0x7f8608101220 2024-09-12T03:40:26.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.833+0000 7f8606ffd640 1 --2- 172.21.15.64:0/1979083624 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f85f00a2d00 0x7f85f00a51c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f85f80058c0 tx=0x7f85f800a2d0 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:40:26.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:26.838+0000 7f85effff640 1 -- 172.21.15.64:0/1979083624 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f85fc014030 con 0x7f8608101220 2024-09-12T03:40:26.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:26 smithi115 ceph-mon[34801]: from='client.16254 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:40:26.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:26 smithi115 ceph-mon[34801]: from='client.16260 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:40:26.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:26 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/121196740' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:40:26.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:26 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/1931036010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:40:26.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:26 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/2949056155' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T03:40:26.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:26 smithi195 ceph-mon[34786]: from='client.16254 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:40:26.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:26 smithi195 ceph-mon[34786]: from='client.16260 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:40:26.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:26 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/121196740' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:40:26.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:26 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/1931036010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:40:26.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:26 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/2949056155' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T03:40:27.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.014+0000 7f8607fff640 1 -- 172.21.15.64:0/1979083624 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f8608109e00 con 0x7f8608101220 2024-09-12T03:40:27.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.014+0000 7f85effff640 1 -- 172.21.15.64:0/1979083624 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v103) ==== 103+0+11 (secure 0 0 0) 0x7f85fc09b6f0 con 0x7f8608101220 2024-09-12T03:40:27.015 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-09-12T03:40:27.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.016+0000 7f8607fff640 1 -- 172.21.15.64:0/1979083624 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f85f00a2d00 msgr2=0x7f85f00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:27.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.016+0000 7f8607fff640 1 --2- 172.21.15.64:0/1979083624 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f85f00a2d00 0x7f85f00a51c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f85f80058c0 tx=0x7f85f800a2d0 comp rx=0 tx=0).stop 2024-09-12T03:40:27.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.016+0000 7f8607fff640 1 -- 172.21.15.64:0/1979083624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8608101220 msgr2=0x7f86080fd170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:27.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.016+0000 7f8607fff640 1 --2- 172.21.15.64:0/1979083624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8608101220 0x7f86080fd170 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f85fc00c970 tx=0x7f85fc00ce30 comp rx=0 tx=0).stop 2024-09-12T03:40:27.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.016+0000 7f8607fff640 1 -- 172.21.15.64:0/1979083624 shutdown_connections 2024-09-12T03:40:27.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.016+0000 7f8607fff640 1 --2- 172.21.15.64:0/1979083624 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f85f00a2d00 0x7f85f00a51c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:27.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.016+0000 7f8607fff640 1 --2- 172.21.15.64:0/1979083624 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8608106890 0x7f86080fd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:27.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.016+0000 7f8607fff640 1 --2- 172.21.15.64:0/1979083624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8608101220 0x7f86080fd170 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:27.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.016+0000 7f8607fff640 1 --2- 172.21.15.64:0/1979083624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8608100880 0x7f86080fcc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:27.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.016+0000 7f8607fff640 1 -- 172.21.15.64:0/1979083624 >> 172.21.15.64:0/1979083624 conn(0x7f86080fc540 msgr2=0x7f860806a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:40:27.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.016+0000 7f8607fff640 1 -- 172.21.15.64:0/1979083624 shutdown_connections 2024-09-12T03:40:27.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.016+0000 7f8607fff640 1 -- 172.21.15.64:0/1979083624 wait complete. 2024-09-12T03:40:27.028 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-12T03:40:27.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.132+0000 7f69dfaa5640 1 -- 172.21.15.64:0/988714632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69d8107ac0 msgr2=0x7f69d8107ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:27.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.132+0000 7f69dfaa5640 1 --2- 172.21.15.64:0/988714632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69d8107ac0 0x7f69d8107ea0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f69d4009930 tx=0x7f69d402a420 comp rx=0 tx=0).stop 2024-09-12T03:40:27.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.132+0000 7f69dfaa5640 1 -- 172.21.15.64:0/988714632 shutdown_connections 2024-09-12T03:40:27.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.132+0000 7f69dfaa5640 1 --2- 172.21.15.64:0/988714632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69d8102450 0x7f69d810e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:27.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.132+0000 7f69dfaa5640 1 --2- 172.21.15.64:0/988714632 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69d8101ab0 0x7f69d8101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:27.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.132+0000 7f69dfaa5640 1 --2- 172.21.15.64:0/988714632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69d8107ac0 0x7f69d8107ea0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:27.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.132+0000 7f69dfaa5640 1 -- 172.21.15.64:0/988714632 >> 172.21.15.64:0/988714632 conn(0x7f69d80fd7d0 msgr2=0x7f69d80ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:40:27.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.132+0000 7f69dfaa5640 1 -- 172.21.15.64:0/988714632 shutdown_connections 2024-09-12T03:40:27.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.132+0000 7f69dfaa5640 1 -- 172.21.15.64:0/988714632 wait complete. 2024-09-12T03:40:27.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.133+0000 7f69dfaa5640 1 Processor -- start 2024-09-12T03:40:27.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.133+0000 7f69dfaa5640 1 -- start start 2024-09-12T03:40:27.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.133+0000 7f69dfaa5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69d8101ab0 0x7f69d81982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:27.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.133+0000 7f69dfaa5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69d8102450 0x7f69d81987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:27.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.133+0000 7f69dfaa5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69d8107ac0 0x7f69d819cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:27.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.133+0000 7f69dfaa5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f69d8111d70 con 0x7f69d8102450 2024-09-12T03:40:27.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.133+0000 7f69dfaa5640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f69d8111bf0 con 0x7f69d8107ac0 2024-09-12T03:40:27.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.133+0000 7f69dfaa5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f69d8111ef0 con 0x7f69d8101ab0 2024-09-12T03:40:27.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.133+0000 7f69dd040640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69d8102450 0x7f69d81987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:27.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.133+0000 7f69dd040640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69d8102450 0x7f69d81987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59516/0 (socket says 172.21.15.64:59516) 2024-09-12T03:40:27.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.133+0000 7f69dd040640 1 -- 172.21.15.64:0/1154641348 learned_addr learned my addr 172.21.15.64:0/1154641348 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:40:27.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.133+0000 7f69dd841640 1 --2- 172.21.15.64:0/1154641348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69d8101ab0 0x7f69d81982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:27.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.133+0000 7f69de042640 1 --2- 172.21.15.64:0/1154641348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69d8107ac0 0x7f69d819cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:27.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.134+0000 7f69dd040640 1 -- 172.21.15.64:0/1154641348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69d8101ab0 msgr2=0x7f69d81982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:27.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.134+0000 7f69dd040640 1 --2- 172.21.15.64:0/1154641348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69d8101ab0 0x7f69d81982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:27.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.134+0000 7f69dd040640 1 -- 172.21.15.64:0/1154641348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69d8107ac0 msgr2=0x7f69d819cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:27.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.134+0000 7f69dd040640 1 --2- 172.21.15.64:0/1154641348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69d8107ac0 0x7f69d819cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:27.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.134+0000 7f69dd040640 1 -- 172.21.15.64:0/1154641348 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69d819d290 con 0x7f69d8102450 2024-09-12T03:40:27.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.134+0000 7f69de042640 1 --2- 172.21.15.64:0/1154641348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69d8107ac0 0x7f69d819cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:40:27.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.134+0000 7f69dd040640 1 --2- 172.21.15.64:0/1154641348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69d8102450 0x7f69d81987e0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f69c800c920 tx=0x7f69c800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:40:27.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.134+0000 7f69ceffd640 1 -- 172.21.15.64:0/1154641348 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c8007c70 con 0x7f69d8102450 2024-09-12T03:40:27.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.134+0000 7f69ceffd640 1 -- 172.21.15.64:0/1154641348 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69c8007e10 con 0x7f69d8102450 2024-09-12T03:40:27.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.134+0000 7f69ceffd640 1 -- 172.21.15.64:0/1154641348 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c8018820 con 0x7f69d8102450 2024-09-12T03:40:27.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.134+0000 7f69dfaa5640 1 -- 172.21.15.64:0/1154641348 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69d819d580 con 0x7f69d8102450 2024-09-12T03:40:27.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.134+0000 7f69dfaa5640 1 -- 172.21.15.64:0/1154641348 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69d81a4e10 con 0x7f69d8102450 2024-09-12T03:40:27.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.135+0000 7f69dfaa5640 1 -- 172.21.15.64:0/1154641348 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69d8101f10 con 0x7f69d8102450 2024-09-12T03:40:27.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.136+0000 7f69ceffd640 1 -- 172.21.15.64:0/1154641348 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f69c80189c0 con 0x7f69d8102450 2024-09-12T03:40:27.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.137+0000 7f69ceffd640 1 --2- 172.21.15.64:0/1154641348 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f69c40a2d00 0x7f69c40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:27.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.137+0000 7f69ceffd640 1 -- 172.21.15.64:0/1154641348 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9583+0+0 (secure 0 0 0) 0x7f69c80cfc80 con 0x7f69d8102450 2024-09-12T03:40:27.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.137+0000 7f69dd841640 1 --2- 172.21.15.64:0/1154641348 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f69c40a2d00 0x7f69c40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:27.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.137+0000 7f69dd841640 1 --2- 172.21.15.64:0/1154641348 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f69c40a2d00 0x7f69c40a51c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f69d4009900 tx=0x7f69d4000bc0 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:40:27.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.140+0000 7f69ceffd640 1 -- 172.21.15.64:0/1154641348 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f69c8014030 con 0x7f69d8102450 2024-09-12T03:40:27.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.366+0000 7f69dfaa5640 1 -- 172.21.15.64:0/1154641348 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f69d81995f0 con 0x7f69d8102450 2024-09-12T03:40:27.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.374+0000 7f69ceffd640 1 -- 172.21.15.64:0/1154641348 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+740902 (secure 0 0 0) 0x7f69d81995f0 con 0x7f69d8102450 2024-09-12T03:40:27.385 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:40:27.386 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T03:40:27.386 INFO:teuthology.orchestra.run.smithi064.stdout: "epoch": 26, 2024-09-12T03:40:27.386 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.386 INFO:teuthology.orchestra.run.smithi064.stdout: "active_gid": 15243, 2024-09-12T03:40:27.386 INFO:teuthology.orchestra.run.smithi064.stdout: "active_name": "x", 2024-09-12T03:40:27.386 INFO:teuthology.orchestra.run.smithi064.stdout: "active_addrs": { 2024-09-12T03:40:27.386 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2024-09-12T03:40:27.386 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:27.386 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2024-09-12T03:40:27.386 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:6800", 2024-09-12T03:40:27.386 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 3960824462 2024-09-12T03:40:27.387 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.387 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:27.387 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2024-09-12T03:40:27.387 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:6801", 2024-09-12T03:40:27.387 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 3960824462 2024-09-12T03:40:27.387 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.387 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T03:40:27.387 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.387 INFO:teuthology.orchestra.run.smithi064.stdout: "active_addr": "172.21.15.64:6801/3960824462", 2024-09-12T03:40:27.387 INFO:teuthology.orchestra.run.smithi064.stdout: "active_change": "2024-09-12T03:29:15.598000+0000", 2024-09-12T03:40:27.387 INFO:teuthology.orchestra.run.smithi064.stdout: "active_mgr_features": 4540701547738038271, 2024-09-12T03:40:27.387 INFO:teuthology.orchestra.run.smithi064.stdout: "available": true, 2024-09-12T03:40:27.387 INFO:teuthology.orchestra.run.smithi064.stdout: "standbys": [ 2024-09-12T03:40:27.387 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:27.388 INFO:teuthology.orchestra.run.smithi064.stdout: "gid": 24931, 2024-09-12T03:40:27.388 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "y", 2024-09-12T03:40:27.388 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr_features": 4540701547738038271, 2024-09-12T03:40:27.388 INFO:teuthology.orchestra.run.smithi064.stdout: "available_modules": [ 2024-09-12T03:40:27.388 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:27.388 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "alerts", 2024-09-12T03:40:27.388 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:27.388 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:27.388 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:27.388 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-12T03:40:27.388 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-12T03:40:27.389 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:27.389 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.389 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.389 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-12T03:40:27.389 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.389 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.389 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.389 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T03:40:27.389 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.389 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.389 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.389 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.389 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:27.389 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:27.390 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.390 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.390 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.390 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.390 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.390 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.390 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.390 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.390 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.390 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.390 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.390 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.391 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.391 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.391 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.391 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.391 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.391 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.391 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.391 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:27.391 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:27.391 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.391 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.391 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.391 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.391 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.392 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.392 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.392 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.392 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.392 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.392 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.392 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.392 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:27.392 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:27.392 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.392 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.392 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.393 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:27.393 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.393 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.393 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.393 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.393 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.393 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.393 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.393 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.393 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.394 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.394 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.394 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.394 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.394 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.394 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.394 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:27.394 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:27.394 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.394 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.394 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.394 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.394 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.394 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.395 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.395 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.395 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.395 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.395 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.395 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.395 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_destination": { 2024-09-12T03:40:27.395 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_destination", 2024-09-12T03:40:27.395 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.395 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.395 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.395 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.395 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.395 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.396 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.396 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Email address to send alerts to", 2024-09-12T03:40:27.396 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.396 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.396 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.396 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.397 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_from_name": { 2024-09-12T03:40:27.397 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_from_name", 2024-09-12T03:40:27.397 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.397 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.397 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.397 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "Ceph", 2024-09-12T03:40:27.397 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.397 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.397 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.397 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Email From: name", 2024-09-12T03:40:27.397 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.397 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.397 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.398 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.398 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_host": { 2024-09-12T03:40:27.398 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_host", 2024-09-12T03:40:27.398 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.398 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.398 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.398 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.398 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.398 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.398 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.398 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP server", 2024-09-12T03:40:27.399 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.399 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.399 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.399 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.399 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_password": { 2024-09-12T03:40:27.399 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_password", 2024-09-12T03:40:27.399 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.399 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.399 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.399 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.399 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.399 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.400 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.400 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Password to authenticate with", 2024-09-12T03:40:27.400 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.400 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.400 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.400 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.400 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_port": { 2024-09-12T03:40:27.400 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_port", 2024-09-12T03:40:27.400 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.400 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.401 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.401 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "465", 2024-09-12T03:40:27.401 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.401 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.401 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.401 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP port", 2024-09-12T03:40:27.401 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.401 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.401 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.401 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.401 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_sender": { 2024-09-12T03:40:27.401 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_sender", 2024-09-12T03:40:27.401 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.401 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.402 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.402 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.402 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.402 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.402 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.402 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP envelope sender", 2024-09-12T03:40:27.402 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.402 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.402 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.402 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.402 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_ssl": { 2024-09-12T03:40:27.403 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_ssl", 2024-09-12T03:40:27.403 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.403 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.403 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.403 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.403 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.403 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.403 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.403 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T03:40:27.403 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.403 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.403 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.403 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.404 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_user": { 2024-09-12T03:40:27.404 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_user", 2024-09-12T03:40:27.404 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.404 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.404 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.404 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.404 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.404 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.404 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.404 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "User to authenticate as", 2024-09-12T03:40:27.404 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.404 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.404 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.405 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.405 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.405 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.405 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:27.405 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "balancer", 2024-09-12T03:40:27.405 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:27.405 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:27.405 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:27.405 INFO:teuthology.orchestra.run.smithi064.stdout: "active": { 2024-09-12T03:40:27.406 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "active", 2024-09-12T03:40:27.406 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.406 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.406 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.406 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.406 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.406 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.406 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.406 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T03:40:27.406 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.406 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.406 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.406 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.407 INFO:teuthology.orchestra.run.smithi064.stdout: "begin_time": { 2024-09-12T03:40:27.407 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "begin_time", 2024-09-12T03:40:27.407 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.407 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.407 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.407 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0000", 2024-09-12T03:40:27.407 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.407 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.407 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.408 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T03:40:27.408 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:40:27.408 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.408 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.408 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.408 INFO:teuthology.orchestra.run.smithi064.stdout: "begin_weekday": { 2024-09-12T03:40:27.408 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "begin_weekday", 2024-09-12T03:40:27.408 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-12T03:40:27.408 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.408 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.408 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-12T03:40:27.408 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0", 2024-09-12T03:40:27.408 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "6", 2024-09-12T03:40:27.409 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.409 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T03:40:27.409 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:40:27.409 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.409 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.409 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.409 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_max_iterations": { 2024-09-12T03:40:27.409 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_max_iterations", 2024-09-12T03:40:27.409 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-12T03:40:27.409 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.409 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.409 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "25", 2024-09-12T03:40:27.409 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-09-12T03:40:27.410 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "250", 2024-09-12T03:40:27.410 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.410 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T03:40:27.410 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.410 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.410 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.410 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.410 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_metrics": { 2024-09-12T03:40:27.410 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_metrics", 2024-09-12T03:40:27.410 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.410 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.410 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.410 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T03:40:27.410 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.411 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.411 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.411 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T03:40:27.411 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T03:40:27.411 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.411 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.411 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.411 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_step": { 2024-09-12T03:40:27.411 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_step", 2024-09-12T03:40:27.411 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-12T03:40:27.411 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.412 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.412 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0.5", 2024-09-12T03:40:27.412 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0.001", 2024-09-12T03:40:27.412 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "0.999", 2024-09-12T03:40:27.412 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.412 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "aggressiveness of optimization", 2024-09-12T03:40:27.412 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T03:40:27.412 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.413 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.413 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.413 INFO:teuthology.orchestra.run.smithi064.stdout: "end_time": { 2024-09-12T03:40:27.413 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "end_time", 2024-09-12T03:40:27.413 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.413 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.413 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.413 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "2359", 2024-09-12T03:40:27.413 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.413 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.413 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.413 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T03:40:27.413 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:40:27.413 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.414 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.414 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.414 INFO:teuthology.orchestra.run.smithi064.stdout: "end_weekday": { 2024-09-12T03:40:27.414 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "end_weekday", 2024-09-12T03:40:27.414 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-12T03:40:27.414 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.414 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.414 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-12T03:40:27.414 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0", 2024-09-12T03:40:27.415 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "6", 2024-09-12T03:40:27.415 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.415 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T03:40:27.415 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:40:27.415 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.415 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.415 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.415 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:27.415 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:27.415 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.415 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.415 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.415 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.415 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.416 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.416 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.416 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.416 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.416 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.416 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.416 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.416 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.416 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.416 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.416 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.416 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.416 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.417 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.417 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:27.417 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:27.417 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.417 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.417 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.417 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.418 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.418 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.418 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.418 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.418 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.418 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.418 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.418 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.418 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:27.418 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:27.418 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.419 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.419 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.419 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:27.419 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.419 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.419 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.419 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.419 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.419 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.419 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.419 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.419 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.419 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.420 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.420 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.420 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.420 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.420 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.420 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:27.420 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:27.420 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.420 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.420 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.420 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.420 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.420 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.421 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.421 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.421 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.421 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.421 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.421 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.421 INFO:teuthology.orchestra.run.smithi064.stdout: "min_score": { 2024-09-12T03:40:27.421 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "min_score", 2024-09-12T03:40:27.421 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-12T03:40:27.421 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.421 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.421 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-12T03:40:27.421 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.422 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.422 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.422 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T03:40:27.422 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.422 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.422 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.422 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.422 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": { 2024-09-12T03:40:27.422 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mode", 2024-09-12T03:40:27.422 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.422 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.422 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.422 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "upmap", 2024-09-12T03:40:27.423 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.423 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.423 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.423 INFO:teuthology.orchestra.run.smithi064.stdout: "crush-compat", 2024-09-12T03:40:27.423 INFO:teuthology.orchestra.run.smithi064.stdout: "none", 2024-09-12T03:40:27.423 INFO:teuthology.orchestra.run.smithi064.stdout: "read", 2024-09-12T03:40:27.423 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap", 2024-09-12T03:40:27.423 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap-read" 2024-09-12T03:40:27.423 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.423 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Balancer mode", 2024-09-12T03:40:27.423 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.423 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.423 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.423 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.424 INFO:teuthology.orchestra.run.smithi064.stdout: "pool_ids": { 2024-09-12T03:40:27.424 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pool_ids", 2024-09-12T03:40:27.424 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.424 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.424 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.424 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.424 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.424 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.424 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.424 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T03:40:27.424 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.424 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.424 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.425 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.425 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-12T03:40:27.425 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-12T03:40:27.425 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:27.425 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.425 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.425 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-12T03:40:27.425 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.425 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.425 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.425 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T03:40:27.425 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.426 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.426 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.426 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.426 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap_max_deviation": { 2024-09-12T03:40:27.426 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "upmap_max_deviation", 2024-09-12T03:40:27.426 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.426 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.426 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.426 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-12T03:40:27.426 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-09-12T03:40:27.427 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.427 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.427 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T03:40:27.427 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T03:40:27.427 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.427 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.427 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.427 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap_max_optimizations": { 2024-09-12T03:40:27.427 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "upmap_max_optimizations", 2024-09-12T03:40:27.427 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-12T03:40:27.427 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.427 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.427 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-12T03:40:27.428 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.428 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.428 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.428 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T03:40:27.428 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.428 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.428 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.428 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.428 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.428 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.428 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:27.428 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephadm", 2024-09-12T03:40:27.428 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:27.428 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:27.429 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:27.429 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_down_multiplier": { 2024-09-12T03:40:27.429 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_down_multiplier", 2024-09-12T03:40:27.429 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-12T03:40:27.429 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.429 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.429 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3.0", 2024-09-12T03:40:27.429 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.429 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.429 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.429 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T03:40:27.429 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.430 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.430 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.430 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.430 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_refresh_rate": { 2024-09-12T03:40:27.430 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_refresh_rate", 2024-09-12T03:40:27.430 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:27.430 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.430 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.430 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "20", 2024-09-12T03:40:27.430 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.430 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.430 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.431 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T03:40:27.431 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.431 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.431 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.431 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.431 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_starting_port": { 2024-09-12T03:40:27.431 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_starting_port", 2024-09-12T03:40:27.431 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.431 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.431 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.431 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "4721", 2024-09-12T03:40:27.431 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.431 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.432 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.432 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T03:40:27.432 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.432 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.432 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.432 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.432 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_ptrace": { 2024-09-12T03:40:27.432 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_ptrace", 2024-09-12T03:40:27.432 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.432 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.433 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.433 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.433 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.433 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.433 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.433 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T03:40:27.433 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T03:40:27.433 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.433 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.433 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.433 INFO:teuthology.orchestra.run.smithi064.stdout: "autotune_interval": { 2024-09-12T03:40:27.433 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "autotune_interval", 2024-09-12T03:40:27.434 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:27.434 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.434 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.434 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-12T03:40:27.434 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.434 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.434 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.434 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T03:40:27.434 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.434 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.434 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.434 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.434 INFO:teuthology.orchestra.run.smithi064.stdout: "autotune_memory_target_ratio": { 2024-09-12T03:40:27.435 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T03:40:27.435 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-12T03:40:27.435 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.435 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.435 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0.7", 2024-09-12T03:40:27.435 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.435 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.435 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.435 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T03:40:27.435 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.435 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.435 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.436 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.436 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm_log_destination": { 2024-09-12T03:40:27.436 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephadm_log_destination", 2024-09-12T03:40:27.436 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.436 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.436 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.436 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.436 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.436 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.437 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.437 INFO:teuthology.orchestra.run.smithi064.stdout: "file", 2024-09-12T03:40:27.437 INFO:teuthology.orchestra.run.smithi064.stdout: "file,syslog", 2024-09-12T03:40:27.437 INFO:teuthology.orchestra.run.smithi064.stdout: "syslog" 2024-09-12T03:40:27.437 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.437 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T03:40:27.437 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.437 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.437 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.437 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.437 INFO:teuthology.orchestra.run.smithi064.stdout: "cgroups_split": { 2024-09-12T03:40:27.437 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cgroups_split", 2024-09-12T03:40:27.437 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.438 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.438 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.438 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.438 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.438 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.438 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.438 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T03:40:27.438 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.438 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.439 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.439 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.439 INFO:teuthology.orchestra.run.smithi064.stdout: "config_checks_enabled": { 2024-09-12T03:40:27.439 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_checks_enabled", 2024-09-12T03:40:27.439 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.439 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.439 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.439 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.439 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.439 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.439 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.439 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T03:40:27.440 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.440 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.440 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.440 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.440 INFO:teuthology.orchestra.run.smithi064.stdout: "config_dashboard": { 2024-09-12T03:40:27.440 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_dashboard", 2024-09-12T03:40:27.440 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.440 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.440 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.440 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.440 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.440 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.441 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.441 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T03:40:27.441 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.441 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.441 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.441 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.441 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_alertmanager": { 2024-09-12T03:40:27.441 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_alertmanager", 2024-09-12T03:40:27.441 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.441 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.441 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.441 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T03:40:27.441 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.442 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.442 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.442 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-09-12T03:40:27.442 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.442 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.442 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.442 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.442 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_base": { 2024-09-12T03:40:27.442 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_base", 2024-09-12T03:40:27.442 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.442 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.442 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.442 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T03:40:27.442 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.443 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.443 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.443 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Container image name, without the tag", 2024-09-12T03:40:27.443 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.443 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.443 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.443 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.443 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_elasticsearch": { 2024-09-12T03:40:27.443 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_elasticsearch", 2024-09-12T03:40:27.443 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.443 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.443 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.443 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T03:40:27.443 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.444 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.444 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.444 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "elasticsearch container image", 2024-09-12T03:40:27.444 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.444 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.444 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.444 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.444 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_grafana": { 2024-09-12T03:40:27.444 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_grafana", 2024-09-12T03:40:27.444 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.444 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.444 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.444 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T03:40:27.445 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.445 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.445 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.445 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-09-12T03:40:27.445 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.445 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.445 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.445 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.445 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_haproxy": { 2024-09-12T03:40:27.445 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_haproxy", 2024-09-12T03:40:27.445 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.445 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.445 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.445 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T03:40:27.446 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.446 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.446 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.446 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "HAproxy container image", 2024-09-12T03:40:27.446 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.446 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.446 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.446 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.446 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_agent": { 2024-09-12T03:40:27.446 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_agent", 2024-09-12T03:40:27.446 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.446 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.447 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.447 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T03:40:27.447 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.447 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.447 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.447 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger agent container image", 2024-09-12T03:40:27.447 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.447 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.447 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.447 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.447 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_collector": { 2024-09-12T03:40:27.447 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_collector", 2024-09-12T03:40:27.447 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.447 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.448 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.448 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T03:40:27.448 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.448 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.448 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.448 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger collector container image", 2024-09-12T03:40:27.448 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.448 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.448 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.448 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.448 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_query": { 2024-09-12T03:40:27.448 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_query", 2024-09-12T03:40:27.448 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.449 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.449 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.449 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T03:40:27.449 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.449 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.449 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.449 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger query container image", 2024-09-12T03:40:27.449 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.449 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.449 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.449 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.449 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_keepalived": { 2024-09-12T03:40:27.449 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_keepalived", 2024-09-12T03:40:27.449 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.450 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.450 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.450 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T03:40:27.450 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.450 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.450 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.450 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Keepalived container image", 2024-09-12T03:40:27.450 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.450 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.450 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.450 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.450 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_loki": { 2024-09-12T03:40:27.450 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_loki", 2024-09-12T03:40:27.451 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.451 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.451 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.451 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T03:40:27.451 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.451 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.451 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.451 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Loki container image", 2024-09-12T03:40:27.451 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.451 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.451 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.451 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.451 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_node_exporter": { 2024-09-12T03:40:27.452 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_node_exporter", 2024-09-12T03:40:27.452 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.452 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.452 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.452 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T03:40:27.452 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.452 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.452 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.452 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-09-12T03:40:27.452 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.452 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.452 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.452 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.453 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_nvmeof": { 2024-09-12T03:40:27.453 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_nvmeof", 2024-09-12T03:40:27.453 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.453 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.453 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.453 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T03:40:27.453 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.453 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.453 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.453 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Nvme-of container image", 2024-09-12T03:40:27.453 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.453 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.453 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.453 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.454 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_prometheus": { 2024-09-12T03:40:27.454 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_prometheus", 2024-09-12T03:40:27.454 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.454 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.454 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.454 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T03:40:27.454 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.454 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.454 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.454 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-09-12T03:40:27.454 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.454 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.454 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.455 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.455 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_promtail": { 2024-09-12T03:40:27.455 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_promtail", 2024-09-12T03:40:27.455 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.455 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.455 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.455 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T03:40:27.455 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.455 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.455 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.455 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Promtail container image", 2024-09-12T03:40:27.455 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.455 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.455 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.456 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.456 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_samba": { 2024-09-12T03:40:27.456 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_samba", 2024-09-12T03:40:27.456 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.456 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.456 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.456 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T03:40:27.456 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.456 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.456 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.456 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Samba/SMB container image", 2024-09-12T03:40:27.456 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.456 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.457 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.457 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.457 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_snmp_gateway": { 2024-09-12T03:40:27.457 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_snmp_gateway", 2024-09-12T03:40:27.457 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.457 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.457 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.457 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T03:40:27.457 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.457 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.457 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.457 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SNMP Gateway container image", 2024-09-12T03:40:27.458 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.458 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.458 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.458 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.458 INFO:teuthology.orchestra.run.smithi064.stdout: "container_init": { 2024-09-12T03:40:27.458 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_init", 2024-09-12T03:40:27.458 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.458 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.458 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.458 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.458 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.458 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.459 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.459 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T03:40:27.459 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.459 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.459 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.459 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.459 INFO:teuthology.orchestra.run.smithi064.stdout: "daemon_cache_timeout": { 2024-09-12T03:40:27.459 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "daemon_cache_timeout", 2024-09-12T03:40:27.459 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:27.459 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.459 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.459 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-12T03:40:27.459 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.460 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.460 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.460 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T03:40:27.460 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.460 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.460 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.460 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.460 INFO:teuthology.orchestra.run.smithi064.stdout: "default_cephadm_command_timeout": { 2024-09-12T03:40:27.460 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T03:40:27.460 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.460 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.460 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.460 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "900", 2024-09-12T03:40:27.461 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.461 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.461 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.461 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T03:40:27.461 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.461 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.461 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.461 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.461 INFO:teuthology.orchestra.run.smithi064.stdout: "default_registry": { 2024-09-12T03:40:27.461 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "default_registry", 2024-09-12T03:40:27.461 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.461 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.461 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.462 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io", 2024-09-12T03:40:27.462 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.462 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.462 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.462 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T03:40:27.462 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.462 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.462 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.462 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.462 INFO:teuthology.orchestra.run.smithi064.stdout: "device_cache_timeout": { 2024-09-12T03:40:27.462 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_cache_timeout", 2024-09-12T03:40:27.462 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:27.462 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.463 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.463 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1800", 2024-09-12T03:40:27.463 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.463 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.463 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.463 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache device inventory", 2024-09-12T03:40:27.463 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.463 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.463 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.463 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.463 INFO:teuthology.orchestra.run.smithi064.stdout: "device_enhanced_scan": { 2024-09-12T03:40:27.464 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_enhanced_scan", 2024-09-12T03:40:27.464 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.464 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.464 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.464 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.464 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.464 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.464 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.464 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T03:40:27.464 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.464 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.464 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.464 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.465 INFO:teuthology.orchestra.run.smithi064.stdout: "facts_cache_timeout": { 2024-09-12T03:40:27.465 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "facts_cache_timeout", 2024-09-12T03:40:27.465 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:27.465 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.465 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.465 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-12T03:40:27.465 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.465 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.465 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.465 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache host facts data", 2024-09-12T03:40:27.465 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.465 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.466 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.466 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.466 INFO:teuthology.orchestra.run.smithi064.stdout: "grafana_dashboards_path": { 2024-09-12T03:40:27.466 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "grafana_dashboards_path", 2024-09-12T03:40:27.466 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.466 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.466 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.466 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T03:40:27.466 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.466 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.466 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.466 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T03:40:27.466 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.466 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.467 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.467 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.467 INFO:teuthology.orchestra.run.smithi064.stdout: "host_check_interval": { 2024-09-12T03:40:27.467 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "host_check_interval", 2024-09-12T03:40:27.467 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:27.467 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.467 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.467 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-12T03:40:27.467 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.467 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.467 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.467 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to perform a host check", 2024-09-12T03:40:27.467 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.467 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.468 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.468 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.468 INFO:teuthology.orchestra.run.smithi064.stdout: "hw_monitoring": { 2024-09-12T03:40:27.468 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "hw_monitoring", 2024-09-12T03:40:27.468 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.468 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.468 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.468 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.468 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.468 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.468 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.468 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T03:40:27.469 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.469 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.469 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.469 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.469 INFO:teuthology.orchestra.run.smithi064.stdout: "inventory_list_all": { 2024-09-12T03:40:27.469 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "inventory_list_all", 2024-09-12T03:40:27.469 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.469 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.469 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.469 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.469 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.469 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.469 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.469 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T03:40:27.470 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.470 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.470 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.470 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.470 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:27.470 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:27.470 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.470 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.470 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.470 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.470 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.470 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.470 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.471 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.471 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.471 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.471 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.471 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.471 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.471 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.471 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.471 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.471 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.471 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.471 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.471 INFO:teuthology.orchestra.run.smithi064.stdout: "log_refresh_metadata": { 2024-09-12T03:40:27.471 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_refresh_metadata", 2024-09-12T03:40:27.472 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.472 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.472 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.472 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.472 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.472 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.472 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.472 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T03:40:27.472 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.472 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.472 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.472 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.473 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:27.473 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:27.473 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.473 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.473 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.473 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.473 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.473 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.473 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.473 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T03:40:27.473 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.473 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.473 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.474 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.474 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:27.474 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:27.474 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.474 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.474 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.474 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:27.474 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.474 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.474 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.474 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.474 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.474 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.475 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.475 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.475 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.475 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.475 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.475 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.475 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.475 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.475 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.475 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:27.475 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:27.475 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.475 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.476 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.476 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.476 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.476 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.476 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.476 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.476 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.476 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.476 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.476 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.476 INFO:teuthology.orchestra.run.smithi064.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T03:40:27.476 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T03:40:27.476 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.477 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.477 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.477 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.477 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.477 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.477 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.477 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T03:40:27.477 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.477 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.477 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.477 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.477 INFO:teuthology.orchestra.run.smithi064.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T03:40:27.477 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T03:40:27.477 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.478 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.478 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.478 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "*", 2024-09-12T03:40:27.478 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.478 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.478 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.478 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T03:40:27.478 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.478 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.478 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.478 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.478 INFO:teuthology.orchestra.run.smithi064.stdout: "max_count_per_host": { 2024-09-12T03:40:27.478 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_count_per_host", 2024-09-12T03:40:27.478 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.479 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.479 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.479 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-12T03:40:27.479 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.479 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.479 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.479 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "max number of daemons per service per host", 2024-09-12T03:40:27.479 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.479 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.479 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.479 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.479 INFO:teuthology.orchestra.run.smithi064.stdout: "max_osd_draining_count": { 2024-09-12T03:40:27.479 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_osd_draining_count", 2024-09-12T03:40:27.479 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.480 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.480 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.480 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-12T03:40:27.480 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.480 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.480 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.480 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T03:40:27.480 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.480 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.480 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.480 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.480 INFO:teuthology.orchestra.run.smithi064.stdout: "migration_current": { 2024-09-12T03:40:27.481 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "migration_current", 2024-09-12T03:40:27.481 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.481 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.481 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.481 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.481 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.481 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.481 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.481 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "internal - do not modify", 2024-09-12T03:40:27.481 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.481 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.481 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.481 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.481 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": { 2024-09-12T03:40:27.482 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mode", 2024-09-12T03:40:27.482 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.482 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.482 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.482 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "root", 2024-09-12T03:40:27.482 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.482 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.482 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.482 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm-package", 2024-09-12T03:40:27.482 INFO:teuthology.orchestra.run.smithi064.stdout: "root" 2024-09-12T03:40:27.482 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.482 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T03:40:27.482 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.483 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.483 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.483 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.483 INFO:teuthology.orchestra.run.smithi064.stdout: "oob_default_addr": { 2024-09-12T03:40:27.483 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "oob_default_addr", 2024-09-12T03:40:27.483 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.483 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.483 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.483 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "169.254.1.1", 2024-09-12T03:40:27.483 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.483 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.483 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.483 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T03:40:27.484 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.484 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.484 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.484 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.484 INFO:teuthology.orchestra.run.smithi064.stdout: "prometheus_alerts_path": { 2024-09-12T03:40:27.484 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prometheus_alerts_path", 2024-09-12T03:40:27.484 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.484 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.484 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.484 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T03:40:27.484 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.484 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.484 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.484 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T03:40:27.485 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.485 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.485 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.485 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.485 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_insecure": { 2024-09-12T03:40:27.485 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_insecure", 2024-09-12T03:40:27.485 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.485 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.485 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.485 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.485 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.486 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.486 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.486 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T03:40:27.486 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.486 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.486 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.486 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.486 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_password": { 2024-09-12T03:40:27.486 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_password", 2024-09-12T03:40:27.486 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.486 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.486 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.486 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.487 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.487 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.487 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.487 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T03:40:27.487 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.487 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.487 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.487 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.487 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_url": { 2024-09-12T03:40:27.487 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_url", 2024-09-12T03:40:27.487 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.487 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.487 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.487 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.487 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.488 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.488 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.488 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T03:40:27.488 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.488 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.488 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.488 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.488 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_username": { 2024-09-12T03:40:27.488 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_username", 2024-09-12T03:40:27.488 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.488 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.488 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.489 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.489 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.489 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.489 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.489 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T03:40:27.489 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.489 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.489 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.489 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.489 INFO:teuthology.orchestra.run.smithi064.stdout: "secure_monitoring_stack": { 2024-09-12T03:40:27.489 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "secure_monitoring_stack", 2024-09-12T03:40:27.489 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.489 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.490 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.490 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.490 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.490 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.490 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.490 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T03:40:27.490 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.490 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.490 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.490 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.490 INFO:teuthology.orchestra.run.smithi064.stdout: "service_discovery_port": { 2024-09-12T03:40:27.490 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "service_discovery_port", 2024-09-12T03:40:27.490 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.490 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.491 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.491 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8765", 2024-09-12T03:40:27.491 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.491 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.491 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.491 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "cephadm service discovery port", 2024-09-12T03:40:27.491 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.491 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.491 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.491 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.491 INFO:teuthology.orchestra.run.smithi064.stdout: "ssh_config_file": { 2024-09-12T03:40:27.491 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssh_config_file", 2024-09-12T03:40:27.491 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.491 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.492 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.492 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.492 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.492 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.492 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.492 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T03:40:27.492 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.492 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.492 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.492 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.492 INFO:teuthology.orchestra.run.smithi064.stdout: "use_agent": { 2024-09-12T03:40:27.492 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "use_agent", 2024-09-12T03:40:27.492 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.492 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.493 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.493 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.493 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.493 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.493 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.493 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T03:40:27.493 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.493 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.493 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.493 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.493 INFO:teuthology.orchestra.run.smithi064.stdout: "use_repo_digest": { 2024-09-12T03:40:27.493 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "use_repo_digest", 2024-09-12T03:40:27.494 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.494 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.494 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.494 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.494 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.494 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.494 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.494 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T03:40:27.494 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.494 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.494 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.494 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.494 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_failed_host_check": { 2024-09-12T03:40:27.494 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_failed_host_check", 2024-09-12T03:40:27.495 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.495 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.495 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.495 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.495 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.495 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.495 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.495 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T03:40:27.495 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.495 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.495 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.495 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.495 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_stray_daemons": { 2024-09-12T03:40:27.495 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_stray_daemons", 2024-09-12T03:40:27.496 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.496 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.496 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.496 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.496 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.496 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.496 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.496 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T03:40:27.496 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.497 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.497 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.497 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.497 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_stray_hosts": { 2024-09-12T03:40:27.497 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_stray_hosts", 2024-09-12T03:40:27.497 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.497 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.497 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.497 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.497 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.497 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.498 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.498 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T03:40:27.498 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.498 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.498 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.498 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.498 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.498 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.498 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:27.498 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crash", 2024-09-12T03:40:27.498 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:27.498 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:27.499 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:27.499 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:27.499 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:27.499 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.499 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.499 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.499 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.499 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.499 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.499 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.499 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.499 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.499 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.499 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.500 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.500 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.500 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.500 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.500 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.500 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.500 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.500 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.500 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:27.500 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:27.500 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.500 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.500 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.501 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.501 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.501 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.501 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.501 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.501 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.501 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.501 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.501 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.501 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:27.501 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:27.501 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.501 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.502 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.502 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:27.502 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.502 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.502 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.502 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.502 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.502 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.502 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.502 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.502 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.502 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.502 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.502 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.503 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.503 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.503 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.503 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:27.503 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:27.503 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.503 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.503 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.503 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.503 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.503 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.503 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.503 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.503 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.504 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.504 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.504 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.504 INFO:teuthology.orchestra.run.smithi064.stdout: "retain_interval": { 2024-09-12T03:40:27.504 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "retain_interval", 2024-09-12T03:40:27.504 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:27.504 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.504 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.504 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "31536000", 2024-09-12T03:40:27.504 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.504 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.504 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.504 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T03:40:27.505 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.505 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.505 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.505 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.505 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_recent_interval": { 2024-09-12T03:40:27.505 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_recent_interval", 2024-09-12T03:40:27.505 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:27.505 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.505 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.505 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1209600", 2024-09-12T03:40:27.505 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.505 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.505 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.506 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T03:40:27.506 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.506 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.506 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.506 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.506 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.506 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.506 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:27.506 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "dashboard", 2024-09-12T03:40:27.506 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:27.506 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:27.506 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:27.506 INFO:teuthology.orchestra.run.smithi064.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T03:40:27.507 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T03:40:27.507 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.507 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.507 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.507 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-12T03:40:27.507 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.507 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.507 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.507 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.507 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.507 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.507 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.507 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.508 INFO:teuthology.orchestra.run.smithi064.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T03:40:27.508 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T03:40:27.508 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.508 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.508 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.508 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.508 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.508 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.508 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.508 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.508 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.508 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.508 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.509 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.509 INFO:teuthology.orchestra.run.smithi064.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T03:40:27.509 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T03:40:27.509 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.509 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.509 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.509 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.509 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.509 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.509 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.509 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.509 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.509 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.509 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.510 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.510 INFO:teuthology.orchestra.run.smithi064.stdout: "AUDIT_API_ENABLED": { 2024-09-12T03:40:27.510 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T03:40:27.510 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.510 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.510 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.510 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.510 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.510 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.510 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.510 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.510 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.510 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.510 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.511 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.511 INFO:teuthology.orchestra.run.smithi064.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T03:40:27.511 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T03:40:27.511 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.511 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.511 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.511 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.511 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.511 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.511 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.511 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.512 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.512 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.512 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.512 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.512 INFO:teuthology.orchestra.run.smithi064.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T03:40:27.512 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T03:40:27.512 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.512 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.512 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.513 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.513 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.513 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.513 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.513 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.513 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.513 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.513 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.513 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.513 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T03:40:27.513 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T03:40:27.513 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.514 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.514 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.514 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.514 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.514 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.514 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.514 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.514 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.514 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.514 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.514 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.514 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T03:40:27.514 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T03:40:27.514 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.515 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.515 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.515 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.515 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.515 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.515 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.515 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.515 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.515 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.515 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.515 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.515 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T03:40:27.515 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T03:40:27.516 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.516 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.516 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.516 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.516 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.516 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.516 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.516 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.516 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.516 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.516 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.516 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.516 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T03:40:27.517 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T03:40:27.517 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.517 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.517 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.517 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.517 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.517 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.517 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.517 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.517 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.517 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.517 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.517 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.518 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T03:40:27.518 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T03:40:27.518 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.518 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.518 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.518 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.518 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.518 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.518 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.518 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.518 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.518 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.518 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.519 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.519 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T03:40:27.519 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T03:40:27.519 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.519 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.519 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.519 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.519 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.519 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.519 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.519 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.519 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.519 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.519 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.520 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.520 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T03:40:27.520 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T03:40:27.520 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.520 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.520 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.520 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.520 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.520 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.520 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.520 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.520 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.520 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.520 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.521 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.521 INFO:teuthology.orchestra.run.smithi064.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T03:40:27.521 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T03:40:27.521 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.521 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.521 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.521 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.521 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.521 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.521 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.521 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.523 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.524 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.524 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.524 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.524 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T03:40:27.524 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T03:40:27.524 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.524 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.524 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.524 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-09-12T03:40:27.524 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.524 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.524 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.525 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.525 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.525 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.525 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.525 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.525 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T03:40:27.525 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T03:40:27.525 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.525 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.525 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.525 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.525 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.525 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.525 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.526 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.526 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.526 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.526 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.526 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.526 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_URL": { 2024-09-12T03:40:27.526 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_URL", 2024-09-12T03:40:27.526 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.526 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.526 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.526 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.526 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.527 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.527 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.527 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.527 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.527 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.527 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.527 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.527 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T03:40:27.527 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T03:40:27.527 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.527 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.527 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.527 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-09-12T03:40:27.527 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.528 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.528 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.528 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.528 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.528 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.528 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.528 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.528 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T03:40:27.528 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T03:40:27.528 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.528 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.528 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.528 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.529 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.529 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.529 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.529 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.529 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.529 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.529 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.529 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.529 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T03:40:27.529 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T03:40:27.529 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.529 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.530 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.530 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.530 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.530 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.530 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.530 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.530 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.530 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.530 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.530 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.530 INFO:teuthology.orchestra.run.smithi064.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T03:40:27.530 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T03:40:27.530 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.531 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.531 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.531 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.531 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.531 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.531 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.531 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.531 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.531 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.531 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.531 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.531 INFO:teuthology.orchestra.run.smithi064.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T03:40:27.531 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T03:40:27.531 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.532 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.532 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.532 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.532 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.532 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.532 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.532 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.532 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.532 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.532 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.532 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.532 INFO:teuthology.orchestra.run.smithi064.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T03:40:27.532 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T03:40:27.533 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.533 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.533 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.533 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.533 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.533 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.533 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.533 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.533 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.533 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.533 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.533 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.534 INFO:teuthology.orchestra.run.smithi064.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T03:40:27.534 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T03:40:27.534 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.534 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.534 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.534 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.534 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.534 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.534 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.534 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.534 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.534 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.534 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.534 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.535 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T03:40:27.535 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T03:40:27.535 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.535 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.535 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.535 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.535 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.535 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.535 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.535 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.535 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.535 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.535 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.536 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.536 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T03:40:27.536 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T03:40:27.536 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.536 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.536 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.536 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.536 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.536 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.536 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.536 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.536 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.536 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.537 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.537 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.537 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T03:40:27.537 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T03:40:27.537 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.537 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.537 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.537 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.537 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.537 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.537 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.537 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.537 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.537 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.538 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.538 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.538 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T03:40:27.538 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T03:40:27.538 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.538 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.538 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.538 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.538 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.538 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.538 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.538 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.538 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.539 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.539 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.539 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.539 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T03:40:27.539 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T03:40:27.539 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.539 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.539 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.539 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.539 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.539 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.539 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.539 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.539 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.540 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.540 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.540 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.540 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T03:40:27.540 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T03:40:27.540 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.540 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.540 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.540 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.540 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.540 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.540 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.540 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.540 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.541 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.541 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.541 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.541 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T03:40:27.541 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T03:40:27.541 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.541 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.541 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.541 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.541 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.541 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.541 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.541 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.542 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.542 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.542 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.542 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.542 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T03:40:27.542 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T03:40:27.542 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.542 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.542 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.542 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.542 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.542 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.542 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.542 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.543 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.543 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.543 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.543 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.543 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T03:40:27.543 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T03:40:27.543 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.543 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.543 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.543 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T03:40:27.543 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.543 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.544 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.544 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.544 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.544 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.544 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.544 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.544 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T03:40:27.544 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T03:40:27.544 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.544 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.544 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.544 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-12T03:40:27.544 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.545 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.545 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.545 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.545 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.545 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.545 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.545 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.545 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T03:40:27.545 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T03:40:27.545 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.545 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.545 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.545 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8", 2024-09-12T03:40:27.545 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.546 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.546 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.546 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.546 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.546 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.546 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.546 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.546 INFO:teuthology.orchestra.run.smithi064.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T03:40:27.546 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T03:40:27.546 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.546 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.546 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.546 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "45", 2024-09-12T03:40:27.547 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.547 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.547 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.547 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.547 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.547 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.547 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.547 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.547 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T03:40:27.547 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T03:40:27.547 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.547 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.547 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.548 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.548 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.548 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.548 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.548 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.548 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.548 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.548 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.548 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.548 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T03:40:27.548 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T03:40:27.548 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.548 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.548 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.549 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-09-12T03:40:27.549 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.549 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.549 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.549 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.549 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.549 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.549 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.549 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.549 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T03:40:27.549 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T03:40:27.549 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.549 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.550 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.550 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.550 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.550 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.550 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.550 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.550 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.550 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.550 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.550 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.550 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T03:40:27.550 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T03:40:27.550 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.550 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.551 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.551 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.551 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.551 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.551 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.551 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.551 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.551 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.551 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.551 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.551 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T03:40:27.551 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T03:40:27.551 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.551 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.552 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.552 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-12T03:40:27.552 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.552 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.552 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.552 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.552 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.552 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.552 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.552 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.552 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T03:40:27.552 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T03:40:27.552 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.553 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.553 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.553 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-12T03:40:27.553 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.553 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.553 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.553 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.553 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.553 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.553 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.553 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.553 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T03:40:27.553 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T03:40:27.554 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.554 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.554 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.554 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-12T03:40:27.554 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.554 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.554 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.554 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.554 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.554 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.554 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.554 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.554 INFO:teuthology.orchestra.run.smithi064.stdout: "cross_origin_url": { 2024-09-12T03:40:27.554 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cross_origin_url", 2024-09-12T03:40:27.555 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.555 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.555 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.555 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.555 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.555 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.555 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.555 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.555 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.555 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.555 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.555 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.555 INFO:teuthology.orchestra.run.smithi064.stdout: "crt_file": { 2024-09-12T03:40:27.556 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crt_file", 2024-09-12T03:40:27.556 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.556 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.556 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.556 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.556 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.556 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.556 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.556 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.556 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.556 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.556 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.556 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.556 INFO:teuthology.orchestra.run.smithi064.stdout: "debug": { 2024-09-12T03:40:27.557 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "debug", 2024-09-12T03:40:27.557 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.557 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.557 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.557 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.557 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.557 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.557 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.557 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable/disable debug options", 2024-09-12T03:40:27.557 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.557 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.557 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.558 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.558 INFO:teuthology.orchestra.run.smithi064.stdout: "jwt_token_ttl": { 2024-09-12T03:40:27.558 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "jwt_token_ttl", 2024-09-12T03:40:27.558 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.558 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.558 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.558 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "28800", 2024-09-12T03:40:27.558 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.558 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.558 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.558 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.558 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.558 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.559 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.559 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.559 INFO:teuthology.orchestra.run.smithi064.stdout: "key_file": { 2024-09-12T03:40:27.559 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "key_file", 2024-09-12T03:40:27.559 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.559 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.559 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.559 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.559 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.559 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.559 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.559 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.559 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.560 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.560 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.560 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.560 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:27.560 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:27.560 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.560 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.560 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.560 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.560 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.560 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.560 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.560 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.561 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.561 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.561 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.561 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.561 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.561 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.561 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.561 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.561 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.561 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.561 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.561 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:27.561 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:27.562 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.562 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.562 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.562 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.562 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.562 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.562 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.562 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.562 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.562 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.562 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.562 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.562 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:27.563 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:27.563 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.563 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.563 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.563 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:27.563 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.563 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.563 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.563 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.563 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.563 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.563 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.563 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.564 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.564 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.564 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.564 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.564 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.564 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.564 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.564 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:27.564 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:27.564 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.564 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.564 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.564 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.565 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.565 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.565 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.565 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.565 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.565 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.565 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.565 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.565 INFO:teuthology.orchestra.run.smithi064.stdout: "motd": { 2024-09-12T03:40:27.565 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "motd", 2024-09-12T03:40:27.565 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.565 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.566 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.566 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.566 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.566 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.566 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.566 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "The message of the day", 2024-09-12T03:40:27.566 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.566 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.566 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.566 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.566 INFO:teuthology.orchestra.run.smithi064.stdout: "redirect_resolve_ip_addr": { 2024-09-12T03:40:27.566 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T03:40:27.566 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.567 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.567 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.567 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.567 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.567 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.567 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.567 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.567 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.567 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.567 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.567 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.567 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-09-12T03:40:27.567 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-09-12T03:40:27.568 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.568 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.568 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.568 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "::", 2024-09-12T03:40:27.568 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.568 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.568 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.568 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.568 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.568 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.568 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.568 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.568 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-09-12T03:40:27.568 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-09-12T03:40:27.569 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.569 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.569 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.569 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8080", 2024-09-12T03:40:27.569 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.569 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.569 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.569 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.569 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.569 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.569 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.569 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.569 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl": { 2024-09-12T03:40:27.570 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl", 2024-09-12T03:40:27.570 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.570 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.570 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.570 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.570 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.570 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.570 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.570 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.570 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.570 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.570 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.570 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.570 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl_server_port": { 2024-09-12T03:40:27.570 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl_server_port", 2024-09-12T03:40:27.571 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.571 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.571 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.571 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8443", 2024-09-12T03:40:27.571 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.571 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.571 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.571 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.571 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.571 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.571 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.571 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.571 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_behaviour": { 2024-09-12T03:40:27.572 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_behaviour", 2024-09-12T03:40:27.572 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.572 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.572 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.572 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "redirect", 2024-09-12T03:40:27.572 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.572 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.572 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.572 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.572 INFO:teuthology.orchestra.run.smithi064.stdout: "redirect" 2024-09-12T03:40:27.572 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.572 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.572 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.573 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.573 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.573 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.573 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_error_status_code": { 2024-09-12T03:40:27.573 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_error_status_code", 2024-09-12T03:40:27.573 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.573 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.573 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.573 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2024-09-12T03:40:27.573 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "400", 2024-09-12T03:40:27.573 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "599", 2024-09-12T03:40:27.573 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.573 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.574 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.574 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.574 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.574 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.574 INFO:teuthology.orchestra.run.smithi064.stdout: "url_prefix": { 2024-09-12T03:40:27.574 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "url_prefix", 2024-09-12T03:40:27.574 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.574 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.574 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.574 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.574 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.574 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.574 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.574 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.575 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.575 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.575 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.575 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.575 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.575 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.575 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:27.575 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "devicehealth", 2024-09-12T03:40:27.575 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:27.575 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:27.575 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:27.575 INFO:teuthology.orchestra.run.smithi064.stdout: "enable_monitoring": { 2024-09-12T03:40:27.575 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enable_monitoring", 2024-09-12T03:40:27.575 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.576 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.576 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.576 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.576 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.576 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.576 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.576 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "monitor device health metrics", 2024-09-12T03:40:27.576 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.576 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.576 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.576 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.576 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:27.576 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:27.577 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.577 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.577 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.577 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.577 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.577 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.577 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.577 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.577 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.577 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.577 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.577 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.577 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.578 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.578 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.578 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.578 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.578 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.578 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.578 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:27.578 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:27.578 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.578 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.578 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.578 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.578 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.578 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.579 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.579 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.579 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.579 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.579 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.579 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.579 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:27.579 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:27.579 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.579 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.579 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.579 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:27.579 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.579 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.580 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.580 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.580 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.580 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.580 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.580 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.580 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.580 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.580 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.580 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.580 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.580 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.580 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.581 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:27.581 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:27.581 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.581 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.581 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.581 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.581 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.581 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.581 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.581 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.581 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.581 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.582 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.582 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.582 INFO:teuthology.orchestra.run.smithi064.stdout: "mark_out_threshold": { 2024-09-12T03:40:27.582 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mark_out_threshold", 2024-09-12T03:40:27.582 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:27.582 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.582 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.582 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "2419200", 2024-09-12T03:40:27.582 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.582 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.582 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.582 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T03:40:27.583 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.583 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.583 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.583 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.583 INFO:teuthology.orchestra.run.smithi064.stdout: "pool_name": { 2024-09-12T03:40:27.583 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pool_name", 2024-09-12T03:40:27.583 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.583 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.583 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.583 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "device_health_metrics", 2024-09-12T03:40:27.583 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.583 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.583 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.583 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T03:40:27.584 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.584 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.584 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.584 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.584 INFO:teuthology.orchestra.run.smithi064.stdout: "retention_period": { 2024-09-12T03:40:27.584 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "retention_period", 2024-09-12T03:40:27.584 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:27.584 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.584 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.584 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15552000", 2024-09-12T03:40:27.584 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.584 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.584 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.585 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long to retain device health metrics", 2024-09-12T03:40:27.585 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.585 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.585 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.585 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.585 INFO:teuthology.orchestra.run.smithi064.stdout: "scrape_frequency": { 2024-09-12T03:40:27.585 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "scrape_frequency", 2024-09-12T03:40:27.585 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:27.585 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.585 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.585 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "86400", 2024-09-12T03:40:27.585 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.585 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.586 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.586 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T03:40:27.586 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.586 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.586 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.586 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.586 INFO:teuthology.orchestra.run.smithi064.stdout: "self_heal": { 2024-09-12T03:40:27.586 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "self_heal", 2024-09-12T03:40:27.586 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.586 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.586 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.586 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.587 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.587 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.587 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.587 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T03:40:27.587 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.587 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.587 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.587 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.587 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-12T03:40:27.587 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-12T03:40:27.587 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:27.587 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.587 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.588 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-12T03:40:27.588 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.588 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.588 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.588 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T03:40:27.588 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.588 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.588 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.588 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.588 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_threshold": { 2024-09-12T03:40:27.588 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_threshold", 2024-09-12T03:40:27.588 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:27.588 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.589 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.589 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "7257600", 2024-09-12T03:40:27.589 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.589 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.589 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.589 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T03:40:27.589 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.589 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.589 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.589 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.589 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.589 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.589 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:27.589 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "diskprediction_local", 2024-09-12T03:40:27.590 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:27.590 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:27.590 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:27.590 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:27.590 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:27.590 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.590 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.590 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.590 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.590 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.590 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.590 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.590 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.591 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.591 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.591 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.591 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.591 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.591 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.591 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.591 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.591 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.591 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.591 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.591 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:27.591 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:27.592 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.592 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.592 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.592 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.592 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.592 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.592 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.592 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.592 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.592 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.593 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.593 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.593 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:27.593 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:27.593 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.593 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.593 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.593 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:27.593 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.593 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.593 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.593 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.593 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.594 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.594 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.594 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.594 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.594 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.594 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.594 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.594 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.594 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.594 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.594 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:27.594 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:27.594 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.595 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.595 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.595 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.595 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.595 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.595 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.595 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.595 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.595 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.595 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.595 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.595 INFO:teuthology.orchestra.run.smithi064.stdout: "predict_interval": { 2024-09-12T03:40:27.595 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "predict_interval", 2024-09-12T03:40:27.595 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.596 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.596 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.596 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "86400", 2024-09-12T03:40:27.596 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.596 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.596 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.596 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.596 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.596 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.596 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.596 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.596 INFO:teuthology.orchestra.run.smithi064.stdout: "predictor_model": { 2024-09-12T03:40:27.596 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "predictor_model", 2024-09-12T03:40:27.597 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.597 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.597 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.597 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "prophetstor", 2024-09-12T03:40:27.597 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.597 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.597 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.597 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.597 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.598 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.598 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.598 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.598 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-12T03:40:27.598 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-12T03:40:27.598 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.598 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.598 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.598 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-12T03:40:27.598 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.598 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.598 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.598 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.598 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.599 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.599 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.599 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.599 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.599 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.599 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:27.599 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "influx", 2024-09-12T03:40:27.599 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": false, 2024-09-12T03:40:27.599 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "influxdb python module not found", 2024-09-12T03:40:27.599 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:27.599 INFO:teuthology.orchestra.run.smithi064.stdout: "batch_size": { 2024-09-12T03:40:27.599 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "batch_size", 2024-09-12T03:40:27.599 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.599 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.600 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.600 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5000", 2024-09-12T03:40:27.600 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.600 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.600 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.600 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T03:40:27.600 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.600 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.600 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.600 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.600 INFO:teuthology.orchestra.run.smithi064.stdout: "database": { 2024-09-12T03:40:27.600 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "database", 2024-09-12T03:40:27.601 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.601 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.601 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.601 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "ceph", 2024-09-12T03:40:27.601 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.601 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.601 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.601 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T03:40:27.601 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.602 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.602 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.602 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.602 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": { 2024-09-12T03:40:27.602 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "hostname", 2024-09-12T03:40:27.602 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.602 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.602 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.602 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.602 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.602 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.602 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.602 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB server hostname", 2024-09-12T03:40:27.602 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.603 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.603 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.603 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.603 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-12T03:40:27.603 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-12T03:40:27.603 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:27.603 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.603 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.603 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "30", 2024-09-12T03:40:27.603 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "5", 2024-09-12T03:40:27.603 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.603 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.603 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T03:40:27.604 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.604 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.604 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.604 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.604 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:27.604 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:27.604 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.604 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.604 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.604 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.604 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.604 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.605 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.605 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.605 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.605 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.605 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.605 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.605 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.605 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.606 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.606 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.606 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.606 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.606 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.606 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:27.606 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:27.606 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.606 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.606 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.606 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.606 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.606 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.606 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.607 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.607 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.607 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.607 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.607 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.607 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:27.607 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:27.607 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.607 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.607 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.607 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:27.607 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.607 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.608 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.608 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.608 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.608 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.608 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.608 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.608 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.608 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.608 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.608 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.608 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.608 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.608 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.609 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:27.609 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:27.609 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.609 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.609 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.609 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.609 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.609 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.609 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.609 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.609 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.609 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.609 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.609 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.610 INFO:teuthology.orchestra.run.smithi064.stdout: "password": { 2024-09-12T03:40:27.610 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "password", 2024-09-12T03:40:27.610 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.610 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.610 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.610 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.610 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.610 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.610 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.610 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "password of InfluxDB server user", 2024-09-12T03:40:27.610 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.610 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.610 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.610 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.611 INFO:teuthology.orchestra.run.smithi064.stdout: "port": { 2024-09-12T03:40:27.611 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "port", 2024-09-12T03:40:27.611 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.611 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.611 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.611 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8086", 2024-09-12T03:40:27.611 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.611 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.611 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.611 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB server port", 2024-09-12T03:40:27.611 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.611 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.611 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.612 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.612 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl": { 2024-09-12T03:40:27.612 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl", 2024-09-12T03:40:27.612 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.612 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.612 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.612 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "false", 2024-09-12T03:40:27.612 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.613 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.613 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.613 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:40:27.613 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.613 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.613 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.613 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.613 INFO:teuthology.orchestra.run.smithi064.stdout: "threads": { 2024-09-12T03:40:27.613 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "threads", 2024-09-12T03:40:27.613 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.613 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.613 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.614 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-12T03:40:27.614 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-09-12T03:40:27.614 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "32", 2024-09-12T03:40:27.614 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.614 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T03:40:27.614 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.614 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.614 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.614 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.614 INFO:teuthology.orchestra.run.smithi064.stdout: "username": { 2024-09-12T03:40:27.614 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "username", 2024-09-12T03:40:27.614 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.614 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.614 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.615 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.615 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.615 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.615 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.615 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "username of InfluxDB server user", 2024-09-12T03:40:27.615 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.615 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.615 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.615 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.615 INFO:teuthology.orchestra.run.smithi064.stdout: "verify_ssl": { 2024-09-12T03:40:27.615 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "verify_ssl", 2024-09-12T03:40:27.615 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.616 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.616 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.616 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "true", 2024-09-12T03:40:27.616 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.616 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.616 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.616 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:40:27.616 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.616 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.616 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.616 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.616 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.616 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.617 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:27.617 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "insights", 2024-09-12T03:40:27.617 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:27.617 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:27.617 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:27.617 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:27.617 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:27.617 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.617 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.617 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.617 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.618 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.618 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.618 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.618 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.618 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.618 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.618 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.618 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.618 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.618 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.618 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.618 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.619 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.619 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.619 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.619 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:27.619 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:27.619 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.619 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.619 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.619 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.619 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.619 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.619 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.619 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.619 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.620 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.620 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.620 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.620 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:27.620 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:27.620 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.620 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.620 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.620 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:27.620 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.620 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.620 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.620 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.621 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.621 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.621 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.621 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.621 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.621 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.621 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.621 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.621 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.621 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.621 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.621 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:27.621 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:27.622 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.622 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.622 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.622 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.622 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.622 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.622 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.622 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.622 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.622 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.622 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.622 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.622 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.622 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.623 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:27.623 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "iostat", 2024-09-12T03:40:27.623 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:27.623 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:27.623 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:27.623 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:27.623 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:27.623 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.623 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.623 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.623 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.623 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.623 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.623 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.624 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.624 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.624 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.624 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.624 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.624 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.624 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.624 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.624 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.624 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.624 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.624 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.624 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:27.625 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:27.625 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.625 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.625 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.625 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.625 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.625 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.625 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.625 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.625 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.625 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.625 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.625 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.625 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:27.626 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:27.626 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.626 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.626 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.626 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:27.626 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.626 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.626 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.626 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.626 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.626 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.626 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.626 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.626 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.627 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.627 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.627 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.627 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.627 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.627 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.627 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:27.627 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:27.627 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.627 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.627 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.627 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.627 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.628 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.628 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.628 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.628 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.628 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.628 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.628 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.628 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.628 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.628 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:27.628 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "k8sevents", 2024-09-12T03:40:27.628 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:27.628 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:27.629 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:27.629 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_event_retention_days": { 2024-09-12T03:40:27.629 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ceph_event_retention_days", 2024-09-12T03:40:27.629 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.629 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.629 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.629 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "7", 2024-09-12T03:40:27.629 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.629 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.629 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.629 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T03:40:27.629 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.629 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.629 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.630 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.630 INFO:teuthology.orchestra.run.smithi064.stdout: "config_check_secs": { 2024-09-12T03:40:27.630 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_check_secs", 2024-09-12T03:40:27.630 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.630 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.630 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.630 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-12T03:40:27.630 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "10", 2024-09-12T03:40:27.630 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.630 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.630 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T03:40:27.631 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.631 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.631 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.631 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.631 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:27.631 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:27.631 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.631 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.631 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.631 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.631 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.631 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.631 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.632 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.632 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.632 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.632 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.632 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.632 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.632 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.632 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.632 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.632 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.632 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.632 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.632 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:27.633 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:27.633 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.633 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.633 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.633 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.633 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.633 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.633 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.633 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.633 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.633 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.634 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.634 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.634 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:27.634 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:27.634 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.634 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.634 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.634 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:27.634 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.634 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.634 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.634 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.634 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.634 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.635 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.635 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.635 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.635 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.635 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.635 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.635 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.635 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.635 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.635 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:27.635 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:27.635 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.635 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.636 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.636 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.636 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.636 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.636 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.636 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.636 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.636 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.636 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.636 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.636 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.636 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.636 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:27.637 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "localpool", 2024-09-12T03:40:27.637 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:27.637 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:27.637 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:27.637 INFO:teuthology.orchestra.run.smithi064.stdout: "failure_domain": { 2024-09-12T03:40:27.637 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "failure_domain", 2024-09-12T03:40:27.637 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.637 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.637 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.637 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "host", 2024-09-12T03:40:27.637 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.637 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.637 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.638 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "failure domain for any created local pool", 2024-09-12T03:40:27.638 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T03:40:27.638 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.638 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.638 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.638 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:27.638 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:27.638 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.638 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.638 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.638 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.638 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.638 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.639 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.639 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.639 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.639 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.639 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.639 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.639 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.639 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.639 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.639 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.639 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.639 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.640 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.640 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:27.640 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:27.640 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.640 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.640 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.640 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.640 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.640 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.640 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.640 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.640 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.640 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.641 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.641 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.641 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:27.641 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:27.641 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.641 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.641 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.641 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:27.641 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.641 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.641 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.641 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.641 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.641 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.642 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.642 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.642 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.642 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.642 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.642 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.642 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.642 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.642 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.642 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:27.642 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:27.642 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.642 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.642 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.643 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.643 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.643 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.643 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.643 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.643 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.643 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.643 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.643 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.643 INFO:teuthology.orchestra.run.smithi064.stdout: "min_size": { 2024-09-12T03:40:27.643 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "min_size", 2024-09-12T03:40:27.643 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.643 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.643 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.644 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.644 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.644 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.644 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.644 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default min_size for any created local pool", 2024-09-12T03:40:27.644 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T03:40:27.644 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.644 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.644 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.644 INFO:teuthology.orchestra.run.smithi064.stdout: "num_rep": { 2024-09-12T03:40:27.644 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "num_rep", 2024-09-12T03:40:27.644 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.644 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.645 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.645 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3", 2024-09-12T03:40:27.645 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.645 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.645 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.645 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default replica count for any created local pool", 2024-09-12T03:40:27.645 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.645 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.645 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.645 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.645 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_num": { 2024-09-12T03:40:27.646 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pg_num", 2024-09-12T03:40:27.646 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.646 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.646 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.646 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "128", 2024-09-12T03:40:27.646 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.646 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.646 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.646 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T03:40:27.646 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.646 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.646 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.646 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.647 INFO:teuthology.orchestra.run.smithi064.stdout: "prefix": { 2024-09-12T03:40:27.647 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prefix", 2024-09-12T03:40:27.647 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.647 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.647 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.647 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.647 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.647 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.647 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.647 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "name prefix for any created local pool", 2024-09-12T03:40:27.647 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.647 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.647 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.648 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.648 INFO:teuthology.orchestra.run.smithi064.stdout: "subtree": { 2024-09-12T03:40:27.648 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "subtree", 2024-09-12T03:40:27.648 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.648 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.648 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.648 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "rack", 2024-09-12T03:40:27.648 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.648 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.648 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.648 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T03:40:27.648 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T03:40:27.648 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.649 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.649 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.649 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.649 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.649 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:27.649 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mds_autoscaler", 2024-09-12T03:40:27.649 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:27.649 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:27.649 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:27.649 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:27.649 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:27.649 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.649 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.650 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.650 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.650 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.650 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.650 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.650 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.650 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.650 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.650 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.650 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.650 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.650 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.650 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.650 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.651 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.651 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.651 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.651 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:27.651 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:27.651 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.651 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.651 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.651 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.651 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.651 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.651 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.652 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.652 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.652 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.652 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.652 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.652 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:27.652 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:27.652 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.652 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.652 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.652 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:27.652 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.652 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.653 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.653 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.653 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.653 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.653 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.653 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.653 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.653 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.653 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.653 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.653 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.653 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.653 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.653 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:27.654 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:27.654 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.654 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.654 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.654 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.654 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.654 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.654 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.654 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.654 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.654 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.654 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.654 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.655 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.655 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.655 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:27.655 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mirroring", 2024-09-12T03:40:27.655 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:27.655 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:27.655 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:27.655 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:27.655 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:27.655 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.655 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.655 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.655 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.655 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.656 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.656 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.656 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.656 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.656 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.656 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.656 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.656 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.656 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.656 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.656 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.656 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.656 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.657 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.657 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:27.657 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:27.657 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.657 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.657 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.657 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.657 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.657 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.657 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.657 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.657 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.657 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.657 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.658 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.658 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:27.658 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:27.658 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.658 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.658 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.658 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:27.658 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.658 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.658 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.658 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.658 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.658 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.658 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.659 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.659 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.659 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.659 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.659 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.659 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.659 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.659 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.659 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:27.659 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:27.659 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.659 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.660 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.660 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.660 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.660 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.660 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.660 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.660 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.660 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.660 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.660 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.660 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.660 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.660 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:27.661 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "nfs", 2024-09-12T03:40:27.661 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:27.661 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:27.661 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:27.661 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:27.661 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:27.661 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.661 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.661 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.661 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.661 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.661 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.661 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.662 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.662 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.662 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.662 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.662 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.662 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.662 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.662 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.662 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.662 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.662 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.662 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.662 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:27.662 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:27.663 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.663 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.663 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.663 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.663 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.663 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.663 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.663 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.663 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.663 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.663 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.663 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.663 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:27.664 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:27.664 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.664 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.664 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.664 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:27.664 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.664 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.664 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.664 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.664 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.664 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.664 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.665 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.665 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.665 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.665 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.665 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.665 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.665 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.665 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.665 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:27.665 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:27.665 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.665 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.665 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.666 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.666 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.666 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.666 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.666 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.666 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.666 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.666 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.666 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.666 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.666 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.666 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:27.666 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "orchestrator", 2024-09-12T03:40:27.667 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:27.667 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:27.667 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:27.667 INFO:teuthology.orchestra.run.smithi064.stdout: "fail_fs": { 2024-09-12T03:40:27.667 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "fail_fs", 2024-09-12T03:40:27.667 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.667 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.667 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.667 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.667 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.667 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.667 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.667 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T03:40:27.668 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.668 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.668 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.668 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.668 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:27.668 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:27.668 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.668 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.668 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.668 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.668 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.668 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.668 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.669 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.669 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.669 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.669 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.669 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.669 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.669 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.669 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.669 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.669 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.669 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.669 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.669 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:27.669 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:27.670 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.670 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.670 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.670 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.670 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.670 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.670 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.670 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.670 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.670 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.670 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.670 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.670 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:27.671 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:27.671 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.671 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.671 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.671 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:27.671 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.671 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.671 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.671 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.671 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.671 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.671 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.672 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.672 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.672 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.672 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.672 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.672 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.672 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.672 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.672 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:27.672 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:27.672 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.672 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.672 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.672 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.673 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.673 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.673 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.673 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.673 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.673 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.673 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.673 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.673 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator": { 2024-09-12T03:40:27.673 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "orchestrator", 2024-09-12T03:40:27.673 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.673 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.673 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.673 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.674 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.674 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.674 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.674 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm", 2024-09-12T03:40:27.674 INFO:teuthology.orchestra.run.smithi064.stdout: "rook", 2024-09-12T03:40:27.674 INFO:teuthology.orchestra.run.smithi064.stdout: "test_orchestrator" 2024-09-12T03:40:27.674 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.674 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Orchestrator backend", 2024-09-12T03:40:27.674 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.674 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.674 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.674 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.674 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.675 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.675 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:27.675 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "osd_perf_query", 2024-09-12T03:40:27.675 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:27.675 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:27.675 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:27.675 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:27.675 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:27.675 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.675 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.675 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.675 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.675 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.676 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.676 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.676 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.676 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.676 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.676 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.676 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.676 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.676 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.676 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.676 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.676 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.677 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.677 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.677 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:27.677 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:27.677 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.677 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.677 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.677 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.677 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.677 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.677 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.677 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.677 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.677 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.678 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.678 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.678 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:27.678 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:27.678 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.678 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.678 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.678 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:27.678 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.678 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.678 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.678 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.678 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.678 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.679 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.679 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.679 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.679 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.679 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.679 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.679 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.679 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.679 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.679 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:27.679 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:27.679 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.679 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.680 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.680 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.680 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.680 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.680 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.680 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.680 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.680 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.680 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.680 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.680 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.680 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.680 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:27.680 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "osd_support", 2024-09-12T03:40:27.681 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:27.681 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:27.681 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:27.681 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:27.681 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:27.681 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.681 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.681 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.681 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.681 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.681 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.681 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.682 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.682 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.682 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.682 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.682 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.682 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.682 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.682 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.682 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.682 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.682 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.682 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.682 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:27.682 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:27.683 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.683 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.683 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.683 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.683 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.683 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.683 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.683 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.683 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.683 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.683 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.683 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.683 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:27.683 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:27.684 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.684 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.684 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.684 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:27.684 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.684 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.684 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.684 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.684 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.684 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.684 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.684 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.684 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.685 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.685 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.685 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.685 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.685 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.685 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.685 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:27.685 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:27.685 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.685 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.685 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.685 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.685 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.685 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.686 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.686 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.686 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.686 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.686 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.686 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.686 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.686 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.686 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:27.686 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pg_autoscaler", 2024-09-12T03:40:27.686 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:27.686 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:27.686 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:27.686 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:27.687 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:27.687 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.687 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.687 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.687 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.687 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.687 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.687 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.687 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.687 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.687 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.687 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.687 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.688 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.688 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.688 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.688 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.688 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.688 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.688 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.688 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:27.688 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:27.688 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.688 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.688 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.688 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.689 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.689 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.689 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.689 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.689 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.689 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.689 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.689 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.689 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:27.689 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:27.689 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.689 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.689 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.690 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:27.690 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.690 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.690 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.690 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.690 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.690 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.690 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.690 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.690 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.690 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.690 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.690 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.690 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.691 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.691 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.691 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:27.691 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:27.691 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.691 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.691 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.691 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.691 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.691 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.691 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.691 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.691 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.692 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.692 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.692 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.692 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-12T03:40:27.692 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-12T03:40:27.692 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:27.692 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.692 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.692 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-12T03:40:27.692 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.692 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.692 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.692 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.692 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.693 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.693 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.693 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.693 INFO:teuthology.orchestra.run.smithi064.stdout: "threshold": { 2024-09-12T03:40:27.693 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "threshold", 2024-09-12T03:40:27.693 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-12T03:40:27.693 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.693 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.693 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3.0", 2024-09-12T03:40:27.693 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1.0", 2024-09-12T03:40:27.693 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.693 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.693 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "scaling threshold", 2024-09-12T03:40:27.693 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T03:40:27.694 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.694 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.694 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.694 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.694 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.694 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:27.694 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "progress", 2024-09-12T03:40:27.694 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:27.694 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:27.694 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:27.694 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_pg_recovery_event": { 2024-09-12T03:40:27.694 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_pg_recovery_event", 2024-09-12T03:40:27.694 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.695 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.695 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.695 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.695 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.695 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.695 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.695 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T03:40:27.695 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.695 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.695 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.695 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.695 INFO:teuthology.orchestra.run.smithi064.stdout: "enabled": { 2024-09-12T03:40:27.695 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enabled", 2024-09-12T03:40:27.696 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.696 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.696 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.696 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.696 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.696 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.696 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.696 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.696 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.696 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.696 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.696 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.696 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:27.696 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:27.697 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.697 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.697 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.697 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.697 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.697 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.697 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.697 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.697 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.697 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.697 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.697 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.697 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.698 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.698 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.698 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.698 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.698 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.698 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.698 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:27.698 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:27.698 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.698 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.698 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.698 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.698 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.698 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.699 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.699 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.699 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.699 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.699 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.699 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.699 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:27.699 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:27.699 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.699 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.699 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.699 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:27.699 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.700 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.700 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.700 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.700 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.700 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.700 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.700 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.700 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.700 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.700 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.700 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.700 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.700 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.700 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.701 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:27.701 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:27.701 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.701 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.701 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.701 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.701 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.701 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.701 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.701 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.701 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.701 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.701 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.701 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.702 INFO:teuthology.orchestra.run.smithi064.stdout: "max_completed_events": { 2024-09-12T03:40:27.702 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_completed_events", 2024-09-12T03:40:27.702 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.702 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.702 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.702 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "50", 2024-09-12T03:40:27.702 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.702 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.702 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.702 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "number of past completed events to remember", 2024-09-12T03:40:27.702 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.702 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.702 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.702 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.703 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-12T03:40:27.703 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-12T03:40:27.703 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:27.703 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.703 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.703 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-12T03:40:27.703 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.703 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.703 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.703 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long the module is going to sleep", 2024-09-12T03:40:27.703 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.703 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.703 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.704 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.704 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.704 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.704 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:27.704 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prometheus", 2024-09-12T03:40:27.704 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:27.704 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:27.704 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:27.704 INFO:teuthology.orchestra.run.smithi064.stdout: "cache": { 2024-09-12T03:40:27.704 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cache", 2024-09-12T03:40:27.704 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.704 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.704 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.705 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.705 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.705 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.705 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.705 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.705 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.705 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.705 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.705 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.705 INFO:teuthology.orchestra.run.smithi064.stdout: "exclude_perf_counters": { 2024-09-12T03:40:27.705 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "exclude_perf_counters", 2024-09-12T03:40:27.705 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.705 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.706 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.706 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.706 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.706 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.706 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.706 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T03:40:27.706 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T03:40:27.706 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.706 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.706 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.706 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:27.706 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:27.706 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.706 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.707 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.707 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.707 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.707 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.707 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.707 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.707 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.707 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.707 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.707 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.707 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.707 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.707 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.708 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.708 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.708 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.708 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.708 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:27.708 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:27.708 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.708 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.708 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.708 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.708 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.708 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.708 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.709 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.709 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.709 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.709 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.709 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.709 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:27.709 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:27.709 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.709 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.709 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.709 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:27.709 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.709 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.710 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.710 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.710 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.710 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.710 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.710 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.710 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.710 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.710 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.710 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.710 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.710 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.710 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.710 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:27.711 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:27.711 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.711 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.711 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.711 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.711 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.711 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.711 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.711 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.711 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.711 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.711 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.711 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.712 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_stats_pools": { 2024-09-12T03:40:27.712 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_stats_pools", 2024-09-12T03:40:27.712 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.712 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.712 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.712 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.712 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.712 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.712 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.712 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.712 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.712 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.713 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.713 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.713 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T03:40:27.713 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T03:40:27.713 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.713 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.713 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.713 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "300", 2024-09-12T03:40:27.713 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.713 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.713 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.714 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.714 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.714 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.714 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.714 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.714 INFO:teuthology.orchestra.run.smithi064.stdout: "scrape_interval": { 2024-09-12T03:40:27.714 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "scrape_interval", 2024-09-12T03:40:27.714 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-12T03:40:27.714 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.714 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.714 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15.0", 2024-09-12T03:40:27.714 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.714 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.714 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.715 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.715 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.715 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.715 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.715 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.715 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-09-12T03:40:27.715 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-09-12T03:40:27.715 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.715 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.715 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.715 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "::", 2024-09-12T03:40:27.715 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.715 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.715 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.716 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T03:40:27.716 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.716 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.716 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.716 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.716 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-09-12T03:40:27.716 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-09-12T03:40:27.716 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.716 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.716 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.716 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "9283", 2024-09-12T03:40:27.716 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.716 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.717 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.717 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T03:40:27.717 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.717 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.717 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.717 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.717 INFO:teuthology.orchestra.run.smithi064.stdout: "stale_cache_strategy": { 2024-09-12T03:40:27.717 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "stale_cache_strategy", 2024-09-12T03:40:27.717 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.717 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.717 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.717 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "log", 2024-09-12T03:40:27.717 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.718 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.718 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.718 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.718 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.718 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.718 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.718 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.718 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_behaviour": { 2024-09-12T03:40:27.718 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_behaviour", 2024-09-12T03:40:27.718 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.718 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.718 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.718 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "default", 2024-09-12T03:40:27.718 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.719 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.719 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.719 INFO:teuthology.orchestra.run.smithi064.stdout: "default", 2024-09-12T03:40:27.719 INFO:teuthology.orchestra.run.smithi064.stdout: "error" 2024-09-12T03:40:27.719 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.719 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.719 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.719 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.719 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.719 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.719 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_error_status_code": { 2024-09-12T03:40:27.719 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_error_status_code", 2024-09-12T03:40:27.719 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.720 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.720 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.720 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2024-09-12T03:40:27.720 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "400", 2024-09-12T03:40:27.720 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "599", 2024-09-12T03:40:27.720 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.720 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.720 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.720 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.720 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.720 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.721 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.721 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.721 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:27.721 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_support", 2024-09-12T03:40:27.721 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:27.721 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:27.721 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:27.721 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:27.721 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:27.721 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.721 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.721 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.721 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.722 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.722 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.722 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.722 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.722 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.722 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.722 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.722 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.722 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.722 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.722 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.722 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.722 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.723 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.723 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.723 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:27.723 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:27.723 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.723 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.723 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.723 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.723 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.723 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.723 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.723 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.723 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.724 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.724 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.724 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.724 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:27.724 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:27.724 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.724 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.724 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.724 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:27.724 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.724 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.724 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.724 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.724 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.725 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.725 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.725 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.725 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.725 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.725 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.725 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.725 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.725 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.725 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.725 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:27.725 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:27.725 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.726 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.726 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.726 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.726 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.726 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.726 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.726 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.726 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.726 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.728 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.728 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.728 INFO:teuthology.orchestra.run.smithi064.stdout: "max_concurrent_snap_create": { 2024-09-12T03:40:27.729 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_concurrent_snap_create", 2024-09-12T03:40:27.729 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.729 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.729 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.729 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-12T03:40:27.730 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.730 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.730 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.730 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.730 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.730 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.730 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.731 INFO:teuthology.orchestra.run.smithi064.stdout: "mirror_snapshot_schedule": { 2024-09-12T03:40:27.731 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T03:40:27.731 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.731 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.731 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.732 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.732 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.732 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.732 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.732 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.732 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.733 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.733 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.733 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.733 INFO:teuthology.orchestra.run.smithi064.stdout: "trash_purge_schedule": { 2024-09-12T03:40:27.733 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "trash_purge_schedule", 2024-09-12T03:40:27.733 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.734 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.734 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.734 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.734 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.734 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.734 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.735 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.735 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.735 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.735 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.735 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.735 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.735 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.736 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:27.736 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "restful", 2024-09-12T03:40:27.736 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:27.736 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:27.736 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:27.736 INFO:teuthology.orchestra.run.smithi064.stdout: "enable_auth": { 2024-09-12T03:40:27.736 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enable_auth", 2024-09-12T03:40:27.737 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.737 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.737 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.737 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.737 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.737 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.738 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.738 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.738 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.738 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.738 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.739 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.739 INFO:teuthology.orchestra.run.smithi064.stdout: "key_file": { 2024-09-12T03:40:27.739 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "key_file", 2024-09-12T03:40:27.739 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.739 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.740 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.740 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.740 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.740 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.740 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.740 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.741 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.741 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.741 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.741 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.742 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:27.742 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:27.742 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.742 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.742 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.742 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.742 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.743 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.743 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.743 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.743 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.743 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.743 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.743 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.743 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.744 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.744 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.744 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.744 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.744 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.744 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.744 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:27.745 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:27.745 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.745 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.745 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.745 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.745 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.745 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.746 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.746 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.746 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.746 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.746 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.746 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.747 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:27.747 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:27.747 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.747 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.747 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.748 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:27.748 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.748 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.748 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.748 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.748 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.748 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.748 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.748 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.748 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.748 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.748 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.749 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.749 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.749 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.749 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.749 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:27.749 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:27.749 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.749 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.749 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.749 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.749 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.749 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.750 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.750 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.750 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.750 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.750 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.750 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.751 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-09-12T03:40:27.751 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-09-12T03:40:27.751 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.751 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.751 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.751 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.751 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.751 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.751 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.751 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.751 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.751 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.752 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.752 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.752 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-09-12T03:40:27.752 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-09-12T03:40:27.752 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.752 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.752 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.752 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.752 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.752 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.752 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.752 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.752 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.753 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.753 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.753 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.753 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.753 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.753 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:27.753 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rgw", 2024-09-12T03:40:27.753 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:27.753 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:27.753 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:27.753 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:27.753 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:27.754 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.754 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.754 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.754 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.754 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.754 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.754 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.754 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.754 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.754 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.754 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.754 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.754 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.754 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.755 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.755 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.755 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.755 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.755 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.755 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:27.755 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:27.755 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.755 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.755 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.755 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.755 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.755 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.756 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.756 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.756 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.756 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.756 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.756 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.756 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:27.756 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:27.756 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.756 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.756 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.756 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:27.756 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.757 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.757 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.757 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.757 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.757 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.757 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.757 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.757 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.757 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.758 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.758 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.758 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.758 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.758 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.758 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:27.758 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:27.758 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.758 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.758 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.758 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.758 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.758 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.758 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.759 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.759 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.759 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.759 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.759 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.759 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.759 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.759 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:27.759 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rook", 2024-09-12T03:40:27.759 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:27.760 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:27.760 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:27.760 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:27.760 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:27.760 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.760 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.760 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.760 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.760 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.760 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.761 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.761 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.761 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.761 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.761 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.762 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.762 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.762 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.762 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.762 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.762 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.762 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.762 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.763 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:27.763 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:27.763 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.763 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.763 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.763 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.763 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.763 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.764 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.764 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.764 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.764 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.764 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.764 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.764 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:27.764 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:27.764 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.764 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.764 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.765 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:27.765 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.765 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.765 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.765 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.765 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.765 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.765 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.765 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.765 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.765 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.766 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.766 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.766 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.766 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.766 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.766 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:27.766 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:27.766 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.766 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.766 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.766 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.766 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.766 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.767 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.767 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.767 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.767 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.767 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.767 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.767 INFO:teuthology.orchestra.run.smithi064.stdout: "storage_class": { 2024-09-12T03:40:27.767 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "storage_class", 2024-09-12T03:40:27.767 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.767 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.767 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.767 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "local", 2024-09-12T03:40:27.768 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.768 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.768 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.768 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T03:40:27.768 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.768 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.768 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.768 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.768 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.768 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.768 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:27.768 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "selftest", 2024-09-12T03:40:27.769 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:27.769 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:27.769 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:27.769 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:27.769 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:27.769 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.769 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.769 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.769 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.769 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.770 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.770 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.770 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.770 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.770 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.770 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.770 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.770 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.770 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.770 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.771 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.771 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.771 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.771 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.771 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:27.771 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:27.771 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.771 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.771 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.771 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.771 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.771 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.771 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.772 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.772 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.772 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.772 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.772 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.772 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:27.772 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:27.772 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.772 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.772 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.772 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:27.772 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.773 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.773 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.773 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.773 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.773 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.773 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.773 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.773 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.773 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.773 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.773 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.773 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.773 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.774 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.774 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:27.774 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:27.774 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.774 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.774 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.774 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.774 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.774 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.774 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.774 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.775 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.775 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.775 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.775 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.775 INFO:teuthology.orchestra.run.smithi064.stdout: "roption1": { 2024-09-12T03:40:27.775 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "roption1", 2024-09-12T03:40:27.775 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.775 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.775 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.775 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.775 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.775 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.775 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.776 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.776 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.776 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.776 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.776 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.776 INFO:teuthology.orchestra.run.smithi064.stdout: "roption2": { 2024-09-12T03:40:27.776 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "roption2", 2024-09-12T03:40:27.776 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.776 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.776 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.776 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "xyz", 2024-09-12T03:40:27.776 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.776 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.777 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.777 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.777 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.777 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.777 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.777 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.777 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption1": { 2024-09-12T03:40:27.777 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption1", 2024-09-12T03:40:27.777 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.777 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.777 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.777 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.778 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.778 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.778 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.778 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.778 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.778 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.778 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.778 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.778 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption2": { 2024-09-12T03:40:27.778 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption2", 2024-09-12T03:40:27.778 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.778 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.778 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.779 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.779 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.779 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.779 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.779 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.779 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.779 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.779 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.779 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.779 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption3": { 2024-09-12T03:40:27.779 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption3", 2024-09-12T03:40:27.779 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-12T03:40:27.779 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.780 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.780 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.780 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.780 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.780 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.780 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.780 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.780 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.780 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.780 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.781 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption4": { 2024-09-12T03:40:27.781 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption4", 2024-09-12T03:40:27.781 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.781 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.781 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.781 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.781 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.781 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.781 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.781 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.782 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.782 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.782 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.782 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.782 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption5": { 2024-09-12T03:40:27.782 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption5", 2024-09-12T03:40:27.782 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.782 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.782 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.782 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.782 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.782 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.782 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.783 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.783 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.783 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.783 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.783 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.783 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption6": { 2024-09-12T03:40:27.783 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption6", 2024-09-12T03:40:27.783 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.783 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.783 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.783 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.784 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.784 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.784 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.784 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.784 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.784 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.784 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.784 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.784 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption7": { 2024-09-12T03:40:27.784 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption7", 2024-09-12T03:40:27.785 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.785 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.785 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.785 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.785 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-09-12T03:40:27.785 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "42", 2024-09-12T03:40:27.785 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.785 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.785 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.785 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.785 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.785 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.785 INFO:teuthology.orchestra.run.smithi064.stdout: "testkey": { 2024-09-12T03:40:27.786 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testkey", 2024-09-12T03:40:27.786 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.786 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.786 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.786 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.786 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.786 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.786 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.786 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.787 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.787 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.787 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.787 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.787 INFO:teuthology.orchestra.run.smithi064.stdout: "testlkey": { 2024-09-12T03:40:27.787 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testlkey", 2024-09-12T03:40:27.787 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.787 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.787 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.787 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.787 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.787 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.787 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.788 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.788 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.788 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.788 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.788 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.788 INFO:teuthology.orchestra.run.smithi064.stdout: "testnewline": { 2024-09-12T03:40:27.788 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testnewline", 2024-09-12T03:40:27.788 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.788 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.788 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.789 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.789 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.789 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.789 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.789 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.789 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.789 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.789 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.789 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.789 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.790 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.790 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:27.790 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snap_schedule", 2024-09-12T03:40:27.790 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:27.790 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:27.790 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:27.790 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_m_granularity": { 2024-09-12T03:40:27.790 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_m_granularity", 2024-09-12T03:40:27.790 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.790 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.790 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.790 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.790 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.791 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.791 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.791 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T03:40:27.791 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.791 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.791 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.791 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.791 INFO:teuthology.orchestra.run.smithi064.stdout: "dump_on_update": { 2024-09-12T03:40:27.791 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "dump_on_update", 2024-09-12T03:40:27.791 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.791 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.791 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.791 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.791 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.792 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.792 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.792 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "dump database to debug log on update", 2024-09-12T03:40:27.792 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.792 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.792 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.792 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.792 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:27.792 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:27.793 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.793 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.793 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.793 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.793 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.793 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.793 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.793 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.793 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.793 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.793 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.793 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.794 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.794 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.794 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.794 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.794 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.794 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.794 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.794 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:27.794 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:27.794 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.795 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.795 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.795 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.795 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.795 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.795 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.795 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.795 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.795 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.795 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.795 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.795 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:27.795 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:27.796 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.796 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.796 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.796 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:27.796 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.796 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.796 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.796 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.796 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.796 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.796 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.796 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.797 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.797 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.797 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.797 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.797 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.797 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.797 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.797 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:27.797 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:27.797 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.797 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.797 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.797 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.798 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.798 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.798 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.798 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.798 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.798 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.798 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.798 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.798 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.798 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.798 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:27.799 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "stats", 2024-09-12T03:40:27.799 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:27.799 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:27.799 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:27.799 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:27.799 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:27.799 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.799 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.799 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.799 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.799 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.799 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.799 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.799 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.800 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.800 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.800 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.800 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.800 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.800 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.800 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.800 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.800 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.800 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.800 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.800 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:27.801 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:27.801 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.801 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.801 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.801 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.801 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.801 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.801 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.802 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.802 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.802 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.802 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.802 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.802 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:27.802 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:27.802 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.802 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.802 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.803 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:27.803 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.803 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.803 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.803 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.803 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.803 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.803 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.803 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.803 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.803 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.804 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.804 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.804 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.804 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.804 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.804 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:27.804 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:27.804 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.805 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.805 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.805 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.805 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.805 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.805 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.805 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.805 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.805 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.806 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.806 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.806 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.806 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.806 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:27.806 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "status", 2024-09-12T03:40:27.806 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:27.806 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:27.806 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:27.806 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:27.807 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:27.807 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.807 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.807 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.807 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.807 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.808 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.808 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.808 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.808 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.808 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.808 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.808 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.809 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.809 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.809 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.809 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.809 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.809 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.809 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.809 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:27.809 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:27.809 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.809 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.809 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.809 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.810 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.810 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.810 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.810 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.810 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.810 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.810 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.810 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.810 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:27.810 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:27.810 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.810 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.811 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.811 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:27.811 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.811 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.811 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.811 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.812 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.812 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.812 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.812 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.812 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.812 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.813 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.813 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.813 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.813 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.814 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.814 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:27.814 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:27.814 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.814 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.814 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.814 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.814 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.814 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.814 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.815 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.815 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.815 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.815 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.815 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.815 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.815 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.815 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:27.815 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "telegraf", 2024-09-12T03:40:27.815 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:27.815 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:27.815 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:27.816 INFO:teuthology.orchestra.run.smithi064.stdout: "address": { 2024-09-12T03:40:27.816 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "address", 2024-09-12T03:40:27.816 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.816 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.816 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.816 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T03:40:27.816 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.816 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.816 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.817 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.817 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.817 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.817 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.817 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.817 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-12T03:40:27.817 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-12T03:40:27.817 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:27.817 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.818 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.818 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15", 2024-09-12T03:40:27.818 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.818 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.818 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.818 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.819 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.819 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.819 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.819 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.819 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:27.820 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:27.820 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.820 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.820 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.820 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.820 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.820 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.820 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.820 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.820 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.821 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.821 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.821 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.821 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.821 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.821 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.821 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.822 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.822 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.822 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.822 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:27.822 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:27.822 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.823 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.823 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.823 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.823 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.823 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.823 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.823 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.824 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.824 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.824 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.824 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.824 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:27.824 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:27.824 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.824 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.824 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.824 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:27.825 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.825 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.825 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.825 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.825 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.825 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.825 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.825 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.826 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.826 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.826 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.826 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.826 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.826 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.826 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.826 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:27.826 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:27.826 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.826 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.827 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.827 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.827 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.827 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.827 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.827 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.827 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.827 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.827 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.827 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.827 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.828 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.828 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:27.828 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "telemetry", 2024-09-12T03:40:27.828 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:27.828 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:27.828 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:27.828 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_basic": { 2024-09-12T03:40:27.828 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_basic", 2024-09-12T03:40:27.828 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.828 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.828 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.828 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.829 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.829 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.829 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.829 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T03:40:27.829 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.829 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.829 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.829 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.829 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_crash": { 2024-09-12T03:40:27.829 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_crash", 2024-09-12T03:40:27.829 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.829 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.829 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.830 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.830 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.830 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.830 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.830 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T03:40:27.830 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.830 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.830 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.830 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.830 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_device": { 2024-09-12T03:40:27.830 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_device", 2024-09-12T03:40:27.830 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.830 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.830 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.831 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.831 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.831 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.831 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.831 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T03:40:27.831 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.831 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.831 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.831 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.831 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_ident": { 2024-09-12T03:40:27.831 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_ident", 2024-09-12T03:40:27.831 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.831 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.832 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.832 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.832 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.832 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.832 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.832 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T03:40:27.832 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.832 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.832 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.832 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.832 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_perf": { 2024-09-12T03:40:27.832 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_perf", 2024-09-12T03:40:27.832 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.833 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.833 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.833 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.833 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.833 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.833 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.834 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T03:40:27.834 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.834 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.834 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.834 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.834 INFO:teuthology.orchestra.run.smithi064.stdout: "contact": { 2024-09-12T03:40:27.835 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "contact", 2024-09-12T03:40:27.835 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.835 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.835 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.835 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.835 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.836 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.836 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.836 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.836 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.836 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.836 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.837 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.837 INFO:teuthology.orchestra.run.smithi064.stdout: "description": { 2024-09-12T03:40:27.837 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "description", 2024-09-12T03:40:27.837 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.837 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.838 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.838 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.838 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.838 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.838 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.838 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.839 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.839 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.839 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.839 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.839 INFO:teuthology.orchestra.run.smithi064.stdout: "device_url": { 2024-09-12T03:40:27.839 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_url", 2024-09-12T03:40:27.840 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.840 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.840 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.840 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T03:40:27.840 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.841 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.841 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.841 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.841 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.841 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.841 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.842 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.842 INFO:teuthology.orchestra.run.smithi064.stdout: "enabled": { 2024-09-12T03:40:27.842 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enabled", 2024-09-12T03:40:27.842 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.842 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.843 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.843 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.843 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.843 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.843 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.843 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.844 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.844 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.844 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.844 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.844 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-12T03:40:27.844 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-12T03:40:27.845 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.845 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.845 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.845 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "24", 2024-09-12T03:40:27.845 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "8", 2024-09-12T03:40:27.846 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.846 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.846 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.846 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.846 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.846 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.847 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.847 INFO:teuthology.orchestra.run.smithi064.stdout: "last_opt_revision": { 2024-09-12T03:40:27.847 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "last_opt_revision", 2024-09-12T03:40:27.847 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.847 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.847 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.848 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1", 2024-09-12T03:40:27.848 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.848 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.848 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.848 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.849 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.849 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.849 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.849 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.849 INFO:teuthology.orchestra.run.smithi064.stdout: "leaderboard": { 2024-09-12T03:40:27.850 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "leaderboard", 2024-09-12T03:40:27.850 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.850 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.850 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.850 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.850 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.851 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.851 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.851 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.851 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.851 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.851 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.852 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.852 INFO:teuthology.orchestra.run.smithi064.stdout: "leaderboard_description": { 2024-09-12T03:40:27.852 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "leaderboard_description", 2024-09-12T03:40:27.852 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.852 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.853 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.853 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.853 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.853 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.853 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.853 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.854 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.854 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.854 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.854 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:27.854 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:27.855 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.855 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.855 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.855 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.855 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.856 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.856 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.856 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.856 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.856 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.857 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.857 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.857 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.857 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.857 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.857 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.858 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.858 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.858 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.858 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:27.858 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:27.859 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.859 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.859 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.859 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.859 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.859 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.860 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.860 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.860 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.860 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.860 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.860 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.861 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:27.861 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:27.861 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.861 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.861 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.861 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:27.861 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.862 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.862 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.862 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.862 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.862 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.862 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.862 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.862 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.863 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.863 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.863 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.863 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.863 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.863 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.863 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:27.864 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:27.864 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.864 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.864 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.864 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.864 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.864 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.865 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.865 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.865 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.865 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.865 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.865 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.865 INFO:teuthology.orchestra.run.smithi064.stdout: "organization": { 2024-09-12T03:40:27.865 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "organization", 2024-09-12T03:40:27.866 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.866 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.866 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.866 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.866 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.866 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.866 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.866 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.866 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.866 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.867 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.867 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.867 INFO:teuthology.orchestra.run.smithi064.stdout: "proxy": { 2024-09-12T03:40:27.867 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "proxy", 2024-09-12T03:40:27.867 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.867 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.867 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.867 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.867 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.867 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.867 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.867 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.868 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.868 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.868 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.868 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.868 INFO:teuthology.orchestra.run.smithi064.stdout: "url": { 2024-09-12T03:40:27.868 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "url", 2024-09-12T03:40:27.868 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.868 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.868 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.868 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T03:40:27.869 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.869 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.869 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.869 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.869 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.869 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.869 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.869 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.869 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.869 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.869 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:27.869 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "test_orchestrator", 2024-09-12T03:40:27.870 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:27.870 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:27.870 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:27.870 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:27.870 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:27.870 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.870 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.870 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.870 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.870 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.870 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.871 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.871 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.871 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.871 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.871 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.871 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.871 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.871 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.871 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.871 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.871 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.871 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.871 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.872 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:27.872 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:27.872 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.872 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.872 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.872 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.872 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.872 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.872 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.872 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.872 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.872 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.873 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.873 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.873 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:27.873 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:27.873 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.873 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.873 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.873 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:27.873 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.873 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.873 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.873 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.874 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.874 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.874 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.874 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.874 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.874 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.874 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.875 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.876 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.876 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.876 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.876 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:27.876 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:27.876 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.876 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.876 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.876 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.876 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.877 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.877 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.877 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.877 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.877 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.877 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.877 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.877 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.877 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.877 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:27.877 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "volumes", 2024-09-12T03:40:27.877 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:27.878 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:27.878 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:27.878 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:27.878 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:27.878 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.878 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.878 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.878 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.878 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.878 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.878 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.878 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.879 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.879 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.879 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.879 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.879 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.879 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.879 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.879 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.879 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.879 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.879 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.879 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:27.879 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:27.880 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.880 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.880 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.880 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.880 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.880 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.880 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.880 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.880 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.880 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.880 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.880 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.881 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:27.881 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:27.881 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.881 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.881 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.881 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:27.881 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.881 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.882 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.882 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.882 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.882 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.882 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.882 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.883 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.883 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.883 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.883 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.883 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.883 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.884 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.884 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:27.884 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:27.884 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.884 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.884 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.885 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.885 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.885 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.885 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.885 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.885 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.885 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.886 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.886 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.886 INFO:teuthology.orchestra.run.smithi064.stdout: "max_concurrent_clones": { 2024-09-12T03:40:27.886 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_concurrent_clones", 2024-09-12T03:40:27.886 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.886 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.886 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.886 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "4", 2024-09-12T03:40:27.887 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.887 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.887 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.887 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T03:40:27.887 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.887 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.887 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.888 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.888 INFO:teuthology.orchestra.run.smithi064.stdout: "periodic_async_work": { 2024-09-12T03:40:27.888 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "periodic_async_work", 2024-09-12T03:40:27.888 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.888 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.889 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.889 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.889 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.889 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.889 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.889 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Periodically check for async work", 2024-09-12T03:40:27.889 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.890 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.890 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.890 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.890 INFO:teuthology.orchestra.run.smithi064.stdout: "snapshot_clone_delay": { 2024-09-12T03:40:27.890 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snapshot_clone_delay", 2024-09-12T03:40:27.891 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.891 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.891 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.891 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-12T03:40:27.891 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.891 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.891 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.891 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T03:40:27.891 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.891 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.891 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.892 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.892 INFO:teuthology.orchestra.run.smithi064.stdout: "snapshot_clone_no_wait": { 2024-09-12T03:40:27.892 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T03:40:27.892 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.892 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.892 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.892 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.892 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.892 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.892 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.893 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T03:40:27.893 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.893 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.893 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.893 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.893 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.893 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.893 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:27.893 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix", 2024-09-12T03:40:27.893 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:27.893 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:27.894 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:27.894 INFO:teuthology.orchestra.run.smithi064.stdout: "discovery_interval": { 2024-09-12T03:40:27.894 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "discovery_interval", 2024-09-12T03:40:27.894 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-12T03:40:27.894 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.894 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.894 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "100", 2024-09-12T03:40:27.894 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.894 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.894 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.894 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.895 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.895 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.895 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.895 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.895 INFO:teuthology.orchestra.run.smithi064.stdout: "identifier": { 2024-09-12T03:40:27.895 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "identifier", 2024-09-12T03:40:27.895 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.895 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.895 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.895 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.895 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.895 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.895 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.896 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.896 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.896 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.896 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.896 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.896 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-12T03:40:27.896 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-12T03:40:27.896 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:27.896 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.897 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.897 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-12T03:40:27.897 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.897 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.897 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.897 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.897 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.897 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.897 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.897 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.897 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:27.897 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:27.898 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.898 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.898 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.898 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.898 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.898 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.898 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.898 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.898 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.898 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.898 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.898 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.898 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.898 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.899 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.899 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.899 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.899 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.899 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.899 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:27.899 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:27.899 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.899 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.899 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.899 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.899 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.899 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.900 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.900 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.900 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.900 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.900 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.900 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.900 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:27.900 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:27.900 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.900 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.900 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.900 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:27.901 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.901 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.901 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.901 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.901 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.901 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.901 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.901 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.901 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.901 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.902 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.902 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.902 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.902 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.902 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.902 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:27.902 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:27.902 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.902 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.902 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.902 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.902 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.903 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.903 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.903 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.903 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.903 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.903 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.903 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.903 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_host": { 2024-09-12T03:40:27.903 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_host", 2024-09-12T03:40:27.903 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.904 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.904 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.904 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.904 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.904 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.904 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.904 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.904 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.905 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.905 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.905 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.905 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_port": { 2024-09-12T03:40:27.905 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_port", 2024-09-12T03:40:27.905 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.905 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.905 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.905 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10051", 2024-09-12T03:40:27.905 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.905 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.906 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.906 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.906 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.906 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.906 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.906 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.906 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_sender": { 2024-09-12T03:40:27.906 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_sender", 2024-09-12T03:40:27.906 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.906 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.906 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.907 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T03:40:27.907 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.907 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.907 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.907 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.907 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.907 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.907 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.908 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.908 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.908 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.908 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T03:40:27.908 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.908 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:27.908 INFO:teuthology.orchestra.run.smithi064.stdout: "gid": 24934, 2024-09-12T03:40:27.908 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "z", 2024-09-12T03:40:27.908 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr_features": 4540701547738038271, 2024-09-12T03:40:27.908 INFO:teuthology.orchestra.run.smithi064.stdout: "available_modules": [ 2024-09-12T03:40:27.908 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:27.908 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "alerts", 2024-09-12T03:40:27.909 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:27.909 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:27.909 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:27.909 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-12T03:40:27.909 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-12T03:40:27.909 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:27.909 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.909 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.909 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-12T03:40:27.909 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.909 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.910 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.910 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T03:40:27.910 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.910 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.910 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.910 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.910 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:27.910 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:27.910 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.910 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.910 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.910 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.910 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.911 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.911 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.911 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.911 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.911 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.911 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.911 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.911 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.911 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.911 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.911 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.911 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.912 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.912 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.912 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:27.912 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:27.912 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.912 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.912 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.912 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.912 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.912 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.912 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.912 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.913 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.913 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.913 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.913 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.913 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:27.913 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:27.913 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.913 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.913 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.913 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:27.913 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.913 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.913 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.914 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.914 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.914 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.914 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.914 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.914 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.914 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.914 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.914 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.914 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.914 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.915 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.915 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:27.915 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:27.915 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.915 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.915 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.915 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.915 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.915 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.915 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.915 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.915 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.915 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.916 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.916 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.916 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_destination": { 2024-09-12T03:40:27.916 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_destination", 2024-09-12T03:40:27.916 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.916 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.916 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.916 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.916 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.916 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.916 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.917 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Email address to send alerts to", 2024-09-12T03:40:27.917 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.917 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.917 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.917 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.917 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_from_name": { 2024-09-12T03:40:27.917 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_from_name", 2024-09-12T03:40:27.917 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.917 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.917 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.918 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "Ceph", 2024-09-12T03:40:27.918 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.918 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.918 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.918 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Email From: name", 2024-09-12T03:40:27.918 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.918 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.918 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.918 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.918 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_host": { 2024-09-12T03:40:27.918 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_host", 2024-09-12T03:40:27.919 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.919 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.919 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.919 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.919 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.919 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.919 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.919 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP server", 2024-09-12T03:40:27.919 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.919 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.919 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.919 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.919 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_password": { 2024-09-12T03:40:27.920 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_password", 2024-09-12T03:40:27.920 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.920 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.920 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.920 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.920 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.920 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.920 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.920 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Password to authenticate with", 2024-09-12T03:40:27.920 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.920 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.920 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.921 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.921 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_port": { 2024-09-12T03:40:27.921 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_port", 2024-09-12T03:40:27.921 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.921 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.921 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.921 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "465", 2024-09-12T03:40:27.921 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.921 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.921 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.921 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP port", 2024-09-12T03:40:27.921 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.922 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.922 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.922 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.922 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_sender": { 2024-09-12T03:40:27.922 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_sender", 2024-09-12T03:40:27.922 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.922 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.922 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.922 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.922 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.922 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.922 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.923 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP envelope sender", 2024-09-12T03:40:27.923 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.923 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.923 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.923 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.923 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_ssl": { 2024-09-12T03:40:27.923 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_ssl", 2024-09-12T03:40:27.923 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.923 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.923 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.923 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.923 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.924 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.924 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.924 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T03:40:27.924 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.924 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.924 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.924 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.924 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_user": { 2024-09-12T03:40:27.925 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_user", 2024-09-12T03:40:27.925 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.925 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.925 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.925 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.925 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.925 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.925 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.925 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "User to authenticate as", 2024-09-12T03:40:27.925 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.925 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.925 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.926 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.926 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.926 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.926 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:27.926 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "balancer", 2024-09-12T03:40:27.926 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:27.926 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:27.926 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:27.926 INFO:teuthology.orchestra.run.smithi064.stdout: "active": { 2024-09-12T03:40:27.926 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "active", 2024-09-12T03:40:27.926 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.926 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.927 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.927 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.927 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.927 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.927 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.927 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T03:40:27.927 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.927 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.927 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.927 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.927 INFO:teuthology.orchestra.run.smithi064.stdout: "begin_time": { 2024-09-12T03:40:27.927 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "begin_time", 2024-09-12T03:40:27.928 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.928 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.928 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.928 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0000", 2024-09-12T03:40:27.928 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.928 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.929 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.929 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T03:40:27.929 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:40:27.929 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.929 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.929 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.929 INFO:teuthology.orchestra.run.smithi064.stdout: "begin_weekday": { 2024-09-12T03:40:27.930 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "begin_weekday", 2024-09-12T03:40:27.930 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-12T03:40:27.930 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.930 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.930 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-12T03:40:27.930 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0", 2024-09-12T03:40:27.930 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "6", 2024-09-12T03:40:27.930 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.930 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T03:40:27.930 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:40:27.930 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.930 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.930 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.931 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_max_iterations": { 2024-09-12T03:40:27.931 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_max_iterations", 2024-09-12T03:40:27.931 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-12T03:40:27.931 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.931 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.931 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "25", 2024-09-12T03:40:27.931 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-09-12T03:40:27.931 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "250", 2024-09-12T03:40:27.931 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.931 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T03:40:27.931 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.931 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.932 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.932 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.932 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_metrics": { 2024-09-12T03:40:27.932 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_metrics", 2024-09-12T03:40:27.932 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.932 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.932 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.932 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T03:40:27.932 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.932 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.932 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.932 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T03:40:27.932 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T03:40:27.933 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.933 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.933 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.933 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_step": { 2024-09-12T03:40:27.933 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_step", 2024-09-12T03:40:27.933 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-12T03:40:27.933 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.933 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.933 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0.5", 2024-09-12T03:40:27.933 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0.001", 2024-09-12T03:40:27.933 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "0.999", 2024-09-12T03:40:27.933 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.933 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "aggressiveness of optimization", 2024-09-12T03:40:27.933 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T03:40:27.934 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.934 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.934 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.934 INFO:teuthology.orchestra.run.smithi064.stdout: "end_time": { 2024-09-12T03:40:27.934 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "end_time", 2024-09-12T03:40:27.934 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.934 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.934 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.934 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "2359", 2024-09-12T03:40:27.934 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.934 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.934 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.934 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T03:40:27.934 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:40:27.935 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.935 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.935 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.935 INFO:teuthology.orchestra.run.smithi064.stdout: "end_weekday": { 2024-09-12T03:40:27.935 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "end_weekday", 2024-09-12T03:40:27.935 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-12T03:40:27.935 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.935 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.935 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-12T03:40:27.935 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0", 2024-09-12T03:40:27.935 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "6", 2024-09-12T03:40:27.935 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.935 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T03:40:27.935 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:40:27.936 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.936 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.936 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.936 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:27.936 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:27.936 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.936 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.936 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.936 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.936 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.936 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.936 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.936 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.937 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.937 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.937 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.937 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.937 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.937 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.937 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.937 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.937 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.937 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.937 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.937 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:27.937 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:27.937 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.938 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.938 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.938 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.938 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.938 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.938 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.938 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.938 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.938 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.938 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.938 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.938 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:27.938 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:27.938 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.939 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.939 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.939 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:27.939 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.939 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.939 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.939 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.939 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.939 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.939 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.939 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.939 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.940 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.940 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.940 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.940 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.940 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.940 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.940 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:27.940 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:27.940 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.940 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.940 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.940 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.940 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.941 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.941 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.941 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.941 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.941 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.941 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.941 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.941 INFO:teuthology.orchestra.run.smithi064.stdout: "min_score": { 2024-09-12T03:40:27.941 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "min_score", 2024-09-12T03:40:27.941 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-12T03:40:27.941 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.941 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.941 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-12T03:40:27.942 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.942 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.942 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.942 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T03:40:27.942 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.942 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.942 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.942 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.942 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": { 2024-09-12T03:40:27.942 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mode", 2024-09-12T03:40:27.942 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.942 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.942 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.942 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "upmap", 2024-09-12T03:40:27.943 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.943 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.943 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.943 INFO:teuthology.orchestra.run.smithi064.stdout: "crush-compat", 2024-09-12T03:40:27.943 INFO:teuthology.orchestra.run.smithi064.stdout: "none", 2024-09-12T03:40:27.943 INFO:teuthology.orchestra.run.smithi064.stdout: "read", 2024-09-12T03:40:27.943 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap", 2024-09-12T03:40:27.943 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap-read" 2024-09-12T03:40:27.943 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.943 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Balancer mode", 2024-09-12T03:40:27.943 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.943 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.943 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.944 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.944 INFO:teuthology.orchestra.run.smithi064.stdout: "pool_ids": { 2024-09-12T03:40:27.944 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pool_ids", 2024-09-12T03:40:27.944 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.944 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.944 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.944 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.944 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.944 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.944 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.944 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T03:40:27.944 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.944 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.944 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.945 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.945 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-12T03:40:27.945 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-12T03:40:27.945 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:27.945 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.945 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.945 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-12T03:40:27.945 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.945 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.945 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.945 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T03:40:27.945 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.946 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.946 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.946 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.946 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap_max_deviation": { 2024-09-12T03:40:27.946 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "upmap_max_deviation", 2024-09-12T03:40:27.946 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.946 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.946 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.946 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-12T03:40:27.946 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-09-12T03:40:27.946 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.946 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.946 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T03:40:27.946 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T03:40:27.947 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.947 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.947 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.947 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap_max_optimizations": { 2024-09-12T03:40:27.947 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "upmap_max_optimizations", 2024-09-12T03:40:27.947 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-12T03:40:27.947 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.947 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.947 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-12T03:40:27.947 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.947 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.948 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.948 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T03:40:27.948 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.948 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.948 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.948 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.948 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:27.948 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.948 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:27.948 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephadm", 2024-09-12T03:40:27.948 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:27.948 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:27.949 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:27.949 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_down_multiplier": { 2024-09-12T03:40:27.949 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_down_multiplier", 2024-09-12T03:40:27.949 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-12T03:40:27.949 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.949 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.949 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3.0", 2024-09-12T03:40:27.949 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.949 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.949 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.949 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T03:40:27.949 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.949 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.950 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.950 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.950 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_refresh_rate": { 2024-09-12T03:40:27.950 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_refresh_rate", 2024-09-12T03:40:27.950 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:27.950 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.950 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.950 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "20", 2024-09-12T03:40:27.950 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.950 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.950 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.950 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T03:40:27.950 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.950 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.950 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.951 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.951 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_starting_port": { 2024-09-12T03:40:27.951 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_starting_port", 2024-09-12T03:40:27.951 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.951 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.951 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.951 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "4721", 2024-09-12T03:40:27.951 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.951 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.951 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.951 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T03:40:27.951 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.951 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.952 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.952 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.952 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_ptrace": { 2024-09-12T03:40:27.952 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_ptrace", 2024-09-12T03:40:27.952 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.952 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.952 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.952 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.952 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.952 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.952 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.952 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T03:40:27.953 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T03:40:27.953 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.953 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.953 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.953 INFO:teuthology.orchestra.run.smithi064.stdout: "autotune_interval": { 2024-09-12T03:40:27.953 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "autotune_interval", 2024-09-12T03:40:27.953 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:27.953 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.953 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.953 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-12T03:40:27.953 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.953 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.953 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.953 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T03:40:27.954 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.954 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.954 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.954 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.954 INFO:teuthology.orchestra.run.smithi064.stdout: "autotune_memory_target_ratio": { 2024-09-12T03:40:27.954 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T03:40:27.954 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-12T03:40:27.954 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.954 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.954 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0.7", 2024-09-12T03:40:27.954 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.954 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.955 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.955 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T03:40:27.955 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.955 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.955 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.955 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.955 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm_log_destination": { 2024-09-12T03:40:27.955 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephadm_log_destination", 2024-09-12T03:40:27.955 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.955 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.955 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.955 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.955 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.955 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.956 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.956 INFO:teuthology.orchestra.run.smithi064.stdout: "file", 2024-09-12T03:40:27.956 INFO:teuthology.orchestra.run.smithi064.stdout: "file,syslog", 2024-09-12T03:40:27.956 INFO:teuthology.orchestra.run.smithi064.stdout: "syslog" 2024-09-12T03:40:27.956 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.956 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T03:40:27.956 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.956 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.956 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.956 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.956 INFO:teuthology.orchestra.run.smithi064.stdout: "cgroups_split": { 2024-09-12T03:40:27.956 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cgroups_split", 2024-09-12T03:40:27.956 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.957 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.957 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.957 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.957 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.957 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.957 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.957 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T03:40:27.957 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.957 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.957 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.957 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.957 INFO:teuthology.orchestra.run.smithi064.stdout: "config_checks_enabled": { 2024-09-12T03:40:27.957 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_checks_enabled", 2024-09-12T03:40:27.958 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.958 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.958 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.958 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.958 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.958 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.958 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.958 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T03:40:27.958 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.958 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.958 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.958 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.958 INFO:teuthology.orchestra.run.smithi064.stdout: "config_dashboard": { 2024-09-12T03:40:27.958 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_dashboard", 2024-09-12T03:40:27.959 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.959 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.959 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.959 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.959 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.959 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.959 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.959 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T03:40:27.959 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.959 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.959 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.959 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.959 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_alertmanager": { 2024-09-12T03:40:27.960 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_alertmanager", 2024-09-12T03:40:27.960 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.960 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.960 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.960 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T03:40:27.960 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.960 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.960 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.960 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-09-12T03:40:27.960 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.960 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.960 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.961 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.961 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_base": { 2024-09-12T03:40:27.961 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_base", 2024-09-12T03:40:27.961 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.961 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.961 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.961 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T03:40:27.961 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.961 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.961 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.961 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Container image name, without the tag", 2024-09-12T03:40:27.961 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.961 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.961 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.962 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.962 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_elasticsearch": { 2024-09-12T03:40:27.962 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_elasticsearch", 2024-09-12T03:40:27.962 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.962 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.962 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.962 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T03:40:27.962 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.962 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.962 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.962 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "elasticsearch container image", 2024-09-12T03:40:27.962 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.962 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.963 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.963 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.963 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_grafana": { 2024-09-12T03:40:27.963 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_grafana", 2024-09-12T03:40:27.963 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.963 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.963 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.963 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T03:40:27.963 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.963 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.963 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.963 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-09-12T03:40:27.963 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.963 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.964 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.964 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.964 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_haproxy": { 2024-09-12T03:40:27.964 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_haproxy", 2024-09-12T03:40:27.964 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.964 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.964 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.964 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T03:40:27.964 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.964 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.964 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.964 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "HAproxy container image", 2024-09-12T03:40:27.964 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.965 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.965 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.965 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.965 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_agent": { 2024-09-12T03:40:27.965 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_agent", 2024-09-12T03:40:27.965 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.965 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.965 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.965 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T03:40:27.965 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.965 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.965 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.966 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger agent container image", 2024-09-12T03:40:27.966 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.966 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.966 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.966 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.966 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_collector": { 2024-09-12T03:40:27.966 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_collector", 2024-09-12T03:40:27.966 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.966 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.966 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.966 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T03:40:27.966 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.966 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.966 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.967 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger collector container image", 2024-09-12T03:40:27.967 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.967 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.967 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.967 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.967 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_query": { 2024-09-12T03:40:27.967 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_query", 2024-09-12T03:40:27.967 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.967 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.967 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.967 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T03:40:27.967 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.967 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.968 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.968 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger query container image", 2024-09-12T03:40:27.968 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.968 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.968 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.968 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.968 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_keepalived": { 2024-09-12T03:40:27.968 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_keepalived", 2024-09-12T03:40:27.968 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.968 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.968 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.968 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T03:40:27.968 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.969 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.969 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.969 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Keepalived container image", 2024-09-12T03:40:27.969 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.969 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.969 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.969 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.969 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_loki": { 2024-09-12T03:40:27.969 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_loki", 2024-09-12T03:40:27.969 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.969 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.969 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.969 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T03:40:27.970 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.970 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.970 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.970 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Loki container image", 2024-09-12T03:40:27.970 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.970 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.970 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.970 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.970 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_node_exporter": { 2024-09-12T03:40:27.970 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_node_exporter", 2024-09-12T03:40:27.970 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.970 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.970 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.971 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T03:40:27.971 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.971 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.971 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.971 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-09-12T03:40:27.971 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.971 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.971 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.971 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.971 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_nvmeof": { 2024-09-12T03:40:27.971 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_nvmeof", 2024-09-12T03:40:27.971 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.971 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.972 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.972 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T03:40:27.972 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.972 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.972 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.972 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Nvme-of container image", 2024-09-12T03:40:27.972 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.972 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.972 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.972 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.972 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_prometheus": { 2024-09-12T03:40:27.972 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_prometheus", 2024-09-12T03:40:27.973 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.973 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.973 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.973 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T03:40:27.973 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.973 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.973 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.973 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-09-12T03:40:27.973 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.973 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.973 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.973 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.973 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_promtail": { 2024-09-12T03:40:27.973 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_promtail", 2024-09-12T03:40:27.974 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.974 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.974 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.974 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T03:40:27.974 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.974 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.974 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.974 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Promtail container image", 2024-09-12T03:40:27.974 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.974 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.974 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.974 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.974 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_samba": { 2024-09-12T03:40:27.975 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_samba", 2024-09-12T03:40:27.975 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.975 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.975 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.975 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T03:40:27.975 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.975 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.975 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.975 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Samba/SMB container image", 2024-09-12T03:40:27.975 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.975 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.975 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.975 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.975 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_snmp_gateway": { 2024-09-12T03:40:27.976 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_snmp_gateway", 2024-09-12T03:40:27.976 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.976 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.976 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.976 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T03:40:27.976 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.976 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.976 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.976 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SNMP Gateway container image", 2024-09-12T03:40:27.976 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.976 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.976 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.976 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.977 INFO:teuthology.orchestra.run.smithi064.stdout: "container_init": { 2024-09-12T03:40:27.977 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_init", 2024-09-12T03:40:27.977 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.977 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.977 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.977 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.977 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.977 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.977 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.977 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T03:40:27.977 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.977 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.977 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.978 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.978 INFO:teuthology.orchestra.run.smithi064.stdout: "daemon_cache_timeout": { 2024-09-12T03:40:27.978 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "daemon_cache_timeout", 2024-09-12T03:40:27.978 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:27.978 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.978 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.978 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-12T03:40:27.978 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.978 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.978 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.978 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T03:40:27.978 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.978 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.979 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.979 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.979 INFO:teuthology.orchestra.run.smithi064.stdout: "default_cephadm_command_timeout": { 2024-09-12T03:40:27.979 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T03:40:27.979 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.979 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.979 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.979 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "900", 2024-09-12T03:40:27.979 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.979 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.979 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.979 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T03:40:27.979 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.980 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.980 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.980 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.980 INFO:teuthology.orchestra.run.smithi064.stdout: "default_registry": { 2024-09-12T03:40:27.980 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "default_registry", 2024-09-12T03:40:27.980 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.980 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.980 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.980 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io", 2024-09-12T03:40:27.980 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.980 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.980 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.980 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T03:40:27.980 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.981 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.981 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.981 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.981 INFO:teuthology.orchestra.run.smithi064.stdout: "device_cache_timeout": { 2024-09-12T03:40:27.981 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_cache_timeout", 2024-09-12T03:40:27.981 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:27.981 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.981 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.981 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1800", 2024-09-12T03:40:27.981 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.981 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.981 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.981 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache device inventory", 2024-09-12T03:40:27.982 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.982 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.982 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.982 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.982 INFO:teuthology.orchestra.run.smithi064.stdout: "device_enhanced_scan": { 2024-09-12T03:40:27.982 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_enhanced_scan", 2024-09-12T03:40:27.982 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.982 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.982 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.982 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.982 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.982 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.982 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.983 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T03:40:27.983 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.983 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.983 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.983 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.983 INFO:teuthology.orchestra.run.smithi064.stdout: "facts_cache_timeout": { 2024-09-12T03:40:27.983 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "facts_cache_timeout", 2024-09-12T03:40:27.983 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:27.983 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.983 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.983 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-12T03:40:27.983 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.983 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.983 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.984 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache host facts data", 2024-09-12T03:40:27.984 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.984 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.984 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.984 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.984 INFO:teuthology.orchestra.run.smithi064.stdout: "grafana_dashboards_path": { 2024-09-12T03:40:27.984 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "grafana_dashboards_path", 2024-09-12T03:40:27.984 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.984 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.984 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.984 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T03:40:27.984 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.984 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.985 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.985 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T03:40:27.985 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.985 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.985 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.985 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.985 INFO:teuthology.orchestra.run.smithi064.stdout: "host_check_interval": { 2024-09-12T03:40:27.985 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "host_check_interval", 2024-09-12T03:40:27.985 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:27.985 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.985 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.985 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-12T03:40:27.985 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.986 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.986 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.986 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to perform a host check", 2024-09-12T03:40:27.986 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.986 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.986 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.986 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.986 INFO:teuthology.orchestra.run.smithi064.stdout: "hw_monitoring": { 2024-09-12T03:40:27.986 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "hw_monitoring", 2024-09-12T03:40:27.986 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.986 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.986 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.986 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.986 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.987 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.987 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.987 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T03:40:27.987 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.987 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.987 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.987 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.987 INFO:teuthology.orchestra.run.smithi064.stdout: "inventory_list_all": { 2024-09-12T03:40:27.987 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "inventory_list_all", 2024-09-12T03:40:27.987 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.987 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.987 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.987 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.988 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.988 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.988 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.988 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T03:40:27.988 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.988 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.988 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.988 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.988 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:27.988 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:27.988 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.988 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.988 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.989 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.989 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.989 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.989 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.989 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.989 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.989 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.989 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.989 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.989 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.989 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.989 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.989 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.989 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.990 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.990 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.990 INFO:teuthology.orchestra.run.smithi064.stdout: "log_refresh_metadata": { 2024-09-12T03:40:27.990 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_refresh_metadata", 2024-09-12T03:40:27.990 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.990 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.990 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.990 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.990 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.990 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.990 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.990 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T03:40:27.991 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.991 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.991 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.991 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.991 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:27.991 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:27.991 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.991 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.991 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.991 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:27.991 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.991 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.992 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.992 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T03:40:27.992 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.992 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.992 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.992 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.992 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:27.992 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:27.992 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.992 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.992 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.992 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:27.992 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.993 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.993 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:27.993 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:27.993 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:27.993 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:27.993 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:27.993 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:27.993 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:27.993 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:27.993 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.993 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.994 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.994 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.994 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.994 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:27.994 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:27.994 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.994 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.994 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:27.994 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.994 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.994 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.994 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.994 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:27.994 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.995 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.995 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.995 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.995 INFO:teuthology.orchestra.run.smithi064.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T03:40:27.995 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T03:40:27.995 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:27.995 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.995 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.995 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:27.995 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.995 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.995 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.995 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T03:40:27.996 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.996 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.996 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.996 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.996 INFO:teuthology.orchestra.run.smithi064.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T03:40:27.996 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T03:40:27.996 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:27.996 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.996 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.996 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "*", 2024-09-12T03:40:27.996 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.996 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.996 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.997 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T03:40:27.997 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.997 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.997 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.997 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.997 INFO:teuthology.orchestra.run.smithi064.stdout: "max_count_per_host": { 2024-09-12T03:40:27.997 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_count_per_host", 2024-09-12T03:40:27.997 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.997 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.997 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.997 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-12T03:40:27.997 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.997 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.997 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.998 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "max number of daemons per service per host", 2024-09-12T03:40:27.998 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.998 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.998 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.998 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.998 INFO:teuthology.orchestra.run.smithi064.stdout: "max_osd_draining_count": { 2024-09-12T03:40:27.998 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_osd_draining_count", 2024-09-12T03:40:27.998 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.998 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.998 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.998 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-12T03:40:27.998 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.998 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:27.999 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:27.999 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T03:40:27.999 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:27.999 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:27.999 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:27.999 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:27.999 INFO:teuthology.orchestra.run.smithi064.stdout: "migration_current": { 2024-09-12T03:40:27.999 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "migration_current", 2024-09-12T03:40:27.999 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:27.999 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:27.999 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:27.999 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:27.999 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:27.999 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.000 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.000 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "internal - do not modify", 2024-09-12T03:40:28.000 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.000 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.000 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.000 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.000 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": { 2024-09-12T03:40:28.000 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mode", 2024-09-12T03:40:28.000 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.000 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.000 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.000 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "root", 2024-09-12T03:40:28.000 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.001 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.001 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.001 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm-package", 2024-09-12T03:40:28.001 INFO:teuthology.orchestra.run.smithi064.stdout: "root" 2024-09-12T03:40:28.001 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.001 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T03:40:28.001 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.001 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.001 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.001 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.001 INFO:teuthology.orchestra.run.smithi064.stdout: "oob_default_addr": { 2024-09-12T03:40:28.001 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "oob_default_addr", 2024-09-12T03:40:28.001 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.001 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.002 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.002 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "169.254.1.1", 2024-09-12T03:40:28.002 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.002 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.002 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.002 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T03:40:28.002 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.002 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.002 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.002 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.002 INFO:teuthology.orchestra.run.smithi064.stdout: "prometheus_alerts_path": { 2024-09-12T03:40:28.002 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prometheus_alerts_path", 2024-09-12T03:40:28.002 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.003 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.003 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.003 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T03:40:28.003 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.003 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.003 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.003 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T03:40:28.003 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.003 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.003 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.003 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.003 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_insecure": { 2024-09-12T03:40:28.003 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_insecure", 2024-09-12T03:40:28.004 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.004 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.004 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.004 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.004 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.004 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.004 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.004 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T03:40:28.004 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.004 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.004 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.005 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_password": { 2024-09-12T03:40:28.005 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_password", 2024-09-12T03:40:28.005 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.005 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.005 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.005 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.005 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.005 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.005 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.005 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T03:40:28.005 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.005 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.005 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.006 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.006 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_url": { 2024-09-12T03:40:28.006 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_url", 2024-09-12T03:40:28.006 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.006 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.006 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.006 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.006 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.006 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.006 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.006 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T03:40:28.006 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.006 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.007 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.007 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.007 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_username": { 2024-09-12T03:40:28.007 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_username", 2024-09-12T03:40:28.007 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.007 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.007 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.007 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.007 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.007 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.007 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.007 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T03:40:28.007 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.007 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.008 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.008 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.008 INFO:teuthology.orchestra.run.smithi064.stdout: "secure_monitoring_stack": { 2024-09-12T03:40:28.008 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "secure_monitoring_stack", 2024-09-12T03:40:28.008 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.008 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.008 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.008 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.008 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.008 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.008 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.008 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T03:40:28.008 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.009 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.009 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.009 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.009 INFO:teuthology.orchestra.run.smithi064.stdout: "service_discovery_port": { 2024-09-12T03:40:28.009 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "service_discovery_port", 2024-09-12T03:40:28.009 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.009 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.009 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.009 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8765", 2024-09-12T03:40:28.009 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.009 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.009 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.009 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "cephadm service discovery port", 2024-09-12T03:40:28.009 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.010 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.010 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.010 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.010 INFO:teuthology.orchestra.run.smithi064.stdout: "ssh_config_file": { 2024-09-12T03:40:28.010 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssh_config_file", 2024-09-12T03:40:28.010 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.010 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.010 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.010 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.010 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.010 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.010 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.010 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T03:40:28.011 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.011 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.011 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.011 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.011 INFO:teuthology.orchestra.run.smithi064.stdout: "use_agent": { 2024-09-12T03:40:28.011 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "use_agent", 2024-09-12T03:40:28.011 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.011 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.011 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.011 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.011 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.011 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.011 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.011 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T03:40:28.012 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.012 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.012 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.012 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.012 INFO:teuthology.orchestra.run.smithi064.stdout: "use_repo_digest": { 2024-09-12T03:40:28.012 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "use_repo_digest", 2024-09-12T03:40:28.012 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.012 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.012 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.012 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.012 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.013 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.013 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.013 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T03:40:28.013 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.013 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.013 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.013 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.013 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_failed_host_check": { 2024-09-12T03:40:28.013 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_failed_host_check", 2024-09-12T03:40:28.013 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.013 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.013 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.013 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.013 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.014 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.014 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.014 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T03:40:28.014 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.014 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.014 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.014 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.014 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_stray_daemons": { 2024-09-12T03:40:28.014 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_stray_daemons", 2024-09-12T03:40:28.014 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.014 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.014 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.014 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.014 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.015 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.015 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.015 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T03:40:28.015 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.015 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.015 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.015 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_stray_hosts": { 2024-09-12T03:40:28.015 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_stray_hosts", 2024-09-12T03:40:28.015 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.015 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.015 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.015 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.016 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.016 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.016 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.016 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T03:40:28.016 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.016 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.016 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.016 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.016 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.016 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.016 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.016 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crash", 2024-09-12T03:40:28.016 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.016 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.017 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.017 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.017 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.017 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.017 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.017 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.017 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.017 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.017 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.017 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.017 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.017 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.017 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.018 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.018 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.018 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.018 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.018 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.018 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.018 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.018 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.018 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.018 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.018 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.018 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.018 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.018 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.019 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.019 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.019 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.019 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.019 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.019 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.019 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.019 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.019 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.019 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.019 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.019 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.019 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.020 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.020 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.020 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.020 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.020 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.020 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.020 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.020 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.020 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.020 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.020 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.020 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.020 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.020 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.021 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.021 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.021 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.021 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.021 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.021 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.021 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.021 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.021 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.021 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.021 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.021 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.021 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.021 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.022 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.022 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.022 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.022 INFO:teuthology.orchestra.run.smithi064.stdout: "retain_interval": { 2024-09-12T03:40:28.022 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "retain_interval", 2024-09-12T03:40:28.022 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:28.022 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.022 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.022 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "31536000", 2024-09-12T03:40:28.022 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.022 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.022 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.022 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T03:40:28.023 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.023 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.023 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.023 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.023 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_recent_interval": { 2024-09-12T03:40:28.023 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_recent_interval", 2024-09-12T03:40:28.023 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:28.023 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.023 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.023 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1209600", 2024-09-12T03:40:28.023 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.023 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.023 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.023 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T03:40:28.024 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.024 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.024 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.024 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.024 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.024 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.024 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.024 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "dashboard", 2024-09-12T03:40:28.024 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.024 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.024 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.024 INFO:teuthology.orchestra.run.smithi064.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T03:40:28.024 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T03:40:28.025 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.025 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.025 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.025 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-12T03:40:28.025 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.025 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.025 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.025 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.025 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.025 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.025 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.025 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.025 INFO:teuthology.orchestra.run.smithi064.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T03:40:28.025 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T03:40:28.026 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.026 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.026 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.026 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.026 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.026 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.026 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.026 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.026 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.026 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.026 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.026 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.026 INFO:teuthology.orchestra.run.smithi064.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T03:40:28.026 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T03:40:28.027 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.027 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.027 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.027 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.027 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.027 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.027 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.027 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.027 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.027 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.027 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.027 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.027 INFO:teuthology.orchestra.run.smithi064.stdout: "AUDIT_API_ENABLED": { 2024-09-12T03:40:28.027 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T03:40:28.028 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.028 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.028 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.028 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.028 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.028 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.028 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.028 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.028 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.028 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.028 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.028 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.028 INFO:teuthology.orchestra.run.smithi064.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T03:40:28.028 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T03:40:28.029 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.029 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.029 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.029 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.029 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.029 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.029 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.029 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.029 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.029 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.029 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.029 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.029 INFO:teuthology.orchestra.run.smithi064.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T03:40:28.029 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T03:40:28.030 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.030 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.030 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.030 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.030 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.030 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.030 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.030 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.030 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.030 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.030 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.030 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.030 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T03:40:28.030 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T03:40:28.031 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.031 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.031 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.031 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.031 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.031 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.031 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.031 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.031 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.031 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.031 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.031 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.031 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T03:40:28.032 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T03:40:28.032 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.032 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.032 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.032 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.032 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.032 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.032 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.032 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.032 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.032 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.033 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.033 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T03:40:28.033 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T03:40:28.033 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.033 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.033 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.033 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.033 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.033 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.033 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.033 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.033 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.033 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.034 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.034 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T03:40:28.034 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T03:40:28.034 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.034 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.034 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.034 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.034 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.034 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.034 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.034 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.035 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.035 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.035 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.035 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.035 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T03:40:28.035 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T03:40:28.035 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.035 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.035 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.035 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.035 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.035 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.035 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.036 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.036 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.036 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.036 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.036 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T03:40:28.036 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T03:40:28.036 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.036 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.036 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.036 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.037 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.037 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.037 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.037 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.037 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.037 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.037 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.037 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.037 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T03:40:28.037 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T03:40:28.037 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.037 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.037 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.038 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.038 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.038 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.038 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.038 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.038 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.038 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.038 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.038 INFO:teuthology.orchestra.run.smithi064.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T03:40:28.038 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T03:40:28.038 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.039 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.039 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.039 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.039 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.039 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.039 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.039 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.039 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.039 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.039 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.039 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.039 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T03:40:28.039 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T03:40:28.040 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.040 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.040 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.040 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-09-12T03:40:28.040 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.040 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.040 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.040 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.040 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.040 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.040 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.040 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.041 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T03:40:28.041 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T03:40:28.041 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.041 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.041 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.041 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.041 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.041 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.041 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.041 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.041 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.041 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.042 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.042 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_URL": { 2024-09-12T03:40:28.042 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_URL", 2024-09-12T03:40:28.042 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.042 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.042 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.042 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.042 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.042 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.042 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.042 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.043 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.043 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.043 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.043 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.043 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T03:40:28.043 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T03:40:28.043 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.043 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.043 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.043 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-09-12T03:40:28.043 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.043 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.044 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.044 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.044 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.044 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.044 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.044 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.044 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T03:40:28.044 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T03:40:28.044 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.045 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.045 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.045 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.045 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.045 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.045 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.045 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.045 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.045 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.045 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.045 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.045 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T03:40:28.045 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T03:40:28.046 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.046 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.046 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.046 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.046 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.046 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.046 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.046 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.046 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.046 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.046 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.046 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.047 INFO:teuthology.orchestra.run.smithi064.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T03:40:28.047 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T03:40:28.047 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.047 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.047 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.047 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.047 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.047 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.047 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.047 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.047 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.047 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.048 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.048 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.048 INFO:teuthology.orchestra.run.smithi064.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T03:40:28.048 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T03:40:28.048 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.048 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.048 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.048 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.048 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.048 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.048 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.048 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.048 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.049 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.049 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.049 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.049 INFO:teuthology.orchestra.run.smithi064.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T03:40:28.049 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T03:40:28.049 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.049 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.049 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.049 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.049 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.049 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.049 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.050 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.050 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.050 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.050 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.050 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.050 INFO:teuthology.orchestra.run.smithi064.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T03:40:28.050 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T03:40:28.050 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.050 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.050 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.050 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.050 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.050 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.051 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.051 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.051 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.051 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.051 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.051 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T03:40:28.051 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T03:40:28.051 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.051 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.051 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.051 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.052 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.052 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.052 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.052 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.052 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.052 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.052 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.052 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.052 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T03:40:28.052 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T03:40:28.052 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.052 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.052 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.053 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.053 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.053 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.053 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.053 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.053 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.053 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.053 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.053 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.053 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T03:40:28.053 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T03:40:28.053 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.053 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.053 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.054 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.054 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.054 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.054 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.054 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.054 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.054 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.054 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.054 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.054 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T03:40:28.054 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T03:40:28.054 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.054 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.054 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.055 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.055 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.055 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.055 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.055 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.055 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.055 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.055 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.055 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T03:40:28.055 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T03:40:28.055 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.056 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.056 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.056 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.056 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.056 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.056 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.056 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.056 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.056 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.056 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.056 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T03:40:28.056 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T03:40:28.056 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.057 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.057 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.057 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.057 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.057 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.057 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.057 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.057 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.057 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.057 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.057 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.057 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T03:40:28.057 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T03:40:28.058 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.058 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.058 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.058 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.058 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.058 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.058 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.058 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.058 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.058 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.058 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.058 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.058 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T03:40:28.058 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T03:40:28.059 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.059 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.059 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.059 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.059 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.059 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.059 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.059 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.059 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.059 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.059 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.059 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.060 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T03:40:28.060 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T03:40:28.060 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.060 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.060 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.060 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T03:40:28.060 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.060 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.060 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.060 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.060 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.060 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.061 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.061 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.061 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T03:40:28.061 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T03:40:28.061 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.061 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.061 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.061 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-12T03:40:28.061 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.061 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.061 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.061 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.061 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.062 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.062 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.062 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.062 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T03:40:28.062 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T03:40:28.062 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.062 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.062 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.062 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8", 2024-09-12T03:40:28.062 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.062 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.062 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.063 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.063 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.063 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.063 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.063 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.063 INFO:teuthology.orchestra.run.smithi064.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T03:40:28.063 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T03:40:28.063 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.063 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.063 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.063 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "45", 2024-09-12T03:40:28.063 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.063 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.064 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.064 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.064 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.064 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.064 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.064 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.064 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T03:40:28.064 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T03:40:28.064 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.064 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.064 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.064 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.065 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.065 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.065 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.065 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.065 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.065 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.065 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.065 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.065 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T03:40:28.065 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T03:40:28.065 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.065 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.065 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.066 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-09-12T03:40:28.066 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.066 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.066 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.066 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.066 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.066 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.066 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.066 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.066 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T03:40:28.066 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T03:40:28.066 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.067 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.067 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.067 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.067 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.067 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.067 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.067 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.067 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.067 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.067 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.067 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.067 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T03:40:28.068 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T03:40:28.068 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.068 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.068 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.068 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.068 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.068 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.068 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.068 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.068 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.068 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.069 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.069 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.069 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T03:40:28.069 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T03:40:28.069 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.069 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.069 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.069 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-12T03:40:28.069 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.069 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.069 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.069 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.069 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.070 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.070 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.070 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.070 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T03:40:28.070 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T03:40:28.070 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.070 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.070 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.070 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-12T03:40:28.070 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.070 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.070 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.070 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.071 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.071 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.071 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.071 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.071 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T03:40:28.071 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T03:40:28.071 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.071 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.071 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.071 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-12T03:40:28.071 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.071 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.071 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.072 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.072 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.072 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.072 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.072 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.072 INFO:teuthology.orchestra.run.smithi064.stdout: "cross_origin_url": { 2024-09-12T03:40:28.072 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cross_origin_url", 2024-09-12T03:40:28.072 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.072 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.072 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.072 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.072 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.072 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.072 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.073 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.073 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.073 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.073 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.073 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.073 INFO:teuthology.orchestra.run.smithi064.stdout: "crt_file": { 2024-09-12T03:40:28.073 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crt_file", 2024-09-12T03:40:28.073 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.073 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.073 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.073 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.073 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.073 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.073 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.074 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.074 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.074 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.074 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.074 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.074 INFO:teuthology.orchestra.run.smithi064.stdout: "debug": { 2024-09-12T03:40:28.074 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "debug", 2024-09-12T03:40:28.074 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.074 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.074 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.074 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.074 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.074 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.075 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.075 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable/disable debug options", 2024-09-12T03:40:28.075 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.075 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.075 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.075 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.075 INFO:teuthology.orchestra.run.smithi064.stdout: "jwt_token_ttl": { 2024-09-12T03:40:28.075 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "jwt_token_ttl", 2024-09-12T03:40:28.075 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.075 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.075 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.075 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "28800", 2024-09-12T03:40:28.075 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.076 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.076 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.076 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.076 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.076 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.076 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.076 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.076 INFO:teuthology.orchestra.run.smithi064.stdout: "key_file": { 2024-09-12T03:40:28.076 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "key_file", 2024-09-12T03:40:28.076 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.076 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.076 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.077 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.077 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.077 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.077 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.077 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.077 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.077 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.077 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.077 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.077 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.077 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.077 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.077 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.078 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.078 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.078 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.078 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.078 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.078 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.078 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.078 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.078 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.078 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.078 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.078 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.079 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.079 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.079 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.079 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.079 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.079 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.079 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.079 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.079 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.079 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.079 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.079 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.079 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.079 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.080 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.080 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.080 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.080 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.080 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.080 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.080 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.080 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.080 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.080 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.080 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.080 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.081 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.081 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.081 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.081 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.081 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.081 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.081 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.081 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.081 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.081 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.081 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.081 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.081 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.081 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.082 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.082 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.082 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.082 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.082 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.082 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.082 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.082 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.082 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.082 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.082 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.082 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.082 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.083 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.083 INFO:teuthology.orchestra.run.smithi064.stdout: "motd": { 2024-09-12T03:40:28.083 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "motd", 2024-09-12T03:40:28.083 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.083 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.083 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.083 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.083 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.083 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.083 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.083 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "The message of the day", 2024-09-12T03:40:28.083 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.083 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.084 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.084 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.084 INFO:teuthology.orchestra.run.smithi064.stdout: "redirect_resolve_ip_addr": { 2024-09-12T03:40:28.084 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T03:40:28.084 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.084 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.084 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.084 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.084 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.084 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.084 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.084 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.084 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.085 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.085 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.085 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.085 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-09-12T03:40:28.085 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-09-12T03:40:28.085 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.085 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.085 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.085 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "::", 2024-09-12T03:40:28.085 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.085 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.085 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.085 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.085 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.086 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.086 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.086 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.086 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-09-12T03:40:28.086 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-09-12T03:40:28.086 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.086 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.086 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.086 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8080", 2024-09-12T03:40:28.086 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.086 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.086 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.086 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.086 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.087 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.087 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.087 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.087 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl": { 2024-09-12T03:40:28.087 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl", 2024-09-12T03:40:28.087 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.087 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.087 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.087 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.087 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.087 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.087 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.087 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.087 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.088 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.088 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.088 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.088 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl_server_port": { 2024-09-12T03:40:28.088 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl_server_port", 2024-09-12T03:40:28.088 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.088 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.088 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.088 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8443", 2024-09-12T03:40:28.088 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.088 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.088 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.088 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.088 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.089 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.089 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.089 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.089 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_behaviour": { 2024-09-12T03:40:28.089 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_behaviour", 2024-09-12T03:40:28.089 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.089 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.089 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.089 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "redirect", 2024-09-12T03:40:28.089 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.089 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.089 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.090 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.090 INFO:teuthology.orchestra.run.smithi064.stdout: "redirect" 2024-09-12T03:40:28.090 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.090 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.090 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.090 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.090 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.090 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.090 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_error_status_code": { 2024-09-12T03:40:28.090 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_error_status_code", 2024-09-12T03:40:28.090 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.090 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.090 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.090 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2024-09-12T03:40:28.091 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "400", 2024-09-12T03:40:28.091 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "599", 2024-09-12T03:40:28.091 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.091 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.091 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.091 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.091 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.091 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.091 INFO:teuthology.orchestra.run.smithi064.stdout: "url_prefix": { 2024-09-12T03:40:28.091 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "url_prefix", 2024-09-12T03:40:28.091 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.091 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.091 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.092 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.092 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.092 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.092 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.092 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.092 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.092 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.092 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.092 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.092 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.092 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.092 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.092 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "devicehealth", 2024-09-12T03:40:28.092 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.093 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.093 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.093 INFO:teuthology.orchestra.run.smithi064.stdout: "enable_monitoring": { 2024-09-12T03:40:28.093 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enable_monitoring", 2024-09-12T03:40:28.093 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.093 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.093 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.093 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.093 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.093 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.093 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.093 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "monitor device health metrics", 2024-09-12T03:40:28.093 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.094 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.094 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.094 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.094 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.094 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.094 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.094 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.094 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.094 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.094 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.094 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.094 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.095 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.095 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.095 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.095 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.095 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.095 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.095 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.095 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.095 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.095 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.095 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.095 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.095 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.096 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.096 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.096 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.096 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.096 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.096 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.096 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.096 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.096 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.096 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.096 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.096 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.096 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.097 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.097 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.097 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.097 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.097 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.097 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.097 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.097 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.097 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.097 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.097 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.097 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.097 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.097 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.098 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.098 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.098 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.098 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.098 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.098 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.098 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.098 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.098 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.098 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.098 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.098 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.098 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.098 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.099 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.099 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.099 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.099 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.099 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.099 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.099 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.099 INFO:teuthology.orchestra.run.smithi064.stdout: "mark_out_threshold": { 2024-09-12T03:40:28.099 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mark_out_threshold", 2024-09-12T03:40:28.099 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:28.099 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.099 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.099 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "2419200", 2024-09-12T03:40:28.099 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.100 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.100 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.100 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T03:40:28.100 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.100 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.100 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.100 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.100 INFO:teuthology.orchestra.run.smithi064.stdout: "pool_name": { 2024-09-12T03:40:28.100 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pool_name", 2024-09-12T03:40:28.100 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.100 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.100 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.100 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "device_health_metrics", 2024-09-12T03:40:28.101 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.101 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.101 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.101 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T03:40:28.101 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.101 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.101 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.101 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.101 INFO:teuthology.orchestra.run.smithi064.stdout: "retention_period": { 2024-09-12T03:40:28.101 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "retention_period", 2024-09-12T03:40:28.101 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:28.101 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.101 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.101 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15552000", 2024-09-12T03:40:28.102 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.102 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.102 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.102 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long to retain device health metrics", 2024-09-12T03:40:28.102 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.102 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.102 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.102 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.102 INFO:teuthology.orchestra.run.smithi064.stdout: "scrape_frequency": { 2024-09-12T03:40:28.102 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "scrape_frequency", 2024-09-12T03:40:28.102 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:28.102 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.102 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.103 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "86400", 2024-09-12T03:40:28.103 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.103 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.103 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.103 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T03:40:28.103 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.103 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.103 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.103 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.103 INFO:teuthology.orchestra.run.smithi064.stdout: "self_heal": { 2024-09-12T03:40:28.103 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "self_heal", 2024-09-12T03:40:28.103 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.103 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.104 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.104 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.104 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.104 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.104 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.104 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T03:40:28.104 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.104 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.104 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.104 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.104 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-12T03:40:28.104 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-12T03:40:28.105 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:28.105 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.105 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.105 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-12T03:40:28.105 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.105 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.105 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.105 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T03:40:28.105 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.105 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.105 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.105 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.105 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_threshold": { 2024-09-12T03:40:28.105 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_threshold", 2024-09-12T03:40:28.106 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:28.106 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.106 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.106 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "7257600", 2024-09-12T03:40:28.106 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.106 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.106 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.106 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T03:40:28.106 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.106 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.106 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.106 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.107 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.107 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.107 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "diskprediction_local", 2024-09-12T03:40:28.107 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.107 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.107 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.107 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.107 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.107 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.107 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.107 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.107 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.108 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.108 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.108 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.108 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.108 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.108 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.108 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.108 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.108 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.108 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.108 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.108 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.108 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.108 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.109 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.109 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.109 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.109 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.109 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.109 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.109 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.109 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.109 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.109 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.109 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.109 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.109 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.110 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.110 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.110 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.110 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.110 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.110 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.110 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.110 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.110 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.110 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.110 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.110 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.111 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.111 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.111 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.111 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.111 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.111 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.111 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.111 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.111 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.111 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.111 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.111 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.111 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.112 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.112 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.112 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.112 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.112 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.112 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.112 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.112 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.112 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.112 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.112 INFO:teuthology.orchestra.run.smithi064.stdout: "predict_interval": { 2024-09-12T03:40:28.112 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "predict_interval", 2024-09-12T03:40:28.112 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.113 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.113 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.113 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "86400", 2024-09-12T03:40:28.113 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.113 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.113 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.113 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.113 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.113 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.113 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.113 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.113 INFO:teuthology.orchestra.run.smithi064.stdout: "predictor_model": { 2024-09-12T03:40:28.113 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "predictor_model", 2024-09-12T03:40:28.113 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.114 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.114 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.114 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "prophetstor", 2024-09-12T03:40:28.114 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.114 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.114 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.114 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.114 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.114 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.114 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.114 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-12T03:40:28.114 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-12T03:40:28.115 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.115 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.115 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.115 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-12T03:40:28.115 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.115 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.115 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.115 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.115 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.115 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.115 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.115 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.115 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.115 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.116 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.116 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "influx", 2024-09-12T03:40:28.116 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": false, 2024-09-12T03:40:28.116 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "influxdb python module not found", 2024-09-12T03:40:28.116 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.116 INFO:teuthology.orchestra.run.smithi064.stdout: "batch_size": { 2024-09-12T03:40:28.116 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "batch_size", 2024-09-12T03:40:28.116 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.116 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.116 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.116 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5000", 2024-09-12T03:40:28.116 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.116 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.116 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.117 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T03:40:28.117 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.117 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.117 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.117 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.117 INFO:teuthology.orchestra.run.smithi064.stdout: "database": { 2024-09-12T03:40:28.117 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "database", 2024-09-12T03:40:28.117 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.117 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.117 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.117 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "ceph", 2024-09-12T03:40:28.117 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.117 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.117 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.118 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T03:40:28.118 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.118 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.118 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.118 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.118 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": { 2024-09-12T03:40:28.118 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "hostname", 2024-09-12T03:40:28.118 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.118 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.118 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.118 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.118 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.119 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.119 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.119 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB server hostname", 2024-09-12T03:40:28.119 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.119 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.119 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.119 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.119 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-12T03:40:28.119 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-12T03:40:28.119 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:28.119 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.119 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.119 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "30", 2024-09-12T03:40:28.119 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "5", 2024-09-12T03:40:28.120 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.120 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.120 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T03:40:28.120 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.120 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.120 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.120 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.120 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.120 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.120 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.120 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.120 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.120 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.121 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.121 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.121 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.121 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.121 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.121 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.121 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.121 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.121 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.121 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.121 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.121 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.121 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.121 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.122 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.122 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.122 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.122 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.122 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.122 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.122 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.122 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.122 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.122 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.122 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.122 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.122 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.122 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.123 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.123 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.123 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.123 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.123 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.123 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.123 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.123 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.123 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.123 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.123 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.123 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.124 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.124 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.124 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.124 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.124 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.124 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.124 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.124 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.124 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.124 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.124 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.124 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.125 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.125 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.125 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.125 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.125 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.125 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.125 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.125 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.125 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.125 INFO:teuthology.orchestra.run.smithi064.stdout: "password": { 2024-09-12T03:40:28.125 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "password", 2024-09-12T03:40:28.125 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.125 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.126 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.126 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.126 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.126 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.126 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.126 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "password of InfluxDB server user", 2024-09-12T03:40:28.126 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.126 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.126 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.126 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.126 INFO:teuthology.orchestra.run.smithi064.stdout: "port": { 2024-09-12T03:40:28.126 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "port", 2024-09-12T03:40:28.126 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.127 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.127 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.127 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8086", 2024-09-12T03:40:28.127 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.127 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.127 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.127 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB server port", 2024-09-12T03:40:28.127 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.127 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.127 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.127 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.127 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl": { 2024-09-12T03:40:28.127 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl", 2024-09-12T03:40:28.127 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.128 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.128 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.128 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "false", 2024-09-12T03:40:28.128 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.128 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.128 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.128 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:40:28.128 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.128 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.128 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.128 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.128 INFO:teuthology.orchestra.run.smithi064.stdout: "threads": { 2024-09-12T03:40:28.128 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "threads", 2024-09-12T03:40:28.128 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.129 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.129 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.129 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-12T03:40:28.129 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-09-12T03:40:28.129 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "32", 2024-09-12T03:40:28.129 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.129 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T03:40:28.129 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.129 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.129 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.129 INFO:teuthology.orchestra.run.smithi064.stdout: "username": { 2024-09-12T03:40:28.129 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "username", 2024-09-12T03:40:28.129 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.130 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.130 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.130 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.130 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.130 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.130 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.130 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "username of InfluxDB server user", 2024-09-12T03:40:28.130 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.130 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.130 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.130 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.130 INFO:teuthology.orchestra.run.smithi064.stdout: "verify_ssl": { 2024-09-12T03:40:28.130 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "verify_ssl", 2024-09-12T03:40:28.130 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.131 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.131 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.131 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "true", 2024-09-12T03:40:28.131 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.131 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.131 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.131 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:40:28.131 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.131 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.131 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.131 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.131 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.131 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.132 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.132 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "insights", 2024-09-12T03:40:28.132 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.132 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.132 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.132 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.132 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.132 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.132 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.132 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.132 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.132 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.132 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.133 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.133 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.133 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.133 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.133 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.133 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.133 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.133 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.133 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.133 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.133 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.133 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.133 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.134 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.134 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.134 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.134 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.134 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.134 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.134 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.134 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.134 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.134 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.134 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.134 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.134 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.135 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.135 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.135 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.135 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.135 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.135 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.135 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.135 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.135 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.135 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.135 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.135 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.136 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.136 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.136 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.136 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.136 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.136 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.136 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.136 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.136 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.136 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.136 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.136 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.136 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.136 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.137 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.137 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.137 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.137 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.137 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.137 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.137 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.137 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.137 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.137 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.137 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "iostat", 2024-09-12T03:40:28.137 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.138 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.138 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.138 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.138 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.138 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.138 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.138 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.138 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.138 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.140 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.140 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.140 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.140 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.140 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.140 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.140 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.140 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.140 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.140 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.140 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.140 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.141 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.141 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.141 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.141 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.141 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.141 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.141 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.141 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.141 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.141 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.141 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.141 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.141 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.142 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.142 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.142 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.142 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.142 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.142 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.142 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.142 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.142 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.142 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.142 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.142 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.142 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.143 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.143 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.143 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.143 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.143 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.143 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.143 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.143 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.143 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.143 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.143 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.143 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.143 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.143 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.144 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.144 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.144 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.144 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.144 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.144 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.144 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.144 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.144 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.144 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.144 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.144 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.144 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "k8sevents", 2024-09-12T03:40:28.144 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.145 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.145 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.145 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_event_retention_days": { 2024-09-12T03:40:28.145 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ceph_event_retention_days", 2024-09-12T03:40:28.145 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.145 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.145 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.145 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "7", 2024-09-12T03:40:28.145 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.145 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.145 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.145 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T03:40:28.145 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.145 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.146 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.146 INFO:teuthology.orchestra.run.smithi064.stdout: "config_check_secs": { 2024-09-12T03:40:28.146 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_check_secs", 2024-09-12T03:40:28.146 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.146 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.146 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.146 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-12T03:40:28.146 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "10", 2024-09-12T03:40:28.146 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.146 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.146 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T03:40:28.146 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.146 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.147 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.147 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.147 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.147 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.147 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.147 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.147 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.147 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.147 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.147 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.147 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.147 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.148 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.148 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.148 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.148 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.148 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.148 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.148 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.148 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.148 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.148 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.148 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.148 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.148 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.148 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.149 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.149 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.149 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.149 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.149 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.149 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.149 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.149 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.149 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.149 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.149 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.149 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.149 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.150 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.150 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.150 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.150 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.150 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.150 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.150 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.150 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.150 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.150 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.150 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.150 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.150 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.150 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.151 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.151 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.151 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.151 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.151 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.151 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.151 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.151 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.151 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.151 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.151 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.151 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.151 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.152 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.152 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.152 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.152 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.152 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.152 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.152 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.152 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "localpool", 2024-09-12T03:40:28.152 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.152 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.152 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.152 INFO:teuthology.orchestra.run.smithi064.stdout: "failure_domain": { 2024-09-12T03:40:28.152 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "failure_domain", 2024-09-12T03:40:28.153 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.153 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.153 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.153 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "host", 2024-09-12T03:40:28.153 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.153 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.153 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.153 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "failure domain for any created local pool", 2024-09-12T03:40:28.153 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T03:40:28.153 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.153 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.153 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.153 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.154 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.154 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.154 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.154 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.154 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.154 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.154 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.154 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.154 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.154 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.154 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.154 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.154 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.155 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.155 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.155 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.155 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.155 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.155 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.155 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.155 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.155 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.155 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.155 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.155 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.155 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.155 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.156 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.156 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.156 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.156 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.156 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.156 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.156 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.156 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.156 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.156 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.156 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.156 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.157 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.157 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.157 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.157 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.157 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.157 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.157 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.157 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.157 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.157 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.157 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.157 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.157 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.157 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.158 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.158 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.158 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.158 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.158 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.158 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.158 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.158 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.158 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.158 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.158 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.158 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.158 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.158 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.159 INFO:teuthology.orchestra.run.smithi064.stdout: "min_size": { 2024-09-12T03:40:28.159 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "min_size", 2024-09-12T03:40:28.159 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.159 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.159 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.159 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.159 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.159 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.159 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.159 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default min_size for any created local pool", 2024-09-12T03:40:28.159 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T03:40:28.159 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.159 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.159 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.160 INFO:teuthology.orchestra.run.smithi064.stdout: "num_rep": { 2024-09-12T03:40:28.160 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "num_rep", 2024-09-12T03:40:28.160 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.160 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.160 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.160 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3", 2024-09-12T03:40:28.160 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.160 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.160 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.160 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default replica count for any created local pool", 2024-09-12T03:40:28.160 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.160 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.160 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.161 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_num": { 2024-09-12T03:40:28.161 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pg_num", 2024-09-12T03:40:28.161 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.161 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.161 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.161 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "128", 2024-09-12T03:40:28.161 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.161 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.161 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.161 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T03:40:28.162 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.162 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.162 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.162 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.162 INFO:teuthology.orchestra.run.smithi064.stdout: "prefix": { 2024-09-12T03:40:28.162 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prefix", 2024-09-12T03:40:28.162 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.162 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.162 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.162 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.162 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.162 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.162 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.162 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "name prefix for any created local pool", 2024-09-12T03:40:28.163 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.163 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.163 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.163 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.163 INFO:teuthology.orchestra.run.smithi064.stdout: "subtree": { 2024-09-12T03:40:28.163 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "subtree", 2024-09-12T03:40:28.163 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.163 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.163 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.163 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "rack", 2024-09-12T03:40:28.163 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.163 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.163 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.163 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T03:40:28.164 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T03:40:28.164 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.164 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.164 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.164 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.164 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.164 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mds_autoscaler", 2024-09-12T03:40:28.164 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.164 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.164 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.164 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.165 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.165 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.165 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.165 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.165 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.165 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.165 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.165 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.165 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.165 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.165 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.165 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.165 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.165 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.166 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.166 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.166 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.166 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.166 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.166 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.166 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.166 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.166 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.166 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.166 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.166 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.167 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.167 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.167 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.167 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.167 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.167 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.167 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.167 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.167 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.167 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.167 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.167 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.167 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.167 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.168 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.168 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.168 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.168 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.168 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.168 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.168 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.168 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.168 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.168 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.168 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.168 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.168 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.168 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.169 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.169 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.169 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.169 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.169 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.169 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.169 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.169 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.169 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.169 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.169 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.169 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.170 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.170 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.170 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.170 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.170 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mirroring", 2024-09-12T03:40:28.170 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.170 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.170 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.170 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.170 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.170 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.170 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.170 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.170 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.171 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.171 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.171 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.171 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.171 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.171 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.171 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.171 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.171 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.171 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.171 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.171 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.171 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.171 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.172 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.172 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.172 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.172 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.172 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.172 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.172 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.172 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.172 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.172 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.172 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.172 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.172 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.173 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.173 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.173 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.173 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.173 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.173 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.173 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.173 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.173 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.173 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.173 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.173 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.173 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.173 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.174 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.174 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.174 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.174 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.174 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.174 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.174 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.174 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.174 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.174 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.174 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.174 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.174 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.175 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.175 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.175 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.175 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.175 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.175 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.175 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.175 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.175 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.175 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.175 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.175 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.175 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "nfs", 2024-09-12T03:40:28.176 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.176 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.176 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.176 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.176 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.176 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.176 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.176 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.176 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.176 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.176 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.176 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.176 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.177 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.177 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.177 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.177 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.177 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.177 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.177 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.177 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.177 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.177 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.177 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.177 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.177 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.178 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.178 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.178 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.178 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.178 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.178 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.178 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.178 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.178 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.178 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.178 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.178 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.179 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.179 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.179 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.179 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.179 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.179 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.179 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.179 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.179 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.179 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.179 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.179 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.179 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.180 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.180 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.180 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.180 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.180 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.180 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.180 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.180 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.180 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.180 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.180 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.180 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.180 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.181 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.181 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.181 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.181 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.181 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.181 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.181 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.181 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.181 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.181 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.182 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.182 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "orchestrator", 2024-09-12T03:40:28.182 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.182 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.182 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.182 INFO:teuthology.orchestra.run.smithi064.stdout: "fail_fs": { 2024-09-12T03:40:28.182 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "fail_fs", 2024-09-12T03:40:28.182 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.182 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.182 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.182 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.182 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.182 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.183 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.183 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T03:40:28.183 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.183 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.183 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.183 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.183 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.183 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.183 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.183 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.183 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.183 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.184 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.184 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.184 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.184 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.184 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.184 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.184 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.184 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.184 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.184 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.184 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.184 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.184 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.185 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.185 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.185 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.185 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.185 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.185 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.185 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.185 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.185 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.185 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.185 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.185 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.185 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.186 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.186 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.186 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.186 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.186 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.186 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.186 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.186 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.186 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.186 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.186 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.186 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.186 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.186 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.187 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.187 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.187 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.187 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.187 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.187 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.187 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.187 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.187 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.187 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.187 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.187 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.187 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.188 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.188 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.188 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.188 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.188 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.188 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.188 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.188 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.188 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.188 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator": { 2024-09-12T03:40:28.188 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "orchestrator", 2024-09-12T03:40:28.188 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.188 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.189 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.189 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.189 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.189 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.189 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.189 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm", 2024-09-12T03:40:28.189 INFO:teuthology.orchestra.run.smithi064.stdout: "rook", 2024-09-12T03:40:28.189 INFO:teuthology.orchestra.run.smithi064.stdout: "test_orchestrator" 2024-09-12T03:40:28.189 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.189 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Orchestrator backend", 2024-09-12T03:40:28.189 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.189 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.189 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.189 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.190 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.190 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.190 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.190 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "osd_perf_query", 2024-09-12T03:40:28.190 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.190 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.190 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.190 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.190 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.190 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.190 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.190 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.190 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.190 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.191 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.191 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.191 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.191 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.191 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.191 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.191 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.191 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.191 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.191 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.191 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.191 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.191 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.192 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.192 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.192 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.192 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.192 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.192 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.192 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.192 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.192 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.192 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.192 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.192 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.192 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.193 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.193 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.193 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.193 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.193 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.193 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.193 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.193 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.193 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.193 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.193 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.193 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.193 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.194 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.194 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.194 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.194 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.194 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.194 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.194 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.194 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.194 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.194 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.194 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.194 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.195 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.195 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.195 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.195 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.195 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.195 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.195 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.195 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.195 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.195 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.195 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.195 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.195 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.195 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.196 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.196 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "osd_support", 2024-09-12T03:40:28.196 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.196 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.196 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.196 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.196 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.196 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.196 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.196 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.196 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.196 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.196 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.197 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.197 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.197 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.197 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.197 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.197 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.197 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.197 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.197 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.197 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.197 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.197 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.197 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.198 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.198 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.198 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.198 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.198 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.198 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.198 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.198 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.198 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.198 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.198 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.198 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.198 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.199 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.199 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.199 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.199 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.199 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.199 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.199 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.199 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.199 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.199 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.199 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.199 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.199 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.200 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.200 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.200 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.200 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.200 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.200 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.200 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.200 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.200 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.200 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.200 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.200 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.200 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.201 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.201 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.201 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.201 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.201 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.201 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.201 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.201 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.201 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.201 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.201 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.201 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.201 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.202 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pg_autoscaler", 2024-09-12T03:40:28.202 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.202 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.202 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.202 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.202 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.202 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.202 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.202 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.202 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.202 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.202 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.202 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.202 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.203 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.203 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.203 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.203 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.203 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.203 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.203 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.203 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.203 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.203 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.203 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.203 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.203 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.203 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.204 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.204 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.204 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.204 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.204 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.204 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.204 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.204 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.204 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.204 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.204 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.204 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.204 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.204 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.205 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.205 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.205 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.205 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.205 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.205 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.205 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.205 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.205 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.205 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.205 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.205 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.205 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.206 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.206 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.206 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.206 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.206 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.206 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.206 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.206 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.206 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.206 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.206 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.206 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.206 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.206 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.207 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.207 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.207 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.207 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.207 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.207 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-12T03:40:28.207 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-12T03:40:28.207 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:28.207 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.207 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.207 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-12T03:40:28.207 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.207 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.208 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.208 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.208 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.208 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.208 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.208 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.208 INFO:teuthology.orchestra.run.smithi064.stdout: "threshold": { 2024-09-12T03:40:28.208 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "threshold", 2024-09-12T03:40:28.208 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-12T03:40:28.208 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.208 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.208 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3.0", 2024-09-12T03:40:28.208 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1.0", 2024-09-12T03:40:28.209 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.209 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.209 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "scaling threshold", 2024-09-12T03:40:28.209 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T03:40:28.209 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.209 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.209 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.209 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.209 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.209 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.209 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "progress", 2024-09-12T03:40:28.209 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.209 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.210 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.210 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_pg_recovery_event": { 2024-09-12T03:40:28.210 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_pg_recovery_event", 2024-09-12T03:40:28.210 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.210 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.210 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.210 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.210 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.210 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.210 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.210 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T03:40:28.210 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.210 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.210 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.211 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.211 INFO:teuthology.orchestra.run.smithi064.stdout: "enabled": { 2024-09-12T03:40:28.211 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enabled", 2024-09-12T03:40:28.211 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.211 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.211 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.211 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.211 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.211 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.211 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.211 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.211 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.211 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.212 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.212 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.212 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.212 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.212 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.212 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.212 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.212 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.212 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.212 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.212 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.212 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.212 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.213 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.213 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.213 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.213 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.213 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.213 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.213 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.213 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.213 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.213 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.213 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.213 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.213 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.213 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.214 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.214 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.214 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.214 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.214 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.214 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.214 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.214 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.214 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.214 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.214 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.214 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.214 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.214 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.215 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.215 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.215 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.215 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.215 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.215 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.215 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.215 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.215 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.215 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.215 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.215 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.215 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.215 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.216 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.216 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.216 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.216 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.216 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.216 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.216 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.216 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.216 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.216 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.216 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.216 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.216 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.217 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.217 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.217 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.217 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.217 INFO:teuthology.orchestra.run.smithi064.stdout: "max_completed_events": { 2024-09-12T03:40:28.217 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_completed_events", 2024-09-12T03:40:28.217 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.217 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.217 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.217 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "50", 2024-09-12T03:40:28.217 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.217 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.217 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.217 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "number of past completed events to remember", 2024-09-12T03:40:28.218 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.218 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.218 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.218 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.218 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-12T03:40:28.218 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-12T03:40:28.218 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:28.218 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.218 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.218 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-12T03:40:28.218 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.218 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.218 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.218 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long the module is going to sleep", 2024-09-12T03:40:28.219 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.219 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.219 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.219 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.219 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.219 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.219 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.219 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prometheus", 2024-09-12T03:40:28.219 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.219 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.219 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.219 INFO:teuthology.orchestra.run.smithi064.stdout: "cache": { 2024-09-12T03:40:28.219 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cache", 2024-09-12T03:40:28.219 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.220 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.220 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.220 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.220 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.220 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.220 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.220 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.220 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.220 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.220 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.220 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.220 INFO:teuthology.orchestra.run.smithi064.stdout: "exclude_perf_counters": { 2024-09-12T03:40:28.220 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "exclude_perf_counters", 2024-09-12T03:40:28.221 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.221 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.221 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.221 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.221 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.221 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.221 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.221 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T03:40:28.221 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T03:40:28.221 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.221 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.221 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.221 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.222 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.222 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.222 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.222 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.222 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.222 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.222 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.222 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.222 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.222 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.222 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.222 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.222 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.223 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.223 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.223 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.223 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.223 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.223 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.223 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.223 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.223 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.223 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.223 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.223 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.223 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.223 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.224 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.224 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.224 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.224 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.224 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.224 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.224 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.224 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.224 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.224 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.224 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.224 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.224 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.225 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.225 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.225 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.225 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.225 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.225 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.225 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.225 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.225 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.225 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.225 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.225 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.225 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.225 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.226 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.226 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.226 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.226 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.226 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.226 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.226 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.226 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.226 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.226 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.226 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.226 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.226 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.226 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.227 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.227 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_stats_pools": { 2024-09-12T03:40:28.227 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_stats_pools", 2024-09-12T03:40:28.227 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.227 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.227 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.227 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.227 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.227 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.227 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.227 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.227 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.227 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.228 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.228 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.228 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T03:40:28.228 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T03:40:28.228 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.228 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.228 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.228 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "300", 2024-09-12T03:40:28.228 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.228 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.228 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.228 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.228 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.228 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.229 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.229 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.229 INFO:teuthology.orchestra.run.smithi064.stdout: "scrape_interval": { 2024-09-12T03:40:28.229 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "scrape_interval", 2024-09-12T03:40:28.229 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-12T03:40:28.229 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.229 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.229 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15.0", 2024-09-12T03:40:28.229 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.229 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.229 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.229 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.229 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.229 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.230 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.230 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.230 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-09-12T03:40:28.230 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-09-12T03:40:28.230 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.230 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.230 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.230 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "::", 2024-09-12T03:40:28.230 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.230 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.230 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.230 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T03:40:28.230 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.230 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.231 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.231 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.231 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-09-12T03:40:28.231 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-09-12T03:40:28.231 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.231 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.231 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.231 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "9283", 2024-09-12T03:40:28.231 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.231 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.231 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.231 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T03:40:28.231 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.231 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.232 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.232 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.232 INFO:teuthology.orchestra.run.smithi064.stdout: "stale_cache_strategy": { 2024-09-12T03:40:28.232 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "stale_cache_strategy", 2024-09-12T03:40:28.232 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.232 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.232 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.232 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "log", 2024-09-12T03:40:28.232 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.232 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.232 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.232 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.232 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.232 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.233 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.233 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.233 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_behaviour": { 2024-09-12T03:40:28.233 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_behaviour", 2024-09-12T03:40:28.233 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.233 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.233 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.233 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "default", 2024-09-12T03:40:28.233 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.233 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.233 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.233 INFO:teuthology.orchestra.run.smithi064.stdout: "default", 2024-09-12T03:40:28.233 INFO:teuthology.orchestra.run.smithi064.stdout: "error" 2024-09-12T03:40:28.234 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.234 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.234 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.234 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.234 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.234 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.234 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_error_status_code": { 2024-09-12T03:40:28.234 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_error_status_code", 2024-09-12T03:40:28.234 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.234 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.234 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.234 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2024-09-12T03:40:28.234 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "400", 2024-09-12T03:40:28.235 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "599", 2024-09-12T03:40:28.235 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.235 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.235 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.235 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.235 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.235 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.235 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.235 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.235 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.235 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_support", 2024-09-12T03:40:28.235 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.235 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.235 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.236 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.236 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.236 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.236 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.236 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.236 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.236 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.236 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.236 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.236 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.236 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.236 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.236 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.236 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.237 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.237 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.237 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.237 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.237 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.237 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.237 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.237 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.237 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.237 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.237 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.237 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.237 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.238 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.238 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.238 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.238 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.238 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.238 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.238 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.238 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.238 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.238 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.238 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.238 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.238 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.238 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.239 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.239 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.239 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.239 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.239 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.239 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.239 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.239 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.239 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.239 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.239 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.239 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.239 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.240 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.240 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.240 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.240 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.240 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.240 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.240 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.240 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.240 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.240 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.240 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.240 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.240 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.240 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.241 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.241 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.241 INFO:teuthology.orchestra.run.smithi064.stdout: "max_concurrent_snap_create": { 2024-09-12T03:40:28.241 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_concurrent_snap_create", 2024-09-12T03:40:28.241 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.241 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.241 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.241 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-12T03:40:28.241 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.241 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.241 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.241 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.241 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.242 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.242 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.242 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.242 INFO:teuthology.orchestra.run.smithi064.stdout: "mirror_snapshot_schedule": { 2024-09-12T03:40:28.242 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T03:40:28.242 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.242 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.242 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.242 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.242 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.242 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.242 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.242 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.242 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.243 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.243 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.243 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.243 INFO:teuthology.orchestra.run.smithi064.stdout: "trash_purge_schedule": { 2024-09-12T03:40:28.243 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "trash_purge_schedule", 2024-09-12T03:40:28.243 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.243 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.243 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.243 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.243 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.243 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.243 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.243 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.243 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.244 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.244 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.244 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.244 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.244 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.244 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.244 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "restful", 2024-09-12T03:40:28.244 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.244 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.244 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.244 INFO:teuthology.orchestra.run.smithi064.stdout: "enable_auth": { 2024-09-12T03:40:28.244 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enable_auth", 2024-09-12T03:40:28.244 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.244 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.245 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.245 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.245 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.245 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.245 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.245 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.245 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.245 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.245 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.245 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.245 INFO:teuthology.orchestra.run.smithi064.stdout: "key_file": { 2024-09-12T03:40:28.245 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "key_file", 2024-09-12T03:40:28.245 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.245 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.246 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.246 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.246 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.246 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.246 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.246 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.246 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.246 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.246 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.246 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.246 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.246 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.246 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.246 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.247 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.247 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.247 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.247 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.247 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.247 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.247 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.247 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.247 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.247 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.247 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.247 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.248 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.248 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.248 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.248 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.248 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.248 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.248 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.248 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.248 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.248 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.248 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.248 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.249 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.249 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.249 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.249 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.249 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.249 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.249 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.249 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.249 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.249 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.249 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.249 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.249 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.249 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.250 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.250 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.250 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.250 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.250 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.250 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.250 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.250 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.250 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.250 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.250 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.250 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.250 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.250 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.251 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.251 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.251 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.251 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.251 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.251 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.251 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.251 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.251 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.251 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.251 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.251 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.251 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.252 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.252 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-09-12T03:40:28.252 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-09-12T03:40:28.252 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.252 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.252 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.252 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.252 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.252 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.252 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.252 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.252 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.252 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.252 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.253 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.253 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-09-12T03:40:28.253 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-09-12T03:40:28.253 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.253 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.253 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.253 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.253 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.253 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.253 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.253 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.253 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.253 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.254 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.254 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.254 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.254 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.254 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.254 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rgw", 2024-09-12T03:40:28.254 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.254 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.254 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.254 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.254 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.254 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.254 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.254 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.255 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.255 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.255 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.255 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.255 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.255 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.255 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.255 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.255 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.255 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.255 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.255 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.255 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.256 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.256 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.256 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.256 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.256 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.256 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.256 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.256 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.256 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.256 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.256 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.256 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.256 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.256 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.257 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.257 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.257 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.257 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.257 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.257 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.257 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.257 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.257 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.257 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.257 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.257 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.257 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.257 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.258 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.258 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.258 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.258 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.258 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.258 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.258 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.258 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.258 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.258 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.258 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.258 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.259 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.259 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.259 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.259 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.259 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.259 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.259 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.259 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.259 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.259 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.259 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.259 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.259 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.259 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.260 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.260 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rook", 2024-09-12T03:40:28.260 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.260 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.260 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.260 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.260 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.260 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.260 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.260 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.260 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.260 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.260 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.260 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.261 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.261 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.261 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.261 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.261 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.261 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.261 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.261 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.261 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.261 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.261 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.261 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.261 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.261 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.262 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.262 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.262 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.262 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.262 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.262 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.262 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.262 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.262 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.262 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.262 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.262 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.263 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.263 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.263 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.263 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.263 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.263 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.263 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.263 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.263 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.263 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.263 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.263 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.263 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.263 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.264 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.264 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.264 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.264 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.264 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.264 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.264 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.264 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.264 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.264 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.264 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.264 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.265 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.265 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.265 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.265 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.265 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.265 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.265 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.265 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.265 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.265 INFO:teuthology.orchestra.run.smithi064.stdout: "storage_class": { 2024-09-12T03:40:28.265 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "storage_class", 2024-09-12T03:40:28.265 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.265 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.265 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.266 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "local", 2024-09-12T03:40:28.266 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.266 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.266 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.266 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T03:40:28.266 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.266 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.266 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.266 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.266 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.266 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.266 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.266 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "selftest", 2024-09-12T03:40:28.267 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.267 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.267 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.267 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.267 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.267 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.267 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.267 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.267 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.267 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.267 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.267 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.267 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.267 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.268 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.268 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.268 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.268 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.268 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.268 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.268 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.268 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.268 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.268 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.268 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.268 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.268 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.269 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.269 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.269 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.269 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.269 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.269 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.269 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.269 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.269 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.269 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.269 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.269 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.269 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.270 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.270 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.270 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.270 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.270 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.270 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.270 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.270 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.270 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.270 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.270 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.270 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.270 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.270 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.271 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.271 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.271 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.271 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.271 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.271 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.271 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.271 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.271 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.271 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.271 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.271 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.271 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.272 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.272 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.272 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.272 INFO:teuthology.orchestra.run.smithi064.stdout: "roption1": { 2024-09-12T03:40:28.272 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "roption1", 2024-09-12T03:40:28.272 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.272 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.272 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.272 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.272 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.272 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.273 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.273 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.273 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.273 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.273 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.273 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.273 INFO:teuthology.orchestra.run.smithi064.stdout: "roption2": { 2024-09-12T03:40:28.273 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "roption2", 2024-09-12T03:40:28.273 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.273 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.273 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.273 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "xyz", 2024-09-12T03:40:28.273 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.273 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.274 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.274 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.274 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.274 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.274 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.274 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.274 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption1": { 2024-09-12T03:40:28.274 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption1", 2024-09-12T03:40:28.274 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.274 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.274 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.274 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.274 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.274 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.275 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.275 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.275 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.275 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.275 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.275 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.275 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption2": { 2024-09-12T03:40:28.275 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption2", 2024-09-12T03:40:28.275 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.275 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.275 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.275 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.275 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.275 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.276 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.276 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.276 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.276 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.276 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.276 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption3": { 2024-09-12T03:40:28.276 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption3", 2024-09-12T03:40:28.276 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-12T03:40:28.276 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.276 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.276 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.277 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.277 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.277 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.277 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.277 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.277 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.277 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.277 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.277 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption4": { 2024-09-12T03:40:28.277 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption4", 2024-09-12T03:40:28.277 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.277 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.277 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.277 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.278 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.278 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.278 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.278 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.278 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.278 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.278 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.278 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption5": { 2024-09-12T03:40:28.278 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption5", 2024-09-12T03:40:28.278 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.278 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.278 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.279 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.279 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.279 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.279 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.279 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.279 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.279 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.279 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.279 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.279 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption6": { 2024-09-12T03:40:28.279 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption6", 2024-09-12T03:40:28.279 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.279 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.279 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.280 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.280 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.280 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.280 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.280 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.280 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.280 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.280 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.280 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.280 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption7": { 2024-09-12T03:40:28.280 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption7", 2024-09-12T03:40:28.280 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.280 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.281 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.281 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.281 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-09-12T03:40:28.281 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "42", 2024-09-12T03:40:28.281 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.281 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.281 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.281 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.281 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.281 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.281 INFO:teuthology.orchestra.run.smithi064.stdout: "testkey": { 2024-09-12T03:40:28.281 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testkey", 2024-09-12T03:40:28.282 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.282 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.282 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.282 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.282 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.282 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.282 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.282 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.282 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.282 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.282 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.282 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.282 INFO:teuthology.orchestra.run.smithi064.stdout: "testlkey": { 2024-09-12T03:40:28.283 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testlkey", 2024-09-12T03:40:28.283 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.283 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.283 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.283 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.283 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.283 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.283 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.283 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.283 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.283 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.283 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.284 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.284 INFO:teuthology.orchestra.run.smithi064.stdout: "testnewline": { 2024-09-12T03:40:28.284 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testnewline", 2024-09-12T03:40:28.284 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.284 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.284 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.284 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.284 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.284 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.284 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.284 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.284 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.285 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.285 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.285 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.285 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.285 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.285 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.285 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snap_schedule", 2024-09-12T03:40:28.285 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.285 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.285 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.285 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_m_granularity": { 2024-09-12T03:40:28.285 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_m_granularity", 2024-09-12T03:40:28.285 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.286 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.286 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.286 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.286 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.286 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.286 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.286 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T03:40:28.286 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.286 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.286 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.286 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.286 INFO:teuthology.orchestra.run.smithi064.stdout: "dump_on_update": { 2024-09-12T03:40:28.286 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "dump_on_update", 2024-09-12T03:40:28.287 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.287 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.287 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.287 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.287 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.287 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.287 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.287 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "dump database to debug log on update", 2024-09-12T03:40:28.287 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.287 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.287 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.287 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.287 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.287 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.288 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.288 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.288 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.288 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.288 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.288 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.288 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.288 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.288 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.288 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.288 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.288 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.288 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.289 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.289 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.289 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.289 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.289 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.289 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.289 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.289 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.289 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.289 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.289 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.289 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.289 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.290 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.290 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.290 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.290 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.290 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.290 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.290 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.290 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.290 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.290 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.290 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.290 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.290 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.290 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.291 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.291 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.291 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.291 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.291 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.291 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.291 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.291 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.291 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.291 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.291 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.291 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.291 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.291 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.292 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.292 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.292 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.292 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.292 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.292 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.292 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.292 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.292 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.292 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.292 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.292 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.292 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.293 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.293 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.293 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.293 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.293 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "stats", 2024-09-12T03:40:28.293 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.293 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.293 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.293 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.293 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.293 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.293 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.293 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.294 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.294 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.294 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.294 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.294 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.294 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.294 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.294 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.294 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.294 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.294 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.294 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.295 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.295 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.295 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.295 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.295 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.295 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.295 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.295 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.295 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.295 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.295 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.295 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.295 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.295 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.296 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.296 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.296 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.296 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.296 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.296 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.296 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.296 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.296 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.296 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.296 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.296 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.297 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.297 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.297 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.297 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.297 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.297 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.297 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.297 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.297 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.297 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.297 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.297 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.297 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.297 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.298 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.298 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.298 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.298 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.298 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.298 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.298 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.298 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.298 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.298 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.298 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.298 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.298 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.298 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.299 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.299 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "status", 2024-09-12T03:40:28.299 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.299 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.299 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.299 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.299 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.299 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.299 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.299 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.299 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.299 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.300 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.300 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.300 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.300 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.300 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.300 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.300 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.300 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.300 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.300 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.300 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.300 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.300 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.301 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.301 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.301 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.301 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.301 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.301 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.301 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.301 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.301 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.301 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.301 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.301 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.301 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.302 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.302 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.302 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.302 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.302 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.302 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.302 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.302 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.302 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.302 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.302 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.302 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.302 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.303 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.303 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.303 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.303 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.303 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.303 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.303 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.303 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.303 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.303 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.303 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.303 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.303 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.304 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.304 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.304 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.304 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.304 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.304 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.304 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.304 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.304 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.304 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.305 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.305 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "telegraf", 2024-09-12T03:40:28.305 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.305 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.305 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.305 INFO:teuthology.orchestra.run.smithi064.stdout: "address": { 2024-09-12T03:40:28.305 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "address", 2024-09-12T03:40:28.305 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.305 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.306 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.306 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T03:40:28.306 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.306 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.306 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.306 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.306 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.306 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.307 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.307 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.307 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-12T03:40:28.307 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-12T03:40:28.307 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:28.307 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.307 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.307 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15", 2024-09-12T03:40:28.308 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.308 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.308 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.308 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.308 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.308 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.308 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.308 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.308 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.308 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.308 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.308 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.308 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.309 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.309 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.309 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.309 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.309 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.309 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.309 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.309 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.309 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.309 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.309 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.309 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.309 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.310 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.310 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.310 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.310 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.310 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.310 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.310 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.310 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.310 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.310 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.310 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.310 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.310 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.310 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.311 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.311 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.311 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.311 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.311 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.311 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.311 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.311 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.311 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.311 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.311 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.311 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.311 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.312 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.312 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.312 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.312 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.312 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.312 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.312 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.312 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.312 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.312 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.312 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.312 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.312 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.312 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.313 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.313 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.313 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.313 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.313 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.313 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.313 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.313 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.313 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.313 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.313 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.313 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.313 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.314 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.314 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "telemetry", 2024-09-12T03:40:28.314 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.314 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.314 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.314 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_basic": { 2024-09-12T03:40:28.314 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_basic", 2024-09-12T03:40:28.314 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.314 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.314 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.314 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.314 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.314 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.314 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.315 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T03:40:28.315 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.315 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.315 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.315 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_crash": { 2024-09-12T03:40:28.315 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_crash", 2024-09-12T03:40:28.315 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.315 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.315 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.315 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.315 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.315 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.315 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.316 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T03:40:28.316 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.316 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.316 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.316 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.316 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_device": { 2024-09-12T03:40:28.316 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_device", 2024-09-12T03:40:28.316 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.316 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.316 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.316 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.316 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.316 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.317 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.317 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T03:40:28.317 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.317 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.317 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.317 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.317 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_ident": { 2024-09-12T03:40:28.317 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_ident", 2024-09-12T03:40:28.317 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.317 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.317 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.317 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.317 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.318 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.318 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.318 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T03:40:28.318 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.318 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.318 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.318 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.318 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_perf": { 2024-09-12T03:40:28.318 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_perf", 2024-09-12T03:40:28.318 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.318 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.318 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.318 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.319 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.319 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.319 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.319 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T03:40:28.319 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.319 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.319 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.319 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.319 INFO:teuthology.orchestra.run.smithi064.stdout: "contact": { 2024-09-12T03:40:28.319 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "contact", 2024-09-12T03:40:28.319 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.319 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.319 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.319 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.320 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.320 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.320 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.320 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.320 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.320 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.320 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.320 INFO:teuthology.orchestra.run.smithi064.stdout: "description": { 2024-09-12T03:40:28.320 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "description", 2024-09-12T03:40:28.320 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.320 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.320 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.321 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.321 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.321 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.321 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.321 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.321 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.321 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.321 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.321 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.321 INFO:teuthology.orchestra.run.smithi064.stdout: "device_url": { 2024-09-12T03:40:28.321 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_url", 2024-09-12T03:40:28.321 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.321 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.322 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.322 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T03:40:28.322 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.322 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.322 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.322 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.322 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.322 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.322 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.322 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.322 INFO:teuthology.orchestra.run.smithi064.stdout: "enabled": { 2024-09-12T03:40:28.322 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enabled", 2024-09-12T03:40:28.322 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.322 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.323 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.323 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.323 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.323 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.323 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.323 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.323 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.323 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.323 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.323 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.323 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-12T03:40:28.323 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-12T03:40:28.323 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.324 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.324 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.324 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "24", 2024-09-12T03:40:28.324 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "8", 2024-09-12T03:40:28.324 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.324 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.324 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.324 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.324 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.324 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.324 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.324 INFO:teuthology.orchestra.run.smithi064.stdout: "last_opt_revision": { 2024-09-12T03:40:28.324 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "last_opt_revision", 2024-09-12T03:40:28.325 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.325 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.325 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.325 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1", 2024-09-12T03:40:28.325 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.325 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.325 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.325 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.325 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.325 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.325 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.325 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.325 INFO:teuthology.orchestra.run.smithi064.stdout: "leaderboard": { 2024-09-12T03:40:28.326 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "leaderboard", 2024-09-12T03:40:28.326 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.326 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.326 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.326 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.326 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.326 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.326 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.326 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.326 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.326 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.326 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.326 INFO:teuthology.orchestra.run.smithi064.stdout: "leaderboard_description": { 2024-09-12T03:40:28.327 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "leaderboard_description", 2024-09-12T03:40:28.327 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.327 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.327 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.327 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.327 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.327 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.327 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.327 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.327 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.327 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.327 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.327 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.327 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.328 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.328 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.328 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.328 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.328 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.328 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.328 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.328 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.328 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.328 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.328 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.328 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.328 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.329 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.329 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.329 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.329 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.329 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.329 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.329 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.329 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.329 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.329 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.329 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.329 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.329 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.329 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.330 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.330 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.330 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.330 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.330 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.330 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.330 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.330 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.330 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.330 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.330 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.330 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.330 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.330 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.331 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.331 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.331 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.331 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.331 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.331 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.331 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.331 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.331 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.331 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.331 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.331 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.331 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.331 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.332 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.332 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.332 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.332 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.332 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.332 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.332 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.332 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.332 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.332 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.332 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.332 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.332 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.333 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.333 INFO:teuthology.orchestra.run.smithi064.stdout: "organization": { 2024-09-12T03:40:28.333 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "organization", 2024-09-12T03:40:28.333 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.333 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.333 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.333 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.333 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.333 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.333 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.333 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.333 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.333 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.333 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.334 INFO:teuthology.orchestra.run.smithi064.stdout: "proxy": { 2024-09-12T03:40:28.334 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "proxy", 2024-09-12T03:40:28.334 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.334 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.334 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.334 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.334 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.334 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.334 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.334 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.334 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.334 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.334 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.335 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.335 INFO:teuthology.orchestra.run.smithi064.stdout: "url": { 2024-09-12T03:40:28.335 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "url", 2024-09-12T03:40:28.335 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.335 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.335 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.335 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T03:40:28.335 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.335 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.335 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.335 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.335 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.335 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.336 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.336 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.336 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.336 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.336 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.336 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "test_orchestrator", 2024-09-12T03:40:28.336 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.336 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.336 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.336 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.336 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.336 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.336 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.336 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.337 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.337 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.337 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.337 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.337 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.337 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.337 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.337 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.337 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.337 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.337 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.337 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.337 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.338 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.338 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.338 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.338 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.338 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.338 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.338 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.338 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.338 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.338 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.338 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.338 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.338 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.338 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.339 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.339 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.339 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.339 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.339 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.339 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.339 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.339 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.339 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.339 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.339 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.339 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.339 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.340 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.340 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.340 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.340 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.340 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.340 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.340 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.340 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.340 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.340 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.340 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.340 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.341 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.341 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.341 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.341 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.341 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.341 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.341 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.341 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.341 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.341 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.341 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.341 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.341 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.341 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.342 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.342 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "volumes", 2024-09-12T03:40:28.342 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.342 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.342 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.342 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.342 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.342 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.342 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.342 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.342 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.342 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.342 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.343 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.343 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.343 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.343 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.343 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.343 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.343 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.343 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.343 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.343 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.343 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.343 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.343 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.344 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.344 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.344 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.344 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.344 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.344 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.344 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.344 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.344 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.344 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.344 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.344 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.344 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.345 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.345 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.345 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.345 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.345 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.345 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.345 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.345 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.345 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.345 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.345 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.345 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.345 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.346 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.346 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.346 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.346 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.346 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.346 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.346 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.348 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.348 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.348 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.348 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.348 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.348 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.348 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.348 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.348 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.348 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.348 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.349 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.349 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.349 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.349 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.349 INFO:teuthology.orchestra.run.smithi064.stdout: "max_concurrent_clones": { 2024-09-12T03:40:28.349 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_concurrent_clones", 2024-09-12T03:40:28.349 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.349 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.349 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.349 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "4", 2024-09-12T03:40:28.349 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.349 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.349 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.350 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T03:40:28.350 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.350 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.350 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.350 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.350 INFO:teuthology.orchestra.run.smithi064.stdout: "periodic_async_work": { 2024-09-12T03:40:28.350 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "periodic_async_work", 2024-09-12T03:40:28.350 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.350 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.350 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.350 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.350 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.351 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.351 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.351 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Periodically check for async work", 2024-09-12T03:40:28.351 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.351 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.351 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.351 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.351 INFO:teuthology.orchestra.run.smithi064.stdout: "snapshot_clone_delay": { 2024-09-12T03:40:28.351 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snapshot_clone_delay", 2024-09-12T03:40:28.351 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.351 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.351 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.351 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-12T03:40:28.351 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.352 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.352 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.352 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T03:40:28.352 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.352 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.352 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.352 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.352 INFO:teuthology.orchestra.run.smithi064.stdout: "snapshot_clone_no_wait": { 2024-09-12T03:40:28.352 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T03:40:28.352 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.352 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.352 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.353 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.353 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.353 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.353 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.353 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T03:40:28.353 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.353 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.353 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.353 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.353 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.353 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.353 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.353 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix", 2024-09-12T03:40:28.354 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.354 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.354 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.354 INFO:teuthology.orchestra.run.smithi064.stdout: "discovery_interval": { 2024-09-12T03:40:28.354 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "discovery_interval", 2024-09-12T03:40:28.354 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-12T03:40:28.354 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.354 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.354 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "100", 2024-09-12T03:40:28.354 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.354 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.354 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.354 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.354 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.355 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.355 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.355 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.355 INFO:teuthology.orchestra.run.smithi064.stdout: "identifier": { 2024-09-12T03:40:28.355 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "identifier", 2024-09-12T03:40:28.355 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.355 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.355 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.355 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.355 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.355 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.355 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.356 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.356 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.356 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.356 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.356 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.356 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-12T03:40:28.356 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-12T03:40:28.356 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:28.356 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.356 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.356 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-12T03:40:28.356 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.356 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.357 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.357 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.357 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.357 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.357 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.357 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.357 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.357 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.357 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.357 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.357 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.357 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.358 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.358 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.358 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.358 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.358 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.358 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.358 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.358 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.358 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.358 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.358 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.358 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.358 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.359 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.359 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.359 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.359 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.359 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.359 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.359 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.359 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.359 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.359 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.359 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.359 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.359 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.360 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.360 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.360 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.360 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.360 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.360 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.360 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.360 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.360 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.360 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.360 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.360 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.360 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.361 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.361 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.361 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.361 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.361 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.361 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.361 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.361 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.361 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.361 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.361 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.361 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.362 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.362 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.362 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.362 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.362 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.362 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.362 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.362 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.362 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.362 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.362 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.362 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.362 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.363 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_host": { 2024-09-12T03:40:28.363 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_host", 2024-09-12T03:40:28.363 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.363 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.363 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.363 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.363 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.363 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.363 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.363 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.363 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.363 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.363 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.364 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.364 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_port": { 2024-09-12T03:40:28.364 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_port", 2024-09-12T03:40:28.364 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.364 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.364 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.364 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10051", 2024-09-12T03:40:28.364 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.364 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.364 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.364 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.364 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.364 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.365 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.365 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.365 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_sender": { 2024-09-12T03:40:28.365 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_sender", 2024-09-12T03:40:28.365 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.365 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.365 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.365 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T03:40:28.365 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.365 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.365 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.365 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.365 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.366 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.366 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.366 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.366 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.366 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.366 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T03:40:28.366 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.366 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.366 INFO:teuthology.orchestra.run.smithi064.stdout: "modules": [ 2024-09-12T03:40:28.366 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm", 2024-09-12T03:40:28.366 INFO:teuthology.orchestra.run.smithi064.stdout: "dashboard", 2024-09-12T03:40:28.366 INFO:teuthology.orchestra.run.smithi064.stdout: "iostat", 2024-09-12T03:40:28.366 INFO:teuthology.orchestra.run.smithi064.stdout: "nfs", 2024-09-12T03:40:28.367 INFO:teuthology.orchestra.run.smithi064.stdout: "restful", 2024-09-12T03:40:28.367 INFO:teuthology.orchestra.run.smithi064.stdout: "snap_schedule" 2024-09-12T03:40:28.367 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.367 INFO:teuthology.orchestra.run.smithi064.stdout: "available_modules": [ 2024-09-12T03:40:28.367 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.367 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "alerts", 2024-09-12T03:40:28.367 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.367 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.367 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.367 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-12T03:40:28.367 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-12T03:40:28.367 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:28.367 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.368 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.368 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-12T03:40:28.368 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.368 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.368 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.368 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T03:40:28.368 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.368 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.368 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.368 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.368 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.368 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.369 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.369 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.369 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.369 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.369 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.369 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.369 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.369 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.369 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.369 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.369 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.369 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.369 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.369 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.370 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.370 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.370 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.370 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.370 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.370 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.370 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.370 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.370 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.370 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.370 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.370 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.370 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.371 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.371 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.371 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.371 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.371 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.371 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.371 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.371 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.371 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.371 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.371 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.371 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.371 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.372 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.372 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.372 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.372 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.372 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.372 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.372 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.372 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.372 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.372 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.372 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.372 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.372 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.372 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.373 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.373 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.373 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.373 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.373 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.373 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.373 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.373 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.373 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.373 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.373 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.373 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.373 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.374 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.374 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_destination": { 2024-09-12T03:40:28.374 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_destination", 2024-09-12T03:40:28.374 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.374 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.374 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.374 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.374 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.374 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.374 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.374 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Email address to send alerts to", 2024-09-12T03:40:28.374 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.374 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.374 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.375 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.375 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_from_name": { 2024-09-12T03:40:28.375 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_from_name", 2024-09-12T03:40:28.375 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.375 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.375 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.375 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "Ceph", 2024-09-12T03:40:28.375 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.375 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.375 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.375 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Email From: name", 2024-09-12T03:40:28.375 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.375 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.376 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.376 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.376 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_host": { 2024-09-12T03:40:28.376 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_host", 2024-09-12T03:40:28.376 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.376 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.376 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.376 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.376 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.376 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.377 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.377 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP server", 2024-09-12T03:40:28.377 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.377 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.377 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.377 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.377 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_password": { 2024-09-12T03:40:28.377 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_password", 2024-09-12T03:40:28.377 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.377 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.377 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.377 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.378 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.378 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.378 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.378 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Password to authenticate with", 2024-09-12T03:40:28.378 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.378 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.378 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.378 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.378 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_port": { 2024-09-12T03:40:28.378 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_port", 2024-09-12T03:40:28.378 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.378 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.378 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.378 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "465", 2024-09-12T03:40:28.379 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.379 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.379 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.379 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP port", 2024-09-12T03:40:28.379 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.379 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.379 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.379 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.379 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_sender": { 2024-09-12T03:40:28.379 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_sender", 2024-09-12T03:40:28.379 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.379 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.379 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.379 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.380 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.380 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.380 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.380 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP envelope sender", 2024-09-12T03:40:28.380 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.380 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.380 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.380 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.380 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_ssl": { 2024-09-12T03:40:28.380 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_ssl", 2024-09-12T03:40:28.380 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.380 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.381 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.381 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.381 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.381 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.381 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.381 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T03:40:28.381 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.381 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.381 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.381 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.381 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_user": { 2024-09-12T03:40:28.381 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_user", 2024-09-12T03:40:28.381 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.381 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.382 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.382 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.382 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.382 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.382 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.382 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "User to authenticate as", 2024-09-12T03:40:28.382 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.382 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.382 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.382 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.382 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.382 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.382 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.382 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "balancer", 2024-09-12T03:40:28.383 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.383 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.383 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.383 INFO:teuthology.orchestra.run.smithi064.stdout: "active": { 2024-09-12T03:40:28.383 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "active", 2024-09-12T03:40:28.383 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.383 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.383 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.383 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.383 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.383 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.383 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.383 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T03:40:28.384 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.384 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.384 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.384 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.384 INFO:teuthology.orchestra.run.smithi064.stdout: "begin_time": { 2024-09-12T03:40:28.384 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "begin_time", 2024-09-12T03:40:28.384 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.384 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.384 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.384 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0000", 2024-09-12T03:40:28.384 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.384 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.384 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.385 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T03:40:28.385 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:40:28.385 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.385 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.385 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.385 INFO:teuthology.orchestra.run.smithi064.stdout: "begin_weekday": { 2024-09-12T03:40:28.385 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "begin_weekday", 2024-09-12T03:40:28.385 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-12T03:40:28.385 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.385 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.385 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-12T03:40:28.385 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0", 2024-09-12T03:40:28.385 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "6", 2024-09-12T03:40:28.386 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.386 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T03:40:28.386 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:40:28.386 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.386 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.386 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.386 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_max_iterations": { 2024-09-12T03:40:28.386 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_max_iterations", 2024-09-12T03:40:28.386 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-12T03:40:28.386 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.386 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.386 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "25", 2024-09-12T03:40:28.386 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-09-12T03:40:28.387 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "250", 2024-09-12T03:40:28.387 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.387 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T03:40:28.387 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.387 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.387 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.387 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.387 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_metrics": { 2024-09-12T03:40:28.387 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_metrics", 2024-09-12T03:40:28.387 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.387 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.387 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.387 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T03:40:28.387 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.388 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.388 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.388 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T03:40:28.388 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T03:40:28.388 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.388 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.388 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.388 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_step": { 2024-09-12T03:40:28.388 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_step", 2024-09-12T03:40:28.388 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-12T03:40:28.388 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.388 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.389 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0.5", 2024-09-12T03:40:28.389 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0.001", 2024-09-12T03:40:28.389 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "0.999", 2024-09-12T03:40:28.389 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.389 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "aggressiveness of optimization", 2024-09-12T03:40:28.389 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T03:40:28.389 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.389 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.389 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.389 INFO:teuthology.orchestra.run.smithi064.stdout: "end_time": { 2024-09-12T03:40:28.389 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "end_time", 2024-09-12T03:40:28.389 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.389 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.390 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.390 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "2359", 2024-09-12T03:40:28.390 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.390 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.390 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.390 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T03:40:28.390 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:40:28.390 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.390 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.390 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.390 INFO:teuthology.orchestra.run.smithi064.stdout: "end_weekday": { 2024-09-12T03:40:28.390 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "end_weekday", 2024-09-12T03:40:28.390 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-12T03:40:28.390 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.391 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.391 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-12T03:40:28.391 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0", 2024-09-12T03:40:28.391 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "6", 2024-09-12T03:40:28.391 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.391 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T03:40:28.391 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:40:28.391 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.391 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.391 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.391 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.391 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.391 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.392 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.392 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.392 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.392 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.392 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.392 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.392 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.392 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.392 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.392 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.392 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.392 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.392 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.393 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.393 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.393 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.393 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.393 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.393 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.393 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.393 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.393 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.393 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.393 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.393 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.393 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.393 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.394 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.394 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.394 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.394 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.394 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.394 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.394 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.394 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.394 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.394 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.394 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.394 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.394 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.394 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.395 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.395 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.395 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.395 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.395 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.395 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.395 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.395 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.395 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.395 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.395 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.395 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.395 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.395 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.396 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.396 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.396 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.396 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.396 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.396 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.396 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.396 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.396 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.396 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.396 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.396 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.396 INFO:teuthology.orchestra.run.smithi064.stdout: "min_score": { 2024-09-12T03:40:28.396 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "min_score", 2024-09-12T03:40:28.397 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-12T03:40:28.397 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.397 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.397 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-12T03:40:28.397 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.397 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.397 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.397 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T03:40:28.397 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.397 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.397 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.397 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.397 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": { 2024-09-12T03:40:28.398 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mode", 2024-09-12T03:40:28.398 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.398 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.398 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.398 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "upmap", 2024-09-12T03:40:28.398 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.398 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.398 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.398 INFO:teuthology.orchestra.run.smithi064.stdout: "crush-compat", 2024-09-12T03:40:28.398 INFO:teuthology.orchestra.run.smithi064.stdout: "none", 2024-09-12T03:40:28.398 INFO:teuthology.orchestra.run.smithi064.stdout: "read", 2024-09-12T03:40:28.398 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap", 2024-09-12T03:40:28.398 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap-read" 2024-09-12T03:40:28.398 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.399 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Balancer mode", 2024-09-12T03:40:28.399 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.399 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.399 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.399 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.399 INFO:teuthology.orchestra.run.smithi064.stdout: "pool_ids": { 2024-09-12T03:40:28.399 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pool_ids", 2024-09-12T03:40:28.399 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.399 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.399 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.399 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.399 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.399 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.399 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.400 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T03:40:28.400 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.400 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.400 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.400 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.400 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-12T03:40:28.400 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-12T03:40:28.400 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:28.400 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.400 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.400 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-12T03:40:28.400 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.400 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.400 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.401 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T03:40:28.401 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.401 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.401 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.401 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.401 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap_max_deviation": { 2024-09-12T03:40:28.401 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "upmap_max_deviation", 2024-09-12T03:40:28.401 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.401 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.401 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.401 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-12T03:40:28.401 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-09-12T03:40:28.401 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.402 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.402 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T03:40:28.402 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T03:40:28.402 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.402 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.402 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.402 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap_max_optimizations": { 2024-09-12T03:40:28.402 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "upmap_max_optimizations", 2024-09-12T03:40:28.402 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-12T03:40:28.402 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.402 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.402 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-12T03:40:28.402 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.403 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.403 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.403 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T03:40:28.403 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.403 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.403 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.403 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.403 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.403 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.403 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.403 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephadm", 2024-09-12T03:40:28.403 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.403 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.403 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.404 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_down_multiplier": { 2024-09-12T03:40:28.404 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_down_multiplier", 2024-09-12T03:40:28.404 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-12T03:40:28.404 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.404 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.404 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3.0", 2024-09-12T03:40:28.404 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.404 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.404 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.404 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T03:40:28.404 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.404 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.404 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.405 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.405 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_refresh_rate": { 2024-09-12T03:40:28.405 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_refresh_rate", 2024-09-12T03:40:28.405 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:28.405 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.405 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.405 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "20", 2024-09-12T03:40:28.405 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.405 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.405 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.405 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T03:40:28.405 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.405 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.406 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.406 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.406 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_starting_port": { 2024-09-12T03:40:28.406 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_starting_port", 2024-09-12T03:40:28.406 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.406 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.406 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.406 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "4721", 2024-09-12T03:40:28.406 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.406 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.406 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.406 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T03:40:28.406 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.407 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.407 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.407 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.407 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_ptrace": { 2024-09-12T03:40:28.407 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_ptrace", 2024-09-12T03:40:28.407 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.407 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.407 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.407 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.407 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.407 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.407 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.407 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T03:40:28.407 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T03:40:28.408 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.408 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.408 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.408 INFO:teuthology.orchestra.run.smithi064.stdout: "autotune_interval": { 2024-09-12T03:40:28.408 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "autotune_interval", 2024-09-12T03:40:28.408 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:28.408 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.408 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.408 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-12T03:40:28.408 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.408 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.408 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.408 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T03:40:28.409 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.409 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.409 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.409 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.409 INFO:teuthology.orchestra.run.smithi064.stdout: "autotune_memory_target_ratio": { 2024-09-12T03:40:28.409 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T03:40:28.409 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-12T03:40:28.409 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.409 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.409 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0.7", 2024-09-12T03:40:28.409 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.409 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.409 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.410 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T03:40:28.410 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.410 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.410 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.410 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.410 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm_log_destination": { 2024-09-12T03:40:28.410 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephadm_log_destination", 2024-09-12T03:40:28.410 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.410 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.410 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.410 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.410 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.410 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.411 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.411 INFO:teuthology.orchestra.run.smithi064.stdout: "file", 2024-09-12T03:40:28.411 INFO:teuthology.orchestra.run.smithi064.stdout: "file,syslog", 2024-09-12T03:40:28.411 INFO:teuthology.orchestra.run.smithi064.stdout: "syslog" 2024-09-12T03:40:28.411 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.411 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T03:40:28.411 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.411 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.411 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.411 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.411 INFO:teuthology.orchestra.run.smithi064.stdout: "cgroups_split": { 2024-09-12T03:40:28.411 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cgroups_split", 2024-09-12T03:40:28.411 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.411 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.412 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.412 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.412 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.412 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.412 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.412 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T03:40:28.412 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.412 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.412 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.412 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.412 INFO:teuthology.orchestra.run.smithi064.stdout: "config_checks_enabled": { 2024-09-12T03:40:28.412 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_checks_enabled", 2024-09-12T03:40:28.413 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.413 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.413 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.413 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.413 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.413 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.413 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.413 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T03:40:28.413 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.413 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.413 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.413 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.413 INFO:teuthology.orchestra.run.smithi064.stdout: "config_dashboard": { 2024-09-12T03:40:28.413 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_dashboard", 2024-09-12T03:40:28.414 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.414 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.414 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.414 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.414 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.414 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.414 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.414 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T03:40:28.414 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.414 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.414 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.414 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.414 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_alertmanager": { 2024-09-12T03:40:28.415 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_alertmanager", 2024-09-12T03:40:28.415 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.415 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.415 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.415 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T03:40:28.415 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.415 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.415 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.415 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-09-12T03:40:28.415 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.415 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.415 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.415 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.415 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_base": { 2024-09-12T03:40:28.416 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_base", 2024-09-12T03:40:28.416 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.416 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.416 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.416 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T03:40:28.416 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.416 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.416 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.416 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Container image name, without the tag", 2024-09-12T03:40:28.416 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.416 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.416 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.416 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.417 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_elasticsearch": { 2024-09-12T03:40:28.417 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_elasticsearch", 2024-09-12T03:40:28.417 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.417 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.417 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.417 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T03:40:28.417 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.417 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.417 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.417 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "elasticsearch container image", 2024-09-12T03:40:28.417 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.417 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.417 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.418 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.418 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_grafana": { 2024-09-12T03:40:28.418 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_grafana", 2024-09-12T03:40:28.418 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.418 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.418 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.418 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T03:40:28.418 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.418 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.418 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.418 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-09-12T03:40:28.418 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.418 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.419 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.419 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.419 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_haproxy": { 2024-09-12T03:40:28.419 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_haproxy", 2024-09-12T03:40:28.419 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.419 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.419 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.419 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T03:40:28.419 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.419 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.419 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.419 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "HAproxy container image", 2024-09-12T03:40:28.420 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.420 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.420 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.420 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.420 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_agent": { 2024-09-12T03:40:28.420 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_agent", 2024-09-12T03:40:28.420 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.420 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.420 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.420 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T03:40:28.420 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.420 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.420 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.421 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger agent container image", 2024-09-12T03:40:28.421 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.421 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.421 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.421 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.421 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_collector": { 2024-09-12T03:40:28.421 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_collector", 2024-09-12T03:40:28.421 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.421 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.421 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.421 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T03:40:28.421 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.421 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.422 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.422 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger collector container image", 2024-09-12T03:40:28.422 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.422 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.422 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.422 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.422 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_query": { 2024-09-12T03:40:28.422 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_query", 2024-09-12T03:40:28.422 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.422 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.422 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.422 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T03:40:28.422 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.423 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.423 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.423 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger query container image", 2024-09-12T03:40:28.423 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.423 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.423 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.423 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.423 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_keepalived": { 2024-09-12T03:40:28.423 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_keepalived", 2024-09-12T03:40:28.423 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.423 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.423 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.423 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T03:40:28.423 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.424 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.424 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.424 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Keepalived container image", 2024-09-12T03:40:28.424 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.424 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.424 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.424 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.424 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_loki": { 2024-09-12T03:40:28.424 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_loki", 2024-09-12T03:40:28.424 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.424 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.424 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.424 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T03:40:28.425 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.425 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.425 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.425 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Loki container image", 2024-09-12T03:40:28.425 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.425 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.425 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.425 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.425 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_node_exporter": { 2024-09-12T03:40:28.425 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_node_exporter", 2024-09-12T03:40:28.425 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.425 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.425 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.426 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T03:40:28.426 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.426 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.426 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.426 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-09-12T03:40:28.426 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.426 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.426 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.426 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.426 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_nvmeof": { 2024-09-12T03:40:28.426 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_nvmeof", 2024-09-12T03:40:28.426 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.426 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.427 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.427 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T03:40:28.427 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.427 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.427 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.427 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Nvme-of container image", 2024-09-12T03:40:28.427 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.427 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.427 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.427 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.427 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_prometheus": { 2024-09-12T03:40:28.427 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_prometheus", 2024-09-12T03:40:28.427 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.428 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.428 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.428 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T03:40:28.428 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.428 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.428 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.428 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-09-12T03:40:28.428 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.428 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.428 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.428 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.429 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_promtail": { 2024-09-12T03:40:28.429 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_promtail", 2024-09-12T03:40:28.429 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.429 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.429 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.429 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T03:40:28.429 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.429 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.429 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.429 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Promtail container image", 2024-09-12T03:40:28.429 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.429 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.429 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.429 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.430 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_samba": { 2024-09-12T03:40:28.430 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_samba", 2024-09-12T03:40:28.430 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.430 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.430 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.430 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T03:40:28.430 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.430 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.430 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.430 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Samba/SMB container image", 2024-09-12T03:40:28.430 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.430 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.430 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.430 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.431 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_snmp_gateway": { 2024-09-12T03:40:28.431 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_snmp_gateway", 2024-09-12T03:40:28.431 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.431 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.431 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.431 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T03:40:28.431 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.431 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.431 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.431 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SNMP Gateway container image", 2024-09-12T03:40:28.431 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.431 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.431 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.431 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.432 INFO:teuthology.orchestra.run.smithi064.stdout: "container_init": { 2024-09-12T03:40:28.432 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_init", 2024-09-12T03:40:28.432 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.432 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.432 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.432 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.432 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.432 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.432 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.432 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T03:40:28.432 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.432 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.433 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.433 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.433 INFO:teuthology.orchestra.run.smithi064.stdout: "daemon_cache_timeout": { 2024-09-12T03:40:28.433 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "daemon_cache_timeout", 2024-09-12T03:40:28.433 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:28.433 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.433 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.433 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-12T03:40:28.433 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.433 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.433 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.433 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T03:40:28.433 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.434 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.434 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.434 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.434 INFO:teuthology.orchestra.run.smithi064.stdout: "default_cephadm_command_timeout": { 2024-09-12T03:40:28.434 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T03:40:28.434 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.434 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.434 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.434 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "900", 2024-09-12T03:40:28.434 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.434 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.434 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.434 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T03:40:28.434 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.435 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.435 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.435 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.435 INFO:teuthology.orchestra.run.smithi064.stdout: "default_registry": { 2024-09-12T03:40:28.435 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "default_registry", 2024-09-12T03:40:28.435 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.435 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.435 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.435 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io", 2024-09-12T03:40:28.435 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.435 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.435 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.435 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T03:40:28.436 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.436 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.436 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.436 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.436 INFO:teuthology.orchestra.run.smithi064.stdout: "device_cache_timeout": { 2024-09-12T03:40:28.436 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_cache_timeout", 2024-09-12T03:40:28.436 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:28.436 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.436 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.436 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1800", 2024-09-12T03:40:28.436 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.436 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.436 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.436 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache device inventory", 2024-09-12T03:40:28.437 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.437 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.437 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.437 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.437 INFO:teuthology.orchestra.run.smithi064.stdout: "device_enhanced_scan": { 2024-09-12T03:40:28.437 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_enhanced_scan", 2024-09-12T03:40:28.437 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.437 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.437 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.437 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.437 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.437 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.437 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.438 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T03:40:28.438 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.438 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.438 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.438 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.438 INFO:teuthology.orchestra.run.smithi064.stdout: "facts_cache_timeout": { 2024-09-12T03:40:28.438 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "facts_cache_timeout", 2024-09-12T03:40:28.438 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:28.438 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.438 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.438 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-12T03:40:28.438 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.438 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.439 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.439 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache host facts data", 2024-09-12T03:40:28.439 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.439 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.439 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.439 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.439 INFO:teuthology.orchestra.run.smithi064.stdout: "grafana_dashboards_path": { 2024-09-12T03:40:28.439 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "grafana_dashboards_path", 2024-09-12T03:40:28.439 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.439 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.439 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.439 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T03:40:28.439 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.440 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.440 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.440 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T03:40:28.440 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.440 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.440 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.440 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.440 INFO:teuthology.orchestra.run.smithi064.stdout: "host_check_interval": { 2024-09-12T03:40:28.440 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "host_check_interval", 2024-09-12T03:40:28.440 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:28.440 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.440 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.440 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-12T03:40:28.440 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.441 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.441 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.441 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to perform a host check", 2024-09-12T03:40:28.441 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.441 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.441 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.441 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.441 INFO:teuthology.orchestra.run.smithi064.stdout: "hw_monitoring": { 2024-09-12T03:40:28.441 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "hw_monitoring", 2024-09-12T03:40:28.441 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.441 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.441 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.441 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.441 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.442 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.442 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.442 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T03:40:28.442 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.442 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.442 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.442 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.442 INFO:teuthology.orchestra.run.smithi064.stdout: "inventory_list_all": { 2024-09-12T03:40:28.442 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "inventory_list_all", 2024-09-12T03:40:28.442 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.442 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.442 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.442 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.442 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.443 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.443 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.443 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T03:40:28.443 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.443 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.443 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.443 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.443 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.443 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.443 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.443 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.443 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.443 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.443 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.444 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.444 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.444 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.444 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.444 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.444 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.444 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.444 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.444 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.444 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.444 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.444 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.444 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.444 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.445 INFO:teuthology.orchestra.run.smithi064.stdout: "log_refresh_metadata": { 2024-09-12T03:40:28.445 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_refresh_metadata", 2024-09-12T03:40:28.445 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.445 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.445 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.445 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.445 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.445 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.445 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.445 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T03:40:28.445 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.445 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.445 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.445 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.446 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.446 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.446 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.446 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.446 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.446 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.446 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.446 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.446 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.446 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T03:40:28.446 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.446 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.446 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.446 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.447 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.447 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.447 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.447 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.447 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.447 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.447 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.447 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.447 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.447 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.447 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.447 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.447 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.448 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.448 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.448 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.448 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.448 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.448 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.448 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.448 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.448 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.448 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.448 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.448 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.448 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.448 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.449 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.449 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.449 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.449 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.449 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.449 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.449 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.449 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.449 INFO:teuthology.orchestra.run.smithi064.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T03:40:28.449 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T03:40:28.449 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.449 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.449 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.450 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.450 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.450 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.450 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.450 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T03:40:28.450 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.450 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.450 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.450 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.450 INFO:teuthology.orchestra.run.smithi064.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T03:40:28.450 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T03:40:28.450 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.450 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.451 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.451 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "*", 2024-09-12T03:40:28.451 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.451 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.451 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.451 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T03:40:28.451 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.451 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.451 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.451 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.451 INFO:teuthology.orchestra.run.smithi064.stdout: "max_count_per_host": { 2024-09-12T03:40:28.451 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_count_per_host", 2024-09-12T03:40:28.451 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.452 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.452 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.452 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-12T03:40:28.452 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.452 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.452 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.452 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "max number of daemons per service per host", 2024-09-12T03:40:28.452 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.452 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.452 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.452 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.452 INFO:teuthology.orchestra.run.smithi064.stdout: "max_osd_draining_count": { 2024-09-12T03:40:28.452 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_osd_draining_count", 2024-09-12T03:40:28.453 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.453 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.453 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.453 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-12T03:40:28.453 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.453 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.453 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.453 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T03:40:28.453 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.453 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.453 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.453 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.453 INFO:teuthology.orchestra.run.smithi064.stdout: "migration_current": { 2024-09-12T03:40:28.453 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "migration_current", 2024-09-12T03:40:28.454 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.454 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.454 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.454 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.454 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.454 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.454 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.454 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "internal - do not modify", 2024-09-12T03:40:28.454 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.454 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.454 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.454 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.454 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": { 2024-09-12T03:40:28.455 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mode", 2024-09-12T03:40:28.455 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.455 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.455 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.455 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "root", 2024-09-12T03:40:28.455 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.455 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.455 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.455 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm-package", 2024-09-12T03:40:28.455 INFO:teuthology.orchestra.run.smithi064.stdout: "root" 2024-09-12T03:40:28.455 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.455 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T03:40:28.455 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.455 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.456 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.456 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.456 INFO:teuthology.orchestra.run.smithi064.stdout: "oob_default_addr": { 2024-09-12T03:40:28.456 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "oob_default_addr", 2024-09-12T03:40:28.456 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.456 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.456 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.456 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "169.254.1.1", 2024-09-12T03:40:28.456 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.456 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.456 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.456 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T03:40:28.457 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.457 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.457 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.457 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.457 INFO:teuthology.orchestra.run.smithi064.stdout: "prometheus_alerts_path": { 2024-09-12T03:40:28.457 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prometheus_alerts_path", 2024-09-12T03:40:28.457 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.457 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.457 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.457 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T03:40:28.457 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.457 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.457 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.458 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T03:40:28.458 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.458 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.458 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.458 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.458 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_insecure": { 2024-09-12T03:40:28.458 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_insecure", 2024-09-12T03:40:28.458 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.458 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.458 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.458 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.458 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.458 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.458 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.459 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T03:40:28.459 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.459 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.459 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.459 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.459 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_password": { 2024-09-12T03:40:28.459 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_password", 2024-09-12T03:40:28.459 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.459 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.459 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.459 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.459 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.459 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.459 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.460 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T03:40:28.460 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.460 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.460 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.460 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.460 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_url": { 2024-09-12T03:40:28.460 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_url", 2024-09-12T03:40:28.460 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.460 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.460 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.460 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.460 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.460 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.460 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.461 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T03:40:28.461 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.461 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.461 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.461 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.461 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_username": { 2024-09-12T03:40:28.461 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_username", 2024-09-12T03:40:28.461 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.461 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.461 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.461 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.461 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.461 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.461 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.462 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T03:40:28.462 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.462 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.462 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.462 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.462 INFO:teuthology.orchestra.run.smithi064.stdout: "secure_monitoring_stack": { 2024-09-12T03:40:28.462 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "secure_monitoring_stack", 2024-09-12T03:40:28.462 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.462 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.462 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.462 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.462 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.462 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.463 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.463 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T03:40:28.463 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.463 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.463 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.463 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.463 INFO:teuthology.orchestra.run.smithi064.stdout: "service_discovery_port": { 2024-09-12T03:40:28.463 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "service_discovery_port", 2024-09-12T03:40:28.463 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.463 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.463 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.463 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8765", 2024-09-12T03:40:28.463 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.463 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.464 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.464 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "cephadm service discovery port", 2024-09-12T03:40:28.464 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.464 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.464 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.464 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.464 INFO:teuthology.orchestra.run.smithi064.stdout: "ssh_config_file": { 2024-09-12T03:40:28.464 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssh_config_file", 2024-09-12T03:40:28.464 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.464 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.464 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.464 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.464 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.465 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.465 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.465 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T03:40:28.465 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.465 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.465 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.465 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.465 INFO:teuthology.orchestra.run.smithi064.stdout: "use_agent": { 2024-09-12T03:40:28.465 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "use_agent", 2024-09-12T03:40:28.465 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.465 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.465 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.465 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.465 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.466 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.466 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.466 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T03:40:28.466 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.466 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.466 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.466 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.466 INFO:teuthology.orchestra.run.smithi064.stdout: "use_repo_digest": { 2024-09-12T03:40:28.466 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "use_repo_digest", 2024-09-12T03:40:28.466 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.466 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.466 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.466 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.466 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.467 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.467 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.467 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T03:40:28.467 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.467 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.467 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.467 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.467 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_failed_host_check": { 2024-09-12T03:40:28.467 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_failed_host_check", 2024-09-12T03:40:28.467 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.467 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.467 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.467 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.467 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.468 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.468 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.468 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T03:40:28.468 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.468 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.468 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.468 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.468 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_stray_daemons": { 2024-09-12T03:40:28.468 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_stray_daemons", 2024-09-12T03:40:28.468 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.468 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.468 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.468 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.469 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.469 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.469 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.469 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T03:40:28.469 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.469 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.469 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.469 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.469 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_stray_hosts": { 2024-09-12T03:40:28.469 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_stray_hosts", 2024-09-12T03:40:28.469 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.469 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.469 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.469 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.470 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.470 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.470 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.470 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T03:40:28.470 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.470 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.470 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.470 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.470 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.470 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.470 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.470 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crash", 2024-09-12T03:40:28.470 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.470 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.471 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.471 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.471 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.471 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.471 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.471 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.471 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.471 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.471 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.471 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.471 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.471 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.471 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.471 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.472 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.472 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.472 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.472 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.472 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.472 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.472 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.472 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.472 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.472 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.472 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.472 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.472 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.472 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.473 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.473 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.473 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.473 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.473 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.473 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.473 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.473 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.473 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.473 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.473 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.473 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.473 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.473 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.474 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.474 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.474 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.474 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.474 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.474 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.474 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.474 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.474 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.474 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.474 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.474 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.474 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.474 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.475 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.475 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.475 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.475 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.475 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.475 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.475 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.475 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.475 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.475 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.475 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.475 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.475 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.476 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.476 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.476 INFO:teuthology.orchestra.run.smithi064.stdout: "retain_interval": { 2024-09-12T03:40:28.476 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "retain_interval", 2024-09-12T03:40:28.476 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:28.476 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.476 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.476 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "31536000", 2024-09-12T03:40:28.476 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.476 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.476 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.476 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T03:40:28.476 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.476 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.477 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.477 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.477 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_recent_interval": { 2024-09-12T03:40:28.477 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_recent_interval", 2024-09-12T03:40:28.477 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:28.477 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.477 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.477 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1209600", 2024-09-12T03:40:28.477 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.477 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.477 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.477 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T03:40:28.478 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.478 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.478 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.478 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.478 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.478 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.478 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.478 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "dashboard", 2024-09-12T03:40:28.478 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.478 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.478 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.478 INFO:teuthology.orchestra.run.smithi064.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T03:40:28.478 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T03:40:28.478 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.479 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.479 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.479 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-12T03:40:28.479 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.479 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.479 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.479 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.479 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.479 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.479 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.479 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.479 INFO:teuthology.orchestra.run.smithi064.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T03:40:28.479 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T03:40:28.480 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.480 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.480 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.480 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.480 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.480 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.480 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.480 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.480 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.480 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.480 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.481 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.481 INFO:teuthology.orchestra.run.smithi064.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T03:40:28.481 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T03:40:28.481 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.481 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.481 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.481 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.481 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.481 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.481 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.481 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.481 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.482 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.482 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.482 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.482 INFO:teuthology.orchestra.run.smithi064.stdout: "AUDIT_API_ENABLED": { 2024-09-12T03:40:28.482 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T03:40:28.482 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.482 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.482 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.482 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.482 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.482 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.482 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.482 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.483 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.483 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.483 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.483 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.483 INFO:teuthology.orchestra.run.smithi064.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T03:40:28.483 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T03:40:28.483 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.483 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.483 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.483 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.483 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.483 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.483 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.484 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.484 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.484 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.484 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.484 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.484 INFO:teuthology.orchestra.run.smithi064.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T03:40:28.484 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T03:40:28.484 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.484 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.484 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.484 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.484 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.484 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.485 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.485 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.485 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.485 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.485 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.485 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.485 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T03:40:28.485 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T03:40:28.485 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.485 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.485 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.485 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.485 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.485 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.486 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.486 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.486 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.486 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.486 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.486 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.486 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T03:40:28.486 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T03:40:28.486 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.486 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.486 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.486 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.486 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.487 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.487 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.487 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.487 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.487 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.487 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.487 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.487 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T03:40:28.487 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T03:40:28.487 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.487 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.487 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.487 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.487 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.488 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.488 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.488 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.488 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.488 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.488 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.488 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.488 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T03:40:28.488 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T03:40:28.488 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.488 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.488 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.488 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.489 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.489 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.489 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.489 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.489 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.489 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.489 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.489 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.489 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T03:40:28.489 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T03:40:28.489 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.489 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.489 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.490 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.490 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.490 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.490 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.490 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.490 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.490 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.490 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.490 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.490 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T03:40:28.490 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T03:40:28.490 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.490 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.490 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.491 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.491 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.491 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.491 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.491 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.491 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.491 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.491 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.491 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.491 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T03:40:28.491 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T03:40:28.491 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.491 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.491 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.492 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.492 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.492 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.492 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.492 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.492 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.492 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.492 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.492 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.492 INFO:teuthology.orchestra.run.smithi064.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T03:40:28.492 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T03:40:28.492 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.492 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.493 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.493 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.493 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.493 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.493 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.493 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.493 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.493 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.493 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.493 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.493 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T03:40:28.493 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T03:40:28.493 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.493 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.494 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.494 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-09-12T03:40:28.494 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.494 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.494 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.494 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.494 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.494 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.494 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.494 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.494 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T03:40:28.494 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T03:40:28.494 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.495 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.495 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.495 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.495 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.495 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.495 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.495 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.495 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.495 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.495 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.495 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.495 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_URL": { 2024-09-12T03:40:28.495 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_URL", 2024-09-12T03:40:28.495 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.496 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.496 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.496 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.496 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.496 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.496 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.496 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.496 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.496 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.496 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.496 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.496 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T03:40:28.496 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T03:40:28.497 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.497 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.497 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.497 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-09-12T03:40:28.497 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.497 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.497 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.497 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.497 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.497 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.497 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.497 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.497 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T03:40:28.498 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T03:40:28.498 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.498 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.498 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.498 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.498 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.498 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.498 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.498 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.498 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.498 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.498 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.498 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.498 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T03:40:28.499 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T03:40:28.499 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.499 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.499 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.499 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.499 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.499 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.499 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.499 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.499 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.499 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.499 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.499 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.500 INFO:teuthology.orchestra.run.smithi064.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T03:40:28.500 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T03:40:28.500 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.500 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.500 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.500 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.500 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.500 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.500 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.500 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.500 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.500 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.501 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.501 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.501 INFO:teuthology.orchestra.run.smithi064.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T03:40:28.501 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T03:40:28.501 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.501 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.501 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.501 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.501 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.501 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.501 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.501 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.501 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.502 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.502 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.502 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.502 INFO:teuthology.orchestra.run.smithi064.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T03:40:28.502 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T03:40:28.502 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.502 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.502 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.502 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.502 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.502 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.502 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.502 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.502 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.503 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.503 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.503 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.503 INFO:teuthology.orchestra.run.smithi064.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T03:40:28.503 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T03:40:28.503 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.503 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.503 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.503 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.503 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.503 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.503 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.503 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.504 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.504 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.504 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.504 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.504 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T03:40:28.504 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T03:40:28.504 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.504 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.504 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.504 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.504 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.504 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.504 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.505 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.505 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.505 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.505 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.505 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.505 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T03:40:28.505 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T03:40:28.505 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.505 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.505 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.505 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.505 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.505 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.506 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.506 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.506 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.506 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.506 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.506 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.506 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T03:40:28.506 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T03:40:28.506 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.506 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.506 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.506 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.506 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.506 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.507 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.507 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.507 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.507 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.507 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.507 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.507 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T03:40:28.507 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T03:40:28.507 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.507 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.507 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.507 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.507 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.507 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.508 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.508 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.508 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.508 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.508 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.508 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.508 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T03:40:28.508 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T03:40:28.508 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.508 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.508 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.508 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.508 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.509 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.509 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.509 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.509 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.509 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.509 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.509 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.509 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T03:40:28.509 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T03:40:28.509 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.509 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.509 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.509 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.509 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.510 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.510 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.510 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.510 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.510 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.510 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.510 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.510 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T03:40:28.510 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T03:40:28.510 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.510 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.510 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.510 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.510 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.511 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.511 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.511 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.511 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.511 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.511 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.511 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.511 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T03:40:28.511 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T03:40:28.511 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.511 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.511 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.511 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.512 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.512 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.512 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.512 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.512 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.512 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.512 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.512 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.512 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T03:40:28.512 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T03:40:28.512 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.512 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.512 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.513 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T03:40:28.513 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.513 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.513 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.513 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.513 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.513 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.513 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.513 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.513 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T03:40:28.513 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T03:40:28.513 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.513 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.514 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.514 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-12T03:40:28.514 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.514 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.514 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.514 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.514 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.514 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.514 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.514 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.514 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T03:40:28.514 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T03:40:28.514 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.514 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.515 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.515 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8", 2024-09-12T03:40:28.515 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.515 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.515 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.515 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.515 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.515 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.515 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.515 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.515 INFO:teuthology.orchestra.run.smithi064.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T03:40:28.515 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T03:40:28.515 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.515 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.516 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.516 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "45", 2024-09-12T03:40:28.516 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.516 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.516 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.516 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.516 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.516 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.516 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.516 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.516 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T03:40:28.516 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T03:40:28.516 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.517 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.517 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.517 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.517 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.517 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.517 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.517 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.517 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.517 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.517 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.517 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.517 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T03:40:28.518 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T03:40:28.518 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.518 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.518 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.518 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-09-12T03:40:28.518 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.518 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.518 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.518 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.518 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.518 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.518 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.518 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.518 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T03:40:28.519 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T03:40:28.519 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.519 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.519 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.519 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.519 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.519 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.519 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.519 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.519 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.519 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.519 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.519 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.519 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T03:40:28.520 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T03:40:28.520 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.520 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.520 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.520 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.520 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.520 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.520 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.520 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.520 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.520 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.520 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.520 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.521 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T03:40:28.521 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T03:40:28.521 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.521 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.521 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.521 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-12T03:40:28.521 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.521 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.521 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.521 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.521 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.521 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.521 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.522 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.522 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T03:40:28.522 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T03:40:28.522 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.522 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.522 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.522 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-12T03:40:28.522 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.522 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.522 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.522 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.522 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.522 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.523 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.523 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.523 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T03:40:28.523 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T03:40:28.523 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.523 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.523 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.523 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-12T03:40:28.523 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.523 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.523 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.523 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.523 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.524 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.524 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.524 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.524 INFO:teuthology.orchestra.run.smithi064.stdout: "cross_origin_url": { 2024-09-12T03:40:28.524 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cross_origin_url", 2024-09-12T03:40:28.524 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.524 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.524 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.524 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.524 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.524 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.524 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.524 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.525 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.525 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.525 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.525 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.525 INFO:teuthology.orchestra.run.smithi064.stdout: "crt_file": { 2024-09-12T03:40:28.525 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crt_file", 2024-09-12T03:40:28.525 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.525 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.525 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.525 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.525 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.525 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.525 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.525 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.526 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.526 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.526 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.526 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.526 INFO:teuthology.orchestra.run.smithi064.stdout: "debug": { 2024-09-12T03:40:28.526 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "debug", 2024-09-12T03:40:28.526 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.526 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.526 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.526 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.526 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.526 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.526 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.527 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable/disable debug options", 2024-09-12T03:40:28.527 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.527 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.527 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.527 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.527 INFO:teuthology.orchestra.run.smithi064.stdout: "jwt_token_ttl": { 2024-09-12T03:40:28.527 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "jwt_token_ttl", 2024-09-12T03:40:28.527 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.527 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.527 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.527 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "28800", 2024-09-12T03:40:28.527 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.527 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.527 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.528 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.528 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.528 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.528 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.528 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.528 INFO:teuthology.orchestra.run.smithi064.stdout: "key_file": { 2024-09-12T03:40:28.528 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "key_file", 2024-09-12T03:40:28.528 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.528 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.528 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.528 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.528 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.528 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.529 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.529 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.529 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.529 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.529 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.529 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.529 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.529 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.529 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.529 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.529 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.529 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.529 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.529 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.530 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.530 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.530 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.530 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.530 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.530 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.530 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.530 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.530 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.530 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.530 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.530 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.530 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.531 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.531 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.531 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.531 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.531 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.531 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.531 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.531 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.531 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.531 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.531 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.531 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.531 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.532 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.532 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.532 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.532 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.532 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.532 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.532 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.532 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.532 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.532 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.532 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.532 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.532 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.533 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.533 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.533 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.533 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.533 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.533 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.533 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.533 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.533 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.533 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.533 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.533 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.533 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.533 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.534 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.534 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.534 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.534 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.534 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.534 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.534 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.534 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.534 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.534 INFO:teuthology.orchestra.run.smithi064.stdout: "motd": { 2024-09-12T03:40:28.534 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "motd", 2024-09-12T03:40:28.534 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.534 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.535 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.535 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.535 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.535 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.535 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.535 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "The message of the day", 2024-09-12T03:40:28.535 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.535 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.535 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.535 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.535 INFO:teuthology.orchestra.run.smithi064.stdout: "redirect_resolve_ip_addr": { 2024-09-12T03:40:28.535 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T03:40:28.535 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.536 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.536 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.536 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.536 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.536 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.536 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.536 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.536 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.536 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.536 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.536 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.536 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-09-12T03:40:28.536 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-09-12T03:40:28.537 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.537 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.537 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.537 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "::", 2024-09-12T03:40:28.537 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.537 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.537 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.537 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.537 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.537 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.537 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.537 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.538 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-09-12T03:40:28.538 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-09-12T03:40:28.538 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.538 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.538 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.538 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8080", 2024-09-12T03:40:28.538 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.538 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.538 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.538 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.538 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.538 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.538 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.539 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.539 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl": { 2024-09-12T03:40:28.539 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl", 2024-09-12T03:40:28.539 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.539 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.539 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.539 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.539 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.539 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.539 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.539 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.540 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.540 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.540 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.540 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.540 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl_server_port": { 2024-09-12T03:40:28.540 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl_server_port", 2024-09-12T03:40:28.540 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.540 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.540 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.540 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8443", 2024-09-12T03:40:28.540 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.541 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.541 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.541 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.541 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.541 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.541 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.541 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.541 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_behaviour": { 2024-09-12T03:40:28.541 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_behaviour", 2024-09-12T03:40:28.541 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.541 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.541 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.541 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "redirect", 2024-09-12T03:40:28.542 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.542 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.542 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.542 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.542 INFO:teuthology.orchestra.run.smithi064.stdout: "redirect" 2024-09-12T03:40:28.542 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.542 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.542 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.542 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.542 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.542 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.542 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_error_status_code": { 2024-09-12T03:40:28.542 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_error_status_code", 2024-09-12T03:40:28.543 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.543 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.543 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.543 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2024-09-12T03:40:28.543 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "400", 2024-09-12T03:40:28.543 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "599", 2024-09-12T03:40:28.543 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.543 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.543 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.543 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.543 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.543 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.543 INFO:teuthology.orchestra.run.smithi064.stdout: "url_prefix": { 2024-09-12T03:40:28.544 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "url_prefix", 2024-09-12T03:40:28.544 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.544 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.544 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.544 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.544 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.544 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.544 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.544 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.544 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.544 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.544 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.545 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.545 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.545 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.545 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.545 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "devicehealth", 2024-09-12T03:40:28.545 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.545 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.545 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.545 INFO:teuthology.orchestra.run.smithi064.stdout: "enable_monitoring": { 2024-09-12T03:40:28.548 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enable_monitoring", 2024-09-12T03:40:28.548 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.548 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.548 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.548 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.548 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.548 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.548 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.549 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "monitor device health metrics", 2024-09-12T03:40:28.549 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.549 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.549 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.549 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.549 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.549 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.549 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.549 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.549 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.549 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.549 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.549 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.550 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.550 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.550 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.550 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.550 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.550 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.550 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.550 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.550 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.550 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.550 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.550 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.550 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.550 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.551 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.551 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.551 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.551 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.551 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.551 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.551 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.551 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.551 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.551 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.551 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.551 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.551 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.552 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.552 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.552 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.552 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.552 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.552 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.552 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.552 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.552 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.552 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.553 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.553 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.553 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.553 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.553 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.553 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.553 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.553 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.553 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.553 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.553 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.553 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.554 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.554 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.554 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.554 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.554 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.554 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.554 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.554 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.554 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.554 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.554 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.554 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.554 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.554 INFO:teuthology.orchestra.run.smithi064.stdout: "mark_out_threshold": { 2024-09-12T03:40:28.555 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mark_out_threshold", 2024-09-12T03:40:28.555 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:28.555 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.555 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.555 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "2419200", 2024-09-12T03:40:28.555 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.555 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.555 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.555 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T03:40:28.555 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.555 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.555 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.555 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.556 INFO:teuthology.orchestra.run.smithi064.stdout: "pool_name": { 2024-09-12T03:40:28.556 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pool_name", 2024-09-12T03:40:28.556 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.556 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.556 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.556 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "device_health_metrics", 2024-09-12T03:40:28.556 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.556 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.556 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.556 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T03:40:28.556 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.556 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.556 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.557 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.557 INFO:teuthology.orchestra.run.smithi064.stdout: "retention_period": { 2024-09-12T03:40:28.557 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "retention_period", 2024-09-12T03:40:28.557 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:28.557 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.557 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.557 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15552000", 2024-09-12T03:40:28.557 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.557 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.557 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.557 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long to retain device health metrics", 2024-09-12T03:40:28.557 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.557 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.558 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.558 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.558 INFO:teuthology.orchestra.run.smithi064.stdout: "scrape_frequency": { 2024-09-12T03:40:28.558 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "scrape_frequency", 2024-09-12T03:40:28.558 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:28.558 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.558 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.558 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "86400", 2024-09-12T03:40:28.558 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.558 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.558 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.558 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T03:40:28.558 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.558 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.559 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.559 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.559 INFO:teuthology.orchestra.run.smithi064.stdout: "self_heal": { 2024-09-12T03:40:28.559 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "self_heal", 2024-09-12T03:40:28.559 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.559 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.559 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.559 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.559 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.559 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.559 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.559 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T03:40:28.559 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.560 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.560 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.560 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.560 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-12T03:40:28.560 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-12T03:40:28.560 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:28.560 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.560 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.560 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-12T03:40:28.560 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.560 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.560 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.560 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T03:40:28.561 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.561 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.561 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.561 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.561 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_threshold": { 2024-09-12T03:40:28.561 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_threshold", 2024-09-12T03:40:28.561 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:28.561 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.561 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.561 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "7257600", 2024-09-12T03:40:28.561 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.561 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.561 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.562 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T03:40:28.562 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.562 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.562 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.562 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.562 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.562 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.562 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.562 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "diskprediction_local", 2024-09-12T03:40:28.562 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.562 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.562 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.562 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.563 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.563 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.563 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.563 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.563 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.563 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.563 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.563 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.563 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.563 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.563 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.563 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.563 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.564 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.564 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.564 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.564 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.564 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.564 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.564 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.564 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.564 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.564 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.564 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.564 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.564 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.565 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.565 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.565 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.565 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.565 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.565 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.565 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.565 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.565 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.565 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.565 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.565 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.565 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.565 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.566 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.566 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.566 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.566 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.566 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.566 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.566 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.566 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.566 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.566 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.566 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.566 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.566 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.566 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.567 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.567 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.567 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.567 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.567 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.567 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.567 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.567 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.567 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.567 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.567 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.567 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.567 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.567 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.568 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.568 INFO:teuthology.orchestra.run.smithi064.stdout: "predict_interval": { 2024-09-12T03:40:28.568 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "predict_interval", 2024-09-12T03:40:28.568 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.568 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.568 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.568 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "86400", 2024-09-12T03:40:28.568 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.568 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.569 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.569 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.569 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.569 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.569 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.569 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.569 INFO:teuthology.orchestra.run.smithi064.stdout: "predictor_model": { 2024-09-12T03:40:28.569 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "predictor_model", 2024-09-12T03:40:28.569 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.569 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.569 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.569 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "prophetstor", 2024-09-12T03:40:28.569 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.569 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.570 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.570 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.570 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.570 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.570 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.570 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.570 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-12T03:40:28.570 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-12T03:40:28.570 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.570 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.570 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.570 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-12T03:40:28.570 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.571 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.571 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.571 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.571 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.571 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.571 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.571 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.571 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.571 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.571 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.571 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "influx", 2024-09-12T03:40:28.571 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": false, 2024-09-12T03:40:28.571 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "influxdb python module not found", 2024-09-12T03:40:28.571 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.572 INFO:teuthology.orchestra.run.smithi064.stdout: "batch_size": { 2024-09-12T03:40:28.572 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "batch_size", 2024-09-12T03:40:28.572 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.572 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.572 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.572 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5000", 2024-09-12T03:40:28.572 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.572 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.572 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.572 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T03:40:28.572 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.572 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.572 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.573 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.573 INFO:teuthology.orchestra.run.smithi064.stdout: "database": { 2024-09-12T03:40:28.573 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "database", 2024-09-12T03:40:28.573 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.573 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.573 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.573 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "ceph", 2024-09-12T03:40:28.573 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.573 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.573 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.573 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T03:40:28.573 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.573 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.574 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.574 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.574 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": { 2024-09-12T03:40:28.574 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "hostname", 2024-09-12T03:40:28.574 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.574 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.574 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.574 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.574 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.574 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.574 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.574 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB server hostname", 2024-09-12T03:40:28.574 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.575 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.575 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.575 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.575 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-12T03:40:28.575 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-12T03:40:28.575 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:28.575 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.575 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.575 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "30", 2024-09-12T03:40:28.575 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "5", 2024-09-12T03:40:28.575 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.575 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.575 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T03:40:28.576 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.576 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.576 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.576 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.576 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.576 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.576 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.576 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.576 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.576 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.576 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.576 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.576 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.576 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.577 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.577 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.577 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.577 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.577 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.577 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.577 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.577 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.577 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.577 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.577 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.577 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.577 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.578 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.578 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.578 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.578 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.578 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.578 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.578 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.578 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.578 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.578 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.578 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.578 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.578 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.578 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.579 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.579 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.579 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.579 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.579 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.579 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.579 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.579 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.579 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.579 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.579 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.579 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.579 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.579 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.580 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.580 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.580 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.580 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.580 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.580 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.580 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.580 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.580 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.580 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.580 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.580 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.580 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.580 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.581 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.581 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.581 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.581 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.581 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.581 INFO:teuthology.orchestra.run.smithi064.stdout: "password": { 2024-09-12T03:40:28.581 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "password", 2024-09-12T03:40:28.581 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.581 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.581 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.581 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.581 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.581 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.581 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.582 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "password of InfluxDB server user", 2024-09-12T03:40:28.582 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.582 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.582 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.582 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.582 INFO:teuthology.orchestra.run.smithi064.stdout: "port": { 2024-09-12T03:40:28.582 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "port", 2024-09-12T03:40:28.582 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.582 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.582 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.582 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8086", 2024-09-12T03:40:28.582 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.582 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.582 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.583 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB server port", 2024-09-12T03:40:28.583 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.583 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.583 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.583 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.583 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl": { 2024-09-12T03:40:28.583 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl", 2024-09-12T03:40:28.583 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.583 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.583 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.583 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "false", 2024-09-12T03:40:28.583 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.583 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.583 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.584 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:40:28.584 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.584 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.584 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.584 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.584 INFO:teuthology.orchestra.run.smithi064.stdout: "threads": { 2024-09-12T03:40:28.584 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "threads", 2024-09-12T03:40:28.584 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.584 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.584 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.584 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-12T03:40:28.584 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-09-12T03:40:28.584 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "32", 2024-09-12T03:40:28.584 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.585 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T03:40:28.585 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.585 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.585 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.585 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.585 INFO:teuthology.orchestra.run.smithi064.stdout: "username": { 2024-09-12T03:40:28.585 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "username", 2024-09-12T03:40:28.585 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.585 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.585 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.585 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.585 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.586 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.586 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.586 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "username of InfluxDB server user", 2024-09-12T03:40:28.586 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.586 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.586 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.586 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.586 INFO:teuthology.orchestra.run.smithi064.stdout: "verify_ssl": { 2024-09-12T03:40:28.586 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "verify_ssl", 2024-09-12T03:40:28.586 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.586 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.586 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.586 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "true", 2024-09-12T03:40:28.587 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.587 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.587 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.587 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:40:28.587 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.587 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.587 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.587 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.587 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.587 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.587 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.587 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "insights", 2024-09-12T03:40:28.587 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.587 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.588 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.588 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.588 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.588 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.588 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.588 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.588 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.588 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.588 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.588 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.588 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.588 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.588 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.589 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.589 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.589 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.589 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.589 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.589 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.589 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.589 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.589 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.589 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.589 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.589 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.589 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.589 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.590 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.590 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.590 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.590 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.590 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.590 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.590 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.590 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.590 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.590 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.590 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.590 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.590 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.591 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.591 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.591 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.591 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.591 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.591 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.591 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.591 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.591 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.591 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.591 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.591 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.591 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.592 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.592 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.592 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.592 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.592 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.592 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.592 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.592 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.592 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.592 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.592 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.592 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.592 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.593 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.593 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.593 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.593 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.593 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.593 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.593 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.593 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.593 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "iostat", 2024-09-12T03:40:28.593 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.593 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.593 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.593 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.593 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.594 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.594 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.594 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.594 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.594 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.594 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.594 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.594 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.594 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.594 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.594 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.594 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.594 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.595 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.595 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.595 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.595 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.595 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.595 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.595 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.595 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.595 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.595 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.595 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.595 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.595 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.596 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.596 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.596 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.596 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.596 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.596 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.596 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.596 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.596 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.596 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.596 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.596 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.596 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.597 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.597 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.597 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.597 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.597 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.597 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.597 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.597 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.597 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.597 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.597 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.597 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.597 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.597 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.598 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.598 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.598 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.598 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.598 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.598 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.598 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.598 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.598 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.598 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.598 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.598 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.598 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.598 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.599 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.599 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.599 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.599 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.599 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "k8sevents", 2024-09-12T03:40:28.599 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.599 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.599 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.599 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_event_retention_days": { 2024-09-12T03:40:28.599 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ceph_event_retention_days", 2024-09-12T03:40:28.599 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.599 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.599 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.599 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "7", 2024-09-12T03:40:28.600 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.600 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.600 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.600 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T03:40:28.600 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.600 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.600 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.600 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.600 INFO:teuthology.orchestra.run.smithi064.stdout: "config_check_secs": { 2024-09-12T03:40:28.600 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_check_secs", 2024-09-12T03:40:28.600 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.600 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.600 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.600 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-12T03:40:28.601 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "10", 2024-09-12T03:40:28.601 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.601 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.601 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T03:40:28.601 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.601 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.601 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.601 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.601 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.601 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.601 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.601 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.601 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.602 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.602 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.602 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.602 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.602 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.602 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.602 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.602 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.602 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.602 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.602 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.602 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.602 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.602 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.602 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.603 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.603 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.603 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.603 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.603 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.603 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.603 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.603 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.603 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.603 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.603 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.603 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.603 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.604 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.604 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.604 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.604 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.604 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.604 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.604 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.604 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.604 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.604 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.604 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.604 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.604 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.605 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.605 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.605 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.605 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.605 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.605 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.605 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.605 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.605 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.605 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.605 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.605 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.605 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.606 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.606 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.606 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.606 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.606 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.606 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.606 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.606 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.606 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.606 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.606 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.606 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.606 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.606 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.607 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "localpool", 2024-09-12T03:40:28.607 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.607 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.607 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.607 INFO:teuthology.orchestra.run.smithi064.stdout: "failure_domain": { 2024-09-12T03:40:28.607 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "failure_domain", 2024-09-12T03:40:28.607 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.607 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.607 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.607 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "host", 2024-09-12T03:40:28.607 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.607 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.607 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.607 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "failure domain for any created local pool", 2024-09-12T03:40:28.608 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T03:40:28.608 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.608 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.608 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.608 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.608 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.608 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.608 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.608 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.608 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.608 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.608 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.608 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.609 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.609 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.609 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.609 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.609 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.609 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.609 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.609 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.609 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.609 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.609 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.609 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.609 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.609 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.610 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.610 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.610 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.610 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.610 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.610 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.610 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.610 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.610 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.610 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.610 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.610 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.610 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.611 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.611 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.611 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.611 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.611 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.611 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.611 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.611 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.611 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.611 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.611 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.611 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.611 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.612 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.612 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.612 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.612 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.612 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.612 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.612 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.612 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.612 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.612 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.612 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.612 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.612 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.612 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.613 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.613 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.613 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.613 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.613 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.613 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.613 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.613 INFO:teuthology.orchestra.run.smithi064.stdout: "min_size": { 2024-09-12T03:40:28.613 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "min_size", 2024-09-12T03:40:28.613 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.613 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.613 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.613 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.614 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.614 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.614 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.614 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default min_size for any created local pool", 2024-09-12T03:40:28.614 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T03:40:28.614 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.614 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.614 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.614 INFO:teuthology.orchestra.run.smithi064.stdout: "num_rep": { 2024-09-12T03:40:28.614 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "num_rep", 2024-09-12T03:40:28.614 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.614 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.614 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.614 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3", 2024-09-12T03:40:28.615 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.615 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.615 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.615 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default replica count for any created local pool", 2024-09-12T03:40:28.615 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.615 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.615 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.615 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.615 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_num": { 2024-09-12T03:40:28.615 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pg_num", 2024-09-12T03:40:28.615 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.615 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.615 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.615 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "128", 2024-09-12T03:40:28.616 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.616 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.616 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.616 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T03:40:28.616 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.616 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.616 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.616 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.616 INFO:teuthology.orchestra.run.smithi064.stdout: "prefix": { 2024-09-12T03:40:28.616 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prefix", 2024-09-12T03:40:28.616 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.616 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.616 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.617 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.617 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.617 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.617 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.617 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "name prefix for any created local pool", 2024-09-12T03:40:28.617 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.617 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.617 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.617 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.617 INFO:teuthology.orchestra.run.smithi064.stdout: "subtree": { 2024-09-12T03:40:28.617 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "subtree", 2024-09-12T03:40:28.617 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.617 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.617 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.618 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "rack", 2024-09-12T03:40:28.618 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.618 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.618 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.618 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T03:40:28.618 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T03:40:28.618 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.618 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.618 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.618 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.618 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.618 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.618 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mds_autoscaler", 2024-09-12T03:40:28.619 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.619 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.619 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.619 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.619 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.619 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.619 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.619 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.619 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.619 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.619 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.619 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.619 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.620 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.620 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.620 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.620 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.620 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.620 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.620 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.620 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.620 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.620 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.620 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.620 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.620 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.621 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.621 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.621 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.621 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.621 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.621 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.621 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.621 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.621 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.621 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.621 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.621 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.621 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.621 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.622 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.622 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.622 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.622 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.622 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.622 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.622 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.622 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.622 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.622 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.622 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.622 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.622 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.622 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.623 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.623 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.623 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.623 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.623 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.623 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.623 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.623 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.623 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.623 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.623 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.623 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.623 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.624 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.624 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.624 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.624 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.624 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.624 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.624 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.624 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.624 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.624 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mirroring", 2024-09-12T03:40:28.624 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.624 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.624 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.624 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.625 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.625 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.625 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.625 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.625 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.625 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.625 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.625 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.625 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.625 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.625 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.625 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.625 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.626 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.626 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.626 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.626 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.626 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.626 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.626 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.626 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.626 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.626 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.626 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.626 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.626 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.626 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.627 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.627 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.627 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.627 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.627 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.627 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.627 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.627 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.627 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.627 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.627 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.627 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.627 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.627 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.628 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.628 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.628 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.628 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.628 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.628 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.628 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.628 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.628 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.628 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.628 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.628 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.628 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.629 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.629 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.629 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.629 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.629 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.629 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.629 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.629 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.629 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.629 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.629 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.629 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.629 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.629 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.630 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.630 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.630 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.630 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.630 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "nfs", 2024-09-12T03:40:28.630 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.630 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.630 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.630 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.630 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.630 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.630 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.630 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.630 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.631 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.631 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.631 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.631 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.631 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.631 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.631 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.631 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.631 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.631 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.631 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.631 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.631 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.631 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.632 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.632 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.632 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.632 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.632 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.632 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.632 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.632 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.632 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.632 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.632 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.632 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.632 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.632 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.633 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.633 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.633 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.633 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.633 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.633 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.633 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.633 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.633 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.633 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.633 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.633 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.633 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.634 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.634 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.634 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.634 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.634 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.634 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.634 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.634 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.634 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.634 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.634 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.634 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.634 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.634 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.635 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.635 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.635 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.635 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.635 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.635 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.635 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.635 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.635 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.635 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.635 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.635 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.636 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "orchestrator", 2024-09-12T03:40:28.636 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.636 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.636 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.636 INFO:teuthology.orchestra.run.smithi064.stdout: "fail_fs": { 2024-09-12T03:40:28.636 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "fail_fs", 2024-09-12T03:40:28.636 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.636 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.636 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.636 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.636 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.636 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.636 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.637 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T03:40:28.637 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.637 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.637 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.637 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.637 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.637 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.637 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.637 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.637 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.637 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.637 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.637 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.638 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.638 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.638 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.638 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.638 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.638 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.638 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.638 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.638 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.638 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.638 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.638 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.638 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.638 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.639 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.639 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.639 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.639 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.639 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.639 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.639 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.639 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.639 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.639 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.639 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.639 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.639 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.640 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.640 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.640 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.640 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.640 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.640 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.640 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.640 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.640 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.640 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.640 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.640 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.640 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.640 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.641 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.641 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.641 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.641 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.641 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.641 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.641 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.641 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.641 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.641 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.641 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.641 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.641 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.641 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.642 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.642 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.642 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.642 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.642 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.642 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.642 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.642 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator": { 2024-09-12T03:40:28.642 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "orchestrator", 2024-09-12T03:40:28.642 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.642 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.642 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.642 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.642 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.643 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.643 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.643 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm", 2024-09-12T03:40:28.643 INFO:teuthology.orchestra.run.smithi064.stdout: "rook", 2024-09-12T03:40:28.643 INFO:teuthology.orchestra.run.smithi064.stdout: "test_orchestrator" 2024-09-12T03:40:28.643 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.643 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Orchestrator backend", 2024-09-12T03:40:28.643 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.643 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.643 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.643 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.643 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.643 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.644 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.644 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "osd_perf_query", 2024-09-12T03:40:28.644 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.644 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.644 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.644 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.644 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.644 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.644 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.644 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.644 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.644 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.644 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.644 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.645 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.645 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.645 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.645 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.645 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.645 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.645 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.645 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.645 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.645 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.645 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.645 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.645 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.646 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.646 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.646 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.646 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.646 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.646 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.646 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.646 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.646 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.646 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.646 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.646 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.647 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.647 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.647 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.647 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.647 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.647 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.647 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.647 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.647 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.647 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.647 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.647 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.647 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.647 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.648 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.648 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.648 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.648 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.648 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.648 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.648 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.648 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.648 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.648 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.648 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.648 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.648 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.649 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.649 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.649 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.649 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.649 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.649 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.649 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.649 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.649 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.649 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.649 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.649 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.650 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "osd_support", 2024-09-12T03:40:28.650 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.650 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.650 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.650 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.650 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.650 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.650 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.650 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.650 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.650 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.650 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.650 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.650 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.651 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.651 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.651 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.651 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.651 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.651 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.651 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.651 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.651 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.651 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.651 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.651 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.652 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.652 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.652 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.652 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.652 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.652 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.652 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.652 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.652 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.652 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.652 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.652 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.652 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.653 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.653 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.653 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.653 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.653 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.653 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.653 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.653 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.653 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.653 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.653 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.653 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.653 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.654 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.654 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.654 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.654 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.654 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.654 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.654 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.654 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.654 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.654 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.654 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.654 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.654 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.655 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.655 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.655 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.655 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.655 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.655 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.655 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.655 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.655 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.655 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.655 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.655 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.655 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pg_autoscaler", 2024-09-12T03:40:28.655 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.656 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.656 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.656 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.656 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.656 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.656 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.656 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.656 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.656 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.656 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.656 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.656 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.656 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.657 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.657 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.657 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.657 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.657 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.657 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.657 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.657 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.657 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.657 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.657 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.657 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.657 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.658 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.658 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.658 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.658 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.658 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.658 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.658 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.658 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.658 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.658 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.658 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.658 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.658 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.659 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.659 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.659 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.659 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.659 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.659 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.659 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.659 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.659 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.659 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.659 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.659 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.659 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.660 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.660 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.660 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.660 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.660 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.660 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.660 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.660 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.660 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.660 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.660 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.660 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.660 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.661 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.661 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.661 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.661 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.661 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.661 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.661 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.661 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-12T03:40:28.661 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-12T03:40:28.661 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:28.661 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.661 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.661 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-12T03:40:28.662 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.662 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.662 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.662 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.662 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.662 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.662 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.662 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.662 INFO:teuthology.orchestra.run.smithi064.stdout: "threshold": { 2024-09-12T03:40:28.662 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "threshold", 2024-09-12T03:40:28.662 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-12T03:40:28.662 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.662 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.663 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3.0", 2024-09-12T03:40:28.663 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1.0", 2024-09-12T03:40:28.663 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.663 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.663 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "scaling threshold", 2024-09-12T03:40:28.663 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T03:40:28.663 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.663 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.663 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.663 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.663 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.663 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.663 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "progress", 2024-09-12T03:40:28.664 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.664 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.664 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.664 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_pg_recovery_event": { 2024-09-12T03:40:28.664 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_pg_recovery_event", 2024-09-12T03:40:28.664 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.664 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.664 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.664 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.664 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.664 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.664 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.665 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T03:40:28.665 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.665 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.665 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.665 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.665 INFO:teuthology.orchestra.run.smithi064.stdout: "enabled": { 2024-09-12T03:40:28.665 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enabled", 2024-09-12T03:40:28.665 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.665 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.665 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.665 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.665 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.665 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.665 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.666 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.666 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.666 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.666 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.666 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.666 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.666 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.666 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.666 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.666 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.666 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.666 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.666 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.667 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.667 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.667 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.667 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.667 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.667 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.667 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.667 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.667 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.667 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.667 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.667 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.667 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.668 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.668 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.668 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.668 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.668 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.668 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.668 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.668 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.668 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.668 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.668 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.668 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.668 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.668 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.669 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.669 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.669 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.669 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.669 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.669 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.669 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.669 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.669 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.669 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.669 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.669 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.669 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.670 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.670 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.670 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.670 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.670 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.670 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.670 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.670 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.670 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.670 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.670 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.670 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.670 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.671 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.671 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.671 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.671 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.671 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.671 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.671 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.671 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.671 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.671 INFO:teuthology.orchestra.run.smithi064.stdout: "max_completed_events": { 2024-09-12T03:40:28.671 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_completed_events", 2024-09-12T03:40:28.671 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.671 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.671 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.672 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "50", 2024-09-12T03:40:28.672 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.672 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.672 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.672 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "number of past completed events to remember", 2024-09-12T03:40:28.672 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.672 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.672 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.672 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.672 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-12T03:40:28.672 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-12T03:40:28.672 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:28.672 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.672 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.673 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-12T03:40:28.673 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.673 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.673 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.673 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long the module is going to sleep", 2024-09-12T03:40:28.673 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.673 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.673 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.673 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.673 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.673 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.673 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.673 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prometheus", 2024-09-12T03:40:28.673 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.674 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.674 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.674 INFO:teuthology.orchestra.run.smithi064.stdout: "cache": { 2024-09-12T03:40:28.674 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cache", 2024-09-12T03:40:28.674 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.674 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.674 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.674 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.674 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.674 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.674 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.674 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.674 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.674 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.675 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.675 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.675 INFO:teuthology.orchestra.run.smithi064.stdout: "exclude_perf_counters": { 2024-09-12T03:40:28.675 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "exclude_perf_counters", 2024-09-12T03:40:28.675 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.675 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.675 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.675 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.675 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.675 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.675 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.675 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T03:40:28.675 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T03:40:28.675 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.676 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.676 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.676 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.676 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.676 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.676 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.676 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.676 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.676 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.676 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.676 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.676 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.676 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.677 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.677 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.677 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.677 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.677 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.677 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.677 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.677 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.677 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.677 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.677 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.677 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.677 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.677 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.678 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.678 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.678 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.678 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.678 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.678 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.678 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.678 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.678 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.678 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.678 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.678 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.678 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.678 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.679 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.679 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.679 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.679 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.679 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.679 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.679 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.679 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.679 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.679 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.679 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.679 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.679 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.680 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.680 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.680 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.680 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.680 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.680 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.680 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.680 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.680 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.680 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.680 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.680 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.680 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.680 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.681 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.681 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.681 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.681 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.681 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_stats_pools": { 2024-09-12T03:40:28.681 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_stats_pools", 2024-09-12T03:40:28.681 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.681 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.681 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.681 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.681 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.681 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.682 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.682 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.682 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.682 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.682 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.682 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.682 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T03:40:28.682 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T03:40:28.682 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.682 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.682 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.682 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "300", 2024-09-12T03:40:28.682 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.683 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.683 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.683 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.683 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.683 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.683 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.683 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.683 INFO:teuthology.orchestra.run.smithi064.stdout: "scrape_interval": { 2024-09-12T03:40:28.683 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "scrape_interval", 2024-09-12T03:40:28.683 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-12T03:40:28.683 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.683 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.683 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15.0", 2024-09-12T03:40:28.683 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.684 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.684 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.684 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.684 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.684 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.684 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.684 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.684 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-09-12T03:40:28.684 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-09-12T03:40:28.684 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.684 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.684 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.684 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "::", 2024-09-12T03:40:28.684 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.685 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.685 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.685 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T03:40:28.685 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.685 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.685 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.685 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.685 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-09-12T03:40:28.685 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-09-12T03:40:28.685 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.685 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.685 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.685 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "9283", 2024-09-12T03:40:28.686 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.686 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.686 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.686 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T03:40:28.686 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.686 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.686 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.686 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.686 INFO:teuthology.orchestra.run.smithi064.stdout: "stale_cache_strategy": { 2024-09-12T03:40:28.686 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "stale_cache_strategy", 2024-09-12T03:40:28.686 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.686 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.686 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.686 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "log", 2024-09-12T03:40:28.687 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.687 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.687 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.687 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.687 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.687 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.687 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.687 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.687 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_behaviour": { 2024-09-12T03:40:28.687 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_behaviour", 2024-09-12T03:40:28.687 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.687 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.687 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.687 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "default", 2024-09-12T03:40:28.688 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.688 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.688 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.688 INFO:teuthology.orchestra.run.smithi064.stdout: "default", 2024-09-12T03:40:28.688 INFO:teuthology.orchestra.run.smithi064.stdout: "error" 2024-09-12T03:40:28.688 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.688 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.688 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.688 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.688 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.688 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.688 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_error_status_code": { 2024-09-12T03:40:28.688 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_error_status_code", 2024-09-12T03:40:28.689 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.689 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.689 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.689 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2024-09-12T03:40:28.689 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "400", 2024-09-12T03:40:28.689 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "599", 2024-09-12T03:40:28.689 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.689 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.689 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.689 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.689 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.689 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.689 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.690 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.690 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.690 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_support", 2024-09-12T03:40:28.690 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.690 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.690 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.690 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.690 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.690 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.690 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.690 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.690 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.690 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.690 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.691 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.691 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.691 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.691 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.691 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.691 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.691 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.691 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.691 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.691 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.691 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.691 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.691 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.692 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.692 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.692 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.692 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.692 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.692 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.692 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.692 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.692 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.692 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.692 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.692 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.692 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.693 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.693 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.693 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.693 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.693 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.693 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.693 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.693 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.693 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.693 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.693 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.693 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.694 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.694 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.694 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.694 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.694 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.694 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.694 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.694 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.694 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.694 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.694 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.694 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.694 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.695 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.695 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.695 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.695 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.695 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.695 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.695 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.695 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.695 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.695 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.695 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.695 INFO:teuthology.orchestra.run.smithi064.stdout: "max_concurrent_snap_create": { 2024-09-12T03:40:28.695 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_concurrent_snap_create", 2024-09-12T03:40:28.696 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.696 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.696 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.696 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-12T03:40:28.696 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.696 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.696 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.696 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.696 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.696 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.696 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.696 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.696 INFO:teuthology.orchestra.run.smithi064.stdout: "mirror_snapshot_schedule": { 2024-09-12T03:40:28.697 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T03:40:28.697 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.697 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.697 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.697 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.697 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.697 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.697 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.697 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.697 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.697 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.697 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.697 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.698 INFO:teuthology.orchestra.run.smithi064.stdout: "trash_purge_schedule": { 2024-09-12T03:40:28.698 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "trash_purge_schedule", 2024-09-12T03:40:28.698 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.698 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.698 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.698 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.698 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.698 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.698 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.698 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.698 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.698 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.698 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.698 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.699 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.699 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.699 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.699 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "restful", 2024-09-12T03:40:28.699 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.699 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.699 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.699 INFO:teuthology.orchestra.run.smithi064.stdout: "enable_auth": { 2024-09-12T03:40:28.699 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enable_auth", 2024-09-12T03:40:28.699 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.699 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.699 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.699 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.700 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.700 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.700 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.700 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.700 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.700 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.700 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.700 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.700 INFO:teuthology.orchestra.run.smithi064.stdout: "key_file": { 2024-09-12T03:40:28.700 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "key_file", 2024-09-12T03:40:28.700 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.700 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.701 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.701 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.701 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.701 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.701 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.701 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.701 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.701 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.701 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.701 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.701 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.701 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.702 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.702 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.702 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.702 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.702 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.702 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.702 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.702 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.702 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.702 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.702 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.702 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.702 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.703 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.703 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.703 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.703 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.703 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.703 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.703 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.703 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.703 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.703 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.703 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.703 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.703 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.704 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.704 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.704 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.704 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.704 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.704 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.704 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.704 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.704 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.704 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.704 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.704 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.705 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.705 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.705 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.705 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.705 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.705 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.705 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.705 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.705 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.705 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.705 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.705 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.705 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.706 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.706 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.706 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.706 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.706 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.706 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.706 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.706 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.706 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.706 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.706 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.706 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.706 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.707 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.707 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.707 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.707 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.707 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-09-12T03:40:28.707 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-09-12T03:40:28.707 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.707 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.707 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.707 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.707 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.707 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.707 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.708 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.708 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.708 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.708 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.708 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.708 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-09-12T03:40:28.708 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-09-12T03:40:28.708 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.708 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.708 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.708 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.708 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.708 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.709 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.709 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.709 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.709 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.709 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.709 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.709 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.709 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.709 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.709 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rgw", 2024-09-12T03:40:28.709 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.709 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.709 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.710 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.710 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.710 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.710 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.710 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.710 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.710 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.710 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.710 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.710 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.710 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.710 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.710 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.711 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.711 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.711 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.711 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.711 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.711 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.711 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.711 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.711 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.711 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.711 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.711 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.711 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.711 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.712 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.712 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.712 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.712 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.712 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.712 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.712 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.712 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.712 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.712 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.712 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.712 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.712 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.713 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.713 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.713 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.713 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.713 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.713 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.713 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.713 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.713 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.713 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.713 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.713 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.713 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.714 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.714 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.714 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.714 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.714 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.714 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.714 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.714 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.714 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.714 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.714 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.714 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.714 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.714 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.715 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.715 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.715 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.715 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.715 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.715 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.715 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rook", 2024-09-12T03:40:28.715 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.715 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.715 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.715 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.715 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.715 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.716 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.716 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.716 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.716 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.716 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.716 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.716 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.716 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.716 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.716 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.716 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.716 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.716 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.716 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.717 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.717 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.717 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.717 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.717 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.717 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.717 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.717 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.717 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.717 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.717 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.717 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.717 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.718 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.718 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.718 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.718 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.718 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.718 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.718 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.718 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.718 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.718 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.718 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.718 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.718 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.718 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.719 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.719 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.719 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.719 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.719 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.719 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.719 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.719 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.719 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.719 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.719 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.719 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.719 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.719 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.720 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.720 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.720 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.720 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.720 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.720 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.720 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.720 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.720 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.720 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.720 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.720 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.720 INFO:teuthology.orchestra.run.smithi064.stdout: "storage_class": { 2024-09-12T03:40:28.721 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "storage_class", 2024-09-12T03:40:28.721 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.721 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.721 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.721 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "local", 2024-09-12T03:40:28.721 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.721 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.721 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.721 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T03:40:28.721 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.721 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.721 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.721 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.722 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.722 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.722 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.722 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "selftest", 2024-09-12T03:40:28.722 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.722 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.722 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.722 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.722 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.722 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.722 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.722 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.722 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.723 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.723 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.723 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.723 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.723 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.723 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.723 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.723 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.723 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.723 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.723 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.723 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.723 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.724 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.724 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.724 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.724 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.724 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.724 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.724 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.724 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.724 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.724 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.724 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.724 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.725 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.725 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.725 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.725 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.725 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.725 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.725 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.725 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.725 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.725 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.725 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.725 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.725 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.726 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.726 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.726 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.726 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.726 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.726 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.726 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.726 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.726 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.726 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.726 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.726 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.726 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.727 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.727 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.727 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.727 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.727 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.727 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.727 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.727 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.727 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.727 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.727 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.727 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.727 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.727 INFO:teuthology.orchestra.run.smithi064.stdout: "roption1": { 2024-09-12T03:40:28.728 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "roption1", 2024-09-12T03:40:28.728 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.728 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.728 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.728 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.728 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.728 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.728 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.728 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.728 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.728 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.728 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.728 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.729 INFO:teuthology.orchestra.run.smithi064.stdout: "roption2": { 2024-09-12T03:40:28.729 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "roption2", 2024-09-12T03:40:28.729 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.729 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.729 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.729 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "xyz", 2024-09-12T03:40:28.729 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.729 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.729 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.729 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.729 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.729 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.729 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.729 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.730 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption1": { 2024-09-12T03:40:28.730 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption1", 2024-09-12T03:40:28.730 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.730 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.730 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.730 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.730 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.730 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.730 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.730 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.730 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.730 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.730 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.731 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption2": { 2024-09-12T03:40:28.731 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption2", 2024-09-12T03:40:28.731 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.731 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.731 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.731 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.731 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.731 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.731 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.731 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.731 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.731 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.732 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.732 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.732 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption3": { 2024-09-12T03:40:28.732 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption3", 2024-09-12T03:40:28.732 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-12T03:40:28.732 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.732 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.732 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.732 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.732 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.732 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.732 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.732 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.733 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.733 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.733 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.733 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption4": { 2024-09-12T03:40:28.733 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption4", 2024-09-12T03:40:28.733 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.733 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.733 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.733 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.733 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.733 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.733 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.733 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.734 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.734 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.734 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.734 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.734 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption5": { 2024-09-12T03:40:28.734 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption5", 2024-09-12T03:40:28.734 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.734 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.734 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.734 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.734 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.734 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.734 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.734 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.735 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.735 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.735 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.735 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.735 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption6": { 2024-09-12T03:40:28.735 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption6", 2024-09-12T03:40:28.735 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.735 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.735 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.735 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.735 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.735 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.736 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.736 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.736 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.736 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.736 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.736 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.736 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption7": { 2024-09-12T03:40:28.736 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption7", 2024-09-12T03:40:28.736 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.736 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.736 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.736 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.736 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-09-12T03:40:28.737 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "42", 2024-09-12T03:40:28.737 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.737 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.737 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.737 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.737 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.737 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.737 INFO:teuthology.orchestra.run.smithi064.stdout: "testkey": { 2024-09-12T03:40:28.737 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testkey", 2024-09-12T03:40:28.737 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.737 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.737 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.737 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.737 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.738 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.738 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.738 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.738 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.738 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.738 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.738 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.738 INFO:teuthology.orchestra.run.smithi064.stdout: "testlkey": { 2024-09-12T03:40:28.738 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testlkey", 2024-09-12T03:40:28.738 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.738 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.738 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.738 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.739 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.739 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.739 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.739 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.739 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.739 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.739 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.739 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.739 INFO:teuthology.orchestra.run.smithi064.stdout: "testnewline": { 2024-09-12T03:40:28.739 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testnewline", 2024-09-12T03:40:28.739 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.739 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.739 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.739 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.740 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.740 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.740 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.740 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.740 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.740 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.740 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.740 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.740 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.740 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.740 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snap_schedule", 2024-09-12T03:40:28.740 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.741 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.741 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.741 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_m_granularity": { 2024-09-12T03:40:28.741 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_m_granularity", 2024-09-12T03:40:28.741 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.741 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.741 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.741 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.741 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.741 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.741 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.741 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T03:40:28.741 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.741 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.742 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.742 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.742 INFO:teuthology.orchestra.run.smithi064.stdout: "dump_on_update": { 2024-09-12T03:40:28.742 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "dump_on_update", 2024-09-12T03:40:28.742 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.742 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.742 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.742 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.742 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.742 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.742 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.742 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "dump database to debug log on update", 2024-09-12T03:40:28.742 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.743 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.743 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.743 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.743 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.743 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.743 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.743 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.743 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.743 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.743 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.743 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.743 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.743 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.743 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.744 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.744 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.744 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.744 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.744 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.744 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.744 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.744 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.744 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.744 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.744 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.744 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.744 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.745 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.745 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.745 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.745 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.745 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.745 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.745 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.745 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.745 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.745 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.745 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.745 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.745 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.746 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.746 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.746 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.746 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.746 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.746 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.746 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.746 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.746 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.746 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.746 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.746 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.746 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.746 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.747 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.747 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.747 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.747 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.747 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.747 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.747 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.747 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.747 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.747 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.747 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.747 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.747 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.748 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.748 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.748 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.748 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.748 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.748 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.748 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.748 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.748 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.748 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "stats", 2024-09-12T03:40:28.748 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.748 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.748 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.749 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.749 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.749 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.749 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.749 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.749 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.749 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.749 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.749 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.749 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.749 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.749 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.749 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.750 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.750 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.750 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.750 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.750 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.750 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.750 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.750 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.750 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.750 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.750 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.750 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.750 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.750 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.751 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.751 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.751 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.751 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.751 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.751 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.751 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.751 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.751 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.751 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.751 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.751 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.751 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.752 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.752 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.752 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.752 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.752 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.752 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.752 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.752 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.752 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.752 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.752 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.752 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.753 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.753 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.753 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.753 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.753 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.753 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.753 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.753 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.753 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.753 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.753 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.753 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.753 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.754 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.754 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.754 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.754 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.754 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.754 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.754 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.754 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.754 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "status", 2024-09-12T03:40:28.754 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.754 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.754 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.754 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.755 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.755 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.755 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.755 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.755 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.755 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.755 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.755 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.755 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.755 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.755 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.755 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.755 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.756 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.756 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.756 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.756 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.756 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.756 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.756 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.756 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.756 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.756 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.756 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.756 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.756 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.757 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.757 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.757 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.757 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.757 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.757 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.757 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.757 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.757 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.757 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.757 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.757 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.757 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.758 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.758 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.758 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.758 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.758 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.758 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.758 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.758 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.758 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.758 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.758 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.758 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.758 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.758 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.759 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.759 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.759 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.759 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.759 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.759 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.759 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.759 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.759 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.759 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.759 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.759 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.759 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.760 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.760 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.760 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.760 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.760 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.760 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.760 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "telegraf", 2024-09-12T03:40:28.760 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.760 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.760 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.760 INFO:teuthology.orchestra.run.smithi064.stdout: "address": { 2024-09-12T03:40:28.760 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "address", 2024-09-12T03:40:28.760 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.760 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.761 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.761 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T03:40:28.761 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.761 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.761 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.761 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.761 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.761 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.761 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.761 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.761 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-12T03:40:28.761 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-12T03:40:28.761 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:28.762 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.762 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.762 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15", 2024-09-12T03:40:28.762 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.762 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.762 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.762 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.762 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.762 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.762 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.762 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.762 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.762 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.762 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.763 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.763 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.763 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.763 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.763 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.763 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.763 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.763 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.763 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.763 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.763 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.763 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.763 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.764 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.764 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.764 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.764 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.764 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.764 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.764 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.764 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.764 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.764 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.764 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.764 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.764 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.765 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.765 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.765 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.765 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.765 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.765 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.765 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.765 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.765 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.765 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.765 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.765 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.765 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.766 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.766 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.766 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.766 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.766 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.766 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.766 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.766 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.766 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.766 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.766 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.766 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.767 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.767 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.767 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.767 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.767 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.767 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.767 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.767 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.767 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.767 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.767 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.768 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.768 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.768 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.768 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.768 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.768 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.768 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.768 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.768 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "telemetry", 2024-09-12T03:40:28.768 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.768 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.768 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.768 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_basic": { 2024-09-12T03:40:28.768 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_basic", 2024-09-12T03:40:28.769 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.769 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.769 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.769 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.769 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.769 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.769 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.769 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T03:40:28.769 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.769 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.769 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.769 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.770 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_crash": { 2024-09-12T03:40:28.770 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_crash", 2024-09-12T03:40:28.770 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.770 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.770 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.770 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.770 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.770 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.770 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.770 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T03:40:28.770 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.770 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.771 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.771 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.771 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_device": { 2024-09-12T03:40:28.771 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_device", 2024-09-12T03:40:28.771 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.771 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.771 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.771 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.771 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.771 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.771 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.771 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T03:40:28.771 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.771 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.772 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.772 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.772 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_ident": { 2024-09-12T03:40:28.772 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_ident", 2024-09-12T03:40:28.772 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.772 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.772 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.772 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.772 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.772 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.772 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.772 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T03:40:28.773 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.773 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.773 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.773 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.773 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_perf": { 2024-09-12T03:40:28.773 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_perf", 2024-09-12T03:40:28.773 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.773 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.773 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.773 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.773 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.773 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.773 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.773 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T03:40:28.774 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.774 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.774 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.774 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.774 INFO:teuthology.orchestra.run.smithi064.stdout: "contact": { 2024-09-12T03:40:28.774 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "contact", 2024-09-12T03:40:28.774 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.774 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.774 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.774 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.774 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.774 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.774 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.775 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.775 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.775 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.775 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.775 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.775 INFO:teuthology.orchestra.run.smithi064.stdout: "description": { 2024-09-12T03:40:28.775 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "description", 2024-09-12T03:40:28.775 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.775 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.775 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.775 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.775 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.775 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.775 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.776 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.776 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.776 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.776 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.776 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.776 INFO:teuthology.orchestra.run.smithi064.stdout: "device_url": { 2024-09-12T03:40:28.776 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_url", 2024-09-12T03:40:28.776 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.776 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.776 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.776 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T03:40:28.776 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.777 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.777 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.777 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.777 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.777 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.777 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.777 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.777 INFO:teuthology.orchestra.run.smithi064.stdout: "enabled": { 2024-09-12T03:40:28.777 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enabled", 2024-09-12T03:40:28.777 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.777 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.777 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.777 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.777 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.778 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.778 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.778 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.778 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.778 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.778 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.778 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.778 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-12T03:40:28.778 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-12T03:40:28.778 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.778 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.778 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.778 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "24", 2024-09-12T03:40:28.778 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "8", 2024-09-12T03:40:28.779 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.779 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.779 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.779 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.779 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.779 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.779 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.779 INFO:teuthology.orchestra.run.smithi064.stdout: "last_opt_revision": { 2024-09-12T03:40:28.779 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "last_opt_revision", 2024-09-12T03:40:28.779 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.779 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.779 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.779 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1", 2024-09-12T03:40:28.779 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.780 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.780 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.780 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.780 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.780 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.780 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.780 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.780 INFO:teuthology.orchestra.run.smithi064.stdout: "leaderboard": { 2024-09-12T03:40:28.780 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "leaderboard", 2024-09-12T03:40:28.780 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.780 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.780 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.780 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.780 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.781 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.781 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.781 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.781 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.781 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.781 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.781 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.781 INFO:teuthology.orchestra.run.smithi064.stdout: "leaderboard_description": { 2024-09-12T03:40:28.781 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "leaderboard_description", 2024-09-12T03:40:28.781 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.781 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.781 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.781 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.781 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.782 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.782 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.782 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.782 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.782 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.782 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.782 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.782 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.782 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.782 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.782 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.782 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.782 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.783 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.783 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.783 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.783 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.783 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.783 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.783 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.783 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.783 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.783 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.783 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.783 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.783 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.783 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.784 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.784 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.784 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.784 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.784 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.784 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.784 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.784 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.784 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.784 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.784 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.784 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.784 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.784 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.785 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.785 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.785 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.785 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.785 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.785 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.785 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.785 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.785 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.785 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.785 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.785 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.785 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.786 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.786 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.786 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.786 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.786 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.786 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.786 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.786 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.786 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.786 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.786 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.786 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.786 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.787 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.787 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.787 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.787 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.787 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.787 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.787 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.787 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.787 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.787 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.787 INFO:teuthology.orchestra.run.smithi064.stdout: "organization": { 2024-09-12T03:40:28.788 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "organization", 2024-09-12T03:40:28.788 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.788 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.788 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.788 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.788 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.788 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.788 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.788 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.788 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.788 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.788 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.789 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.789 INFO:teuthology.orchestra.run.smithi064.stdout: "proxy": { 2024-09-12T03:40:28.789 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "proxy", 2024-09-12T03:40:28.789 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.789 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.789 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.789 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.789 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.789 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.789 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.789 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.789 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.789 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.790 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.790 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.790 INFO:teuthology.orchestra.run.smithi064.stdout: "url": { 2024-09-12T03:40:28.790 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "url", 2024-09-12T03:40:28.790 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.790 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.790 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.790 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T03:40:28.790 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.790 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.790 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.790 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.790 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.791 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.791 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.791 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.791 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.791 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.791 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.791 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "test_orchestrator", 2024-09-12T03:40:28.791 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.791 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.791 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.791 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.791 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.791 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.792 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.792 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.792 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.792 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.792 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.792 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.792 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.792 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.792 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.792 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.792 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.792 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.792 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.793 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.793 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.793 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.793 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.793 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.793 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.793 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.793 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.793 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.793 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.793 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.793 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.793 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.794 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.794 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.794 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.794 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.794 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.794 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.794 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.794 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.794 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.794 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.794 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.794 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.795 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.795 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.795 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.795 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.795 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.795 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.795 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.795 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.795 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.795 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.795 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.795 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.795 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.795 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.796 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.796 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.796 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.796 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.796 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.796 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.796 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.796 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.796 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.796 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.796 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.796 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.796 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.797 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.797 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.797 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.797 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.797 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.797 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "volumes", 2024-09-12T03:40:28.797 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.797 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.797 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.797 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.797 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.797 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.797 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.798 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.798 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.798 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.798 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.798 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.798 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.798 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.798 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.798 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.798 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.798 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.798 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.799 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.799 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.799 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.799 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.799 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.799 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.799 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.799 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.799 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.799 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.799 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.799 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.799 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.800 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.800 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.800 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.800 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.800 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.800 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.800 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.800 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.800 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.800 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.800 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.800 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.800 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.801 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.801 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.801 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.801 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.801 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.801 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.801 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.801 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.801 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.801 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.801 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.801 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.801 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.802 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.802 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.802 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.802 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.802 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.802 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.802 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.802 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.802 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.802 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.802 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.802 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.802 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.802 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.803 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.803 INFO:teuthology.orchestra.run.smithi064.stdout: "max_concurrent_clones": { 2024-09-12T03:40:28.803 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_concurrent_clones", 2024-09-12T03:40:28.803 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.803 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.803 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.803 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "4", 2024-09-12T03:40:28.803 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.803 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.803 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.803 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T03:40:28.803 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.803 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.803 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.804 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.804 INFO:teuthology.orchestra.run.smithi064.stdout: "periodic_async_work": { 2024-09-12T03:40:28.804 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "periodic_async_work", 2024-09-12T03:40:28.804 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.804 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.804 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.804 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.804 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.804 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.804 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.804 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Periodically check for async work", 2024-09-12T03:40:28.804 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.804 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.805 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.805 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.805 INFO:teuthology.orchestra.run.smithi064.stdout: "snapshot_clone_delay": { 2024-09-12T03:40:28.805 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snapshot_clone_delay", 2024-09-12T03:40:28.805 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.805 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.805 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.805 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-12T03:40:28.805 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.805 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.805 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.805 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T03:40:28.805 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.805 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.806 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.806 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.806 INFO:teuthology.orchestra.run.smithi064.stdout: "snapshot_clone_no_wait": { 2024-09-12T03:40:28.806 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T03:40:28.806 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.806 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.806 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.806 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-12T03:40:28.806 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.806 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.806 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.806 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T03:40:28.807 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.807 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.807 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.807 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.807 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.807 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.807 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.807 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix", 2024-09-12T03:40:28.807 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-12T03:40:28.807 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-12T03:40:28.807 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-12T03:40:28.807 INFO:teuthology.orchestra.run.smithi064.stdout: "discovery_interval": { 2024-09-12T03:40:28.807 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "discovery_interval", 2024-09-12T03:40:28.808 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-12T03:40:28.808 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.808 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.808 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "100", 2024-09-12T03:40:28.808 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.808 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.808 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.808 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.808 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.808 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.808 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.808 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.808 INFO:teuthology.orchestra.run.smithi064.stdout: "identifier": { 2024-09-12T03:40:28.808 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "identifier", 2024-09-12T03:40:28.809 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.809 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.809 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.809 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.809 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.809 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.809 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.809 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.809 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.809 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.809 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.809 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.809 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-12T03:40:28.809 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-12T03:40:28.810 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-12T03:40:28.810 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.810 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.810 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-12T03:40:28.810 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.810 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.810 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.810 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.810 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.810 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.810 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.810 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.810 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-12T03:40:28.811 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-12T03:40:28.811 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.811 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.811 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.811 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.811 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.811 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.811 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.811 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.811 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.811 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.811 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.811 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.812 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.812 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.812 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.812 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.812 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.812 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.812 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-12T03:40:28.812 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-12T03:40:28.812 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.812 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.812 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.812 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.812 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.813 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.813 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.813 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.813 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.813 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.813 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.813 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.813 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-12T03:40:28.813 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-12T03:40:28.813 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.813 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.813 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.813 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-12T03:40:28.814 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.814 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.814 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-12T03:40:28.814 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-12T03:40:28.814 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-12T03:40:28.814 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-12T03:40:28.814 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-12T03:40:28.814 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-12T03:40:28.814 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-12T03:40:28.814 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.814 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.814 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.814 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.814 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.815 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.815 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-12T03:40:28.815 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-12T03:40:28.815 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-12T03:40:28.815 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.815 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-12T03:40:28.815 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-12T03:40:28.815 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.815 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.815 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.815 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.815 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.815 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.816 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.816 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.816 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_host": { 2024-09-12T03:40:28.816 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_host", 2024-09-12T03:40:28.816 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.816 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.816 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.816 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-12T03:40:28.816 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.816 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.816 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.816 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.816 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.817 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.817 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.817 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.817 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_port": { 2024-09-12T03:40:28.817 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_port", 2024-09-12T03:40:28.817 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-12T03:40:28.817 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.817 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.817 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10051", 2024-09-12T03:40:28.817 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.817 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.817 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.817 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.817 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.818 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.818 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.818 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.818 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_sender": { 2024-09-12T03:40:28.818 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_sender", 2024-09-12T03:40:28.818 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-12T03:40:28.818 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-12T03:40:28.818 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-12T03:40:28.818 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T03:40:28.818 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-12T03:40:28.818 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-12T03:40:28.818 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-12T03:40:28.818 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-12T03:40:28.819 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-12T03:40:28.819 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-12T03:40:28.819 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-12T03:40:28.819 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.819 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.819 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.819 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.819 INFO:teuthology.orchestra.run.smithi064.stdout: "services": { 2024-09-12T03:40:28.819 INFO:teuthology.orchestra.run.smithi064.stdout: "dashboard": "https://172.21.15.64:8443/" 2024-09-12T03:40:28.819 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.819 INFO:teuthology.orchestra.run.smithi064.stdout: "always_on_modules": { 2024-09-12T03:40:28.819 INFO:teuthology.orchestra.run.smithi064.stdout: "octopus": [ 2024-09-12T03:40:28.819 INFO:teuthology.orchestra.run.smithi064.stdout: "balancer", 2024-09-12T03:40:28.819 INFO:teuthology.orchestra.run.smithi064.stdout: "crash", 2024-09-12T03:40:28.820 INFO:teuthology.orchestra.run.smithi064.stdout: "devicehealth", 2024-09-12T03:40:28.820 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator", 2024-09-12T03:40:28.820 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_autoscaler", 2024-09-12T03:40:28.820 INFO:teuthology.orchestra.run.smithi064.stdout: "progress", 2024-09-12T03:40:28.820 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_support", 2024-09-12T03:40:28.820 INFO:teuthology.orchestra.run.smithi064.stdout: "status", 2024-09-12T03:40:28.820 INFO:teuthology.orchestra.run.smithi064.stdout: "telemetry", 2024-09-12T03:40:28.820 INFO:teuthology.orchestra.run.smithi064.stdout: "volumes" 2024-09-12T03:40:28.820 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.820 INFO:teuthology.orchestra.run.smithi064.stdout: "pacific": [ 2024-09-12T03:40:28.820 INFO:teuthology.orchestra.run.smithi064.stdout: "balancer", 2024-09-12T03:40:28.820 INFO:teuthology.orchestra.run.smithi064.stdout: "crash", 2024-09-12T03:40:28.820 INFO:teuthology.orchestra.run.smithi064.stdout: "devicehealth", 2024-09-12T03:40:28.820 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator", 2024-09-12T03:40:28.821 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_autoscaler", 2024-09-12T03:40:28.821 INFO:teuthology.orchestra.run.smithi064.stdout: "progress", 2024-09-12T03:40:28.821 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_support", 2024-09-12T03:40:28.821 INFO:teuthology.orchestra.run.smithi064.stdout: "status", 2024-09-12T03:40:28.821 INFO:teuthology.orchestra.run.smithi064.stdout: "telemetry", 2024-09-12T03:40:28.821 INFO:teuthology.orchestra.run.smithi064.stdout: "volumes" 2024-09-12T03:40:28.821 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.821 INFO:teuthology.orchestra.run.smithi064.stdout: "quincy": [ 2024-09-12T03:40:28.821 INFO:teuthology.orchestra.run.smithi064.stdout: "balancer", 2024-09-12T03:40:28.821 INFO:teuthology.orchestra.run.smithi064.stdout: "crash", 2024-09-12T03:40:28.821 INFO:teuthology.orchestra.run.smithi064.stdout: "devicehealth", 2024-09-12T03:40:28.821 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator", 2024-09-12T03:40:28.821 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_autoscaler", 2024-09-12T03:40:28.821 INFO:teuthology.orchestra.run.smithi064.stdout: "progress", 2024-09-12T03:40:28.822 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_support", 2024-09-12T03:40:28.822 INFO:teuthology.orchestra.run.smithi064.stdout: "status", 2024-09-12T03:40:28.822 INFO:teuthology.orchestra.run.smithi064.stdout: "telemetry", 2024-09-12T03:40:28.822 INFO:teuthology.orchestra.run.smithi064.stdout: "volumes" 2024-09-12T03:40:28.822 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.822 INFO:teuthology.orchestra.run.smithi064.stdout: "reef": [ 2024-09-12T03:40:28.822 INFO:teuthology.orchestra.run.smithi064.stdout: "balancer", 2024-09-12T03:40:28.822 INFO:teuthology.orchestra.run.smithi064.stdout: "crash", 2024-09-12T03:40:28.822 INFO:teuthology.orchestra.run.smithi064.stdout: "devicehealth", 2024-09-12T03:40:28.822 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator", 2024-09-12T03:40:28.822 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_autoscaler", 2024-09-12T03:40:28.822 INFO:teuthology.orchestra.run.smithi064.stdout: "progress", 2024-09-12T03:40:28.822 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_support", 2024-09-12T03:40:28.823 INFO:teuthology.orchestra.run.smithi064.stdout: "status", 2024-09-12T03:40:28.823 INFO:teuthology.orchestra.run.smithi064.stdout: "telemetry", 2024-09-12T03:40:28.823 INFO:teuthology.orchestra.run.smithi064.stdout: "volumes" 2024-09-12T03:40:28.823 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-12T03:40:28.823 INFO:teuthology.orchestra.run.smithi064.stdout: "squid": [ 2024-09-12T03:40:28.823 INFO:teuthology.orchestra.run.smithi064.stdout: "balancer", 2024-09-12T03:40:28.823 INFO:teuthology.orchestra.run.smithi064.stdout: "crash", 2024-09-12T03:40:28.823 INFO:teuthology.orchestra.run.smithi064.stdout: "devicehealth", 2024-09-12T03:40:28.823 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator", 2024-09-12T03:40:28.823 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_autoscaler", 2024-09-12T03:40:28.823 INFO:teuthology.orchestra.run.smithi064.stdout: "progress", 2024-09-12T03:40:28.823 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_support", 2024-09-12T03:40:28.823 INFO:teuthology.orchestra.run.smithi064.stdout: "status", 2024-09-12T03:40:28.823 INFO:teuthology.orchestra.run.smithi064.stdout: "telemetry", 2024-09-12T03:40:28.823 INFO:teuthology.orchestra.run.smithi064.stdout: "volumes" 2024-09-12T03:40:28.824 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T03:40:28.824 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.824 INFO:teuthology.orchestra.run.smithi064.stdout: "last_failure_osd_epoch": 93, 2024-09-12T03:40:28.824 INFO:teuthology.orchestra.run.smithi064.stdout: "active_clients": [ 2024-09-12T03:40:28.824 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.824 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "devicehealth", 2024-09-12T03:40:28.824 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2024-09-12T03:40:28.824 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.824 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2024-09-12T03:40:28.824 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:0", 2024-09-12T03:40:28.824 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 3999373582 2024-09-12T03:40:28.824 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.824 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T03:40:28.825 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.825 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.825 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "libcephsqlite", 2024-09-12T03:40:28.825 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2024-09-12T03:40:28.825 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.825 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2024-09-12T03:40:28.825 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:0", 2024-09-12T03:40:28.825 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 1984591835 2024-09-12T03:40:28.825 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.825 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T03:40:28.825 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.825 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.825 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_support", 2024-09-12T03:40:28.826 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2024-09-12T03:40:28.826 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.826 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2024-09-12T03:40:28.826 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:0", 2024-09-12T03:40:28.826 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 4244331121 2024-09-12T03:40:28.826 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.826 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T03:40:28.826 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.826 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.826 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snap_schedule", 2024-09-12T03:40:28.826 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2024-09-12T03:40:28.826 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.826 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2024-09-12T03:40:28.826 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:0", 2024-09-12T03:40:28.827 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 2701114818 2024-09-12T03:40:28.827 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.827 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T03:40:28.827 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.827 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.827 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snap_schedule", 2024-09-12T03:40:28.827 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2024-09-12T03:40:28.827 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.827 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2024-09-12T03:40:28.827 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:0", 2024-09-12T03:40:28.827 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 1351167108 2024-09-12T03:40:28.827 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.827 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T03:40:28.827 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:28.828 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.828 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "volumes", 2024-09-12T03:40:28.828 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2024-09-12T03:40:28.828 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-12T03:40:28.828 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2024-09-12T03:40:28.828 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:0", 2024-09-12T03:40:28.828 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 889643926 2024-09-12T03:40:28.828 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.828 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T03:40:28.828 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:28.828 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-12T03:40:28.828 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T03:40:28.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.379+0000 7f69dfaa5640 1 -- 172.21.15.64:0/1154641348 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f69c40a2d00 msgr2=0x7f69c40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:28.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.379+0000 7f69dfaa5640 1 --2- 172.21.15.64:0/1154641348 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f69c40a2d00 0x7f69c40a51c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f69d4009900 tx=0x7f69d4000bc0 comp rx=0 tx=0).stop 2024-09-12T03:40:28.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.379+0000 7f69dfaa5640 1 -- 172.21.15.64:0/1154641348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69d8102450 msgr2=0x7f69d81987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:28.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.379+0000 7f69dfaa5640 1 --2- 172.21.15.64:0/1154641348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69d8102450 0x7f69d81987e0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f69c800c920 tx=0x7f69c800cde0 comp rx=0 tx=0).stop 2024-09-12T03:40:28.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.379+0000 7f69dfaa5640 1 -- 172.21.15.64:0/1154641348 shutdown_connections 2024-09-12T03:40:28.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.379+0000 7f69dfaa5640 1 --2- 172.21.15.64:0/1154641348 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f69c40a2d00 0x7f69c40a51c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:28.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.379+0000 7f69dfaa5640 1 --2- 172.21.15.64:0/1154641348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69d8107ac0 0x7f69d819cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:28.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.379+0000 7f69dfaa5640 1 --2- 172.21.15.64:0/1154641348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69d8102450 0x7f69d81987e0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:28.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.379+0000 7f69dfaa5640 1 --2- 172.21.15.64:0/1154641348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69d8101ab0 0x7f69d81982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:28.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.379+0000 7f69dfaa5640 1 -- 172.21.15.64:0/1154641348 >> 172.21.15.64:0/1154641348 conn(0x7f69d80fd7d0 msgr2=0x7f69d80fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:40:28.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.379+0000 7f69dfaa5640 1 -- 172.21.15.64:0/1154641348 shutdown_connections 2024-09-12T03:40:28.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:27.379+0000 7f69dfaa5640 1 -- 172.21.15.64:0/1154641348 wait complete. 2024-09-12T03:40:28.831 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:27 smithi195 ceph-mon[34786]: pgmap v348: 129 pgs: 129 active+clean; 5.2 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-12T03:40:28.831 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:27 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/770743821' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T03:40:28.831 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:27 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/1979083624' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-12T03:40:28.831 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:27 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/1154641348' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T03:40:28.832 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:27 smithi115 ceph-mon[34801]: pgmap v348: 129 pgs: 129 active+clean; 5.2 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-12T03:40:28.832 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:27 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/770743821' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T03:40:28.832 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:27 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/1979083624' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-12T03:40:28.832 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:27 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/1154641348' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T03:40:28.838 DEBUG:tasks.check_counter:Getting stats from a 2024-09-12T03:40:28.839 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-12T03:40:28.839 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:27 smithi064 ceph-mon[30952]: pgmap v348: 129 pgs: 129 active+clean; 5.2 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-09-12T03:40:28.839 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:27 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/770743821' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T03:40:28.839 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:27 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/1979083624' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-12T03:40:28.839 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:27 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/1154641348' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T03:40:28.863 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:28 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:40:28.863 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:28 smithi064 ceph-mon[30952]: pgmap v349: 129 pgs: 129 active+clean; 5.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 21 op/s 2024-09-12T03:40:28.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:28 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:40:28.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:28 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:40:28.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:28 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:40:28.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:28 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:40:28.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:28 smithi064 ceph-mon[30952]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:40:28.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:28 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:40:28.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:28 smithi115 ceph-mon[34801]: pgmap v349: 129 pgs: 129 active+clean; 5.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 21 op/s 2024-09-12T03:40:28.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:28 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:40:28.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:28 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:40:28.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:28 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:40:28.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:28 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:40:28.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:28 smithi115 ceph-mon[34801]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:40:28.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:28 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:40:28.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:28 smithi195 ceph-mon[34786]: pgmap v349: 129 pgs: 129 active+clean; 5.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 21 op/s 2024-09-12T03:40:28.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:28 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:40:28.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:28 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:40:28.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:28 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:40:28.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:28 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:40:28.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:28 smithi195 ceph-mon[34786]: from='mgr.15243 ' entity='mgr.x' 2024-09-12T03:40:29.099 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:40:29.623 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T03:40:29.624 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:40:29.624 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 233, 2024-09-12T03:40:29.624 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 222, 2024-09-12T03:40:29.624 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 1583808, 2024-09-12T03:40:29.624 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 136384, 2024-09-12T03:40:29.624 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 4, 2024-09-12T03:40:29.624 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 0, 2024-09-12T03:40:29.624 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.044881530, 2024-09-12T03:40:29.624 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.016134185, 2024-09-12T03:40:29.624 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.039229163, 2024-09-12T03:40:29.624 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:40:29.625 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:40:29.625 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 221, 2024-09-12T03:40:29.625 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.013723184, 2024-09-12T03:40:29.625 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000062095 2024-09-12T03:40:29.625 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:29.625 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:40:29.625 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-12T03:40:29.625 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-12T03:40:29.625 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-12T03:40:29.625 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:29.625 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 1583808, 2024-09-12T03:40:29.625 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 136384 2024-09-12T03:40:29.625 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:29.625 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:40:29.626 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 2, 2024-09-12T03:40:29.626 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 169, 2024-09-12T03:40:29.626 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 320, 2024-09-12T03:40:29.626 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 186880, 2024-09-12T03:40:29.626 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 5, 2024-09-12T03:40:29.626 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 3, 2024-09-12T03:40:29.626 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.015678214, 2024-09-12T03:40:29.626 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.010983166, 2024-09-12T03:40:29.626 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.002116387, 2024-09-12T03:40:29.626 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:40:29.626 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:40:29.626 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 166, 2024-09-12T03:40:29.627 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.014856376, 2024-09-12T03:40:29.627 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000089496 2024-09-12T03:40:29.627 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:29.627 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:40:29.627 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-12T03:40:29.627 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-12T03:40:29.627 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-12T03:40:29.627 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:29.627 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T03:40:29.628 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 186880 2024-09-12T03:40:29.628 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:29.628 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:40:29.628 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 4, 2024-09-12T03:40:29.628 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 4, 2024-09-12T03:40:29.628 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 2992, 2024-09-12T03:40:29.628 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 448, 2024-09-12T03:40:29.628 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 2, 2024-09-12T03:40:29.628 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 1, 2024-09-12T03:40:29.629 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.002493650, 2024-09-12T03:40:29.629 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.000053265, 2024-09-12T03:40:29.629 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.002418514, 2024-09-12T03:40:29.629 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:40:29.629 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:40:29.629 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 3, 2024-09-12T03:40:29.629 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.004318326, 2024-09-12T03:40:29.629 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.001439442 2024-09-12T03:40:29.629 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:29.629 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:40:29.629 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-12T03:40:29.630 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-12T03:40:29.630 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-12T03:40:29.630 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:29.630 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-12T03:40:29.630 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T03:40:29.630 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:29.630 INFO:teuthology.orchestra.run.smithi064.stdout: "cct": { 2024-09-12T03:40:29.630 INFO:teuthology.orchestra.run.smithi064.stdout: "total_workers": 0, 2024-09-12T03:40:29.630 INFO:teuthology.orchestra.run.smithi064.stdout: "unhealthy_workers": 0 2024-09-12T03:40:29.630 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:29.630 INFO:teuthology.orchestra.run.smithi064.stdout: "mempool": { 2024-09-12T03:40:29.630 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:40:29.630 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_items": 0, 2024-09-12T03:40:29.630 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:40:29.630 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:40:29.631 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:40:29.631 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:40:29.631 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:40:29.631 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:40:29.631 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:40:29.631 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:40:29.631 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:40:29.631 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:40:29.631 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:40:29.631 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:40:29.631 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:40:29.631 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_items": 0, 2024-09-12T03:40:29.631 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:40:29.632 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_items": 0, 2024-09-12T03:40:29.632 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:40:29.632 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:40:29.632 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:40:29.632 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:40:29.632 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:40:29.632 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:40:29.632 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:40:29.632 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_items": 0, 2024-09-12T03:40:29.632 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:40:29.632 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:40:29.632 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:40:29.632 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_items": 0, 2024-09-12T03:40:29.633 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_bytes": 0, 2024-09-12T03:40:29.633 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_items": 0, 2024-09-12T03:40:29.633 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:40:29.633 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:40:29.633 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:40:29.633 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:40:29.633 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_bytes": 148994, 2024-09-12T03:40:29.633 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_items": 31, 2024-09-12T03:40:29.633 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:40:29.633 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_items": 0, 2024-09-12T03:40:29.633 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_bytes": 0, 2024-09-12T03:40:29.633 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_items": 0, 2024-09-12T03:40:29.633 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:40:29.633 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_items": 0, 2024-09-12T03:40:29.634 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:40:29.634 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_items": 0, 2024-09-12T03:40:29.634 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_bytes": 0, 2024-09-12T03:40:29.634 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_items": 0, 2024-09-12T03:40:29.634 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:40:29.634 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:40:29.634 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_bytes": 0, 2024-09-12T03:40:29.634 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_items": 0, 2024-09-12T03:40:29.634 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_bytes": 928, 2024-09-12T03:40:29.634 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_items": 2, 2024-09-12T03:40:29.634 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_bytes": 0, 2024-09-12T03:40:29.634 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_items": 0, 2024-09-12T03:40:29.635 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_bytes": 0, 2024-09-12T03:40:29.635 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_items": 0 2024-09-12T03:40:29.635 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:29.635 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:40:29.635 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-09-12T03:40:29.635 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-09-12T03:40:29.635 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-09-12T03:40:29.635 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 235, 2024-09-12T03:40:29.635 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 1550955, 2024-09-12T03:40:29.636 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-09-12T03:40:29.636 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 235, 2024-09-12T03:40:29.636 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-09-12T03:40:29.636 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-09-12T03:40:29.636 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 235, 2024-09-12T03:40:29.636 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 1550955, 2024-09-12T03:40:29.636 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-09-12T03:40:29.636 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-12T03:40:29.636 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-12T03:40:29.636 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-12T03:40:29.636 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:29.636 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:29.636 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T03:40:30.376 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-09-12T03:40:30.376 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-09-12T03:40:30.376 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-09-12T03:40:30.376 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-09-12T03:40:30.376 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-09-12T03:40:30.376 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-09-12T03:40:30.376 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-09-12T03:40:30.376 DEBUG:tasks.check_counter:Getting stats from d 2024-09-12T03:40:30.377 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-12T03:40:30.676 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:40:31.096 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:30 smithi064 ceph-mon[30952]: pgmap v350: 129 pgs: 129 active+clean; 5.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-09-12T03:40:31.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:30 smithi115 ceph-mon[34801]: pgmap v350: 129 pgs: 129 active+clean; 5.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-09-12T03:40:31.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:30 smithi195 ceph-mon[34786]: pgmap v350: 129 pgs: 129 active+clean; 5.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-09-12T03:40:31.349 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T03:40:31.350 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:40:31.350 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 238, 2024-09-12T03:40:31.350 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 227, 2024-09-12T03:40:31.350 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 1585168, 2024-09-12T03:40:31.350 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 139584, 2024-09-12T03:40:31.350 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 4, 2024-09-12T03:40:31.350 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 0, 2024-09-12T03:40:31.350 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.044080664, 2024-09-12T03:40:31.350 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.016733146, 2024-09-12T03:40:31.350 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.039150771, 2024-09-12T03:40:31.350 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:40:31.350 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:40:31.350 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 226, 2024-09-12T03:40:31.351 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.013439656, 2024-09-12T03:40:31.351 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000059467 2024-09-12T03:40:31.351 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:31.351 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:40:31.351 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-12T03:40:31.351 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-12T03:40:31.351 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-12T03:40:31.351 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:31.351 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 1585168, 2024-09-12T03:40:31.351 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 139584 2024-09-12T03:40:31.351 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:31.351 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:40:31.351 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 6, 2024-09-12T03:40:31.351 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 175, 2024-09-12T03:40:31.352 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 3312, 2024-09-12T03:40:31.352 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 188032, 2024-09-12T03:40:31.352 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 5, 2024-09-12T03:40:31.352 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 3, 2024-09-12T03:40:31.352 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.019064194, 2024-09-12T03:40:31.352 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.012174216, 2024-09-12T03:40:31.352 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.004059085, 2024-09-12T03:40:31.352 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:40:31.352 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:40:31.352 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 173, 2024-09-12T03:40:31.352 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.017325357, 2024-09-12T03:40:31.352 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000100146 2024-09-12T03:40:31.352 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:31.353 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:40:31.353 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-12T03:40:31.353 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-12T03:40:31.353 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-12T03:40:31.353 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:31.353 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-09-12T03:40:31.353 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 188032 2024-09-12T03:40:31.353 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:31.353 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:40:31.353 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 0, 2024-09-12T03:40:31.353 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 2, 2024-09-12T03:40:31.353 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 0, 2024-09-12T03:40:31.353 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 0, 2024-09-12T03:40:31.354 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 2, 2024-09-12T03:40:31.354 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 1, 2024-09-12T03:40:31.354 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.000319003, 2024-09-12T03:40:31.354 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T03:40:31.354 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.000240375, 2024-09-12T03:40:31.354 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:40:31.354 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:40:31.354 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-12T03:40:31.354 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-12T03:40:31.354 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-12T03:40:31.354 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:31.354 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:40:31.354 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-12T03:40:31.354 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-12T03:40:31.355 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-12T03:40:31.355 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:31.355 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:40:31.355 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:40:31.355 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:31.355 INFO:teuthology.orchestra.run.smithi064.stdout: "cct": { 2024-09-12T03:40:31.355 INFO:teuthology.orchestra.run.smithi064.stdout: "total_workers": 0, 2024-09-12T03:40:31.355 INFO:teuthology.orchestra.run.smithi064.stdout: "unhealthy_workers": 0 2024-09-12T03:40:31.355 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:31.355 INFO:teuthology.orchestra.run.smithi064.stdout: "mempool": { 2024-09-12T03:40:31.355 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:40:31.355 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_items": 0, 2024-09-12T03:40:31.355 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:40:31.355 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:40:31.356 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:40:31.356 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:40:31.356 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:40:31.356 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:40:31.356 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:40:31.356 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:40:31.356 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:40:31.356 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:40:31.356 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:40:31.356 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:40:31.356 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:40:31.356 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_items": 0, 2024-09-12T03:40:31.356 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:40:31.356 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_items": 0, 2024-09-12T03:40:31.357 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:40:31.357 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:40:31.357 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:40:31.357 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:40:31.357 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:40:31.357 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:40:31.357 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:40:31.357 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_items": 0, 2024-09-12T03:40:31.357 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:40:31.357 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:40:31.357 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:40:31.357 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_items": 0, 2024-09-12T03:40:31.357 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_bytes": 0, 2024-09-12T03:40:31.358 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_items": 0, 2024-09-12T03:40:31.358 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:40:31.358 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:40:31.358 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:40:31.358 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:40:31.358 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_bytes": 148977, 2024-09-12T03:40:31.358 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_items": 30, 2024-09-12T03:40:31.358 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:40:31.358 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_items": 0, 2024-09-12T03:40:31.358 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_bytes": 0, 2024-09-12T03:40:31.358 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_items": 0, 2024-09-12T03:40:31.358 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:40:31.358 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_items": 0, 2024-09-12T03:40:31.358 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:40:31.359 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_items": 0, 2024-09-12T03:40:31.359 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_bytes": 0, 2024-09-12T03:40:31.359 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_items": 0, 2024-09-12T03:40:31.359 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:40:31.359 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:40:31.359 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_bytes": 0, 2024-09-12T03:40:31.359 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_items": 0, 2024-09-12T03:40:31.359 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_bytes": 928, 2024-09-12T03:40:31.359 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_items": 2, 2024-09-12T03:40:31.359 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_bytes": 0, 2024-09-12T03:40:31.359 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_items": 0, 2024-09-12T03:40:31.359 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_bytes": 0, 2024-09-12T03:40:31.359 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_items": 0 2024-09-12T03:40:31.360 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:31.360 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:40:31.360 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-09-12T03:40:31.360 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-09-12T03:40:31.360 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-09-12T03:40:31.360 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 240, 2024-09-12T03:40:31.360 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 1551605, 2024-09-12T03:40:31.360 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-09-12T03:40:31.360 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 240, 2024-09-12T03:40:31.360 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-09-12T03:40:31.360 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-09-12T03:40:31.360 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 240, 2024-09-12T03:40:31.360 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 1551605, 2024-09-12T03:40:31.361 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-09-12T03:40:31.361 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-12T03:40:31.361 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-12T03:40:31.361 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-12T03:40:31.361 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:31.361 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:31.361 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T03:40:31.985 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-12T03:40:31.985 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-12T03:40:31.985 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-09-12T03:40:31.985 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-09-12T03:40:31.985 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-12T03:40:31.985 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-12T03:40:31.985 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-12T03:40:31.985 DEBUG:tasks.check_counter:Getting stats from g 2024-09-12T03:40:31.986 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-12T03:40:32.243 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:40:32.777 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T03:40:32.777 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:40:32.777 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 4, 2024-09-12T03:40:32.777 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 4, 2024-09-12T03:40:32.777 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 2992, 2024-09-12T03:40:32.778 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 448, 2024-09-12T03:40:32.778 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 4, 2024-09-12T03:40:32.778 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 0, 2024-09-12T03:40:32.778 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.003832351, 2024-09-12T03:40:32.778 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.000048001, 2024-09-12T03:40:32.778 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.002674464, 2024-09-12T03:40:32.778 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:40:32.778 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:40:32.778 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 3, 2024-09-12T03:40:32.778 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.004674574, 2024-09-12T03:40:32.778 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.001558191 2024-09-12T03:40:32.779 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:32.779 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:40:32.779 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-12T03:40:32.779 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-12T03:40:32.779 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-12T03:40:32.779 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:32.779 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-12T03:40:32.779 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T03:40:32.779 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:32.779 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:40:32.779 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 2, 2024-09-12T03:40:32.779 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 171, 2024-09-12T03:40:32.780 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 320, 2024-09-12T03:40:32.780 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 187232, 2024-09-12T03:40:32.780 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 5, 2024-09-12T03:40:32.780 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 3, 2024-09-12T03:40:32.780 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.016878885, 2024-09-12T03:40:32.780 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.012060073, 2024-09-12T03:40:32.780 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.002124355, 2024-09-12T03:40:32.780 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:40:32.780 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:40:32.780 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 168, 2024-09-12T03:40:32.780 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.016239781, 2024-09-12T03:40:32.780 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000096665 2024-09-12T03:40:32.780 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:32.781 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:40:32.781 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-12T03:40:32.781 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-12T03:40:32.781 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-12T03:40:32.781 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:32.781 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T03:40:32.781 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 187232 2024-09-12T03:40:32.781 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:32.781 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:40:32.781 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 234, 2024-09-12T03:40:32.781 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 223, 2024-09-12T03:40:32.781 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 1584080, 2024-09-12T03:40:32.782 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 137024, 2024-09-12T03:40:32.782 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 2, 2024-09-12T03:40:32.782 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 1, 2024-09-12T03:40:32.782 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.043545823, 2024-09-12T03:40:32.782 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.017119953, 2024-09-12T03:40:32.782 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.038690979, 2024-09-12T03:40:32.782 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:40:32.782 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:40:32.782 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 222, 2024-09-12T03:40:32.782 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.013583281, 2024-09-12T03:40:32.782 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000061185 2024-09-12T03:40:32.782 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:32.782 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:40:32.783 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-12T03:40:32.783 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-12T03:40:32.783 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-12T03:40:32.783 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:32.783 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 1584080, 2024-09-12T03:40:32.783 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 137024 2024-09-12T03:40:32.783 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:32.783 INFO:teuthology.orchestra.run.smithi064.stdout: "cct": { 2024-09-12T03:40:32.783 INFO:teuthology.orchestra.run.smithi064.stdout: "total_workers": 0, 2024-09-12T03:40:32.783 INFO:teuthology.orchestra.run.smithi064.stdout: "unhealthy_workers": 0 2024-09-12T03:40:32.783 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:32.783 INFO:teuthology.orchestra.run.smithi064.stdout: "mempool": { 2024-09-12T03:40:32.784 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:40:32.784 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_items": 0, 2024-09-12T03:40:32.784 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:40:32.784 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:40:32.784 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:40:32.784 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:40:32.784 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:40:32.784 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:40:32.784 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:40:32.784 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:40:32.784 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:40:32.784 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:40:32.784 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:40:32.785 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:40:32.785 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:40:32.785 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_items": 0, 2024-09-12T03:40:32.785 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:40:32.785 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_items": 0, 2024-09-12T03:40:32.785 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:40:32.785 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:40:32.785 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:40:32.785 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:40:32.785 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:40:32.785 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:40:32.785 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:40:32.785 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_items": 0, 2024-09-12T03:40:32.785 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:40:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:40:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:40:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_items": 0, 2024-09-12T03:40:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_bytes": 0, 2024-09-12T03:40:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_items": 0, 2024-09-12T03:40:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:40:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:40:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:40:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:40:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_bytes": 149034, 2024-09-12T03:40:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_items": 32, 2024-09-12T03:40:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:40:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_items": 0, 2024-09-12T03:40:32.786 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_bytes": 0, 2024-09-12T03:40:32.787 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_items": 0, 2024-09-12T03:40:32.787 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:40:32.787 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_items": 0, 2024-09-12T03:40:32.787 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:40:32.787 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_items": 0, 2024-09-12T03:40:32.787 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_bytes": 0, 2024-09-12T03:40:32.787 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_items": 0, 2024-09-12T03:40:32.787 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:40:32.787 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:40:32.787 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_bytes": 0, 2024-09-12T03:40:32.787 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_items": 0, 2024-09-12T03:40:32.787 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_bytes": 928, 2024-09-12T03:40:32.787 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_items": 2, 2024-09-12T03:40:32.787 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_bytes": 0, 2024-09-12T03:40:32.788 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_items": 0, 2024-09-12T03:40:32.788 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_bytes": 0, 2024-09-12T03:40:32.788 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_items": 0 2024-09-12T03:40:32.788 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:32.788 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:40:32.788 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-09-12T03:40:32.788 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-09-12T03:40:32.788 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-09-12T03:40:32.788 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 236, 2024-09-12T03:40:32.788 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 1551085, 2024-09-12T03:40:32.788 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-09-12T03:40:32.788 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 236, 2024-09-12T03:40:32.788 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-09-12T03:40:32.789 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-09-12T03:40:32.789 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 236, 2024-09-12T03:40:32.789 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 1551085, 2024-09-12T03:40:32.789 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-09-12T03:40:32.789 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-12T03:40:32.789 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-12T03:40:32.789 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-12T03:40:32.789 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:32.789 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:32.789 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T03:40:33.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:32 smithi064 ceph-mon[30952]: pgmap v351: 129 pgs: 129 active+clean; 5.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 16 op/s 2024-09-12T03:40:33.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:32 smithi115 ceph-mon[34801]: pgmap v351: 129 pgs: 129 active+clean; 5.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 16 op/s 2024-09-12T03:40:33.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:32 smithi195 ceph-mon[34786]: pgmap v351: 129 pgs: 129 active+clean; 5.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 16 op/s 2024-09-12T03:40:33.460 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-09-12T03:40:33.460 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-09-12T03:40:33.460 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-09-12T03:40:33.460 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-09-12T03:40:33.460 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-09-12T03:40:33.460 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-09-12T03:40:33.460 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-09-12T03:40:33.461 DEBUG:tasks.check_counter:Getting stats from j 2024-09-12T03:40:33.461 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-12T03:40:33.717 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:40:34.241 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-12T03:40:34.241 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:40:34.241 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 0, 2024-09-12T03:40:34.242 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 2, 2024-09-12T03:40:34.242 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 0, 2024-09-12T03:40:34.242 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 0, 2024-09-12T03:40:34.242 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 4, 2024-09-12T03:40:34.242 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 0, 2024-09-12T03:40:34.242 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.001513046, 2024-09-12T03:40:34.242 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T03:40:34.242 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.000349939, 2024-09-12T03:40:34.242 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:40:34.242 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:40:34.242 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-12T03:40:34.242 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-12T03:40:34.243 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-12T03:40:34.243 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:34.243 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:40:34.243 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-12T03:40:34.243 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-12T03:40:34.243 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-12T03:40:34.243 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:34.243 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:40:34.243 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:40:34.243 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:34.243 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:40:34.243 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 6, 2024-09-12T03:40:34.243 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 177, 2024-09-12T03:40:34.244 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 3312, 2024-09-12T03:40:34.244 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 188384, 2024-09-12T03:40:34.244 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 5, 2024-09-12T03:40:34.244 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 3, 2024-09-12T03:40:34.244 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.019015941, 2024-09-12T03:40:34.244 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.011524404, 2024-09-12T03:40:34.244 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.004807267, 2024-09-12T03:40:34.244 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:40:34.244 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:40:34.244 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 175, 2024-09-12T03:40:34.244 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.023903977, 2024-09-12T03:40:34.244 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000136594 2024-09-12T03:40:34.244 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:34.244 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:40:34.245 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-12T03:40:34.245 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-12T03:40:34.245 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-12T03:40:34.245 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:34.245 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-09-12T03:40:34.245 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 188384 2024-09-12T03:40:34.245 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:34.245 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:40:34.245 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 240, 2024-09-12T03:40:34.245 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 229, 2024-09-12T03:40:34.245 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 1585712, 2024-09-12T03:40:34.245 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 140864, 2024-09-12T03:40:34.245 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 2, 2024-09-12T03:40:34.246 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 1, 2024-09-12T03:40:34.246 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.047087870, 2024-09-12T03:40:34.246 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.017104822, 2024-09-12T03:40:34.246 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.042980729, 2024-09-12T03:40:34.246 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:40:34.246 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:40:34.246 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 228, 2024-09-12T03:40:34.246 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.013657619, 2024-09-12T03:40:34.246 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000059901 2024-09-12T03:40:34.246 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:34.246 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:40:34.246 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-12T03:40:34.246 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-12T03:40:34.246 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-12T03:40:34.247 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:34.247 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 1585712, 2024-09-12T03:40:34.247 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 140864 2024-09-12T03:40:34.247 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:34.247 INFO:teuthology.orchestra.run.smithi064.stdout: "cct": { 2024-09-12T03:40:34.247 INFO:teuthology.orchestra.run.smithi064.stdout: "total_workers": 0, 2024-09-12T03:40:34.247 INFO:teuthology.orchestra.run.smithi064.stdout: "unhealthy_workers": 0 2024-09-12T03:40:34.247 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:34.247 INFO:teuthology.orchestra.run.smithi064.stdout: "mempool": { 2024-09-12T03:40:34.247 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:40:34.247 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_items": 0, 2024-09-12T03:40:34.247 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:40:34.247 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:40:34.247 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:40:34.248 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:40:34.248 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:40:34.248 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:40:34.248 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:40:34.248 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:40:34.248 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:40:34.248 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:40:34.248 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:40:34.248 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:40:34.249 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:40:34.249 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_items": 0, 2024-09-12T03:40:34.249 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:40:34.249 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_items": 0, 2024-09-12T03:40:34.249 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:40:34.249 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:40:34.249 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:40:34.249 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:40:34.249 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:40:34.249 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:40:34.250 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:40:34.250 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_items": 0, 2024-09-12T03:40:34.250 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:40:34.250 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:40:34.250 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:40:34.250 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_items": 0, 2024-09-12T03:40:34.250 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_bytes": 0, 2024-09-12T03:40:34.250 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_items": 0, 2024-09-12T03:40:34.250 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:40:34.250 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:40:34.250 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:40:34.250 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:40:34.250 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_bytes": 148994, 2024-09-12T03:40:34.250 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_items": 31, 2024-09-12T03:40:34.250 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:40:34.251 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_items": 0, 2024-09-12T03:40:34.251 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_bytes": 0, 2024-09-12T03:40:34.251 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_items": 0, 2024-09-12T03:40:34.251 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:40:34.251 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_items": 0, 2024-09-12T03:40:34.251 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:40:34.251 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_items": 0, 2024-09-12T03:40:34.251 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_bytes": 0, 2024-09-12T03:40:34.251 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_items": 0, 2024-09-12T03:40:34.251 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:40:34.251 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:40:34.251 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_bytes": 0, 2024-09-12T03:40:34.251 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_items": 0, 2024-09-12T03:40:34.252 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_bytes": 928, 2024-09-12T03:40:34.252 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_items": 2, 2024-09-12T03:40:34.252 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_bytes": 0, 2024-09-12T03:40:34.252 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_items": 0, 2024-09-12T03:40:34.252 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_bytes": 0, 2024-09-12T03:40:34.252 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_items": 0 2024-09-12T03:40:34.252 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-12T03:40:34.252 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:40:34.252 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-09-12T03:40:34.252 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-09-12T03:40:34.252 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-09-12T03:40:34.252 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 242, 2024-09-12T03:40:34.252 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 1551865, 2024-09-12T03:40:34.252 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-09-12T03:40:34.252 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 242, 2024-09-12T03:40:34.253 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-09-12T03:40:34.253 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-09-12T03:40:34.253 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 242, 2024-09-12T03:40:34.253 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 1551865, 2024-09-12T03:40:34.253 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-09-12T03:40:34.253 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-12T03:40:34.253 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-12T03:40:34.253 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-12T03:40:34.253 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:34.253 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-12T03:40:34.253 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-12T03:40:34.998 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-12T03:40:34.998 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-12T03:40:34.998 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-12T03:40:34.999 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-12T03:40:34.999 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-12T03:40:34.999 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-12T03:40:34.999 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-12T03:40:34.999 DEBUG:tasks.check_counter:Getting stats from b 2024-09-12T03:40:34.999 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-12T03:40:35.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:34 smithi115 ceph-mon[34801]: pgmap v352: 129 pgs: 129 active+clean; 5.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:40:35.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:34 smithi195 ceph-mon[34786]: pgmap v352: 129 pgs: 129 active+clean; 5.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:40:35.260 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:40:35.271 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:34 smithi064 ceph-mon[30952]: pgmap v352: 129 pgs: 129 active+clean; 5.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:40:35.775 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T03:40:35.775 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:40:35.775 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 2669, 2024-09-12T03:40:35.775 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 3019, 2024-09-12T03:40:35.775 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 2456748, 2024-09-12T03:40:35.776 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 6592959, 2024-09-12T03:40:35.776 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 104, 2024-09-12T03:40:35.776 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 3, 2024-09-12T03:40:35.776 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.292605277, 2024-09-12T03:40:35.776 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.105389135, 2024-09-12T03:40:35.776 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.159283882, 2024-09-12T03:40:35.776 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.037213358, 2024-09-12T03:40:35.776 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:40:35.777 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 3019, 2024-09-12T03:40:35.777 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.207223761, 2024-09-12T03:40:35.777 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000068639 2024-09-12T03:40:35.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.777 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:40:35.777 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 2178, 2024-09-12T03:40:35.777 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000508086, 2024-09-12T03:40:35.777 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000233 2024-09-12T03:40:35.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.777 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 1690800, 2024-09-12T03:40:35.778 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 149856 2024-09-12T03:40:35.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.778 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:40:35.778 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 2921, 2024-09-12T03:40:35.778 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 5280, 2024-09-12T03:40:35.778 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 981328, 2024-09-12T03:40:35.778 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 6899465, 2024-09-12T03:40:35.779 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 15, 2024-09-12T03:40:35.779 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 5, 2024-09-12T03:40:35.780 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.359067461, 2024-09-12T03:40:35.780 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.129276557, 2024-09-12T03:40:35.780 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.155229761, 2024-09-12T03:40:35.780 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.073120635, 2024-09-12T03:40:35.780 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:40:35.781 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 5277, 2024-09-12T03:40:35.781 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.341799893, 2024-09-12T03:40:35.781 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000064771 2024-09-12T03:40:35.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.782 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:40:35.782 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 5914, 2024-09-12T03:40:35.782 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.001351215, 2024-09-12T03:40:35.783 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000228 2024-09-12T03:40:35.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.783 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 100688, 2024-09-12T03:40:35.783 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 502720 2024-09-12T03:40:35.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.784 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:40:35.784 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 3757, 2024-09-12T03:40:35.784 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 3604, 2024-09-12T03:40:35.784 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 1261549, 2024-09-12T03:40:35.784 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 14345403, 2024-09-12T03:40:35.784 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 20, 2024-09-12T03:40:35.784 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 7, 2024-09-12T03:40:35.784 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.367624210, 2024-09-12T03:40:35.784 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.129023668, 2024-09-12T03:40:35.784 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.144168371, 2024-09-12T03:40:35.784 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.094635563, 2024-09-12T03:40:35.784 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:40:35.785 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 3602, 2024-09-12T03:40:35.785 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.265925920, 2024-09-12T03:40:35.785 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000073827 2024-09-12T03:40:35.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.785 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:40:35.785 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 2945, 2024-09-12T03:40:35.785 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000769312, 2024-09-12T03:40:35.785 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000261 2024-09-12T03:40:35.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.785 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 334672, 2024-09-12T03:40:35.785 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 97296 2024-09-12T03:40:35.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.786 INFO:teuthology.orchestra.run.smithi115.stdout: "cct": { 2024-09-12T03:40:35.786 INFO:teuthology.orchestra.run.smithi115.stdout: "total_workers": 1, 2024-09-12T03:40:35.786 INFO:teuthology.orchestra.run.smithi115.stdout: "unhealthy_workers": 0 2024-09-12T03:40:35.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.786 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-MDSRank": { 2024-09-12T03:40:35.786 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2024-09-12T03:40:35.786 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2024-09-12T03:40:35.786 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 6057, 2024-09-12T03:40:35.786 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 1.597441401, 2024-09-12T03:40:35.786 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000263734 2024-09-12T03:40:35.786 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T03:40:35.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.787 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-PurgeQueue": { 2024-09-12T03:40:35.787 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2024-09-12T03:40:35.787 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2024-09-12T03:40:35.787 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 14, 2024-09-12T03:40:35.787 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000371583, 2024-09-12T03:40:35.787 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000026541 2024-09-12T03:40:35.787 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T03:40:35.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.787 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-09-12T03:40:35.787 INFO:teuthology.orchestra.run.smithi115.stdout: "request": 773, 2024-09-12T03:40:35.787 INFO:teuthology.orchestra.run.smithi115.stdout: "reply": 773, 2024-09-12T03:40:35.787 INFO:teuthology.orchestra.run.smithi115.stdout: "reply_latency": { 2024-09-12T03:40:35.788 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 773, 2024-09-12T03:40:35.788 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 3.190564016, 2024-09-12T03:40:35.788 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.004127508 2024-09-12T03:40:35.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.788 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_reply": 0, 2024-09-12T03:40:35.788 INFO:teuthology.orchestra.run.smithi115.stdout: "forward": 0, 2024-09-12T03:40:35.788 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_complete": 0, 2024-09-12T03:40:35.788 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_keys": 0, 2024-09-12T03:40:35.788 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_commit": 347, 2024-09-12T03:40:35.788 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_split": 0, 2024-09-12T03:40:35.788 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_merge": 0, 2024-09-12T03:40:35.788 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes": 41, 2024-09-12T03:40:35.789 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_top": 13, 2024-09-12T03:40:35.789 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_bottom": 28, 2024-09-12T03:40:35.789 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pin_tail": 0, 2024-09-12T03:40:35.789 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pinned": 22, 2024-09-12T03:40:35.789 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_expired": 0, 2024-09-12T03:40:35.789 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_with_caps": 3, 2024-09-12T03:40:35.789 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": 3, 2024-09-12T03:40:35.789 INFO:teuthology.orchestra.run.smithi115.stdout: "subtrees": 2, 2024-09-12T03:40:35.789 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse": 820, 2024-09-12T03:40:35.789 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_hit": 806, 2024-09-12T03:40:35.789 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_forward": 0, 2024-09-12T03:40:35.789 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_discover": 0, 2024-09-12T03:40:35.790 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:40:35.790 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_remote_ino": 0, 2024-09-12T03:40:35.790 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_lock": 0, 2024-09-12T03:40:35.790 INFO:teuthology.orchestra.run.smithi115.stdout: "load_cent": 6946, 2024-09-12T03:40:35.790 INFO:teuthology.orchestra.run.smithi115.stdout: "q": 3, 2024-09-12T03:40:35.790 INFO:teuthology.orchestra.run.smithi115.stdout: "exported": 0, 2024-09-12T03:40:35.790 INFO:teuthology.orchestra.run.smithi115.stdout: "exported_inodes": 0, 2024-09-12T03:40:35.790 INFO:teuthology.orchestra.run.smithi115.stdout: "imported": 0, 2024-09-12T03:40:35.790 INFO:teuthology.orchestra.run.smithi115.stdout: "imported_inodes": 0, 2024-09-12T03:40:35.790 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_dir_fetch": 0, 2024-09-12T03:40:35.790 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:40:35.790 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_peer_discover": 0, 2024-09-12T03:40:35.790 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rfiles": 4, 2024-09-12T03:40:35.791 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rbytes": 234, 2024-09-12T03:40:35.791 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rsnaps": 6, 2024-09-12T03:40:35.791 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:40:35.791 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_set_tag": 0, 2024-09-12T03:40:35.791 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:40:35.791 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:40:35.791 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:40:35.791 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:40:35.791 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:40:35.791 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_file_inodes": 0, 2024-09-12T03:40:35.791 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:40:35.791 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_revoke": 621, 2024-09-12T03:40:35.792 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_grant": 1208, 2024-09-12T03:40:35.792 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_trunc": 599, 2024-09-12T03:40:35.792 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:40:35.792 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:40:35.792 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps": 638, 2024-09-12T03:40:35.792 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps_dirty": 613, 2024-09-12T03:40:35.792 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_cap_release": 2, 2024-09-12T03:40:35.792 INFO:teuthology.orchestra.run.smithi115.stdout: "process_request_cap_release": 26 2024-09-12T03:40:35.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.792 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cache": { 2024-09-12T03:40:35.792 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update": 10, 2024-09-12T03:40:35.792 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update_receipt": 0, 2024-09-12T03:40:35.793 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_try_discover": 0, 2024-09-12T03:40:35.793 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_send_discover": 0, 2024-09-12T03:40:35.793 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_handle_discover": 20, 2024-09-12T03:40:35.793 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays": 2, 2024-09-12T03:40:35.793 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_delayed": 0, 2024-09-12T03:40:35.793 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:40:35.793 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_created": 3, 2024-09-12T03:40:35.793 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_enqueued": 1, 2024-09-12T03:40:35.793 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_reintegrated": 0, 2024-09-12T03:40:35.793 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_migrated": 0, 2024-09-12T03:40:35.793 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_processing": 0, 2024-09-12T03:40:35.793 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:40:35.794 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:40:35.794 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_started": 0, 2024-09-12T03:40:35.794 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_completed": 0, 2024-09-12T03:40:35.794 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_quiesce_path": 0, 2024-09-12T03:40:35.794 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_quiesce_inode": 0, 2024-09-12T03:40:35.794 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:40:35.794 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_exportdir": 0, 2024-09-12T03:40:35.794 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_flush": 0, 2024-09-12T03:40:35.794 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:40:35.794 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragstats": 0, 2024-09-12T03:40:35.794 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_inodestats": 0 2024-09-12T03:40:35.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.794 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_log": { 2024-09-12T03:40:35.795 INFO:teuthology.orchestra.run.smithi115.stdout: "evlrg": 0, 2024-09-12T03:40:35.795 INFO:teuthology.orchestra.run.smithi115.stdout: "evadd": 2481, 2024-09-12T03:40:35.795 INFO:teuthology.orchestra.run.smithi115.stdout: "evex": 2435, 2024-09-12T03:40:35.795 INFO:teuthology.orchestra.run.smithi115.stdout: "evtrm": 2435, 2024-09-12T03:40:35.795 INFO:teuthology.orchestra.run.smithi115.stdout: "ev": 46, 2024-09-12T03:40:35.795 INFO:teuthology.orchestra.run.smithi115.stdout: "evexg": 0, 2024-09-12T03:40:35.795 INFO:teuthology.orchestra.run.smithi115.stdout: "evexd": 52, 2024-09-12T03:40:35.795 INFO:teuthology.orchestra.run.smithi115.stdout: "segadd": 57, 2024-09-12T03:40:35.795 INFO:teuthology.orchestra.run.smithi115.stdout: "segex": 55, 2024-09-12T03:40:35.795 INFO:teuthology.orchestra.run.smithi115.stdout: "segtrm": 55, 2024-09-12T03:40:35.795 INFO:teuthology.orchestra.run.smithi115.stdout: "seg": 2, 2024-09-12T03:40:35.795 INFO:teuthology.orchestra.run.smithi115.stdout: "segmjr": 1, 2024-09-12T03:40:35.796 INFO:teuthology.orchestra.run.smithi115.stdout: "segexg": 0, 2024-09-12T03:40:35.796 INFO:teuthology.orchestra.run.smithi115.stdout: "segexd": 1, 2024-09-12T03:40:35.796 INFO:teuthology.orchestra.run.smithi115.stdout: "expos": 24826276, 2024-09-12T03:40:35.796 INFO:teuthology.orchestra.run.smithi115.stdout: "wrpos": 25270003, 2024-09-12T03:40:35.796 INFO:teuthology.orchestra.run.smithi115.stdout: "rdpos": 0, 2024-09-12T03:40:35.796 INFO:teuthology.orchestra.run.smithi115.stdout: "jlat": { 2024-09-12T03:40:35.796 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 2422, 2024-09-12T03:40:35.796 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 5.091003681, 2024-09-12T03:40:35.796 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.002101983 2024-09-12T03:40:35.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.796 INFO:teuthology.orchestra.run.smithi115.stdout: "replayed": 0 2024-09-12T03:40:35.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.796 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_mem": { 2024-09-12T03:40:35.797 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 35, 2024-09-12T03:40:35.797 INFO:teuthology.orchestra.run.smithi115.stdout: "ino+": 36, 2024-09-12T03:40:35.797 INFO:teuthology.orchestra.run.smithi115.stdout: "ino-": 1, 2024-09-12T03:40:35.797 INFO:teuthology.orchestra.run.smithi115.stdout: "dir": 20, 2024-09-12T03:40:35.797 INFO:teuthology.orchestra.run.smithi115.stdout: "dir+": 20, 2024-09-12T03:40:35.797 INFO:teuthology.orchestra.run.smithi115.stdout: "dir-": 0, 2024-09-12T03:40:35.797 INFO:teuthology.orchestra.run.smithi115.stdout: "dn": 41, 2024-09-12T03:40:35.797 INFO:teuthology.orchestra.run.smithi115.stdout: "dn+": 42, 2024-09-12T03:40:35.797 INFO:teuthology.orchestra.run.smithi115.stdout: "dn-": 1, 2024-09-12T03:40:35.797 INFO:teuthology.orchestra.run.smithi115.stdout: "cap": 3, 2024-09-12T03:40:35.797 INFO:teuthology.orchestra.run.smithi115.stdout: "cap+": 20, 2024-09-12T03:40:35.797 INFO:teuthology.orchestra.run.smithi115.stdout: "cap-": 17, 2024-09-12T03:40:35.797 INFO:teuthology.orchestra.run.smithi115.stdout: "rss": 56540, 2024-09-12T03:40:35.798 INFO:teuthology.orchestra.run.smithi115.stdout: "heap": 231712 2024-09-12T03:40:35.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.798 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_server": { 2024-09-12T03:40:35.798 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_client_request": 2031, 2024-09-12T03:40:35.798 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_server_request": 1, 2024-09-12T03:40:35.798 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_request": 773, 2024-09-12T03:40:35.798 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_session": 35, 2024-09-12T03:40:35.798 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_peer_request": 2, 2024-09-12T03:40:35.798 INFO:teuthology.orchestra.run.smithi115.stdout: "req_create_latency": { 2024-09-12T03:40:35.798 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 5, 2024-09-12T03:40:35.798 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.003415138, 2024-09-12T03:40:35.798 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000683027 2024-09-12T03:40:35.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.799 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getattr_latency": { 2024-09-12T03:40:35.799 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 114, 2024-09-12T03:40:35.799 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.019894023, 2024-09-12T03:40:35.799 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000174508 2024-09-12T03:40:35.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.799 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getfilelock_latency": { 2024-09-12T03:40:35.799 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:35.799 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:35.799 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:35.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.799 INFO:teuthology.orchestra.run.smithi115.stdout: "req_link_latency": { 2024-09-12T03:40:35.799 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:35.799 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:35.800 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:35.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.800 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookup_latency": { 2024-09-12T03:40:35.800 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 3, 2024-09-12T03:40:35.800 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000473928, 2024-09-12T03:40:35.800 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000157976 2024-09-12T03:40:35.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.800 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookuphash_latency": { 2024-09-12T03:40:35.800 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:35.800 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:35.800 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:35.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.800 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupino_latency": { 2024-09-12T03:40:35.801 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:35.801 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:35.801 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:35.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.801 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupname_latency": { 2024-09-12T03:40:35.801 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:35.801 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:35.801 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:35.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.801 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupparent_latency": { 2024-09-12T03:40:35.801 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:35.801 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:35.801 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:35.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.802 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupsnap_latency": { 2024-09-12T03:40:35.802 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:35.802 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:35.802 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:35.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.802 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lssnap_latency": { 2024-09-12T03:40:35.802 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 11, 2024-09-12T03:40:35.802 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.001414132, 2024-09-12T03:40:35.803 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000128557 2024-09-12T03:40:35.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.803 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mkdir_latency": { 2024-09-12T03:40:35.803 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 8, 2024-09-12T03:40:35.803 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.012296459, 2024-09-12T03:40:35.803 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.001537057 2024-09-12T03:40:35.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.804 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mknod_latency": { 2024-09-12T03:40:35.804 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:35.804 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:35.804 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:35.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.804 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mksnap_latency": { 2024-09-12T03:40:35.804 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 11, 2024-09-12T03:40:35.804 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.051631701, 2024-09-12T03:40:35.804 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.004693791 2024-09-12T03:40:35.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.804 INFO:teuthology.orchestra.run.smithi115.stdout: "req_open_latency": { 2024-09-12T03:40:35.804 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 599, 2024-09-12T03:40:35.805 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 3.036960844, 2024-09-12T03:40:35.805 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.005070051 2024-09-12T03:40:35.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.805 INFO:teuthology.orchestra.run.smithi115.stdout: "req_readdir_latency": { 2024-09-12T03:40:35.805 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 2, 2024-09-12T03:40:35.805 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.003960086, 2024-09-12T03:40:35.805 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.001980043 2024-09-12T03:40:35.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.805 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rename_latency": { 2024-09-12T03:40:35.805 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 2, 2024-09-12T03:40:35.805 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.002105031, 2024-09-12T03:40:35.805 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.001052515 2024-09-12T03:40:35.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.806 INFO:teuthology.orchestra.run.smithi115.stdout: "req_renamesnap_latency": { 2024-09-12T03:40:35.806 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:35.806 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:35.806 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:35.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.806 INFO:teuthology.orchestra.run.smithi115.stdout: "req_snapdiff_latency": { 2024-09-12T03:40:35.806 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:35.806 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:35.806 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:35.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.806 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmdir_latency": { 2024-09-12T03:40:35.806 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 2, 2024-09-12T03:40:35.807 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.011410677, 2024-09-12T03:40:35.807 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.005705338 2024-09-12T03:40:35.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.807 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmsnap_latency": { 2024-09-12T03:40:35.807 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 5, 2024-09-12T03:40:35.807 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.022354124, 2024-09-12T03:40:35.807 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.004470824 2024-09-12T03:40:35.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.807 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmxattr_latency": { 2024-09-12T03:40:35.807 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:35.807 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:35.807 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:35.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.808 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setattr_latency": { 2024-09-12T03:40:35.808 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 2, 2024-09-12T03:40:35.808 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.001187572, 2024-09-12T03:40:35.808 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000593786 2024-09-12T03:40:35.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.808 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setdirlayout_latency": { 2024-09-12T03:40:35.808 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:35.808 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:35.808 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:35.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.808 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setfilelock_latency": { 2024-09-12T03:40:35.808 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:35.809 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:35.809 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:35.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.809 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setlayout_latency": { 2024-09-12T03:40:35.809 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:35.809 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:35.809 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:35.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.809 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setxattr_latency": { 2024-09-12T03:40:35.809 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 8, 2024-09-12T03:40:35.809 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.021507526, 2024-09-12T03:40:35.809 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.002688440 2024-09-12T03:40:35.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.810 INFO:teuthology.orchestra.run.smithi115.stdout: "req_symlink_latency": { 2024-09-12T03:40:35.810 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:35.810 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:35.810 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:35.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.810 INFO:teuthology.orchestra.run.smithi115.stdout: "req_unlink_latency": { 2024-09-12T03:40:35.810 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1, 2024-09-12T03:40:35.810 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.001952775, 2024-09-12T03:40:35.810 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.001952775 2024-09-12T03:40:35.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.810 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:40:35.810 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:40:35.811 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getvxattr_latency": { 2024-09-12T03:40:35.811 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:35.811 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:35.811 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:35.811 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T03:40:35.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.811 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_sessions": { 2024-09-12T03:40:35.811 INFO:teuthology.orchestra.run.smithi115.stdout: "session_count": 2, 2024-09-12T03:40:35.811 INFO:teuthology.orchestra.run.smithi115.stdout: "session_add": 6, 2024-09-12T03:40:35.811 INFO:teuthology.orchestra.run.smithi115.stdout: "session_remove": 4, 2024-09-12T03:40:35.811 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_open": 2, 2024-09-12T03:40:35.811 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_stale": 0, 2024-09-12T03:40:35.812 INFO:teuthology.orchestra.run.smithi115.stdout: "total_load": 116, 2024-09-12T03:40:35.812 INFO:teuthology.orchestra.run.smithi115.stdout: "average_load": 58, 2024-09-12T03:40:35.812 INFO:teuthology.orchestra.run.smithi115.stdout: "avg_session_uptime": 656, 2024-09-12T03:40:35.812 INFO:teuthology.orchestra.run.smithi115.stdout: "mdthresh_evicted": 0 2024-09-12T03:40:35.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.812 INFO:teuthology.orchestra.run.smithi115.stdout: "mempool": { 2024-09-12T03:40:35.812 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:40:35.812 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_items": 0, 2024-09-12T03:40:35.812 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:40:35.812 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:40:35.812 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:40:35.812 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:40:35.812 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:40:35.813 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:40:35.813 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:40:35.813 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:40:35.813 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:40:35.813 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:40:35.813 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:40:35.813 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:40:35.813 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:40:35.813 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_items": 0, 2024-09-12T03:40:35.813 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:40:35.813 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_items": 0, 2024-09-12T03:40:35.813 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:40:35.814 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:40:35.814 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:40:35.814 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:40:35.814 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:40:35.814 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:40:35.814 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:40:35.814 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_items": 0, 2024-09-12T03:40:35.814 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:40:35.814 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:40:35.814 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:40:35.814 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_items": 0, 2024-09-12T03:40:35.814 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_bytes": 0, 2024-09-12T03:40:35.814 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_items": 0, 2024-09-12T03:40:35.815 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:40:35.815 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:40:35.815 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:40:35.815 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:40:35.815 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_bytes": 330706, 2024-09-12T03:40:35.815 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_items": 127, 2024-09-12T03:40:35.815 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:40:35.815 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_items": 0, 2024-09-12T03:40:35.815 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_bytes": 0, 2024-09-12T03:40:35.815 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_items": 0, 2024-09-12T03:40:35.815 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:40:35.815 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_items": 0, 2024-09-12T03:40:35.815 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:40:35.815 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_items": 0, 2024-09-12T03:40:35.816 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_bytes": 9896, 2024-09-12T03:40:35.816 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_items": 200, 2024-09-12T03:40:35.816 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:40:35.816 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:40:35.816 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_bytes": 0, 2024-09-12T03:40:35.816 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_items": 0, 2024-09-12T03:40:35.816 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_bytes": 170276, 2024-09-12T03:40:35.816 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_items": 1049, 2024-09-12T03:40:35.816 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_bytes": 0, 2024-09-12T03:40:35.816 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_items": 0, 2024-09-12T03:40:35.816 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_bytes": 0, 2024-09-12T03:40:35.816 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_items": 0 2024-09-12T03:40:35.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.817 INFO:teuthology.orchestra.run.smithi115.stdout: "objecter": { 2024-09-12T03:40:35.817 INFO:teuthology.orchestra.run.smithi115.stdout: "op_active": 0, 2024-09-12T03:40:35.817 INFO:teuthology.orchestra.run.smithi115.stdout: "op_laggy": 0, 2024-09-12T03:40:35.817 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send": 4232, 2024-09-12T03:40:35.817 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send_bytes": 22168148, 2024-09-12T03:40:35.817 INFO:teuthology.orchestra.run.smithi115.stdout: "op_resend": 37, 2024-09-12T03:40:35.817 INFO:teuthology.orchestra.run.smithi115.stdout: "op_reply": 4232, 2024-09-12T03:40:35.817 INFO:teuthology.orchestra.run.smithi115.stdout: "op_latency": { 2024-09-12T03:40:35.817 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 4232, 2024-09-12T03:40:35.817 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 26.017536553, 2024-09-12T03:40:35.817 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.006147811 2024-09-12T03:40:35.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.818 INFO:teuthology.orchestra.run.smithi115.stdout: "op_inflight": 0, 2024-09-12T03:40:35.818 INFO:teuthology.orchestra.run.smithi115.stdout: "oplen_avg": { 2024-09-12T03:40:35.818 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 4232, 2024-09-12T03:40:35.818 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 4980 2024-09-12T03:40:35.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.818 INFO:teuthology.orchestra.run.smithi115.stdout: "op": 4232, 2024-09-12T03:40:35.818 INFO:teuthology.orchestra.run.smithi115.stdout: "op_r": 1, 2024-09-12T03:40:35.818 INFO:teuthology.orchestra.run.smithi115.stdout: "op_w": 4231, 2024-09-12T03:40:35.818 INFO:teuthology.orchestra.run.smithi115.stdout: "op_rmw": 0, 2024-09-12T03:40:35.818 INFO:teuthology.orchestra.run.smithi115.stdout: "op_pg": 0, 2024-09-12T03:40:35.818 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_stat": 327, 2024-09-12T03:40:35.818 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_create": 30, 2024-09-12T03:40:35.818 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_read": 1, 2024-09-12T03:40:35.819 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_write": 2423, 2024-09-12T03:40:35.819 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writefull": 180, 2024-09-12T03:40:35.819 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writesame": 0, 2024-09-12T03:40:35.819 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_append": 0, 2024-09-12T03:40:35.819 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_zero": 0, 2024-09-12T03:40:35.819 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_truncate": 0, 2024-09-12T03:40:35.819 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_delete": 523, 2024-09-12T03:40:35.819 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_mapext": 0, 2024-09-12T03:40:35.819 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_sparse_read": 0, 2024-09-12T03:40:35.819 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_clonerange": 0, 2024-09-12T03:40:35.819 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_getxattr": 0, 2024-09-12T03:40:35.819 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_setxattr": 55, 2024-09-12T03:40:35.819 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:40:35.820 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_rmxattr": 0, 2024-09-12T03:40:35.820 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:40:35.820 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_call": 0, 2024-09-12T03:40:35.820 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_watch": 0, 2024-09-12T03:40:35.820 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_notify": 0, 2024-09-12T03:40:35.820 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:40:35.820 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls": 0, 2024-09-12T03:40:35.820 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:40:35.820 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_other": 599, 2024-09-12T03:40:35.820 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_active": 0, 2024-09-12T03:40:35.820 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_send": 0, 2024-09-12T03:40:35.820 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_resend": 0, 2024-09-12T03:40:35.820 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_ping": 0, 2024-09-12T03:40:35.821 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_active": 0, 2024-09-12T03:40:35.821 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_send": 0, 2024-09-12T03:40:35.821 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_resend": 0, 2024-09-12T03:40:35.821 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_active": 0, 2024-09-12T03:40:35.821 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_send": 0, 2024-09-12T03:40:35.821 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_resend": 0, 2024-09-12T03:40:35.821 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_active": 0, 2024-09-12T03:40:35.821 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_send": 0, 2024-09-12T03:40:35.821 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_resend": 0, 2024-09-12T03:40:35.821 INFO:teuthology.orchestra.run.smithi115.stdout: "command_active": 0, 2024-09-12T03:40:35.822 INFO:teuthology.orchestra.run.smithi115.stdout: "command_send": 0, 2024-09-12T03:40:35.822 INFO:teuthology.orchestra.run.smithi115.stdout: "command_resend": 0, 2024-09-12T03:40:35.822 INFO:teuthology.orchestra.run.smithi115.stdout: "map_epoch": 103, 2024-09-12T03:40:35.822 INFO:teuthology.orchestra.run.smithi115.stdout: "map_full": 0, 2024-09-12T03:40:35.822 INFO:teuthology.orchestra.run.smithi115.stdout: "map_inc": 21, 2024-09-12T03:40:35.822 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_sessions": 12, 2024-09-12T03:40:35.822 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_open": 12, 2024-09-12T03:40:35.822 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_close": 0, 2024-09-12T03:40:35.822 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_laggy": 0, 2024-09-12T03:40:35.822 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_wr": 826, 2024-09-12T03:40:35.822 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_rd": 0, 2024-09-12T03:40:35.822 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_del": 16 2024-09-12T03:40:35.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.823 INFO:teuthology.orchestra.run.smithi115.stdout: "oft": { 2024-09-12T03:40:35.823 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_objs": 1, 2024-09-12T03:40:35.823 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:40:35.823 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_updates": 0, 2024-09-12T03:40:35.823 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_removes": 0 2024-09-12T03:40:35.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.823 INFO:teuthology.orchestra.run.smithi115.stdout: "osd-slow-ops": { 2024-09-12T03:40:35.823 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_ops_count": 0 2024-09-12T03:40:35.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.823 INFO:teuthology.orchestra.run.smithi115.stdout: "purge_queue": { 2024-09-12T03:40:35.823 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops": 0, 2024-09-12T03:40:35.823 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops_high_water": 2, 2024-09-12T03:40:35.823 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing": 0, 2024-09-12T03:40:35.824 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_high_water": 1, 2024-09-12T03:40:35.824 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed_ops": 2, 2024-09-12T03:40:35.824 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed": 1, 2024-09-12T03:40:35.824 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_item_in_journal": 0 2024-09-12T03:40:35.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.824 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:40:35.824 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-12T03:40:35.824 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-09-12T03:40:35.824 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-12T03:40:35.824 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 9343, 2024-09-12T03:40:35.824 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 3790243, 2024-09-12T03:40:35.824 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-12T03:40:35.824 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 9343, 2024-09-12T03:40:35.825 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-09-12T03:40:35.825 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-09-12T03:40:35.825 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 9343, 2024-09-12T03:40:35.825 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 3790243, 2024-09-12T03:40:35.825 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-12T03:40:35.825 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:35.825 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:35.825 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:35.825 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T03:40:35.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.825 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_bytes": { 2024-09-12T03:40:35.825 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-12T03:40:35.825 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-09-12T03:40:35.825 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-12T03:40:35.826 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-09-12T03:40:35.826 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-09-12T03:40:35.826 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-12T03:40:35.826 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-09-12T03:40:35.826 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 4232, 2024-09-12T03:40:35.826 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 22168305, 2024-09-12T03:40:35.826 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 4232, 2024-09-12T03:40:35.826 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 22168305, 2024-09-12T03:40:35.826 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-12T03:40:35.826 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:35.826 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:35.826 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:35.826 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T03:40:35.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.827 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_ops": { 2024-09-12T03:40:35.827 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-12T03:40:35.827 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 1024, 2024-09-12T03:40:35.827 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-12T03:40:35.827 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-09-12T03:40:35.827 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-09-12T03:40:35.827 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-12T03:40:35.827 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-09-12T03:40:35.827 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 4232, 2024-09-12T03:40:35.827 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 4232, 2024-09-12T03:40:35.827 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 4232, 2024-09-12T03:40:35.827 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 4232, 2024-09-12T03:40:35.827 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-12T03:40:35.828 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:35.828 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:35.828 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:35.828 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T03:40:35.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.828 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:40:35.828 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-12T03:40:35.828 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2024-09-12T03:40:35.828 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-12T03:40:35.828 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 1, 2024-09-12T03:40:35.828 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 157, 2024-09-12T03:40:35.828 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-12T03:40:35.828 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 1, 2024-09-12T03:40:35.829 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-09-12T03:40:35.829 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-09-12T03:40:35.829 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 1, 2024-09-12T03:40:35.829 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 157, 2024-09-12T03:40:35.829 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-12T03:40:35.829 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:35.829 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:35.829 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:35.829 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T03:40:35.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:35.829 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle-0x55a8866d2960": { 2024-09-12T03:40:35.829 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-12T03:40:35.829 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2024-09-12T03:40:35.830 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-12T03:40:35.830 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 2481, 2024-09-12T03:40:35.830 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 21075699, 2024-09-12T03:40:35.830 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-12T03:40:35.830 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 2481, 2024-09-12T03:40:35.830 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-09-12T03:40:35.830 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-09-12T03:40:35.830 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 2422, 2024-09-12T03:40:35.830 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 21075699, 2024-09-12T03:40:35.830 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-12T03:40:35.830 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:35.830 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:35.830 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:35.830 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T03:40:35.831 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T03:40:35.831 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T03:40:36.670 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-09-12T03:40:36.670 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-09-12T03:40:36.670 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=10 2024-09-12T03:40:36.670 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-09-12T03:40:36.671 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-09-12T03:40:36.671 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2024-09-12T03:40:36.671 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=5 2024-09-12T03:40:36.671 DEBUG:tasks.check_counter:Getting stats from e 2024-09-12T03:40:36.671 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-12T03:40:36.908 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:36 smithi115 ceph-mon[34801]: pgmap v353: 129 pgs: 129 active+clean; 5.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:40:36.933 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:40:37.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:36 smithi195 ceph-mon[34786]: pgmap v353: 129 pgs: 129 active+clean; 5.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:40:37.271 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:36 smithi064 ceph-mon[30952]: pgmap v353: 129 pgs: 129 active+clean; 5.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:40:37.475 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T03:40:37.476 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:40:37.476 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 262, 2024-09-12T03:40:37.476 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 249, 2024-09-12T03:40:37.476 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 1620770, 2024-09-12T03:40:37.476 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 144020, 2024-09-12T03:40:37.476 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 5, 2024-09-12T03:40:37.476 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 1, 2024-09-12T03:40:37.476 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.047924014, 2024-09-12T03:40:37.476 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.017200709, 2024-09-12T03:40:37.477 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.045575360, 2024-09-12T03:40:37.477 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.000020245, 2024-09-12T03:40:37.477 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:40:37.477 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 248, 2024-09-12T03:40:37.477 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.015492692, 2024-09-12T03:40:37.477 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000062470 2024-09-12T03:40:37.477 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.477 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:40:37.477 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:37.477 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:37.477 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:37.478 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.478 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 1619936, 2024-09-12T03:40:37.478 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 143696 2024-09-12T03:40:37.478 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.478 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:40:37.478 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 697, 2024-09-12T03:40:37.478 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 868, 2024-09-12T03:40:37.478 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 173320, 2024-09-12T03:40:37.478 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 678783, 2024-09-12T03:40:37.478 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 7, 2024-09-12T03:40:37.478 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 5, 2024-09-12T03:40:37.479 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.078267628, 2024-09-12T03:40:37.479 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.028062432, 2024-09-12T03:40:37.479 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.018131116, 2024-09-12T03:40:37.479 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.023418754, 2024-09-12T03:40:37.479 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:40:37.479 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 866, 2024-09-12T03:40:37.479 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.047935516, 2024-09-12T03:40:37.479 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000055352 2024-09-12T03:40:37.479 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.479 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:40:37.479 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:37.479 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:37.480 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:37.480 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.480 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-09-12T03:40:37.480 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 459040 2024-09-12T03:40:37.480 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.480 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:40:37.480 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 1573, 2024-09-12T03:40:37.480 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 1575, 2024-09-12T03:40:37.480 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 472493, 2024-09-12T03:40:37.480 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 500272, 2024-09-12T03:40:37.481 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 4, 2024-09-12T03:40:37.481 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 3, 2024-09-12T03:40:37.481 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.164309773, 2024-09-12T03:40:37.481 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.058911277, 2024-09-12T03:40:37.481 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.040932638, 2024-09-12T03:40:37.481 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.049089883, 2024-09-12T03:40:37.481 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:40:37.481 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1573, 2024-09-12T03:40:37.481 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.071143599, 2024-09-12T03:40:37.481 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000045227 2024-09-12T03:40:37.481 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.482 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:40:37.482 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:37.482 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:37.482 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:37.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.482 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:40:37.482 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:40:37.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.482 INFO:teuthology.orchestra.run.smithi115.stdout: "cct": { 2024-09-12T03:40:37.482 INFO:teuthology.orchestra.run.smithi115.stdout: "total_workers": 1, 2024-09-12T03:40:37.482 INFO:teuthology.orchestra.run.smithi115.stdout: "unhealthy_workers": 0 2024-09-12T03:40:37.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.483 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-MDSRank": { 2024-09-12T03:40:37.483 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2024-09-12T03:40:37.483 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2024-09-12T03:40:37.483 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 3774, 2024-09-12T03:40:37.483 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.110336637, 2024-09-12T03:40:37.483 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000029235 2024-09-12T03:40:37.483 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T03:40:37.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.483 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-PurgeQueue": { 2024-09-12T03:40:37.484 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2024-09-12T03:40:37.484 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2024-09-12T03:40:37.486 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:37.486 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:37.486 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:37.486 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T03:40:37.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.487 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-09-12T03:40:37.487 INFO:teuthology.orchestra.run.smithi115.stdout: "request": 0, 2024-09-12T03:40:37.487 INFO:teuthology.orchestra.run.smithi115.stdout: "reply": 0, 2024-09-12T03:40:37.487 INFO:teuthology.orchestra.run.smithi115.stdout: "reply_latency": { 2024-09-12T03:40:37.487 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:37.487 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:37.487 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:37.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.487 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_reply": 0, 2024-09-12T03:40:37.488 INFO:teuthology.orchestra.run.smithi115.stdout: "forward": 0, 2024-09-12T03:40:37.488 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_complete": 0, 2024-09-12T03:40:37.488 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_keys": 0, 2024-09-12T03:40:37.488 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_commit": 0, 2024-09-12T03:40:37.488 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_split": 0, 2024-09-12T03:40:37.488 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_merge": 0, 2024-09-12T03:40:37.488 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes": 0, 2024-09-12T03:40:37.488 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_top": 0, 2024-09-12T03:40:37.488 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_bottom": 0, 2024-09-12T03:40:37.488 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pin_tail": 0, 2024-09-12T03:40:37.489 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pinned": 0, 2024-09-12T03:40:37.489 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_expired": 0, 2024-09-12T03:40:37.489 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_with_caps": 0, 2024-09-12T03:40:37.489 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": 0, 2024-09-12T03:40:37.489 INFO:teuthology.orchestra.run.smithi115.stdout: "subtrees": 1, 2024-09-12T03:40:37.489 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse": 0, 2024-09-12T03:40:37.489 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_hit": 0, 2024-09-12T03:40:37.489 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_forward": 0, 2024-09-12T03:40:37.489 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_discover": 0, 2024-09-12T03:40:37.489 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:40:37.489 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_remote_ino": 0, 2024-09-12T03:40:37.489 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_lock": 0, 2024-09-12T03:40:37.490 INFO:teuthology.orchestra.run.smithi115.stdout: "load_cent": 0, 2024-09-12T03:40:37.490 INFO:teuthology.orchestra.run.smithi115.stdout: "q": 0, 2024-09-12T03:40:37.490 INFO:teuthology.orchestra.run.smithi115.stdout: "exported": 0, 2024-09-12T03:40:37.490 INFO:teuthology.orchestra.run.smithi115.stdout: "exported_inodes": 0, 2024-09-12T03:40:37.490 INFO:teuthology.orchestra.run.smithi115.stdout: "imported": 0, 2024-09-12T03:40:37.490 INFO:teuthology.orchestra.run.smithi115.stdout: "imported_inodes": 0, 2024-09-12T03:40:37.490 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_dir_fetch": 0, 2024-09-12T03:40:37.490 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:40:37.490 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_peer_discover": 0, 2024-09-12T03:40:37.490 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rfiles": 0, 2024-09-12T03:40:37.491 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rbytes": 0, 2024-09-12T03:40:37.491 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rsnaps": 0, 2024-09-12T03:40:37.491 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:40:37.491 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_set_tag": 0, 2024-09-12T03:40:37.491 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:40:37.491 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:40:37.491 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:40:37.491 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:40:37.491 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:40:37.491 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_file_inodes": 0, 2024-09-12T03:40:37.491 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:40:37.491 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T03:40:37.491 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_grant": 0, 2024-09-12T03:40:37.492 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:40:37.492 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:40:37.492 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:40:37.492 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps": 0, 2024-09-12T03:40:37.492 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps_dirty": 0, 2024-09-12T03:40:37.492 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_cap_release": 0, 2024-09-12T03:40:37.492 INFO:teuthology.orchestra.run.smithi115.stdout: "process_request_cap_release": 0 2024-09-12T03:40:37.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.492 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cache": { 2024-09-12T03:40:37.492 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update": 0, 2024-09-12T03:40:37.492 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update_receipt": 0, 2024-09-12T03:40:37.493 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_try_discover": 0, 2024-09-12T03:40:37.493 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_send_discover": 0, 2024-09-12T03:40:37.493 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_handle_discover": 0, 2024-09-12T03:40:37.493 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays": 0, 2024-09-12T03:40:37.493 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_delayed": 0, 2024-09-12T03:40:37.493 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:40:37.493 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_created": 0, 2024-09-12T03:40:37.493 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_enqueued": 0, 2024-09-12T03:40:37.493 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_reintegrated": 0, 2024-09-12T03:40:37.493 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_migrated": 0, 2024-09-12T03:40:37.493 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_processing": 0, 2024-09-12T03:40:37.493 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:40:37.493 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:40:37.494 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_started": 0, 2024-09-12T03:40:37.494 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_completed": 0, 2024-09-12T03:40:37.494 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_quiesce_path": 0, 2024-09-12T03:40:37.494 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_quiesce_inode": 0, 2024-09-12T03:40:37.494 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:40:37.494 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_exportdir": 0, 2024-09-12T03:40:37.494 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_flush": 0, 2024-09-12T03:40:37.494 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:40:37.494 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragstats": 0, 2024-09-12T03:40:37.494 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_inodestats": 0 2024-09-12T03:40:37.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.494 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_log": { 2024-09-12T03:40:37.494 INFO:teuthology.orchestra.run.smithi115.stdout: "evlrg": 0, 2024-09-12T03:40:37.494 INFO:teuthology.orchestra.run.smithi115.stdout: "evadd": 0, 2024-09-12T03:40:37.495 INFO:teuthology.orchestra.run.smithi115.stdout: "evex": 0, 2024-09-12T03:40:37.495 INFO:teuthology.orchestra.run.smithi115.stdout: "evtrm": 0, 2024-09-12T03:40:37.495 INFO:teuthology.orchestra.run.smithi115.stdout: "ev": 8, 2024-09-12T03:40:37.495 INFO:teuthology.orchestra.run.smithi115.stdout: "evexg": 0, 2024-09-12T03:40:37.495 INFO:teuthology.orchestra.run.smithi115.stdout: "evexd": 0, 2024-09-12T03:40:37.495 INFO:teuthology.orchestra.run.smithi115.stdout: "segadd": 0, 2024-09-12T03:40:37.495 INFO:teuthology.orchestra.run.smithi115.stdout: "segex": 0, 2024-09-12T03:40:37.495 INFO:teuthology.orchestra.run.smithi115.stdout: "segtrm": 0, 2024-09-12T03:40:37.495 INFO:teuthology.orchestra.run.smithi115.stdout: "seg": 2, 2024-09-12T03:40:37.495 INFO:teuthology.orchestra.run.smithi115.stdout: "segmjr": 2, 2024-09-12T03:40:37.495 INFO:teuthology.orchestra.run.smithi115.stdout: "segexg": 0, 2024-09-12T03:40:37.495 INFO:teuthology.orchestra.run.smithi115.stdout: "segexd": 0, 2024-09-12T03:40:37.495 INFO:teuthology.orchestra.run.smithi115.stdout: "expos": 4194304, 2024-09-12T03:40:37.495 INFO:teuthology.orchestra.run.smithi115.stdout: "wrpos": 4196848, 2024-09-12T03:40:37.496 INFO:teuthology.orchestra.run.smithi115.stdout: "rdpos": 4196707, 2024-09-12T03:40:37.496 INFO:teuthology.orchestra.run.smithi115.stdout: "jlat": { 2024-09-12T03:40:37.496 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:37.496 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:37.496 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:37.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.496 INFO:teuthology.orchestra.run.smithi115.stdout: "replayed": 8 2024-09-12T03:40:37.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.496 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_mem": { 2024-09-12T03:40:37.496 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 3, 2024-09-12T03:40:37.496 INFO:teuthology.orchestra.run.smithi115.stdout: "ino+": 3, 2024-09-12T03:40:37.496 INFO:teuthology.orchestra.run.smithi115.stdout: "ino-": 0, 2024-09-12T03:40:37.496 INFO:teuthology.orchestra.run.smithi115.stdout: "dir": 1, 2024-09-12T03:40:37.497 INFO:teuthology.orchestra.run.smithi115.stdout: "dir+": 1, 2024-09-12T03:40:37.497 INFO:teuthology.orchestra.run.smithi115.stdout: "dir-": 0, 2024-09-12T03:40:37.497 INFO:teuthology.orchestra.run.smithi115.stdout: "dn": 0, 2024-09-12T03:40:37.497 INFO:teuthology.orchestra.run.smithi115.stdout: "dn+": 0, 2024-09-12T03:40:37.497 INFO:teuthology.orchestra.run.smithi115.stdout: "dn-": 0, 2024-09-12T03:40:37.497 INFO:teuthology.orchestra.run.smithi115.stdout: "cap": 0, 2024-09-12T03:40:37.497 INFO:teuthology.orchestra.run.smithi115.stdout: "cap+": 0, 2024-09-12T03:40:37.497 INFO:teuthology.orchestra.run.smithi115.stdout: "cap-": 0, 2024-09-12T03:40:37.497 INFO:teuthology.orchestra.run.smithi115.stdout: "rss": 48836, 2024-09-12T03:40:37.497 INFO:teuthology.orchestra.run.smithi115.stdout: "heap": 215328 2024-09-12T03:40:37.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.497 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_server": { 2024-09-12T03:40:37.498 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_client_request": 0, 2024-09-12T03:40:37.498 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_server_request": 0, 2024-09-12T03:40:37.498 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_request": 0, 2024-09-12T03:40:37.498 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_session": 0, 2024-09-12T03:40:37.498 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_peer_request": 0, 2024-09-12T03:40:37.499 INFO:teuthology.orchestra.run.smithi115.stdout: "req_create_latency": { 2024-09-12T03:40:37.499 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:37.499 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:37.499 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:37.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.499 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getattr_latency": { 2024-09-12T03:40:37.499 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:37.499 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:37.499 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:37.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.499 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getfilelock_latency": { 2024-09-12T03:40:37.499 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:37.499 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:37.500 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:37.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.500 INFO:teuthology.orchestra.run.smithi115.stdout: "req_link_latency": { 2024-09-12T03:40:37.500 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:37.500 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:37.500 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:37.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.500 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookup_latency": { 2024-09-12T03:40:37.500 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:37.500 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:37.500 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:37.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.501 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookuphash_latency": { 2024-09-12T03:40:37.501 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:37.501 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:37.501 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:37.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.501 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupino_latency": { 2024-09-12T03:40:37.501 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:37.501 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:37.501 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:37.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.502 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupname_latency": { 2024-09-12T03:40:37.502 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:37.502 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:37.502 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:37.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.502 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupparent_latency": { 2024-09-12T03:40:37.502 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:37.502 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:37.502 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:37.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.502 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupsnap_latency": { 2024-09-12T03:40:37.502 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:37.502 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:37.503 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:37.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.503 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lssnap_latency": { 2024-09-12T03:40:37.503 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:37.503 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:37.503 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:37.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.503 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mkdir_latency": { 2024-09-12T03:40:37.503 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:37.504 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:37.504 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:37.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.504 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mknod_latency": { 2024-09-12T03:40:37.505 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:37.505 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:37.505 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:37.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.505 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mksnap_latency": { 2024-09-12T03:40:37.505 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:37.505 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:37.505 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:37.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.506 INFO:teuthology.orchestra.run.smithi115.stdout: "req_open_latency": { 2024-09-12T03:40:37.506 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:37.506 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:37.506 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:37.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.506 INFO:teuthology.orchestra.run.smithi115.stdout: "req_readdir_latency": { 2024-09-12T03:40:37.506 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:37.506 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:37.506 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:37.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.507 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rename_latency": { 2024-09-12T03:40:37.507 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:37.507 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:37.507 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:37.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.507 INFO:teuthology.orchestra.run.smithi115.stdout: "req_renamesnap_latency": { 2024-09-12T03:40:37.507 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:37.507 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:37.507 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:37.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.507 INFO:teuthology.orchestra.run.smithi115.stdout: "req_snapdiff_latency": { 2024-09-12T03:40:37.508 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:37.508 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:37.508 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:37.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.509 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmdir_latency": { 2024-09-12T03:40:37.509 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:37.509 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:37.509 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:37.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.509 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmsnap_latency": { 2024-09-12T03:40:37.509 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:37.509 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:37.509 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:37.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.510 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmxattr_latency": { 2024-09-12T03:40:37.510 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:37.510 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:37.510 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:37.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.510 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setattr_latency": { 2024-09-12T03:40:37.510 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:37.510 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:37.510 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:37.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.510 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setdirlayout_latency": { 2024-09-12T03:40:37.511 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:37.511 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:37.511 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:37.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.511 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setfilelock_latency": { 2024-09-12T03:40:37.511 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:37.511 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:37.511 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:37.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.511 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setlayout_latency": { 2024-09-12T03:40:37.511 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:37.511 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:37.511 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:37.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.512 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setxattr_latency": { 2024-09-12T03:40:37.512 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:37.512 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:37.512 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:37.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.512 INFO:teuthology.orchestra.run.smithi115.stdout: "req_symlink_latency": { 2024-09-12T03:40:37.512 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:37.512 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:37.512 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:37.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.512 INFO:teuthology.orchestra.run.smithi115.stdout: "req_unlink_latency": { 2024-09-12T03:40:37.512 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:37.513 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:37.513 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:37.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.513 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:40:37.513 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:40:37.513 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getvxattr_latency": { 2024-09-12T03:40:37.513 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:37.513 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:37.513 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:37.513 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T03:40:37.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.514 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_sessions": { 2024-09-12T03:40:37.514 INFO:teuthology.orchestra.run.smithi115.stdout: "session_count": 2, 2024-09-12T03:40:37.514 INFO:teuthology.orchestra.run.smithi115.stdout: "session_add": 4, 2024-09-12T03:40:37.514 INFO:teuthology.orchestra.run.smithi115.stdout: "session_remove": 2, 2024-09-12T03:40:37.514 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_open": 3, 2024-09-12T03:40:37.514 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_stale": 0, 2024-09-12T03:40:37.514 INFO:teuthology.orchestra.run.smithi115.stdout: "total_load": 0, 2024-09-12T03:40:37.514 INFO:teuthology.orchestra.run.smithi115.stdout: "average_load": 0, 2024-09-12T03:40:37.514 INFO:teuthology.orchestra.run.smithi115.stdout: "avg_session_uptime": 2218, 2024-09-12T03:40:37.514 INFO:teuthology.orchestra.run.smithi115.stdout: "mdthresh_evicted": 0 2024-09-12T03:40:37.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.514 INFO:teuthology.orchestra.run.smithi115.stdout: "mempool": { 2024-09-12T03:40:37.515 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:40:37.515 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_items": 0, 2024-09-12T03:40:37.515 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:40:37.515 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:40:37.515 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:40:37.515 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:40:37.515 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:40:37.515 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:40:37.515 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:40:37.515 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:40:37.515 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:40:37.516 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:40:37.516 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:40:37.516 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:40:37.516 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:40:37.516 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_items": 0, 2024-09-12T03:40:37.516 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:40:37.516 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_items": 0, 2024-09-12T03:40:37.516 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:40:37.516 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:40:37.516 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:40:37.517 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:40:37.517 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:40:37.517 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:40:37.517 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:40:37.517 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_items": 0, 2024-09-12T03:40:37.517 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:40:37.517 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:40:37.517 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:40:37.517 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_items": 0, 2024-09-12T03:40:37.517 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_bytes": 0, 2024-09-12T03:40:37.517 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_items": 0, 2024-09-12T03:40:37.517 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:40:37.517 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:40:37.517 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:40:37.518 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:40:37.518 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_bytes": 189427, 2024-09-12T03:40:37.518 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_items": 53, 2024-09-12T03:40:37.518 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:40:37.518 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_items": 0, 2024-09-12T03:40:37.518 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_bytes": 0, 2024-09-12T03:40:37.518 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_items": 0, 2024-09-12T03:40:37.518 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:40:37.518 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_items": 0, 2024-09-12T03:40:37.519 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:40:37.519 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_items": 0, 2024-09-12T03:40:37.519 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_bytes": 9896, 2024-09-12T03:40:37.519 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_items": 200, 2024-09-12T03:40:37.519 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:40:37.519 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:40:37.519 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_bytes": 0, 2024-09-12T03:40:37.519 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_items": 0, 2024-09-12T03:40:37.519 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_bytes": 8512, 2024-09-12T03:40:37.520 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_items": 13, 2024-09-12T03:40:37.520 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_bytes": 0, 2024-09-12T03:40:37.520 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_items": 0, 2024-09-12T03:40:37.520 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_bytes": 0, 2024-09-12T03:40:37.520 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_items": 0 2024-09-12T03:40:37.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.520 INFO:teuthology.orchestra.run.smithi115.stdout: "objecter": { 2024-09-12T03:40:37.520 INFO:teuthology.orchestra.run.smithi115.stdout: "op_active": 0, 2024-09-12T03:40:37.520 INFO:teuthology.orchestra.run.smithi115.stdout: "op_laggy": 0, 2024-09-12T03:40:37.520 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send": 2265, 2024-09-12T03:40:37.520 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send_bytes": 21, 2024-09-12T03:40:37.520 INFO:teuthology.orchestra.run.smithi115.stdout: "op_resend": 3, 2024-09-12T03:40:37.520 INFO:teuthology.orchestra.run.smithi115.stdout: "op_reply": 2265, 2024-09-12T03:40:37.520 INFO:teuthology.orchestra.run.smithi115.stdout: "op_latency": { 2024-09-12T03:40:37.521 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 2265, 2024-09-12T03:40:37.521 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 1.074980729, 2024-09-12T03:40:37.521 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000474605 2024-09-12T03:40:37.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.521 INFO:teuthology.orchestra.run.smithi115.stdout: "op_inflight": 0, 2024-09-12T03:40:37.521 INFO:teuthology.orchestra.run.smithi115.stdout: "oplen_avg": { 2024-09-12T03:40:37.521 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 2265, 2024-09-12T03:40:37.521 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 2266 2024-09-12T03:40:37.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.521 INFO:teuthology.orchestra.run.smithi115.stdout: "op": 2265, 2024-09-12T03:40:37.521 INFO:teuthology.orchestra.run.smithi115.stdout: "op_r": 2265, 2024-09-12T03:40:37.521 INFO:teuthology.orchestra.run.smithi115.stdout: "op_w": 0, 2024-09-12T03:40:37.521 INFO:teuthology.orchestra.run.smithi115.stdout: "op_rmw": 0, 2024-09-12T03:40:37.521 INFO:teuthology.orchestra.run.smithi115.stdout: "op_pg": 0, 2024-09-12T03:40:37.522 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_stat": 1471, 2024-09-12T03:40:37.522 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_create": 0, 2024-09-12T03:40:37.522 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_read": 792, 2024-09-12T03:40:37.522 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_write": 0, 2024-09-12T03:40:37.522 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writefull": 0, 2024-09-12T03:40:37.522 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writesame": 0, 2024-09-12T03:40:37.522 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_append": 0, 2024-09-12T03:40:37.522 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_zero": 0, 2024-09-12T03:40:37.522 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_truncate": 0, 2024-09-12T03:40:37.522 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_delete": 0, 2024-09-12T03:40:37.522 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_mapext": 0, 2024-09-12T03:40:37.522 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_sparse_read": 0, 2024-09-12T03:40:37.522 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_clonerange": 0, 2024-09-12T03:40:37.522 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_getxattr": 1, 2024-09-12T03:40:37.523 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_setxattr": 0, 2024-09-12T03:40:37.523 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:40:37.523 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_rmxattr": 0, 2024-09-12T03:40:37.523 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:40:37.523 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_call": 0, 2024-09-12T03:40:37.523 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_watch": 0, 2024-09-12T03:40:37.523 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_notify": 0, 2024-09-12T03:40:37.523 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:40:37.523 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls": 0, 2024-09-12T03:40:37.523 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:40:37.524 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_other": 0, 2024-09-12T03:40:37.524 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_active": 0, 2024-09-12T03:40:37.524 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_send": 0, 2024-09-12T03:40:37.524 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_resend": 0, 2024-09-12T03:40:37.524 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_ping": 0, 2024-09-12T03:40:37.524 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_active": 0, 2024-09-12T03:40:37.524 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_send": 0, 2024-09-12T03:40:37.524 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_resend": 0, 2024-09-12T03:40:37.524 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_active": 0, 2024-09-12T03:40:37.524 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_send": 0, 2024-09-12T03:40:37.524 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_resend": 0, 2024-09-12T03:40:37.524 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_active": 0, 2024-09-12T03:40:37.524 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_send": 0, 2024-09-12T03:40:37.525 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_resend": 0, 2024-09-12T03:40:37.525 INFO:teuthology.orchestra.run.smithi115.stdout: "command_active": 0, 2024-09-12T03:40:37.525 INFO:teuthology.orchestra.run.smithi115.stdout: "command_send": 0, 2024-09-12T03:40:37.525 INFO:teuthology.orchestra.run.smithi115.stdout: "command_resend": 0, 2024-09-12T03:40:37.525 INFO:teuthology.orchestra.run.smithi115.stdout: "map_epoch": 103, 2024-09-12T03:40:37.525 INFO:teuthology.orchestra.run.smithi115.stdout: "map_full": 0, 2024-09-12T03:40:37.525 INFO:teuthology.orchestra.run.smithi115.stdout: "map_inc": 15, 2024-09-12T03:40:37.525 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_sessions": 5, 2024-09-12T03:40:37.525 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_open": 5, 2024-09-12T03:40:37.525 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_close": 0, 2024-09-12T03:40:37.525 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_laggy": 0, 2024-09-12T03:40:37.525 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_wr": 0, 2024-09-12T03:40:37.525 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_rd": 2, 2024-09-12T03:40:37.526 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_del": 0 2024-09-12T03:40:37.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.526 INFO:teuthology.orchestra.run.smithi115.stdout: "oft": { 2024-09-12T03:40:37.526 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_objs": 0, 2024-09-12T03:40:37.526 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:40:37.526 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_updates": 0, 2024-09-12T03:40:37.526 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_removes": 0 2024-09-12T03:40:37.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.526 INFO:teuthology.orchestra.run.smithi115.stdout: "osd-slow-ops": { 2024-09-12T03:40:37.526 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_ops_count": 0 2024-09-12T03:40:37.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.526 INFO:teuthology.orchestra.run.smithi115.stdout: "purge_queue": { 2024-09-12T03:40:37.526 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops": 0, 2024-09-12T03:40:37.526 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T03:40:37.527 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing": 0, 2024-09-12T03:40:37.527 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_high_water": 0, 2024-09-12T03:40:37.527 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed_ops": 0, 2024-09-12T03:40:37.527 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed": 0, 2024-09-12T03:40:37.527 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_item_in_journal": 0 2024-09-12T03:40:37.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.527 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:40:37.527 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-12T03:40:37.527 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-09-12T03:40:37.527 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-12T03:40:37.527 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 2528, 2024-09-12T03:40:37.527 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 2022698, 2024-09-12T03:40:37.527 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-12T03:40:37.527 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 2528, 2024-09-12T03:40:37.528 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-09-12T03:40:37.528 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-09-12T03:40:37.528 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 2528, 2024-09-12T03:40:37.528 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 2022698, 2024-09-12T03:40:37.528 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-12T03:40:37.528 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:37.528 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:37.528 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:37.528 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T03:40:37.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.528 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_bytes": { 2024-09-12T03:40:37.528 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-12T03:40:37.528 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-09-12T03:40:37.528 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-12T03:40:37.529 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-09-12T03:40:37.529 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-09-12T03:40:37.529 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-12T03:40:37.529 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-09-12T03:40:37.529 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 2265, 2024-09-12T03:40:37.529 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 2549, 2024-09-12T03:40:37.529 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 2265, 2024-09-12T03:40:37.529 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 2549, 2024-09-12T03:40:37.529 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-12T03:40:37.529 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:37.529 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:37.530 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:37.530 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T03:40:37.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.530 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_ops": { 2024-09-12T03:40:37.530 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-12T03:40:37.530 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 1024, 2024-09-12T03:40:37.530 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-12T03:40:37.530 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-09-12T03:40:37.530 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-09-12T03:40:37.530 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-12T03:40:37.530 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-09-12T03:40:37.530 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 2265, 2024-09-12T03:40:37.530 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 2265, 2024-09-12T03:40:37.530 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 2265, 2024-09-12T03:40:37.531 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 2265, 2024-09-12T03:40:37.532 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-12T03:40:37.532 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:37.532 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:37.532 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:37.532 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T03:40:37.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.532 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:40:37.532 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-12T03:40:37.532 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2024-09-12T03:40:37.532 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-12T03:40:37.532 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-09-12T03:40:37.532 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-09-12T03:40:37.532 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-12T03:40:37.533 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-09-12T03:40:37.533 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-09-12T03:40:37.533 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-09-12T03:40:37.533 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 0, 2024-09-12T03:40:37.533 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 0, 2024-09-12T03:40:37.533 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-12T03:40:37.533 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:37.533 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:37.533 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:37.533 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T03:40:37.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:37.533 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle-0x55613581ab40": { 2024-09-12T03:40:37.533 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-12T03:40:37.533 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2024-09-12T03:40:37.534 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-12T03:40:37.534 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-09-12T03:40:37.534 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-09-12T03:40:37.534 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-12T03:40:37.534 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-09-12T03:40:37.534 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-09-12T03:40:37.534 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-09-12T03:40:37.534 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 0, 2024-09-12T03:40:37.534 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 0, 2024-09-12T03:40:37.534 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-12T03:40:37.534 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:37.534 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:37.534 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:37.535 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T03:40:37.535 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T03:40:37.535 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T03:40:38.257 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-12T03:40:38.258 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-12T03:40:38.258 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-09-12T03:40:38.258 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-09-12T03:40:38.258 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-09-12T03:40:38.258 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-09-12T03:40:38.258 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-09-12T03:40:38.258 DEBUG:tasks.check_counter:Getting stats from h 2024-09-12T03:40:38.258 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-12T03:40:38.506 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:40:39.073 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T03:40:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:40:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 1598, 2024-09-12T03:40:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 1142, 2024-09-12T03:40:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 1979793, 2024-09-12T03:40:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 569962, 2024-09-12T03:40:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 9, 2024-09-12T03:40:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 4, 2024-09-12T03:40:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.153536207, 2024-09-12T03:40:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.066703526, 2024-09-12T03:40:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.109086856, 2024-09-12T03:40:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.004384644, 2024-09-12T03:40:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:40:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1141, 2024-09-12T03:40:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.060466828, 2024-09-12T03:40:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000052994 2024-09-12T03:40:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:40:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 2087, 2024-09-12T03:40:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000548963, 2024-09-12T03:40:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000263 2024-09-12T03:40:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 1630448, 2024-09-12T03:40:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 152768 2024-09-12T03:40:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:40:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 81, 2024-09-12T03:40:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 114, 2024-09-12T03:40:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 26245, 2024-09-12T03:40:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 208792, 2024-09-12T03:40:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 14, 2024-09-12T03:40:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 5, 2024-09-12T03:40:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.011665396, 2024-09-12T03:40:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.003706832, 2024-09-12T03:40:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.007130523, 2024-09-12T03:40:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.000357468, 2024-09-12T03:40:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:40:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 113, 2024-09-12T03:40:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.015692530, 2024-09-12T03:40:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000138871 2024-09-12T03:40:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:40:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 116, 2024-09-12T03:40:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000020906, 2024-09-12T03:40:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000180 2024-09-12T03:40:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 2016, 2024-09-12T03:40:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 127152 2024-09-12T03:40:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:40:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 33, 2024-09-12T03:40:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 188, 2024-09-12T03:40:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 9720, 2024-09-12T03:40:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 346769, 2024-09-12T03:40:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 9, 2024-09-12T03:40:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 6, 2024-09-12T03:40:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.014268242, 2024-09-12T03:40:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.006894020, 2024-09-12T03:40:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.005753963, 2024-09-12T03:40:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.000238630, 2024-09-12T03:40:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:40:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 185, 2024-09-12T03:40:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.034477227, 2024-09-12T03:40:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000186363 2024-09-12T03:40:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:40:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 45, 2024-09-12T03:40:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000010431, 2024-09-12T03:40:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000231 2024-09-12T03:40:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 4736, 2024-09-12T03:40:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 335680 2024-09-12T03:40:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "cct": { 2024-09-12T03:40:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "total_workers": 1, 2024-09-12T03:40:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "unhealthy_workers": 0 2024-09-12T03:40:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-MDSRank": { 2024-09-12T03:40:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2024-09-12T03:40:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2024-09-12T03:40:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 63, 2024-09-12T03:40:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.001472366, 2024-09-12T03:40:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000023370 2024-09-12T03:40:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T03:40:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-PurgeQueue": { 2024-09-12T03:40:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2024-09-12T03:40:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2024-09-12T03:40:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 2, 2024-09-12T03:40:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000006691, 2024-09-12T03:40:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000003345 2024-09-12T03:40:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T03:40:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-09-12T03:40:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "request": 5, 2024-09-12T03:40:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "reply": 3, 2024-09-12T03:40:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "reply_latency": { 2024-09-12T03:40:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 3, 2024-09-12T03:40:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000548721, 2024-09-12T03:40:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000182907 2024-09-12T03:40:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_reply": 0, 2024-09-12T03:40:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "forward": 2, 2024-09-12T03:40:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_complete": 0, 2024-09-12T03:40:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_keys": 0, 2024-09-12T03:40:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_commit": 11, 2024-09-12T03:40:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_split": 0, 2024-09-12T03:40:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_merge": 0, 2024-09-12T03:40:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes": 16, 2024-09-12T03:40:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_top": 0, 2024-09-12T03:40:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_bottom": 16, 2024-09-12T03:40:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pin_tail": 0, 2024-09-12T03:40:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pinned": 13, 2024-09-12T03:40:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_expired": 4, 2024-09-12T03:40:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_with_caps": 0, 2024-09-12T03:40:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": 0, 2024-09-12T03:40:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "subtrees": 3, 2024-09-12T03:40:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse": 17, 2024-09-12T03:40:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_hit": 10, 2024-09-12T03:40:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_forward": 1, 2024-09-12T03:40:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_discover": 5, 2024-09-12T03:40:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:40:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_remote_ino": 0, 2024-09-12T03:40:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_lock": 0, 2024-09-12T03:40:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "load_cent": 0, 2024-09-12T03:40:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "q": 0, 2024-09-12T03:40:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "exported": 0, 2024-09-12T03:40:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "exported_inodes": 0, 2024-09-12T03:40:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "imported": 0, 2024-09-12T03:40:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "imported_inodes": 0, 2024-09-12T03:40:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_dir_fetch": 0, 2024-09-12T03:40:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:40:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_peer_discover": 0, 2024-09-12T03:40:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rfiles": 5, 2024-09-12T03:40:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rbytes": 298, 2024-09-12T03:40:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rsnaps": 6, 2024-09-12T03:40:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:40:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_set_tag": 0, 2024-09-12T03:40:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:40:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:40:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:40:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:40:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:40:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_file_inodes": 0, 2024-09-12T03:40:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:40:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T03:40:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_grant": 0, 2024-09-12T03:40:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:40:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:40:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:40:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps": 0, 2024-09-12T03:40:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps_dirty": 0, 2024-09-12T03:40:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_cap_release": 0, 2024-09-12T03:40:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "process_request_cap_release": 1 2024-09-12T03:40:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cache": { 2024-09-12T03:40:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update": 0, 2024-09-12T03:40:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update_receipt": 12, 2024-09-12T03:40:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_try_discover": 8, 2024-09-12T03:40:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_send_discover": 10, 2024-09-12T03:40:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_handle_discover": 0, 2024-09-12T03:40:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays": 0, 2024-09-12T03:40:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_delayed": 0, 2024-09-12T03:40:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:40:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_created": 0, 2024-09-12T03:40:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_enqueued": 0, 2024-09-12T03:40:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_reintegrated": 0, 2024-09-12T03:40:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_migrated": 0, 2024-09-12T03:40:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_processing": 0, 2024-09-12T03:40:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:40:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:40:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_started": 0, 2024-09-12T03:40:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_completed": 0, 2024-09-12T03:40:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_quiesce_path": 0, 2024-09-12T03:40:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_quiesce_inode": 0, 2024-09-12T03:40:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:40:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_exportdir": 0, 2024-09-12T03:40:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_flush": 0, 2024-09-12T03:40:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:40:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragstats": 0, 2024-09-12T03:40:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_inodestats": 0 2024-09-12T03:40:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_log": { 2024-09-12T03:40:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "evlrg": 0, 2024-09-12T03:40:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "evadd": 8, 2024-09-12T03:40:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "evex": 0, 2024-09-12T03:40:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "evtrm": 0, 2024-09-12T03:40:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "ev": 8, 2024-09-12T03:40:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "evexg": 0, 2024-09-12T03:40:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "evexd": 0, 2024-09-12T03:40:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "segadd": 2, 2024-09-12T03:40:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "segex": 0, 2024-09-12T03:40:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "segtrm": 0, 2024-09-12T03:40:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "seg": 2, 2024-09-12T03:40:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "segmjr": 2, 2024-09-12T03:40:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "segexg": 0, 2024-09-12T03:40:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "segexd": 0, 2024-09-12T03:40:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "expos": 4194304, 2024-09-12T03:40:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "wrpos": 4196848, 2024-09-12T03:40:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "rdpos": 0, 2024-09-12T03:40:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "jlat": { 2024-09-12T03:40:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 7, 2024-09-12T03:40:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.011252170, 2024-09-12T03:40:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.001607452 2024-09-12T03:40:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "replayed": 0 2024-09-12T03:40:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_mem": { 2024-09-12T03:40:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 20, 2024-09-12T03:40:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "ino+": 22, 2024-09-12T03:40:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "ino-": 2, 2024-09-12T03:40:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "dir": 19, 2024-09-12T03:40:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "dir+": 22, 2024-09-12T03:40:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "dir-": 3, 2024-09-12T03:40:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "dn": 16, 2024-09-12T03:40:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "dn+": 20, 2024-09-12T03:40:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "dn-": 4, 2024-09-12T03:40:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "cap": 0, 2024-09-12T03:40:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "cap+": 3, 2024-09-12T03:40:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "cap-": 3, 2024-09-12T03:40:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "rss": 52672, 2024-09-12T03:40:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "heap": 223520 2024-09-12T03:40:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_server": { 2024-09-12T03:40:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_client_request": 6, 2024-09-12T03:40:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_server_request": 0, 2024-09-12T03:40:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_request": 5, 2024-09-12T03:40:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_session": 26, 2024-09-12T03:40:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_peer_request": 0, 2024-09-12T03:40:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "req_create_latency": { 2024-09-12T03:40:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getattr_latency": { 2024-09-12T03:40:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 2, 2024-09-12T03:40:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000436538, 2024-09-12T03:40:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000218269 2024-09-12T03:40:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getfilelock_latency": { 2024-09-12T03:40:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: "req_link_latency": { 2024-09-12T03:40:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookup_latency": { 2024-09-12T03:40:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1, 2024-09-12T03:40:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000112183, 2024-09-12T03:40:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000112183 2024-09-12T03:40:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookuphash_latency": { 2024-09-12T03:40:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupino_latency": { 2024-09-12T03:40:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupname_latency": { 2024-09-12T03:40:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupparent_latency": { 2024-09-12T03:40:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupsnap_latency": { 2024-09-12T03:40:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lssnap_latency": { 2024-09-12T03:40:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mkdir_latency": { 2024-09-12T03:40:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mknod_latency": { 2024-09-12T03:40:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mksnap_latency": { 2024-09-12T03:40:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "req_open_latency": { 2024-09-12T03:40:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "req_readdir_latency": { 2024-09-12T03:40:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rename_latency": { 2024-09-12T03:40:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: "req_renamesnap_latency": { 2024-09-12T03:40:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: "req_snapdiff_latency": { 2024-09-12T03:40:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmdir_latency": { 2024-09-12T03:40:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmsnap_latency": { 2024-09-12T03:40:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmxattr_latency": { 2024-09-12T03:40:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setattr_latency": { 2024-09-12T03:40:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setdirlayout_latency": { 2024-09-12T03:40:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setfilelock_latency": { 2024-09-12T03:40:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setlayout_latency": { 2024-09-12T03:40:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setxattr_latency": { 2024-09-12T03:40:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "req_symlink_latency": { 2024-09-12T03:40:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "req_unlink_latency": { 2024-09-12T03:40:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:40:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:40:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getvxattr_latency": { 2024-09-12T03:40:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T03:40:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_sessions": { 2024-09-12T03:40:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: "session_count": 2, 2024-09-12T03:40:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: "session_add": 4, 2024-09-12T03:40:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: "session_remove": 2, 2024-09-12T03:40:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_open": 2, 2024-09-12T03:40:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_stale": 0, 2024-09-12T03:40:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: "total_load": 2, 2024-09-12T03:40:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: "average_load": 1, 2024-09-12T03:40:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: "avg_session_uptime": 658, 2024-09-12T03:40:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: "mdthresh_evicted": 0 2024-09-12T03:40:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: "mempool": { 2024-09-12T03:40:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:40:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_items": 0, 2024-09-12T03:40:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:40:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:40:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:40:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:40:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:40:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:40:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:40:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:40:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:40:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:40:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:40:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:40:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:40:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_items": 0, 2024-09-12T03:40:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:40:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_items": 0, 2024-09-12T03:40:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:40:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:40:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:40:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:40:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:40:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:40:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:40:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_items": 0, 2024-09-12T03:40:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:40:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:40:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:40:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_items": 0, 2024-09-12T03:40:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_bytes": 0, 2024-09-12T03:40:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_items": 0, 2024-09-12T03:40:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:40:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:40:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:40:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:40:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_bytes": 247495, 2024-09-12T03:40:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_items": 87, 2024-09-12T03:40:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:40:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_items": 0, 2024-09-12T03:40:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_bytes": 0, 2024-09-12T03:40:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_items": 0, 2024-09-12T03:40:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:40:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_items": 0, 2024-09-12T03:40:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:40:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_items": 0, 2024-09-12T03:40:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_bytes": 9896, 2024-09-12T03:40:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_items": 200, 2024-09-12T03:40:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:40:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:40:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_bytes": 0, 2024-09-12T03:40:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_items": 0, 2024-09-12T03:40:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_bytes": 86373, 2024-09-12T03:40:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_items": 337, 2024-09-12T03:40:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_bytes": 0, 2024-09-12T03:40:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_items": 0, 2024-09-12T03:40:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_bytes": 0, 2024-09-12T03:40:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_items": 0 2024-09-12T03:40:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "objecter": { 2024-09-12T03:40:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "op_active": 0, 2024-09-12T03:40:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "op_laggy": 0, 2024-09-12T03:40:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send": 46, 2024-09-12T03:40:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send_bytes": 13081, 2024-09-12T03:40:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "op_resend": 27, 2024-09-12T03:40:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "op_reply": 46, 2024-09-12T03:40:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "op_latency": { 2024-09-12T03:40:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 46, 2024-09-12T03:40:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.222996044, 2024-09-12T03:40:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.004847740 2024-09-12T03:40:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "op_inflight": 0, 2024-09-12T03:40:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "oplen_avg": { 2024-09-12T03:40:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 46, 2024-09-12T03:40:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 70 2024-09-12T03:40:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "op": 46, 2024-09-12T03:40:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "op_r": 0, 2024-09-12T03:40:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "op_w": 46, 2024-09-12T03:40:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "op_rmw": 0, 2024-09-12T03:40:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "op_pg": 0, 2024-09-12T03:40:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_stat": 0, 2024-09-12T03:40:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_create": 10, 2024-09-12T03:40:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_read": 0, 2024-09-12T03:40:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_write": 7, 2024-09-12T03:40:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writefull": 8, 2024-09-12T03:40:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writesame": 0, 2024-09-12T03:40:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_append": 0, 2024-09-12T03:40:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_zero": 0, 2024-09-12T03:40:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_truncate": 0, 2024-09-12T03:40:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_delete": 6, 2024-09-12T03:40:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_mapext": 0, 2024-09-12T03:40:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_sparse_read": 0, 2024-09-12T03:40:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_clonerange": 0, 2024-09-12T03:40:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_getxattr": 0, 2024-09-12T03:40:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_setxattr": 20, 2024-09-12T03:40:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:40:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_rmxattr": 0, 2024-09-12T03:40:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:40:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_call": 0, 2024-09-12T03:40:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_watch": 0, 2024-09-12T03:40:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_notify": 0, 2024-09-12T03:40:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:40:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls": 0, 2024-09-12T03:40:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:40:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_other": 0, 2024-09-12T03:40:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_active": 0, 2024-09-12T03:40:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_send": 0, 2024-09-12T03:40:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_resend": 0, 2024-09-12T03:40:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_ping": 0, 2024-09-12T03:40:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_active": 0, 2024-09-12T03:40:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_send": 0, 2024-09-12T03:40:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_resend": 0, 2024-09-12T03:40:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_active": 0, 2024-09-12T03:40:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_send": 0, 2024-09-12T03:40:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_resend": 0, 2024-09-12T03:40:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_active": 0, 2024-09-12T03:40:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_send": 0, 2024-09-12T03:40:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_resend": 0, 2024-09-12T03:40:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "command_active": 0, 2024-09-12T03:40:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "command_send": 0, 2024-09-12T03:40:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "command_resend": 0, 2024-09-12T03:40:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "map_epoch": 103, 2024-09-12T03:40:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "map_full": 0, 2024-09-12T03:40:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "map_inc": 15, 2024-09-12T03:40:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_sessions": 9, 2024-09-12T03:40:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_open": 9, 2024-09-12T03:40:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_close": 0, 2024-09-12T03:40:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_laggy": 0, 2024-09-12T03:40:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_wr": 17, 2024-09-12T03:40:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_rd": 0, 2024-09-12T03:40:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_del": 2 2024-09-12T03:40:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "oft": { 2024-09-12T03:40:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_objs": 1, 2024-09-12T03:40:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:40:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_updates": 0, 2024-09-12T03:40:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_removes": 0 2024-09-12T03:40:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "osd-slow-ops": { 2024-09-12T03:40:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_ops_count": 0 2024-09-12T03:40:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "purge_queue": { 2024-09-12T03:40:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops": 0, 2024-09-12T03:40:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T03:40:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing": 0, 2024-09-12T03:40:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_high_water": 0, 2024-09-12T03:40:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed_ops": 0, 2024-09-12T03:40:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed": 0, 2024-09-12T03:40:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_item_in_journal": 0 2024-09-12T03:40:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:40:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-12T03:40:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-09-12T03:40:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-12T03:40:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 1708, 2024-09-12T03:40:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 1843578, 2024-09-12T03:40:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-12T03:40:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 1708, 2024-09-12T03:40:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-09-12T03:40:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-09-12T03:40:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 1708, 2024-09-12T03:40:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 1843578, 2024-09-12T03:40:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-12T03:40:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T03:40:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_bytes": { 2024-09-12T03:40:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-12T03:40:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-09-12T03:40:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-12T03:40:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-09-12T03:40:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-09-12T03:40:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-12T03:40:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-09-12T03:40:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 46, 2024-09-12T03:40:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 13081, 2024-09-12T03:40:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 46, 2024-09-12T03:40:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 13081, 2024-09-12T03:40:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-12T03:40:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T03:40:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_ops": { 2024-09-12T03:40:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-12T03:40:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 1024, 2024-09-12T03:40:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-12T03:40:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-09-12T03:40:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-09-12T03:40:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-12T03:40:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-09-12T03:40:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 46, 2024-09-12T03:40:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 46, 2024-09-12T03:40:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 46, 2024-09-12T03:40:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 46, 2024-09-12T03:40:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-12T03:40:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T03:40:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:40:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-12T03:40:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2024-09-12T03:40:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-12T03:40:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-09-12T03:40:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-09-12T03:40:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-12T03:40:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-09-12T03:40:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-09-12T03:40:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-09-12T03:40:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 0, 2024-09-12T03:40:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 0, 2024-09-12T03:40:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-12T03:40:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T03:40:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle-0x5598c6792960": { 2024-09-12T03:40:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-12T03:40:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2024-09-12T03:40:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-12T03:40:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 8, 2024-09-12T03:40:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 2544, 2024-09-12T03:40:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-12T03:40:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 8, 2024-09-12T03:40:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-09-12T03:40:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-09-12T03:40:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 7, 2024-09-12T03:40:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 2544, 2024-09-12T03:40:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-12T03:40:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T03:40:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T03:40:39.122 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T03:40:39.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:38 smithi115 ceph-mon[34801]: pgmap v354: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-09-12T03:40:39.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:38 smithi195 ceph-mon[34786]: pgmap v354: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-09-12T03:40:39.271 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:38 smithi064 ceph-mon[30952]: pgmap v354: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-09-12T03:40:39.865 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-12T03:40:39.865 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-12T03:40:39.865 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-09-12T03:40:39.865 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=12 2024-09-12T03:40:39.865 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-09-12T03:40:39.865 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-12T03:40:39.865 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-12T03:40:39.866 DEBUG:tasks.check_counter:Getting stats from k 2024-09-12T03:40:39.866 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-12T03:40:40.110 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:40:40.658 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T03:40:40.658 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:40:40.658 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 4, 2024-09-12T03:40:40.658 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 4, 2024-09-12T03:40:40.658 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 2992, 2024-09-12T03:40:40.658 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 448, 2024-09-12T03:40:40.658 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 4, 2024-09-12T03:40:40.659 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 0, 2024-09-12T03:40:40.659 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.002959460, 2024-09-12T03:40:40.659 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.000036057, 2024-09-12T03:40:40.659 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.002165269, 2024-09-12T03:40:40.659 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:40:40.659 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:40:40.659 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 3, 2024-09-12T03:40:40.659 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.004017250, 2024-09-12T03:40:40.659 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.001339083 2024-09-12T03:40:40.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:40.659 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:40:40.659 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:40.659 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:40.660 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:40.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:40.660 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-12T03:40:40.660 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T03:40:40.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:40.660 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:40:40.660 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 242, 2024-09-12T03:40:40.660 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 231, 2024-09-12T03:40:40.660 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 1586256, 2024-09-12T03:40:40.660 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 142144, 2024-09-12T03:40:40.660 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 2, 2024-09-12T03:40:40.660 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 1, 2024-09-12T03:40:40.660 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.045482597, 2024-09-12T03:40:40.661 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.016131727, 2024-09-12T03:40:40.661 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.044557497, 2024-09-12T03:40:40.661 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:40:40.661 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:40:40.661 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 230, 2024-09-12T03:40:40.661 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.012573226, 2024-09-12T03:40:40.661 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000054666 2024-09-12T03:40:40.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:40.661 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:40:40.661 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:40.661 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:40.661 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:40.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:40.662 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 1586256, 2024-09-12T03:40:40.662 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 142144 2024-09-12T03:40:40.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:40.662 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:40:40.662 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 2, 2024-09-12T03:40:40.662 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 177, 2024-09-12T03:40:40.662 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 320, 2024-09-12T03:40:40.662 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 188288, 2024-09-12T03:40:40.662 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 5, 2024-09-12T03:40:40.662 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 3, 2024-09-12T03:40:40.662 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.012786273, 2024-09-12T03:40:40.662 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.008883431, 2024-09-12T03:40:40.662 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.001400353, 2024-09-12T03:40:40.662 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:40:40.663 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:40:40.663 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 174, 2024-09-12T03:40:40.663 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.021443356, 2024-09-12T03:40:40.663 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000123237 2024-09-12T03:40:40.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:40.663 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:40:40.663 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:40.663 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:40.663 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:40.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:40.663 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T03:40:40.663 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 188288 2024-09-12T03:40:40.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:40.663 INFO:teuthology.orchestra.run.smithi115.stdout: "cct": { 2024-09-12T03:40:40.664 INFO:teuthology.orchestra.run.smithi115.stdout: "total_workers": 0, 2024-09-12T03:40:40.664 INFO:teuthology.orchestra.run.smithi115.stdout: "unhealthy_workers": 0 2024-09-12T03:40:40.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:40.664 INFO:teuthology.orchestra.run.smithi115.stdout: "mempool": { 2024-09-12T03:40:40.664 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:40:40.664 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_items": 0, 2024-09-12T03:40:40.664 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:40:40.664 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:40:40.664 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:40:40.664 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:40:40.664 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:40:40.664 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:40:40.664 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:40:40.664 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:40:40.665 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:40:40.665 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:40:40.665 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:40:40.665 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:40:40.665 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:40:40.665 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_items": 0, 2024-09-12T03:40:40.665 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:40:40.665 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_items": 0, 2024-09-12T03:40:40.665 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:40:40.665 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:40:40.665 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:40:40.665 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:40:40.665 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:40:40.666 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:40:40.666 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:40:40.666 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_items": 0, 2024-09-12T03:40:40.666 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:40:40.666 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:40:40.666 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:40:40.666 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_items": 0, 2024-09-12T03:40:40.666 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_bytes": 0, 2024-09-12T03:40:40.666 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_items": 0, 2024-09-12T03:40:40.666 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:40:40.666 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:40:40.666 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:40:40.666 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:40:40.667 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_bytes": 148967, 2024-09-12T03:40:40.667 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_items": 29, 2024-09-12T03:40:40.667 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:40:40.667 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_items": 0, 2024-09-12T03:40:40.667 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_bytes": 0, 2024-09-12T03:40:40.667 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_items": 0, 2024-09-12T03:40:40.667 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:40:40.667 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_items": 0, 2024-09-12T03:40:40.667 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:40:40.667 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_items": 0, 2024-09-12T03:40:40.667 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_bytes": 0, 2024-09-12T03:40:40.667 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_items": 0, 2024-09-12T03:40:40.667 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:40:40.667 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:40:40.668 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_bytes": 0, 2024-09-12T03:40:40.668 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_items": 0, 2024-09-12T03:40:40.668 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_bytes": 928, 2024-09-12T03:40:40.668 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_items": 2, 2024-09-12T03:40:40.668 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_bytes": 0, 2024-09-12T03:40:40.668 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_items": 0, 2024-09-12T03:40:40.668 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_bytes": 0, 2024-09-12T03:40:40.668 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_items": 0 2024-09-12T03:40:40.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T03:40:40.668 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:40:40.668 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-09-12T03:40:40.668 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-09-12T03:40:40.668 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-09-12T03:40:40.669 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 244, 2024-09-12T03:40:40.669 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 1552125, 2024-09-12T03:40:40.669 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-09-12T03:40:40.669 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 244, 2024-09-12T03:40:40.669 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-09-12T03:40:40.669 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-09-12T03:40:40.669 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 244, 2024-09-12T03:40:40.669 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 1552125, 2024-09-12T03:40:40.669 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-09-12T03:40:40.669 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-09-12T03:40:40.669 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-09-12T03:40:40.669 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-09-12T03:40:40.670 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T03:40:40.670 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T03:40:40.670 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T03:40:41.143 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:40 smithi115 ceph-mon[34801]: pgmap v355: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.8 KiB/s wr, 6 op/s 2024-09-12T03:40:41.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:40 smithi195 ceph-mon[34786]: pgmap v355: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.8 KiB/s wr, 6 op/s 2024-09-12T03:40:41.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:40 smithi064 ceph-mon[30952]: pgmap v355: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.8 KiB/s wr, 6 op/s 2024-09-12T03:40:41.356 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-09-12T03:40:41.356 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-09-12T03:40:41.356 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-09-12T03:40:41.357 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-09-12T03:40:41.357 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-09-12T03:40:41.357 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-09-12T03:40:41.357 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-09-12T03:40:41.357 DEBUG:tasks.check_counter:Getting stats from c 2024-09-12T03:40:41.357 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-12T03:40:41.620 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:40:42.146 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-12T03:40:42.146 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:40:42.146 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 1645, 2024-09-12T03:40:42.146 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 1647, 2024-09-12T03:40:42.147 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 8871925, 2024-09-12T03:40:42.147 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 523120, 2024-09-12T03:40:42.147 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 6, 2024-09-12T03:40:42.147 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 2, 2024-09-12T03:40:42.147 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.178421646, 2024-09-12T03:40:42.147 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.050839428, 2024-09-12T03:40:42.147 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.056863721, 2024-09-12T03:40:42.147 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.054578409, 2024-09-12T03:40:42.147 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:40:42.147 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 1645, 2024-09-12T03:40:42.147 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.070405530, 2024-09-12T03:40:42.147 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000042799 2024-09-12T03:40:42.147 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.148 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:40:42.148 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:42.148 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:42.148 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:42.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.148 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:40:42.148 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:40:42.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.148 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:40:42.148 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 635, 2024-09-12T03:40:42.148 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 808, 2024-09-12T03:40:42.148 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 8548598, 2024-09-12T03:40:42.149 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 661568, 2024-09-12T03:40:42.149 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 7, 2024-09-12T03:40:42.149 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 5, 2024-09-12T03:40:42.149 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.075071432, 2024-09-12T03:40:42.149 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.025924054, 2024-09-12T03:40:42.149 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.024510025, 2024-09-12T03:40:42.149 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.017831679, 2024-09-12T03:40:42.149 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:40:42.149 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 805, 2024-09-12T03:40:42.149 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.039235607, 2024-09-12T03:40:42.149 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000048739 2024-09-12T03:40:42.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.150 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:40:42.150 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:42.150 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:42.150 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:42.150 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.150 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T03:40:42.150 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 460208 2024-09-12T03:40:42.150 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.150 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:40:42.150 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 983, 2024-09-12T03:40:42.150 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 967, 2024-09-12T03:40:42.150 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 6100444, 2024-09-12T03:40:42.151 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 371993, 2024-09-12T03:40:42.151 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 4, 2024-09-12T03:40:42.151 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 3, 2024-09-12T03:40:42.151 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.119426916, 2024-09-12T03:40:42.151 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.041505365, 2024-09-12T03:40:42.151 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.065178825, 2024-09-12T03:40:42.151 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.020852124, 2024-09-12T03:40:42.151 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:40:42.151 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 967, 2024-09-12T03:40:42.151 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.046144586, 2024-09-12T03:40:42.151 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000047719 2024-09-12T03:40:42.151 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.151 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:40:42.152 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:42.152 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:42.152 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:42.152 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.152 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 1623088, 2024-09-12T03:40:42.152 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 144304 2024-09-12T03:40:42.152 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.152 INFO:teuthology.orchestra.run.smithi195.stdout: "cct": { 2024-09-12T03:40:42.152 INFO:teuthology.orchestra.run.smithi195.stdout: "total_workers": 1, 2024-09-12T03:40:42.152 INFO:teuthology.orchestra.run.smithi195.stdout: "unhealthy_workers": 0 2024-09-12T03:40:42.152 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.152 INFO:teuthology.orchestra.run.smithi195.stdout: "finisher-MDSRank": { 2024-09-12T03:40:42.152 INFO:teuthology.orchestra.run.smithi195.stdout: "queue_len": 0, 2024-09-12T03:40:42.153 INFO:teuthology.orchestra.run.smithi195.stdout: "complete_latency": { 2024-09-12T03:40:42.153 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 5127, 2024-09-12T03:40:42.153 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.139807602, 2024-09-12T03:40:42.153 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000027268 2024-09-12T03:40:42.153 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-12T03:40:42.153 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.153 INFO:teuthology.orchestra.run.smithi195.stdout: "finisher-PurgeQueue": { 2024-09-12T03:40:42.153 INFO:teuthology.orchestra.run.smithi195.stdout: "queue_len": 0, 2024-09-12T03:40:42.153 INFO:teuthology.orchestra.run.smithi195.stdout: "complete_latency": { 2024-09-12T03:40:42.153 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:42.153 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:42.153 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:42.153 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-12T03:40:42.154 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.154 INFO:teuthology.orchestra.run.smithi195.stdout: "mds": { 2024-09-12T03:40:42.154 INFO:teuthology.orchestra.run.smithi195.stdout: "request": 0, 2024-09-12T03:40:42.154 INFO:teuthology.orchestra.run.smithi195.stdout: "reply": 0, 2024-09-12T03:40:42.154 INFO:teuthology.orchestra.run.smithi195.stdout: "reply_latency": { 2024-09-12T03:40:42.154 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:42.154 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:42.154 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:42.154 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.154 INFO:teuthology.orchestra.run.smithi195.stdout: "slow_reply": 0, 2024-09-12T03:40:42.154 INFO:teuthology.orchestra.run.smithi195.stdout: "forward": 0, 2024-09-12T03:40:42.154 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_fetch_complete": 0, 2024-09-12T03:40:42.155 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_fetch_keys": 0, 2024-09-12T03:40:42.155 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_commit": 0, 2024-09-12T03:40:42.155 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_split": 0, 2024-09-12T03:40:42.155 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_merge": 0, 2024-09-12T03:40:42.155 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes": 31, 2024-09-12T03:40:42.155 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_top": 12, 2024-09-12T03:40:42.155 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_bottom": 19, 2024-09-12T03:40:42.155 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_pin_tail": 0, 2024-09-12T03:40:42.155 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_pinned": 14, 2024-09-12T03:40:42.156 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_expired": 0, 2024-09-12T03:40:42.156 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_with_caps": 0, 2024-09-12T03:40:42.156 INFO:teuthology.orchestra.run.smithi195.stdout: "caps": 0, 2024-09-12T03:40:42.156 INFO:teuthology.orchestra.run.smithi195.stdout: "subtrees": 2, 2024-09-12T03:40:42.156 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse": 0, 2024-09-12T03:40:42.156 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_hit": 0, 2024-09-12T03:40:42.156 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_forward": 0, 2024-09-12T03:40:42.156 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_discover": 0, 2024-09-12T03:40:42.156 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:40:42.156 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_remote_ino": 0, 2024-09-12T03:40:42.156 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_lock": 0, 2024-09-12T03:40:42.157 INFO:teuthology.orchestra.run.smithi195.stdout: "load_cent": 0, 2024-09-12T03:40:42.157 INFO:teuthology.orchestra.run.smithi195.stdout: "q": 0, 2024-09-12T03:40:42.157 INFO:teuthology.orchestra.run.smithi195.stdout: "exported": 0, 2024-09-12T03:40:42.157 INFO:teuthology.orchestra.run.smithi195.stdout: "exported_inodes": 0, 2024-09-12T03:40:42.157 INFO:teuthology.orchestra.run.smithi195.stdout: "imported": 0, 2024-09-12T03:40:42.157 INFO:teuthology.orchestra.run.smithi195.stdout: "imported_inodes": 0, 2024-09-12T03:40:42.157 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_dir_fetch": 0, 2024-09-12T03:40:42.157 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:40:42.157 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_peer_discover": 0, 2024-09-12T03:40:42.157 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rfiles": 4, 2024-09-12T03:40:42.157 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rbytes": 234, 2024-09-12T03:40:42.157 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rsnaps": 6, 2024-09-12T03:40:42.157 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:40:42.158 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_set_tag": 0, 2024-09-12T03:40:42.158 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:40:42.158 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:40:42.158 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:40:42.158 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:40:42.158 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:40:42.158 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_file_inodes": 0, 2024-09-12T03:40:42.158 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:40:42.158 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T03:40:42.158 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_grant": 0, 2024-09-12T03:40:42.158 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:40:42.158 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:40:42.158 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:40:42.159 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_caps": 0, 2024-09-12T03:40:42.159 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_caps_dirty": 0, 2024-09-12T03:40:42.159 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_cap_release": 0, 2024-09-12T03:40:42.159 INFO:teuthology.orchestra.run.smithi195.stdout: "process_request_cap_release": 0 2024-09-12T03:40:42.159 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.159 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_cache": { 2024-09-12T03:40:42.159 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_update": 0, 2024-09-12T03:40:42.159 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_update_receipt": 0, 2024-09-12T03:40:42.159 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_try_discover": 0, 2024-09-12T03:40:42.159 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_send_discover": 0, 2024-09-12T03:40:42.159 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_handle_discover": 0, 2024-09-12T03:40:42.159 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays": 0, 2024-09-12T03:40:42.159 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays_delayed": 0, 2024-09-12T03:40:42.159 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:40:42.160 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_created": 0, 2024-09-12T03:40:42.160 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_enqueued": 0, 2024-09-12T03:40:42.160 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_reintegrated": 0, 2024-09-12T03:40:42.160 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_migrated": 0, 2024-09-12T03:40:42.160 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_processing": 0, 2024-09-12T03:40:42.160 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:40:42.160 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:40:42.160 INFO:teuthology.orchestra.run.smithi195.stdout: "recovery_started": 0, 2024-09-12T03:40:42.160 INFO:teuthology.orchestra.run.smithi195.stdout: "recovery_completed": 0, 2024-09-12T03:40:42.160 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_quiesce_path": 0, 2024-09-12T03:40:42.160 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_quiesce_inode": 0, 2024-09-12T03:40:42.161 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:40:42.161 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_exportdir": 0, 2024-09-12T03:40:42.161 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_flush": 0, 2024-09-12T03:40:42.161 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:40:42.161 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_fragstats": 0, 2024-09-12T03:40:42.161 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_inodestats": 0 2024-09-12T03:40:42.161 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.161 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_log": { 2024-09-12T03:40:42.161 INFO:teuthology.orchestra.run.smithi195.stdout: "evlrg": 0, 2024-09-12T03:40:42.161 INFO:teuthology.orchestra.run.smithi195.stdout: "evadd": 0, 2024-09-12T03:40:42.161 INFO:teuthology.orchestra.run.smithi195.stdout: "evex": 0, 2024-09-12T03:40:42.161 INFO:teuthology.orchestra.run.smithi195.stdout: "evtrm": 0, 2024-09-12T03:40:42.161 INFO:teuthology.orchestra.run.smithi195.stdout: "ev": 2481, 2024-09-12T03:40:42.162 INFO:teuthology.orchestra.run.smithi195.stdout: "evexg": 0, 2024-09-12T03:40:42.162 INFO:teuthology.orchestra.run.smithi195.stdout: "evexd": 0, 2024-09-12T03:40:42.162 INFO:teuthology.orchestra.run.smithi195.stdout: "segadd": 0, 2024-09-12T03:40:42.162 INFO:teuthology.orchestra.run.smithi195.stdout: "segex": 0, 2024-09-12T03:40:42.162 INFO:teuthology.orchestra.run.smithi195.stdout: "segtrm": 0, 2024-09-12T03:40:42.162 INFO:teuthology.orchestra.run.smithi195.stdout: "seg": 2, 2024-09-12T03:40:42.162 INFO:teuthology.orchestra.run.smithi195.stdout: "segmjr": 52, 2024-09-12T03:40:42.162 INFO:teuthology.orchestra.run.smithi195.stdout: "segexg": 0, 2024-09-12T03:40:42.162 INFO:teuthology.orchestra.run.smithi195.stdout: "segexd": 0, 2024-09-12T03:40:42.162 INFO:teuthology.orchestra.run.smithi195.stdout: "expos": 24826276, 2024-09-12T03:40:42.162 INFO:teuthology.orchestra.run.smithi195.stdout: "wrpos": 25270003, 2024-09-12T03:40:42.162 INFO:teuthology.orchestra.run.smithi195.stdout: "rdpos": 25258611, 2024-09-12T03:40:42.162 INFO:teuthology.orchestra.run.smithi195.stdout: "jlat": { 2024-09-12T03:40:42.163 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:42.163 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:42.163 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:42.163 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.163 INFO:teuthology.orchestra.run.smithi195.stdout: "replayed": 2481 2024-09-12T03:40:42.163 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.163 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_mem": { 2024-09-12T03:40:42.163 INFO:teuthology.orchestra.run.smithi195.stdout: "ino": 26, 2024-09-12T03:40:42.163 INFO:teuthology.orchestra.run.smithi195.stdout: "ino+": 27, 2024-09-12T03:40:42.163 INFO:teuthology.orchestra.run.smithi195.stdout: "ino-": 1, 2024-09-12T03:40:42.163 INFO:teuthology.orchestra.run.smithi195.stdout: "dir": 11, 2024-09-12T03:40:42.163 INFO:teuthology.orchestra.run.smithi195.stdout: "dir+": 11, 2024-09-12T03:40:42.163 INFO:teuthology.orchestra.run.smithi195.stdout: "dir-": 0, 2024-09-12T03:40:42.164 INFO:teuthology.orchestra.run.smithi195.stdout: "dn": 31, 2024-09-12T03:40:42.164 INFO:teuthology.orchestra.run.smithi195.stdout: "dn+": 31, 2024-09-12T03:40:42.164 INFO:teuthology.orchestra.run.smithi195.stdout: "dn-": 0, 2024-09-12T03:40:42.164 INFO:teuthology.orchestra.run.smithi195.stdout: "cap": 0, 2024-09-12T03:40:42.164 INFO:teuthology.orchestra.run.smithi195.stdout: "cap+": 0, 2024-09-12T03:40:42.164 INFO:teuthology.orchestra.run.smithi195.stdout: "cap-": 0, 2024-09-12T03:40:42.164 INFO:teuthology.orchestra.run.smithi195.stdout: "rss": 49000, 2024-09-12T03:40:42.164 INFO:teuthology.orchestra.run.smithi195.stdout: "heap": 215328 2024-09-12T03:40:42.164 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.164 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_server": { 2024-09-12T03:40:42.164 INFO:teuthology.orchestra.run.smithi195.stdout: "dispatch_client_request": 0, 2024-09-12T03:40:42.164 INFO:teuthology.orchestra.run.smithi195.stdout: "dispatch_server_request": 0, 2024-09-12T03:40:42.165 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_request": 0, 2024-09-12T03:40:42.165 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_session": 0, 2024-09-12T03:40:42.165 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_peer_request": 0, 2024-09-12T03:40:42.165 INFO:teuthology.orchestra.run.smithi195.stdout: "req_create_latency": { 2024-09-12T03:40:42.165 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:42.165 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:42.165 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:42.165 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.165 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getattr_latency": { 2024-09-12T03:40:42.165 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:42.165 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:42.165 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:42.165 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.166 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getfilelock_latency": { 2024-09-12T03:40:42.166 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:42.166 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:42.166 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:42.166 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.166 INFO:teuthology.orchestra.run.smithi195.stdout: "req_link_latency": { 2024-09-12T03:40:42.166 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:42.166 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:42.166 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:42.166 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.166 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookup_latency": { 2024-09-12T03:40:42.166 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:42.167 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:42.167 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:42.167 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.167 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookuphash_latency": { 2024-09-12T03:40:42.167 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:42.167 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:42.167 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:42.167 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.167 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupino_latency": { 2024-09-12T03:40:42.167 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:42.167 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:42.167 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:42.167 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.168 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupname_latency": { 2024-09-12T03:40:42.168 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:42.168 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:42.168 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:42.168 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.168 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupparent_latency": { 2024-09-12T03:40:42.168 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:42.168 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:42.168 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:42.168 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.168 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupsnap_latency": { 2024-09-12T03:40:42.168 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:42.168 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:42.169 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:42.169 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.169 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lssnap_latency": { 2024-09-12T03:40:42.169 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:42.169 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:42.169 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:42.169 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.169 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mkdir_latency": { 2024-09-12T03:40:42.169 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:42.169 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:42.169 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:42.169 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.170 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mknod_latency": { 2024-09-12T03:40:42.170 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:42.170 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:42.170 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:42.170 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.170 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mksnap_latency": { 2024-09-12T03:40:42.170 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:42.170 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:42.170 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:42.170 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.170 INFO:teuthology.orchestra.run.smithi195.stdout: "req_open_latency": { 2024-09-12T03:40:42.170 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:42.171 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:42.171 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:42.171 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.171 INFO:teuthology.orchestra.run.smithi195.stdout: "req_readdir_latency": { 2024-09-12T03:40:42.171 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:42.171 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:42.171 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:42.171 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.171 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rename_latency": { 2024-09-12T03:40:42.171 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:42.171 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:42.171 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:42.171 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.172 INFO:teuthology.orchestra.run.smithi195.stdout: "req_renamesnap_latency": { 2024-09-12T03:40:42.172 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:42.172 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:42.172 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:42.172 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.172 INFO:teuthology.orchestra.run.smithi195.stdout: "req_snapdiff_latency": { 2024-09-12T03:40:42.173 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:42.173 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:42.173 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:42.173 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.173 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmdir_latency": { 2024-09-12T03:40:42.173 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:42.173 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:42.173 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:42.173 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.173 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmsnap_latency": { 2024-09-12T03:40:42.173 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:42.174 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:42.174 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:42.174 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.174 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmxattr_latency": { 2024-09-12T03:40:42.174 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:42.174 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:42.174 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:42.174 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.174 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setattr_latency": { 2024-09-12T03:40:42.174 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:42.174 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:42.174 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:42.174 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.175 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setdirlayout_latency": { 2024-09-12T03:40:42.175 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:42.175 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:42.175 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:42.175 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.175 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setfilelock_latency": { 2024-09-12T03:40:42.175 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:42.175 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:42.175 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:42.175 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.175 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setlayout_latency": { 2024-09-12T03:40:42.175 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:42.175 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:42.175 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:42.176 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.176 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setxattr_latency": { 2024-09-12T03:40:42.176 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:42.176 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:42.176 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:42.176 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.176 INFO:teuthology.orchestra.run.smithi195.stdout: "req_symlink_latency": { 2024-09-12T03:40:42.176 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:42.176 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:42.176 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:42.176 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.176 INFO:teuthology.orchestra.run.smithi195.stdout: "req_unlink_latency": { 2024-09-12T03:40:42.176 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:42.176 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:42.177 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:42.177 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.178 INFO:teuthology.orchestra.run.smithi195.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:40:42.178 INFO:teuthology.orchestra.run.smithi195.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:40:42.178 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getvxattr_latency": { 2024-09-12T03:40:42.178 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:42.178 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:42.178 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:42.179 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-12T03:40:42.179 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.179 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_sessions": { 2024-09-12T03:40:42.179 INFO:teuthology.orchestra.run.smithi195.stdout: "session_count": 2, 2024-09-12T03:40:42.179 INFO:teuthology.orchestra.run.smithi195.stdout: "session_add": 6, 2024-09-12T03:40:42.179 INFO:teuthology.orchestra.run.smithi195.stdout: "session_remove": 4, 2024-09-12T03:40:42.179 INFO:teuthology.orchestra.run.smithi195.stdout: "sessions_open": 4, 2024-09-12T03:40:42.179 INFO:teuthology.orchestra.run.smithi195.stdout: "sessions_stale": 0, 2024-09-12T03:40:42.179 INFO:teuthology.orchestra.run.smithi195.stdout: "total_load": 0, 2024-09-12T03:40:42.179 INFO:teuthology.orchestra.run.smithi195.stdout: "average_load": 0, 2024-09-12T03:40:42.179 INFO:teuthology.orchestra.run.smithi195.stdout: "avg_session_uptime": 2706, 2024-09-12T03:40:42.179 INFO:teuthology.orchestra.run.smithi195.stdout: "mdthresh_evicted": 0 2024-09-12T03:40:42.179 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.179 INFO:teuthology.orchestra.run.smithi195.stdout: "mempool": { 2024-09-12T03:40:42.180 INFO:teuthology.orchestra.run.smithi195.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:40:42.180 INFO:teuthology.orchestra.run.smithi195.stdout: "bloom_filter_items": 0, 2024-09-12T03:40:42.180 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:40:42.181 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:40:42.181 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:40:42.181 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:40:42.181 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:40:42.181 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:40:42.181 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:40:42.181 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:40:42.181 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:40:42.181 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:40:42.181 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:40:42.181 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:40:42.182 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:40:42.182 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_extent_items": 0, 2024-09-12T03:40:42.182 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:40:42.182 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_blob_items": 0, 2024-09-12T03:40:42.182 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:40:42.182 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:40:42.182 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:40:42.182 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:40:42.182 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:40:42.182 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:40:42.182 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:40:42.182 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_txc_items": 0, 2024-09-12T03:40:42.182 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:40:42.182 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:40:42.183 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:40:42.183 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_items": 0, 2024-09-12T03:40:42.183 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_bytes": 0, 2024-09-12T03:40:42.183 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_items": 0, 2024-09-12T03:40:42.183 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:40:42.183 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:40:42.183 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:40:42.183 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:40:42.183 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_anon_bytes": 197530, 2024-09-12T03:40:42.183 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_anon_items": 57, 2024-09-12T03:40:42.183 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:40:42.183 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_meta_items": 0, 2024-09-12T03:40:42.183 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_bytes": 0, 2024-09-12T03:40:42.183 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_items": 0, 2024-09-12T03:40:42.184 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:40:42.184 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_mapbl_items": 0, 2024-09-12T03:40:42.184 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:40:42.184 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_pglog_items": 0, 2024-09-12T03:40:42.184 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_bytes": 9896, 2024-09-12T03:40:42.184 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_items": 200, 2024-09-12T03:40:42.184 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:40:42.184 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:40:42.184 INFO:teuthology.orchestra.run.smithi195.stdout: "pgmap_bytes": 0, 2024-09-12T03:40:42.184 INFO:teuthology.orchestra.run.smithi195.stdout: "pgmap_items": 0, 2024-09-12T03:40:42.184 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_co_bytes": 126908, 2024-09-12T03:40:42.184 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_co_items": 564, 2024-09-12T03:40:42.184 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_1_bytes": 0, 2024-09-12T03:40:42.184 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_1_items": 0, 2024-09-12T03:40:42.185 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_2_bytes": 0, 2024-09-12T03:40:42.185 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_2_items": 0 2024-09-12T03:40:42.185 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.185 INFO:teuthology.orchestra.run.smithi195.stdout: "objecter": { 2024-09-12T03:40:42.185 INFO:teuthology.orchestra.run.smithi195.stdout: "op_active": 0, 2024-09-12T03:40:42.185 INFO:teuthology.orchestra.run.smithi195.stdout: "op_laggy": 0, 2024-09-12T03:40:42.185 INFO:teuthology.orchestra.run.smithi195.stdout: "op_send": 2994, 2024-09-12T03:40:42.185 INFO:teuthology.orchestra.run.smithi195.stdout: "op_send_bytes": 26, 2024-09-12T03:40:42.185 INFO:teuthology.orchestra.run.smithi195.stdout: "op_resend": 4, 2024-09-12T03:40:42.185 INFO:teuthology.orchestra.run.smithi195.stdout: "op_reply": 2994, 2024-09-12T03:40:42.185 INFO:teuthology.orchestra.run.smithi195.stdout: "op_latency": { 2024-09-12T03:40:42.185 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 2994, 2024-09-12T03:40:42.185 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 1.611968223, 2024-09-12T03:40:42.186 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000538399 2024-09-12T03:40:42.186 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.186 INFO:teuthology.orchestra.run.smithi195.stdout: "op_inflight": 0, 2024-09-12T03:40:42.186 INFO:teuthology.orchestra.run.smithi195.stdout: "oplen_avg": { 2024-09-12T03:40:42.186 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 2994, 2024-09-12T03:40:42.186 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 2995 2024-09-12T03:40:42.186 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.186 INFO:teuthology.orchestra.run.smithi195.stdout: "op": 2994, 2024-09-12T03:40:42.186 INFO:teuthology.orchestra.run.smithi195.stdout: "op_r": 2994, 2024-09-12T03:40:42.186 INFO:teuthology.orchestra.run.smithi195.stdout: "op_w": 0, 2024-09-12T03:40:42.186 INFO:teuthology.orchestra.run.smithi195.stdout: "op_rmw": 0, 2024-09-12T03:40:42.186 INFO:teuthology.orchestra.run.smithi195.stdout: "op_pg": 0, 2024-09-12T03:40:42.186 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_stat": 1570, 2024-09-12T03:40:42.187 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_create": 0, 2024-09-12T03:40:42.187 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_read": 1421, 2024-09-12T03:40:42.187 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_write": 0, 2024-09-12T03:40:42.187 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_writefull": 0, 2024-09-12T03:40:42.187 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_writesame": 0, 2024-09-12T03:40:42.187 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_append": 0, 2024-09-12T03:40:42.187 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_zero": 0, 2024-09-12T03:40:42.187 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_truncate": 0, 2024-09-12T03:40:42.187 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_delete": 0, 2024-09-12T03:40:42.187 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_mapext": 0, 2024-09-12T03:40:42.187 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_sparse_read": 0, 2024-09-12T03:40:42.187 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_clonerange": 0, 2024-09-12T03:40:42.187 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_getxattr": 2, 2024-09-12T03:40:42.188 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_setxattr": 0, 2024-09-12T03:40:42.188 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:40:42.188 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_rmxattr": 0, 2024-09-12T03:40:42.188 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:40:42.188 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_call": 0, 2024-09-12T03:40:42.188 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_watch": 0, 2024-09-12T03:40:42.188 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_notify": 0, 2024-09-12T03:40:42.188 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:40:42.188 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_pgls": 0, 2024-09-12T03:40:42.188 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:40:42.188 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_other": 0, 2024-09-12T03:40:42.188 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_active": 0, 2024-09-12T03:40:42.188 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_send": 0, 2024-09-12T03:40:42.189 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_resend": 0, 2024-09-12T03:40:42.189 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_ping": 0, 2024-09-12T03:40:42.189 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_active": 0, 2024-09-12T03:40:42.189 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_send": 0, 2024-09-12T03:40:42.189 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_resend": 0, 2024-09-12T03:40:42.189 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_active": 0, 2024-09-12T03:40:42.189 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_send": 0, 2024-09-12T03:40:42.189 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_resend": 0, 2024-09-12T03:40:42.189 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_active": 0, 2024-09-12T03:40:42.189 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_send": 0, 2024-09-12T03:40:42.189 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_resend": 0, 2024-09-12T03:40:42.189 INFO:teuthology.orchestra.run.smithi195.stdout: "command_active": 0, 2024-09-12T03:40:42.189 INFO:teuthology.orchestra.run.smithi195.stdout: "command_send": 0, 2024-09-12T03:40:42.190 INFO:teuthology.orchestra.run.smithi195.stdout: "command_resend": 0, 2024-09-12T03:40:42.190 INFO:teuthology.orchestra.run.smithi195.stdout: "map_epoch": 103, 2024-09-12T03:40:42.190 INFO:teuthology.orchestra.run.smithi195.stdout: "map_full": 0, 2024-09-12T03:40:42.190 INFO:teuthology.orchestra.run.smithi195.stdout: "map_inc": 15, 2024-09-12T03:40:42.190 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_sessions": 6, 2024-09-12T03:40:42.190 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_session_open": 6, 2024-09-12T03:40:42.190 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_session_close": 0, 2024-09-12T03:40:42.190 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_laggy": 0, 2024-09-12T03:40:42.190 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_wr": 0, 2024-09-12T03:40:42.190 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_rd": 2, 2024-09-12T03:40:42.190 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_del": 0 2024-09-12T03:40:42.190 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.190 INFO:teuthology.orchestra.run.smithi195.stdout: "oft": { 2024-09-12T03:40:42.190 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_objs": 0, 2024-09-12T03:40:42.191 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:40:42.191 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_updates": 0, 2024-09-12T03:40:42.191 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_removes": 0 2024-09-12T03:40:42.191 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.191 INFO:teuthology.orchestra.run.smithi195.stdout: "osd-slow-ops": { 2024-09-12T03:40:42.191 INFO:teuthology.orchestra.run.smithi195.stdout: "slow_ops_count": 0 2024-09-12T03:40:42.191 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.191 INFO:teuthology.orchestra.run.smithi195.stdout: "purge_queue": { 2024-09-12T03:40:42.191 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_ops": 0, 2024-09-12T03:40:42.191 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T03:40:42.191 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing": 0, 2024-09-12T03:40:42.191 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_high_water": 0, 2024-09-12T03:40:42.191 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executed_ops": 0, 2024-09-12T03:40:42.192 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executed": 0, 2024-09-12T03:40:42.192 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_item_in_journal": 0 2024-09-12T03:40:42.192 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.192 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:40:42.192 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-09-12T03:40:42.192 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 104857600, 2024-09-12T03:40:42.192 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-09-12T03:40:42.192 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 3259, 2024-09-12T03:40:42.192 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 23211246, 2024-09-12T03:40:42.192 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-09-12T03:40:42.192 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 3259, 2024-09-12T03:40:42.192 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-09-12T03:40:42.192 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-09-12T03:40:42.193 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 3259, 2024-09-12T03:40:42.193 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 23211246, 2024-09-12T03:40:42.193 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-09-12T03:40:42.193 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:42.193 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:42.193 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:42.193 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-12T03:40:42.193 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.193 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-objecter_bytes": { 2024-09-12T03:40:42.193 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-09-12T03:40:42.193 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 104857600, 2024-09-12T03:40:42.193 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-09-12T03:40:42.193 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-09-12T03:40:42.194 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-09-12T03:40:42.194 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-09-12T03:40:42.194 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-09-12T03:40:42.194 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 2994, 2024-09-12T03:40:42.194 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 21075709, 2024-09-12T03:40:42.195 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 2994, 2024-09-12T03:40:42.195 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 21075709, 2024-09-12T03:40:42.195 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-09-12T03:40:42.195 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:42.195 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:42.195 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:42.195 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-12T03:40:42.195 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.196 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-objecter_ops": { 2024-09-12T03:40:42.196 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-09-12T03:40:42.196 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 1024, 2024-09-12T03:40:42.196 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-09-12T03:40:42.196 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-09-12T03:40:42.196 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-09-12T03:40:42.196 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-09-12T03:40:42.196 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-09-12T03:40:42.196 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 2994, 2024-09-12T03:40:42.196 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 2994, 2024-09-12T03:40:42.197 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 2994, 2024-09-12T03:40:42.197 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 2994, 2024-09-12T03:40:42.197 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-09-12T03:40:42.197 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:42.197 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:42.197 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:42.197 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-12T03:40:42.197 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.197 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:40:42.197 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-09-12T03:40:42.197 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 3758096384, 2024-09-12T03:40:42.197 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-09-12T03:40:42.197 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-09-12T03:40:42.198 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-09-12T03:40:42.198 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-09-12T03:40:42.198 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-09-12T03:40:42.198 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-09-12T03:40:42.198 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-09-12T03:40:42.198 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 0, 2024-09-12T03:40:42.198 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 0, 2024-09-12T03:40:42.198 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-09-12T03:40:42.198 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:42.198 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:42.198 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:42.198 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-12T03:40:42.198 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:42.199 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-write_buf_throttle-0x55dd09a4eb40": { 2024-09-12T03:40:42.199 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-09-12T03:40:42.199 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 3758096384, 2024-09-12T03:40:42.199 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-09-12T03:40:42.199 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-09-12T03:40:42.199 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-09-12T03:40:42.199 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-09-12T03:40:42.199 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-09-12T03:40:42.199 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-09-12T03:40:42.199 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-09-12T03:40:42.199 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 0, 2024-09-12T03:40:42.199 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 0, 2024-09-12T03:40:42.199 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-09-12T03:40:42.199 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:42.200 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:42.200 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:42.200 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-12T03:40:42.200 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-12T03:40:42.200 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-12T03:40:43.022 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-12T03:40:43.022 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-12T03:40:43.022 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-09-12T03:40:43.022 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-09-12T03:40:43.022 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-09-12T03:40:43.022 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-12T03:40:43.023 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-12T03:40:43.023 DEBUG:tasks.check_counter:Getting stats from f 2024-09-12T03:40:43.023 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-12T03:40:43.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:42 smithi115 ceph-mon[34801]: pgmap v356: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2024-09-12T03:40:43.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:42 smithi195 ceph-mon[34786]: pgmap v356: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2024-09-12T03:40:43.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:42 smithi064 ceph-mon[30952]: pgmap v356: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2024-09-12T03:40:43.299 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:40:43.857 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-12T03:40:43.857 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:40:43.857 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 2271, 2024-09-12T03:40:43.857 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 2272, 2024-09-12T03:40:43.857 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 644457, 2024-09-12T03:40:43.857 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 721867, 2024-09-12T03:40:43.857 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 6, 2024-09-12T03:40:43.857 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 2, 2024-09-12T03:40:43.857 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.205042425, 2024-09-12T03:40:43.857 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.068926550, 2024-09-12T03:40:43.858 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.052264516, 2024-09-12T03:40:43.858 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.067274607, 2024-09-12T03:40:43.858 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:40:43.858 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 2270, 2024-09-12T03:40:43.858 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.092247993, 2024-09-12T03:40:43.858 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000040637 2024-09-12T03:40:43.858 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.858 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:40:43.858 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:43.858 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:43.858 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:43.858 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.858 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:40:43.858 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:40:43.859 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.859 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:40:43.859 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 8, 2024-09-12T03:40:43.859 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 179, 2024-09-12T03:40:43.859 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 3868, 2024-09-12T03:40:43.859 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 461354, 2024-09-12T03:40:43.859 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 6, 2024-09-12T03:40:43.859 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 4, 2024-09-12T03:40:43.859 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.013615312, 2024-09-12T03:40:43.859 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.007308677, 2024-09-12T03:40:43.859 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.004114195, 2024-09-12T03:40:43.859 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.000064859, 2024-09-12T03:40:43.860 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:40:43.860 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 177, 2024-09-12T03:40:43.860 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.024177018, 2024-09-12T03:40:43.860 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000136593 2024-09-12T03:40:43.860 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.860 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:40:43.860 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:43.860 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:43.860 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:43.860 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.860 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-09-12T03:40:43.860 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 460656 2024-09-12T03:40:43.860 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.861 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:40:43.861 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 263, 2024-09-12T03:40:43.861 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 250, 2024-09-12T03:40:43.861 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 1620476, 2024-09-12T03:40:43.861 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 144654, 2024-09-12T03:40:43.861 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 3, 2024-09-12T03:40:43.861 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 2, 2024-09-12T03:40:43.861 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.045167210, 2024-09-12T03:40:43.861 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.016470660, 2024-09-12T03:40:43.861 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.042362571, 2024-09-12T03:40:43.861 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.000021050, 2024-09-12T03:40:43.862 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:40:43.862 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 249, 2024-09-12T03:40:43.862 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.014750517, 2024-09-12T03:40:43.862 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000059239 2024-09-12T03:40:43.862 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.862 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:40:43.862 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:43.862 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:43.862 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:43.862 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.862 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 1620208, 2024-09-12T03:40:43.863 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 144336 2024-09-12T03:40:43.863 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.863 INFO:teuthology.orchestra.run.smithi195.stdout: "cct": { 2024-09-12T03:40:43.863 INFO:teuthology.orchestra.run.smithi195.stdout: "total_workers": 1, 2024-09-12T03:40:43.863 INFO:teuthology.orchestra.run.smithi195.stdout: "unhealthy_workers": 0 2024-09-12T03:40:43.863 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.863 INFO:teuthology.orchestra.run.smithi195.stdout: "finisher-MDSRank": { 2024-09-12T03:40:43.863 INFO:teuthology.orchestra.run.smithi195.stdout: "queue_len": 0, 2024-09-12T03:40:43.863 INFO:teuthology.orchestra.run.smithi195.stdout: "complete_latency": { 2024-09-12T03:40:43.864 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 3850, 2024-09-12T03:40:43.864 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.107420233, 2024-09-12T03:40:43.864 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000027901 2024-09-12T03:40:43.864 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-12T03:40:43.864 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.864 INFO:teuthology.orchestra.run.smithi195.stdout: "finisher-PurgeQueue": { 2024-09-12T03:40:43.864 INFO:teuthology.orchestra.run.smithi195.stdout: "queue_len": 0, 2024-09-12T03:40:43.864 INFO:teuthology.orchestra.run.smithi195.stdout: "complete_latency": { 2024-09-12T03:40:43.864 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:43.864 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:43.864 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:43.864 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-12T03:40:43.864 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.864 INFO:teuthology.orchestra.run.smithi195.stdout: "mds": { 2024-09-12T03:40:43.865 INFO:teuthology.orchestra.run.smithi195.stdout: "request": 0, 2024-09-12T03:40:43.865 INFO:teuthology.orchestra.run.smithi195.stdout: "reply": 0, 2024-09-12T03:40:43.865 INFO:teuthology.orchestra.run.smithi195.stdout: "reply_latency": { 2024-09-12T03:40:43.865 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:43.865 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:43.865 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:43.865 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.865 INFO:teuthology.orchestra.run.smithi195.stdout: "slow_reply": 0, 2024-09-12T03:40:43.865 INFO:teuthology.orchestra.run.smithi195.stdout: "forward": 0, 2024-09-12T03:40:43.865 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_fetch_complete": 0, 2024-09-12T03:40:43.865 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_fetch_keys": 0, 2024-09-12T03:40:43.865 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_commit": 0, 2024-09-12T03:40:43.865 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_split": 0, 2024-09-12T03:40:43.865 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_merge": 0, 2024-09-12T03:40:43.866 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes": 0, 2024-09-12T03:40:43.866 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_top": 0, 2024-09-12T03:40:43.866 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_bottom": 0, 2024-09-12T03:40:43.866 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_pin_tail": 0, 2024-09-12T03:40:43.866 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_pinned": 0, 2024-09-12T03:40:43.866 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_expired": 0, 2024-09-12T03:40:43.866 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_with_caps": 0, 2024-09-12T03:40:43.866 INFO:teuthology.orchestra.run.smithi195.stdout: "caps": 0, 2024-09-12T03:40:43.866 INFO:teuthology.orchestra.run.smithi195.stdout: "subtrees": 1, 2024-09-12T03:40:43.866 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse": 0, 2024-09-12T03:40:43.866 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_hit": 0, 2024-09-12T03:40:43.866 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_forward": 0, 2024-09-12T03:40:43.866 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_discover": 0, 2024-09-12T03:40:43.867 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:40:43.867 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_remote_ino": 0, 2024-09-12T03:40:43.867 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_lock": 0, 2024-09-12T03:40:43.867 INFO:teuthology.orchestra.run.smithi195.stdout: "load_cent": 0, 2024-09-12T03:40:43.867 INFO:teuthology.orchestra.run.smithi195.stdout: "q": 0, 2024-09-12T03:40:43.867 INFO:teuthology.orchestra.run.smithi195.stdout: "exported": 0, 2024-09-12T03:40:43.867 INFO:teuthology.orchestra.run.smithi195.stdout: "exported_inodes": 0, 2024-09-12T03:40:43.867 INFO:teuthology.orchestra.run.smithi195.stdout: "imported": 0, 2024-09-12T03:40:43.867 INFO:teuthology.orchestra.run.smithi195.stdout: "imported_inodes": 0, 2024-09-12T03:40:43.867 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_dir_fetch": 0, 2024-09-12T03:40:43.867 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:40:43.867 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_peer_discover": 0, 2024-09-12T03:40:43.867 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rfiles": 0, 2024-09-12T03:40:43.867 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rbytes": 0, 2024-09-12T03:40:43.868 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rsnaps": 0, 2024-09-12T03:40:43.868 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:40:43.868 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_set_tag": 0, 2024-09-12T03:40:43.868 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:40:43.868 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:40:43.868 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:40:43.868 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:40:43.868 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:40:43.868 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_file_inodes": 0, 2024-09-12T03:40:43.868 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:40:43.868 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T03:40:43.868 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_grant": 0, 2024-09-12T03:40:43.869 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:40:43.869 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:40:43.869 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:40:43.869 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_caps": 0, 2024-09-12T03:40:43.869 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_caps_dirty": 0, 2024-09-12T03:40:43.869 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_cap_release": 0, 2024-09-12T03:40:43.869 INFO:teuthology.orchestra.run.smithi195.stdout: "process_request_cap_release": 0 2024-09-12T03:40:43.869 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.869 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_cache": { 2024-09-12T03:40:43.869 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_update": 0, 2024-09-12T03:40:43.869 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_update_receipt": 0, 2024-09-12T03:40:43.869 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_try_discover": 0, 2024-09-12T03:40:43.869 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_send_discover": 0, 2024-09-12T03:40:43.870 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_handle_discover": 0, 2024-09-12T03:40:43.870 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays": 0, 2024-09-12T03:40:43.870 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays_delayed": 0, 2024-09-12T03:40:43.870 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:40:43.870 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_created": 0, 2024-09-12T03:40:43.870 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_enqueued": 0, 2024-09-12T03:40:43.870 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_reintegrated": 0, 2024-09-12T03:40:43.870 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_migrated": 0, 2024-09-12T03:40:43.870 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_processing": 0, 2024-09-12T03:40:43.870 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:40:43.870 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:40:43.870 INFO:teuthology.orchestra.run.smithi195.stdout: "recovery_started": 0, 2024-09-12T03:40:43.870 INFO:teuthology.orchestra.run.smithi195.stdout: "recovery_completed": 0, 2024-09-12T03:40:43.870 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_quiesce_path": 0, 2024-09-12T03:40:43.871 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_quiesce_inode": 0, 2024-09-12T03:40:43.871 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:40:43.871 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_exportdir": 0, 2024-09-12T03:40:43.871 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_flush": 0, 2024-09-12T03:40:43.871 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:40:43.871 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_fragstats": 0, 2024-09-12T03:40:43.871 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_inodestats": 0 2024-09-12T03:40:43.871 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.871 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_log": { 2024-09-12T03:40:43.871 INFO:teuthology.orchestra.run.smithi195.stdout: "evlrg": 0, 2024-09-12T03:40:43.871 INFO:teuthology.orchestra.run.smithi195.stdout: "evadd": 0, 2024-09-12T03:40:43.871 INFO:teuthology.orchestra.run.smithi195.stdout: "evex": 0, 2024-09-12T03:40:43.871 INFO:teuthology.orchestra.run.smithi195.stdout: "evtrm": 0, 2024-09-12T03:40:43.871 INFO:teuthology.orchestra.run.smithi195.stdout: "ev": 4, 2024-09-12T03:40:43.872 INFO:teuthology.orchestra.run.smithi195.stdout: "evexg": 0, 2024-09-12T03:40:43.872 INFO:teuthology.orchestra.run.smithi195.stdout: "evexd": 0, 2024-09-12T03:40:43.872 INFO:teuthology.orchestra.run.smithi195.stdout: "segadd": 0, 2024-09-12T03:40:43.872 INFO:teuthology.orchestra.run.smithi195.stdout: "segex": 0, 2024-09-12T03:40:43.872 INFO:teuthology.orchestra.run.smithi195.stdout: "segtrm": 0, 2024-09-12T03:40:43.872 INFO:teuthology.orchestra.run.smithi195.stdout: "seg": 2, 2024-09-12T03:40:43.872 INFO:teuthology.orchestra.run.smithi195.stdout: "segmjr": 2, 2024-09-12T03:40:43.872 INFO:teuthology.orchestra.run.smithi195.stdout: "segexg": 0, 2024-09-12T03:40:43.872 INFO:teuthology.orchestra.run.smithi195.stdout: "segexd": 0, 2024-09-12T03:40:43.872 INFO:teuthology.orchestra.run.smithi195.stdout: "expos": 4194304, 2024-09-12T03:40:43.873 INFO:teuthology.orchestra.run.smithi195.stdout: "wrpos": 4195780, 2024-09-12T03:40:43.873 INFO:teuthology.orchestra.run.smithi195.stdout: "rdpos": 4195387, 2024-09-12T03:40:43.873 INFO:teuthology.orchestra.run.smithi195.stdout: "jlat": { 2024-09-12T03:40:43.873 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:43.873 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:43.873 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:43.873 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.873 INFO:teuthology.orchestra.run.smithi195.stdout: "replayed": 4 2024-09-12T03:40:43.873 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.873 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_mem": { 2024-09-12T03:40:43.873 INFO:teuthology.orchestra.run.smithi195.stdout: "ino": 3, 2024-09-12T03:40:43.873 INFO:teuthology.orchestra.run.smithi195.stdout: "ino+": 3, 2024-09-12T03:40:43.874 INFO:teuthology.orchestra.run.smithi195.stdout: "ino-": 0, 2024-09-12T03:40:43.874 INFO:teuthology.orchestra.run.smithi195.stdout: "dir": 1, 2024-09-12T03:40:43.874 INFO:teuthology.orchestra.run.smithi195.stdout: "dir+": 1, 2024-09-12T03:40:43.874 INFO:teuthology.orchestra.run.smithi195.stdout: "dir-": 0, 2024-09-12T03:40:43.874 INFO:teuthology.orchestra.run.smithi195.stdout: "dn": 0, 2024-09-12T03:40:43.874 INFO:teuthology.orchestra.run.smithi195.stdout: "dn+": 0, 2024-09-12T03:40:43.874 INFO:teuthology.orchestra.run.smithi195.stdout: "dn-": 0, 2024-09-12T03:40:43.874 INFO:teuthology.orchestra.run.smithi195.stdout: "cap": 0, 2024-09-12T03:40:43.874 INFO:teuthology.orchestra.run.smithi195.stdout: "cap+": 0, 2024-09-12T03:40:43.874 INFO:teuthology.orchestra.run.smithi195.stdout: "cap-": 0, 2024-09-12T03:40:43.874 INFO:teuthology.orchestra.run.smithi195.stdout: "rss": 48860, 2024-09-12T03:40:43.874 INFO:teuthology.orchestra.run.smithi195.stdout: "heap": 215328 2024-09-12T03:40:43.874 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.875 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_server": { 2024-09-12T03:40:43.875 INFO:teuthology.orchestra.run.smithi195.stdout: "dispatch_client_request": 0, 2024-09-12T03:40:43.875 INFO:teuthology.orchestra.run.smithi195.stdout: "dispatch_server_request": 0, 2024-09-12T03:40:43.875 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_request": 0, 2024-09-12T03:40:43.875 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_session": 0, 2024-09-12T03:40:43.875 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_peer_request": 0, 2024-09-12T03:40:43.875 INFO:teuthology.orchestra.run.smithi195.stdout: "req_create_latency": { 2024-09-12T03:40:43.875 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:43.875 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:43.875 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:43.875 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.876 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getattr_latency": { 2024-09-12T03:40:43.876 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:43.876 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:43.876 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:43.876 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.876 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getfilelock_latency": { 2024-09-12T03:40:43.876 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:43.876 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:43.876 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:43.876 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.876 INFO:teuthology.orchestra.run.smithi195.stdout: "req_link_latency": { 2024-09-12T03:40:43.876 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:43.876 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:43.876 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:43.877 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.877 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookup_latency": { 2024-09-12T03:40:43.877 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:43.877 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:43.877 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:43.877 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.877 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookuphash_latency": { 2024-09-12T03:40:43.877 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:43.877 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:43.878 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:43.878 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.878 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupino_latency": { 2024-09-12T03:40:43.878 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:43.878 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:43.878 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:43.878 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.878 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupname_latency": { 2024-09-12T03:40:43.878 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:43.878 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:43.878 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:43.878 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.878 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupparent_latency": { 2024-09-12T03:40:43.879 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:43.879 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:43.879 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:43.879 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.879 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupsnap_latency": { 2024-09-12T03:40:43.879 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:43.879 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:43.879 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:43.879 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.879 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lssnap_latency": { 2024-09-12T03:40:43.879 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:43.880 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:43.880 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:43.880 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.880 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mkdir_latency": { 2024-09-12T03:40:43.880 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:43.880 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:43.880 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:43.880 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.880 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mknod_latency": { 2024-09-12T03:40:43.880 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:43.880 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:43.880 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:43.880 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.881 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mksnap_latency": { 2024-09-12T03:40:43.881 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:43.881 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:43.881 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:43.881 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.881 INFO:teuthology.orchestra.run.smithi195.stdout: "req_open_latency": { 2024-09-12T03:40:43.881 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:43.881 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:43.881 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:43.881 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.882 INFO:teuthology.orchestra.run.smithi195.stdout: "req_readdir_latency": { 2024-09-12T03:40:43.882 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:43.882 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:43.882 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:43.882 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.882 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rename_latency": { 2024-09-12T03:40:43.882 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:43.882 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:43.882 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:43.882 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.882 INFO:teuthology.orchestra.run.smithi195.stdout: "req_renamesnap_latency": { 2024-09-12T03:40:43.882 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:43.883 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:43.883 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:43.883 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.883 INFO:teuthology.orchestra.run.smithi195.stdout: "req_snapdiff_latency": { 2024-09-12T03:40:43.883 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:43.883 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:43.883 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:43.883 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.883 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmdir_latency": { 2024-09-12T03:40:43.883 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:43.883 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:43.883 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:43.884 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.884 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmsnap_latency": { 2024-09-12T03:40:43.884 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:43.884 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:43.884 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:43.884 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.884 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmxattr_latency": { 2024-09-12T03:40:43.884 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:43.884 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:43.885 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:43.885 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.885 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setattr_latency": { 2024-09-12T03:40:43.885 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:43.885 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:43.885 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:43.885 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.885 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setdirlayout_latency": { 2024-09-12T03:40:43.886 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:43.886 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:43.886 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:43.886 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.886 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setfilelock_latency": { 2024-09-12T03:40:43.886 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:43.886 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:43.886 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:43.886 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.886 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setlayout_latency": { 2024-09-12T03:40:43.886 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:43.886 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:43.887 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:43.887 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.887 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setxattr_latency": { 2024-09-12T03:40:43.887 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:43.887 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:43.887 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:43.887 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.887 INFO:teuthology.orchestra.run.smithi195.stdout: "req_symlink_latency": { 2024-09-12T03:40:43.887 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:43.887 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:43.887 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:43.887 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.887 INFO:teuthology.orchestra.run.smithi195.stdout: "req_unlink_latency": { 2024-09-12T03:40:43.887 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:43.888 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:43.888 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:43.888 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.888 INFO:teuthology.orchestra.run.smithi195.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:40:43.888 INFO:teuthology.orchestra.run.smithi195.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:40:43.888 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getvxattr_latency": { 2024-09-12T03:40:43.888 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:43.888 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:43.888 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:43.888 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-12T03:40:43.888 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.888 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_sessions": { 2024-09-12T03:40:43.888 INFO:teuthology.orchestra.run.smithi195.stdout: "session_count": 2, 2024-09-12T03:40:43.889 INFO:teuthology.orchestra.run.smithi195.stdout: "session_add": 2, 2024-09-12T03:40:43.889 INFO:teuthology.orchestra.run.smithi195.stdout: "session_remove": 0, 2024-09-12T03:40:43.889 INFO:teuthology.orchestra.run.smithi195.stdout: "sessions_open": 2, 2024-09-12T03:40:43.889 INFO:teuthology.orchestra.run.smithi195.stdout: "sessions_stale": 0, 2024-09-12T03:40:43.889 INFO:teuthology.orchestra.run.smithi195.stdout: "total_load": 0, 2024-09-12T03:40:43.889 INFO:teuthology.orchestra.run.smithi195.stdout: "average_load": 0, 2024-09-12T03:40:43.889 INFO:teuthology.orchestra.run.smithi195.stdout: "avg_session_uptime": 1696, 2024-09-12T03:40:43.889 INFO:teuthology.orchestra.run.smithi195.stdout: "mdthresh_evicted": 0 2024-09-12T03:40:43.889 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.889 INFO:teuthology.orchestra.run.smithi195.stdout: "mempool": { 2024-09-12T03:40:43.889 INFO:teuthology.orchestra.run.smithi195.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:40:43.889 INFO:teuthology.orchestra.run.smithi195.stdout: "bloom_filter_items": 0, 2024-09-12T03:40:43.889 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:40:43.889 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:40:43.890 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:40:43.890 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:40:43.890 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:40:43.890 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:40:43.890 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:40:43.890 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:40:43.890 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:40:43.890 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:40:43.890 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:40:43.890 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:40:43.891 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:40:43.891 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_extent_items": 0, 2024-09-12T03:40:43.891 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:40:43.891 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_blob_items": 0, 2024-09-12T03:40:43.891 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:40:43.891 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:40:43.891 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:40:43.891 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:40:43.891 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:40:43.891 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:40:43.891 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:40:43.891 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_txc_items": 0, 2024-09-12T03:40:43.891 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:40:43.892 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:40:43.892 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:40:43.892 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_items": 0, 2024-09-12T03:40:43.892 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_bytes": 0, 2024-09-12T03:40:43.892 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_items": 0, 2024-09-12T03:40:43.892 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:40:43.892 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:40:43.893 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:40:43.893 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:40:43.893 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_anon_bytes": 181330, 2024-09-12T03:40:43.893 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_anon_items": 48, 2024-09-12T03:40:43.893 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:40:43.893 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_meta_items": 0, 2024-09-12T03:40:43.893 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_bytes": 0, 2024-09-12T03:40:43.893 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_items": 0, 2024-09-12T03:40:43.893 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:40:43.893 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_mapbl_items": 0, 2024-09-12T03:40:43.893 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:40:43.893 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_pglog_items": 0, 2024-09-12T03:40:43.893 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_bytes": 9896, 2024-09-12T03:40:43.893 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_items": 200, 2024-09-12T03:40:43.894 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:40:43.894 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:40:43.894 INFO:teuthology.orchestra.run.smithi195.stdout: "pgmap_bytes": 0, 2024-09-12T03:40:43.894 INFO:teuthology.orchestra.run.smithi195.stdout: "pgmap_items": 0, 2024-09-12T03:40:43.894 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_co_bytes": 8512, 2024-09-12T03:40:43.894 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_co_items": 13, 2024-09-12T03:40:43.894 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_1_bytes": 0, 2024-09-12T03:40:43.894 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_1_items": 0, 2024-09-12T03:40:43.894 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_2_bytes": 0, 2024-09-12T03:40:43.894 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_2_items": 0 2024-09-12T03:40:43.894 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.894 INFO:teuthology.orchestra.run.smithi195.stdout: "objecter": { 2024-09-12T03:40:43.894 INFO:teuthology.orchestra.run.smithi195.stdout: "op_active": 0, 2024-09-12T03:40:43.894 INFO:teuthology.orchestra.run.smithi195.stdout: "op_laggy": 0, 2024-09-12T03:40:43.895 INFO:teuthology.orchestra.run.smithi195.stdout: "op_send": 2273, 2024-09-12T03:40:43.895 INFO:teuthology.orchestra.run.smithi195.stdout: "op_send_bytes": 21, 2024-09-12T03:40:43.895 INFO:teuthology.orchestra.run.smithi195.stdout: "op_resend": 3, 2024-09-12T03:40:43.895 INFO:teuthology.orchestra.run.smithi195.stdout: "op_reply": 2273, 2024-09-12T03:40:43.895 INFO:teuthology.orchestra.run.smithi195.stdout: "op_latency": { 2024-09-12T03:40:43.895 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 2273, 2024-09-12T03:40:43.895 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 1.239975589, 2024-09-12T03:40:43.895 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000545523 2024-09-12T03:40:43.895 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.895 INFO:teuthology.orchestra.run.smithi195.stdout: "op_inflight": 0, 2024-09-12T03:40:43.895 INFO:teuthology.orchestra.run.smithi195.stdout: "oplen_avg": { 2024-09-12T03:40:43.895 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 2273, 2024-09-12T03:40:43.895 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 2274 2024-09-12T03:40:43.896 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.896 INFO:teuthology.orchestra.run.smithi195.stdout: "op": 2273, 2024-09-12T03:40:43.896 INFO:teuthology.orchestra.run.smithi195.stdout: "op_r": 2273, 2024-09-12T03:40:43.896 INFO:teuthology.orchestra.run.smithi195.stdout: "op_w": 0, 2024-09-12T03:40:43.896 INFO:teuthology.orchestra.run.smithi195.stdout: "op_rmw": 0, 2024-09-12T03:40:43.896 INFO:teuthology.orchestra.run.smithi195.stdout: "op_pg": 0, 2024-09-12T03:40:43.896 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_stat": 1477, 2024-09-12T03:40:43.896 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_create": 0, 2024-09-12T03:40:43.896 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_read": 794, 2024-09-12T03:40:43.896 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_write": 0, 2024-09-12T03:40:43.896 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_writefull": 0, 2024-09-12T03:40:43.896 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_writesame": 0, 2024-09-12T03:40:43.896 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_append": 0, 2024-09-12T03:40:43.896 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_zero": 0, 2024-09-12T03:40:43.897 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_truncate": 0, 2024-09-12T03:40:43.897 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_delete": 0, 2024-09-12T03:40:43.897 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_mapext": 0, 2024-09-12T03:40:43.897 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_sparse_read": 0, 2024-09-12T03:40:43.897 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_clonerange": 0, 2024-09-12T03:40:43.897 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_getxattr": 1, 2024-09-12T03:40:43.897 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_setxattr": 0, 2024-09-12T03:40:43.897 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:40:43.897 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_rmxattr": 0, 2024-09-12T03:40:43.897 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:40:43.897 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_call": 0, 2024-09-12T03:40:43.897 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_watch": 0, 2024-09-12T03:40:43.897 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_notify": 0, 2024-09-12T03:40:43.897 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:40:43.898 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_pgls": 0, 2024-09-12T03:40:43.898 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:40:43.898 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_other": 0, 2024-09-12T03:40:43.898 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_active": 0, 2024-09-12T03:40:43.898 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_send": 0, 2024-09-12T03:40:43.898 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_resend": 0, 2024-09-12T03:40:43.898 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_ping": 0, 2024-09-12T03:40:43.898 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_active": 0, 2024-09-12T03:40:43.898 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_send": 0, 2024-09-12T03:40:43.898 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_resend": 0, 2024-09-12T03:40:43.898 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_active": 0, 2024-09-12T03:40:43.898 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_send": 0, 2024-09-12T03:40:43.898 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_resend": 0, 2024-09-12T03:40:43.898 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_active": 0, 2024-09-12T03:40:43.899 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_send": 0, 2024-09-12T03:40:43.899 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_resend": 0, 2024-09-12T03:40:43.899 INFO:teuthology.orchestra.run.smithi195.stdout: "command_active": 0, 2024-09-12T03:40:43.899 INFO:teuthology.orchestra.run.smithi195.stdout: "command_send": 0, 2024-09-12T03:40:43.899 INFO:teuthology.orchestra.run.smithi195.stdout: "command_resend": 0, 2024-09-12T03:40:43.899 INFO:teuthology.orchestra.run.smithi195.stdout: "map_epoch": 103, 2024-09-12T03:40:43.899 INFO:teuthology.orchestra.run.smithi195.stdout: "map_full": 0, 2024-09-12T03:40:43.899 INFO:teuthology.orchestra.run.smithi195.stdout: "map_inc": 15, 2024-09-12T03:40:43.899 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_sessions": 4, 2024-09-12T03:40:43.899 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_session_open": 4, 2024-09-12T03:40:43.899 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_session_close": 0, 2024-09-12T03:40:43.899 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_laggy": 0, 2024-09-12T03:40:43.899 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_wr": 0, 2024-09-12T03:40:43.899 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_rd": 2, 2024-09-12T03:40:43.900 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_del": 0 2024-09-12T03:40:43.900 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.900 INFO:teuthology.orchestra.run.smithi195.stdout: "oft": { 2024-09-12T03:40:43.900 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_objs": 0, 2024-09-12T03:40:43.900 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:40:43.900 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_updates": 0, 2024-09-12T03:40:43.900 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_removes": 0 2024-09-12T03:40:43.900 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.900 INFO:teuthology.orchestra.run.smithi195.stdout: "osd-slow-ops": { 2024-09-12T03:40:43.900 INFO:teuthology.orchestra.run.smithi195.stdout: "slow_ops_count": 0 2024-09-12T03:40:43.900 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.900 INFO:teuthology.orchestra.run.smithi195.stdout: "purge_queue": { 2024-09-12T03:40:43.900 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_ops": 0, 2024-09-12T03:40:43.900 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T03:40:43.901 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing": 0, 2024-09-12T03:40:43.901 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_high_water": 0, 2024-09-12T03:40:43.901 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executed_ops": 0, 2024-09-12T03:40:43.901 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executed": 0, 2024-09-12T03:40:43.901 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_item_in_journal": 0 2024-09-12T03:40:43.901 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.901 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:40:43.901 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-09-12T03:40:43.901 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 104857600, 2024-09-12T03:40:43.901 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-09-12T03:40:43.901 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 2538, 2024-09-12T03:40:43.901 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 2023964, 2024-09-12T03:40:43.901 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-09-12T03:40:43.902 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 2538, 2024-09-12T03:40:43.902 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-09-12T03:40:43.902 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-09-12T03:40:43.902 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 2538, 2024-09-12T03:40:43.902 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 2023964, 2024-09-12T03:40:43.902 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-09-12T03:40:43.902 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:43.902 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:43.902 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:43.902 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-12T03:40:43.902 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.902 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-objecter_bytes": { 2024-09-12T03:40:43.902 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-09-12T03:40:43.902 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 104857600, 2024-09-12T03:40:43.903 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-09-12T03:40:43.903 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-09-12T03:40:43.903 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-09-12T03:40:43.903 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-09-12T03:40:43.903 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-09-12T03:40:43.903 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 2273, 2024-09-12T03:40:43.903 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 1481, 2024-09-12T03:40:43.903 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 2273, 2024-09-12T03:40:43.903 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 1481, 2024-09-12T03:40:43.903 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-09-12T03:40:43.903 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:43.903 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:43.903 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:43.903 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-12T03:40:43.904 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.904 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-objecter_ops": { 2024-09-12T03:40:43.904 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-09-12T03:40:43.904 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 1024, 2024-09-12T03:40:43.904 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-09-12T03:40:43.904 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-09-12T03:40:43.904 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-09-12T03:40:43.904 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-09-12T03:40:43.904 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-09-12T03:40:43.904 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 2273, 2024-09-12T03:40:43.904 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 2273, 2024-09-12T03:40:43.904 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 2273, 2024-09-12T03:40:43.904 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 2273, 2024-09-12T03:40:43.905 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-09-12T03:40:43.905 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:43.905 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:43.905 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:43.905 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-12T03:40:43.905 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.905 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:40:43.905 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-09-12T03:40:43.905 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 3758096384, 2024-09-12T03:40:43.905 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-09-12T03:40:43.905 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-09-12T03:40:43.905 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-09-12T03:40:43.905 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-09-12T03:40:43.905 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-09-12T03:40:43.906 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-09-12T03:40:43.906 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-09-12T03:40:43.906 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 0, 2024-09-12T03:40:43.906 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 0, 2024-09-12T03:40:43.906 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-09-12T03:40:43.906 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:43.906 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:43.906 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:43.906 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-12T03:40:43.906 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:43.906 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-write_buf_throttle-0x558545610b40": { 2024-09-12T03:40:43.906 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-09-12T03:40:43.906 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 3758096384, 2024-09-12T03:40:43.906 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-09-12T03:40:43.906 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-09-12T03:40:43.907 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-09-12T03:40:43.907 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-09-12T03:40:43.907 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-09-12T03:40:43.907 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-09-12T03:40:43.907 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-09-12T03:40:43.907 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 0, 2024-09-12T03:40:43.907 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 0, 2024-09-12T03:40:43.907 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-09-12T03:40:43.907 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:43.907 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:43.907 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:43.907 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-12T03:40:43.907 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-12T03:40:43.907 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-12T03:40:44.539 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-12T03:40:44.539 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-12T03:40:44.539 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-09-12T03:40:44.539 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-09-12T03:40:44.539 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-09-12T03:40:44.539 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-12T03:40:44.539 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-12T03:40:44.540 DEBUG:tasks.check_counter:Getting stats from i 2024-09-12T03:40:44.540 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-12T03:40:44.785 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:40:45.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:44 smithi115 ceph-mon[34801]: pgmap v357: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.9 KiB/s wr, 5 op/s 2024-09-12T03:40:45.191 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:44 smithi195 ceph-mon[34786]: pgmap v357: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.9 KiB/s wr, 5 op/s 2024-09-12T03:40:45.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:44 smithi064 ceph-mon[30952]: pgmap v357: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.9 KiB/s wr, 5 op/s 2024-09-12T03:40:45.299 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-12T03:40:45.300 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:40:45.300 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 328, 2024-09-12T03:40:45.300 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 484, 2024-09-12T03:40:45.300 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 1641583, 2024-09-12T03:40:45.300 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 553219, 2024-09-12T03:40:45.300 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 15, 2024-09-12T03:40:45.300 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 8, 2024-09-12T03:40:45.300 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.063952053, 2024-09-12T03:40:45.300 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.024260255, 2024-09-12T03:40:45.300 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.051532608, 2024-09-12T03:40:45.300 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.000595266, 2024-09-12T03:40:45.300 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:40:45.301 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 480, 2024-09-12T03:40:45.301 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.036105603, 2024-09-12T03:40:45.301 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000075220 2024-09-12T03:40:45.301 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.301 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:40:45.301 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 55, 2024-09-12T03:40:45.301 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000009284, 2024-09-12T03:40:45.301 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000168 2024-09-12T03:40:45.301 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.301 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 1637392, 2024-09-12T03:40:45.301 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 496272 2024-09-12T03:40:45.301 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.302 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:40:45.302 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 1317, 2024-09-12T03:40:45.302 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 929, 2024-09-12T03:40:45.302 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 353475, 2024-09-12T03:40:45.302 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 524851, 2024-09-12T03:40:45.302 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 8, 2024-09-12T03:40:45.302 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 4, 2024-09-12T03:40:45.302 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.093143365, 2024-09-12T03:40:45.302 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.036643034, 2024-09-12T03:40:45.302 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.060683925, 2024-09-12T03:40:45.302 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.004834280, 2024-09-12T03:40:45.302 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:40:45.302 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 930, 2024-09-12T03:40:45.302 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.052656578, 2024-09-12T03:40:45.303 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000056619 2024-09-12T03:40:45.303 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.303 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:40:45.303 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 2105, 2024-09-12T03:40:45.303 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000434813, 2024-09-12T03:40:45.303 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000206 2024-09-12T03:40:45.303 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.303 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-09-12T03:40:45.303 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 126512 2024-09-12T03:40:45.303 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.303 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:40:45.303 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 69, 2024-09-12T03:40:45.304 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 71, 2024-09-12T03:40:45.304 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 23692, 2024-09-12T03:40:45.304 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 32265, 2024-09-12T03:40:45.304 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 7, 2024-09-12T03:40:45.304 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 5, 2024-09-12T03:40:45.304 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.006686218, 2024-09-12T03:40:45.304 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.001933814, 2024-09-12T03:40:45.304 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.004805078, 2024-09-12T03:40:45.304 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.000456930, 2024-09-12T03:40:45.304 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:40:45.304 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 69, 2024-09-12T03:40:45.304 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.013639138, 2024-09-12T03:40:45.304 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000197668 2024-09-12T03:40:45.305 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.305 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:40:45.305 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 104, 2024-09-12T03:40:45.305 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000019423, 2024-09-12T03:40:45.305 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000186 2024-09-12T03:40:45.305 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.305 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:40:45.305 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:40:45.305 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.305 INFO:teuthology.orchestra.run.smithi195.stdout: "cct": { 2024-09-12T03:40:45.305 INFO:teuthology.orchestra.run.smithi195.stdout: "total_workers": 1, 2024-09-12T03:40:45.305 INFO:teuthology.orchestra.run.smithi195.stdout: "unhealthy_workers": 0 2024-09-12T03:40:45.305 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.306 INFO:teuthology.orchestra.run.smithi195.stdout: "finisher-MDSRank": { 2024-09-12T03:40:45.306 INFO:teuthology.orchestra.run.smithi195.stdout: "queue_len": 0, 2024-09-12T03:40:45.306 INFO:teuthology.orchestra.run.smithi195.stdout: "complete_latency": { 2024-09-12T03:40:45.306 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 48, 2024-09-12T03:40:45.306 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.001371835, 2024-09-12T03:40:45.306 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000028579 2024-09-12T03:40:45.306 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-12T03:40:45.306 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.306 INFO:teuthology.orchestra.run.smithi195.stdout: "finisher-PurgeQueue": { 2024-09-12T03:40:45.306 INFO:teuthology.orchestra.run.smithi195.stdout: "queue_len": 0, 2024-09-12T03:40:45.306 INFO:teuthology.orchestra.run.smithi195.stdout: "complete_latency": { 2024-09-12T03:40:45.306 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 2, 2024-09-12T03:40:45.306 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000008937, 2024-09-12T03:40:45.307 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000004468 2024-09-12T03:40:45.307 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-12T03:40:45.307 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.307 INFO:teuthology.orchestra.run.smithi195.stdout: "mds": { 2024-09-12T03:40:45.307 INFO:teuthology.orchestra.run.smithi195.stdout: "request": 9, 2024-09-12T03:40:45.307 INFO:teuthology.orchestra.run.smithi195.stdout: "reply": 6, 2024-09-12T03:40:45.307 INFO:teuthology.orchestra.run.smithi195.stdout: "reply_latency": { 2024-09-12T03:40:45.307 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 6, 2024-09-12T03:40:45.307 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.001550064, 2024-09-12T03:40:45.307 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000258344 2024-09-12T03:40:45.307 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.307 INFO:teuthology.orchestra.run.smithi195.stdout: "slow_reply": 0, 2024-09-12T03:40:45.307 INFO:teuthology.orchestra.run.smithi195.stdout: "forward": 3, 2024-09-12T03:40:45.308 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_fetch_complete": 0, 2024-09-12T03:40:45.308 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_fetch_keys": 0, 2024-09-12T03:40:45.308 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_commit": 11, 2024-09-12T03:40:45.308 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_split": 0, 2024-09-12T03:40:45.308 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_merge": 0, 2024-09-12T03:40:45.308 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes": 16, 2024-09-12T03:40:45.308 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_top": 0, 2024-09-12T03:40:45.308 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_bottom": 16, 2024-09-12T03:40:45.308 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_pin_tail": 0, 2024-09-12T03:40:45.308 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_pinned": 13, 2024-09-12T03:40:45.308 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_expired": 4, 2024-09-12T03:40:45.309 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_with_caps": 3, 2024-09-12T03:40:45.309 INFO:teuthology.orchestra.run.smithi195.stdout: "caps": 4, 2024-09-12T03:40:45.309 INFO:teuthology.orchestra.run.smithi195.stdout: "subtrees": 3, 2024-09-12T03:40:45.309 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse": 22, 2024-09-12T03:40:45.309 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_hit": 15, 2024-09-12T03:40:45.309 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_forward": 1, 2024-09-12T03:40:45.309 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_discover": 5, 2024-09-12T03:40:45.309 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:40:45.309 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_remote_ino": 0, 2024-09-12T03:40:45.309 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_lock": 0, 2024-09-12T03:40:45.309 INFO:teuthology.orchestra.run.smithi195.stdout: "load_cent": 1000, 2024-09-12T03:40:45.309 INFO:teuthology.orchestra.run.smithi195.stdout: "q": 1, 2024-09-12T03:40:45.309 INFO:teuthology.orchestra.run.smithi195.stdout: "exported": 0, 2024-09-12T03:40:45.310 INFO:teuthology.orchestra.run.smithi195.stdout: "exported_inodes": 0, 2024-09-12T03:40:45.310 INFO:teuthology.orchestra.run.smithi195.stdout: "imported": 0, 2024-09-12T03:40:45.310 INFO:teuthology.orchestra.run.smithi195.stdout: "imported_inodes": 0, 2024-09-12T03:40:45.310 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_dir_fetch": 0, 2024-09-12T03:40:45.310 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:40:45.310 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_peer_discover": 0, 2024-09-12T03:40:45.310 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rfiles": 5, 2024-09-12T03:40:45.310 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rbytes": 298, 2024-09-12T03:40:45.310 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rsnaps": 6, 2024-09-12T03:40:45.310 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:40:45.310 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_set_tag": 0, 2024-09-12T03:40:45.310 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:40:45.310 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:40:45.310 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:40:45.311 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:40:45.311 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:40:45.311 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_file_inodes": 0, 2024-09-12T03:40:45.311 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:40:45.311 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T03:40:45.311 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_grant": 0, 2024-09-12T03:40:45.311 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:40:45.311 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:40:45.311 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:40:45.311 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_caps": 0, 2024-09-12T03:40:45.311 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_caps_dirty": 0, 2024-09-12T03:40:45.311 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_cap_release": 0, 2024-09-12T03:40:45.311 INFO:teuthology.orchestra.run.smithi195.stdout: "process_request_cap_release": 0 2024-09-12T03:40:45.311 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.312 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_cache": { 2024-09-12T03:40:45.312 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_update": 0, 2024-09-12T03:40:45.312 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_update_receipt": 16, 2024-09-12T03:40:45.312 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_try_discover": 10, 2024-09-12T03:40:45.312 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_send_discover": 10, 2024-09-12T03:40:45.312 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_handle_discover": 0, 2024-09-12T03:40:45.312 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays": 0, 2024-09-12T03:40:45.312 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays_delayed": 0, 2024-09-12T03:40:45.312 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:40:45.312 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_created": 0, 2024-09-12T03:40:45.312 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_enqueued": 0, 2024-09-12T03:40:45.313 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_reintegrated": 0, 2024-09-12T03:40:45.313 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_migrated": 0, 2024-09-12T03:40:45.313 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_processing": 0, 2024-09-12T03:40:45.313 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:40:45.313 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:40:45.313 INFO:teuthology.orchestra.run.smithi195.stdout: "recovery_started": 0, 2024-09-12T03:40:45.314 INFO:teuthology.orchestra.run.smithi195.stdout: "recovery_completed": 0, 2024-09-12T03:40:45.314 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_quiesce_path": 0, 2024-09-12T03:40:45.314 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_quiesce_inode": 0, 2024-09-12T03:40:45.314 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:40:45.314 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_exportdir": 0, 2024-09-12T03:40:45.314 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_flush": 0, 2024-09-12T03:40:45.314 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:40:45.314 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_fragstats": 0, 2024-09-12T03:40:45.314 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_inodestats": 0 2024-09-12T03:40:45.314 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.315 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_log": { 2024-09-12T03:40:45.315 INFO:teuthology.orchestra.run.smithi195.stdout: "evlrg": 0, 2024-09-12T03:40:45.315 INFO:teuthology.orchestra.run.smithi195.stdout: "evadd": 4, 2024-09-12T03:40:45.315 INFO:teuthology.orchestra.run.smithi195.stdout: "evex": 0, 2024-09-12T03:40:45.315 INFO:teuthology.orchestra.run.smithi195.stdout: "evtrm": 0, 2024-09-12T03:40:45.315 INFO:teuthology.orchestra.run.smithi195.stdout: "ev": 4, 2024-09-12T03:40:45.315 INFO:teuthology.orchestra.run.smithi195.stdout: "evexg": 0, 2024-09-12T03:40:45.315 INFO:teuthology.orchestra.run.smithi195.stdout: "evexd": 0, 2024-09-12T03:40:45.315 INFO:teuthology.orchestra.run.smithi195.stdout: "segadd": 2, 2024-09-12T03:40:45.315 INFO:teuthology.orchestra.run.smithi195.stdout: "segex": 0, 2024-09-12T03:40:45.315 INFO:teuthology.orchestra.run.smithi195.stdout: "segtrm": 0, 2024-09-12T03:40:45.315 INFO:teuthology.orchestra.run.smithi195.stdout: "seg": 2, 2024-09-12T03:40:45.315 INFO:teuthology.orchestra.run.smithi195.stdout: "segmjr": 2, 2024-09-12T03:40:45.315 INFO:teuthology.orchestra.run.smithi195.stdout: "segexg": 0, 2024-09-12T03:40:45.316 INFO:teuthology.orchestra.run.smithi195.stdout: "segexd": 0, 2024-09-12T03:40:45.316 INFO:teuthology.orchestra.run.smithi195.stdout: "expos": 4194304, 2024-09-12T03:40:45.316 INFO:teuthology.orchestra.run.smithi195.stdout: "wrpos": 4195780, 2024-09-12T03:40:45.316 INFO:teuthology.orchestra.run.smithi195.stdout: "rdpos": 0, 2024-09-12T03:40:45.316 INFO:teuthology.orchestra.run.smithi195.stdout: "jlat": { 2024-09-12T03:40:45.316 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 3, 2024-09-12T03:40:45.316 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.004385562, 2024-09-12T03:40:45.316 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.001461854 2024-09-12T03:40:45.316 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.316 INFO:teuthology.orchestra.run.smithi195.stdout: "replayed": 0 2024-09-12T03:40:45.316 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.316 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_mem": { 2024-09-12T03:40:45.316 INFO:teuthology.orchestra.run.smithi195.stdout: "ino": 20, 2024-09-12T03:40:45.317 INFO:teuthology.orchestra.run.smithi195.stdout: "ino+": 22, 2024-09-12T03:40:45.317 INFO:teuthology.orchestra.run.smithi195.stdout: "ino-": 2, 2024-09-12T03:40:45.317 INFO:teuthology.orchestra.run.smithi195.stdout: "dir": 19, 2024-09-12T03:40:45.317 INFO:teuthology.orchestra.run.smithi195.stdout: "dir+": 23, 2024-09-12T03:40:45.317 INFO:teuthology.orchestra.run.smithi195.stdout: "dir-": 4, 2024-09-12T03:40:45.317 INFO:teuthology.orchestra.run.smithi195.stdout: "dn": 16, 2024-09-12T03:40:45.317 INFO:teuthology.orchestra.run.smithi195.stdout: "dn+": 20, 2024-09-12T03:40:45.317 INFO:teuthology.orchestra.run.smithi195.stdout: "dn-": 4, 2024-09-12T03:40:45.317 INFO:teuthology.orchestra.run.smithi195.stdout: "cap": 4, 2024-09-12T03:40:45.317 INFO:teuthology.orchestra.run.smithi195.stdout: "cap+": 4, 2024-09-12T03:40:45.317 INFO:teuthology.orchestra.run.smithi195.stdout: "cap-": 0, 2024-09-12T03:40:45.318 INFO:teuthology.orchestra.run.smithi195.stdout: "rss": 53056, 2024-09-12T03:40:45.318 INFO:teuthology.orchestra.run.smithi195.stdout: "heap": 223520 2024-09-12T03:40:45.318 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.318 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_server": { 2024-09-12T03:40:45.318 INFO:teuthology.orchestra.run.smithi195.stdout: "dispatch_client_request": 12, 2024-09-12T03:40:45.318 INFO:teuthology.orchestra.run.smithi195.stdout: "dispatch_server_request": 0, 2024-09-12T03:40:45.318 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_request": 9, 2024-09-12T03:40:45.318 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_session": 18, 2024-09-12T03:40:45.318 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_peer_request": 1, 2024-09-12T03:40:45.318 INFO:teuthology.orchestra.run.smithi195.stdout: "req_create_latency": { 2024-09-12T03:40:45.318 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:45.318 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:45.318 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:45.319 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.319 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getattr_latency": { 2024-09-12T03:40:45.319 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 6, 2024-09-12T03:40:45.319 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.001550064, 2024-09-12T03:40:45.319 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000258344 2024-09-12T03:40:45.319 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.319 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getfilelock_latency": { 2024-09-12T03:40:45.319 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:45.319 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:45.319 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:45.319 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.320 INFO:teuthology.orchestra.run.smithi195.stdout: "req_link_latency": { 2024-09-12T03:40:45.320 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:45.320 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:45.320 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:45.320 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.320 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookup_latency": { 2024-09-12T03:40:45.320 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:45.320 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:45.320 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:45.320 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.320 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookuphash_latency": { 2024-09-12T03:40:45.320 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:45.320 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:45.321 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:45.321 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.321 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupino_latency": { 2024-09-12T03:40:45.321 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:45.321 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:45.321 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:45.321 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.321 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupname_latency": { 2024-09-12T03:40:45.321 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:45.321 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:45.321 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:45.321 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.321 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupparent_latency": { 2024-09-12T03:40:45.321 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:45.322 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:45.322 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:45.322 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.322 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupsnap_latency": { 2024-09-12T03:40:45.322 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:45.322 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:45.322 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:45.322 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.322 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lssnap_latency": { 2024-09-12T03:40:45.322 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:45.322 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:45.322 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:45.322 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.322 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mkdir_latency": { 2024-09-12T03:40:45.323 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:45.323 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:45.323 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:45.323 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.323 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mknod_latency": { 2024-09-12T03:40:45.323 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:45.323 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:45.323 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:45.323 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.323 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mksnap_latency": { 2024-09-12T03:40:45.323 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:45.323 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:45.323 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:45.324 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.324 INFO:teuthology.orchestra.run.smithi195.stdout: "req_open_latency": { 2024-09-12T03:40:45.324 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:45.324 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:45.324 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:45.324 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.324 INFO:teuthology.orchestra.run.smithi195.stdout: "req_readdir_latency": { 2024-09-12T03:40:45.324 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:45.324 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:45.324 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:45.324 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.324 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rename_latency": { 2024-09-12T03:40:45.324 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:45.324 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:45.325 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:45.325 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.325 INFO:teuthology.orchestra.run.smithi195.stdout: "req_renamesnap_latency": { 2024-09-12T03:40:45.325 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:45.325 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:45.325 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:45.325 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.325 INFO:teuthology.orchestra.run.smithi195.stdout: "req_snapdiff_latency": { 2024-09-12T03:40:45.325 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:45.325 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:45.325 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:45.325 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.325 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmdir_latency": { 2024-09-12T03:40:45.326 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:45.326 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:45.326 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:45.326 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.326 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmsnap_latency": { 2024-09-12T03:40:45.326 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:45.326 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:45.326 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:45.326 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.326 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmxattr_latency": { 2024-09-12T03:40:45.326 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:45.326 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:45.326 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:45.326 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.327 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setattr_latency": { 2024-09-12T03:40:45.327 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:45.327 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:45.327 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:45.327 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.327 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setdirlayout_latency": { 2024-09-12T03:40:45.327 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:45.327 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:45.327 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:45.327 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.328 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setfilelock_latency": { 2024-09-12T03:40:45.328 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:45.328 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:45.328 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:45.328 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.328 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setlayout_latency": { 2024-09-12T03:40:45.328 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:45.328 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:45.328 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:45.328 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.328 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setxattr_latency": { 2024-09-12T03:40:45.328 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:45.328 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:45.329 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:45.329 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.329 INFO:teuthology.orchestra.run.smithi195.stdout: "req_symlink_latency": { 2024-09-12T03:40:45.329 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:45.329 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:45.329 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:45.329 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.329 INFO:teuthology.orchestra.run.smithi195.stdout: "req_unlink_latency": { 2024-09-12T03:40:45.329 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:45.329 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:45.329 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:45.329 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.329 INFO:teuthology.orchestra.run.smithi195.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:40:45.329 INFO:teuthology.orchestra.run.smithi195.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:40:45.330 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getvxattr_latency": { 2024-09-12T03:40:45.330 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:45.330 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:45.330 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:45.330 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-12T03:40:45.330 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.330 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_sessions": { 2024-09-12T03:40:45.330 INFO:teuthology.orchestra.run.smithi195.stdout: "session_count": 2, 2024-09-12T03:40:45.330 INFO:teuthology.orchestra.run.smithi195.stdout: "session_add": 2, 2024-09-12T03:40:45.330 INFO:teuthology.orchestra.run.smithi195.stdout: "session_remove": 0, 2024-09-12T03:40:45.330 INFO:teuthology.orchestra.run.smithi195.stdout: "sessions_open": 2, 2024-09-12T03:40:45.330 INFO:teuthology.orchestra.run.smithi195.stdout: "sessions_stale": 0, 2024-09-12T03:40:45.330 INFO:teuthology.orchestra.run.smithi195.stdout: "total_load": 4, 2024-09-12T03:40:45.330 INFO:teuthology.orchestra.run.smithi195.stdout: "average_load": 2, 2024-09-12T03:40:45.331 INFO:teuthology.orchestra.run.smithi195.stdout: "avg_session_uptime": 665, 2024-09-12T03:40:45.331 INFO:teuthology.orchestra.run.smithi195.stdout: "mdthresh_evicted": 0 2024-09-12T03:40:45.331 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.331 INFO:teuthology.orchestra.run.smithi195.stdout: "mempool": { 2024-09-12T03:40:45.331 INFO:teuthology.orchestra.run.smithi195.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:40:45.331 INFO:teuthology.orchestra.run.smithi195.stdout: "bloom_filter_items": 0, 2024-09-12T03:40:45.331 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:40:45.331 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:40:45.331 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:40:45.331 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:40:45.331 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:40:45.331 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:40:45.331 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:40:45.331 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:40:45.332 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:40:45.332 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:40:45.332 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:40:45.332 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:40:45.332 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:40:45.332 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_extent_items": 0, 2024-09-12T03:40:45.332 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:40:45.332 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_blob_items": 0, 2024-09-12T03:40:45.332 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:40:45.332 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:40:45.332 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:40:45.332 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:40:45.332 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:40:45.333 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:40:45.333 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:40:45.333 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_txc_items": 0, 2024-09-12T03:40:45.333 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:40:45.333 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:40:45.333 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:40:45.333 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_items": 0, 2024-09-12T03:40:45.334 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_bytes": 0, 2024-09-12T03:40:45.334 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_items": 0, 2024-09-12T03:40:45.334 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:40:45.334 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:40:45.334 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:40:45.334 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:40:45.334 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_anon_bytes": 256310, 2024-09-12T03:40:45.334 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_anon_items": 99, 2024-09-12T03:40:45.334 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_meta_bytes": 88, 2024-09-12T03:40:45.334 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_meta_items": 1, 2024-09-12T03:40:45.334 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_bytes": 0, 2024-09-12T03:40:45.334 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_items": 0, 2024-09-12T03:40:45.334 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:40:45.334 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_mapbl_items": 0, 2024-09-12T03:40:45.335 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:40:45.335 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_pglog_items": 0, 2024-09-12T03:40:45.335 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_bytes": 9896, 2024-09-12T03:40:45.335 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_items": 200, 2024-09-12T03:40:45.335 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:40:45.335 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:40:45.335 INFO:teuthology.orchestra.run.smithi195.stdout: "pgmap_bytes": 0, 2024-09-12T03:40:45.335 INFO:teuthology.orchestra.run.smithi195.stdout: "pgmap_items": 0, 2024-09-12T03:40:45.335 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_co_bytes": 87853, 2024-09-12T03:40:45.335 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_co_items": 346, 2024-09-12T03:40:45.335 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_1_bytes": 0, 2024-09-12T03:40:45.335 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_1_items": 0, 2024-09-12T03:40:45.336 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_2_bytes": 0, 2024-09-12T03:40:45.336 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_2_items": 0 2024-09-12T03:40:45.336 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.336 INFO:teuthology.orchestra.run.smithi195.stdout: "objecter": { 2024-09-12T03:40:45.336 INFO:teuthology.orchestra.run.smithi195.stdout: "op_active": 0, 2024-09-12T03:40:45.336 INFO:teuthology.orchestra.run.smithi195.stdout: "op_laggy": 0, 2024-09-12T03:40:45.336 INFO:teuthology.orchestra.run.smithi195.stdout: "op_send": 40, 2024-09-12T03:40:45.336 INFO:teuthology.orchestra.run.smithi195.stdout: "op_send_bytes": 11937, 2024-09-12T03:40:45.336 INFO:teuthology.orchestra.run.smithi195.stdout: "op_resend": 33, 2024-09-12T03:40:45.336 INFO:teuthology.orchestra.run.smithi195.stdout: "op_reply": 40, 2024-09-12T03:40:45.336 INFO:teuthology.orchestra.run.smithi195.stdout: "op_latency": { 2024-09-12T03:40:45.336 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 40, 2024-09-12T03:40:45.336 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.200996090, 2024-09-12T03:40:45.337 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.005024902 2024-09-12T03:40:45.337 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.337 INFO:teuthology.orchestra.run.smithi195.stdout: "op_inflight": 0, 2024-09-12T03:40:45.337 INFO:teuthology.orchestra.run.smithi195.stdout: "oplen_avg": { 2024-09-12T03:40:45.337 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 40, 2024-09-12T03:40:45.337 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 63 2024-09-12T03:40:45.337 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.337 INFO:teuthology.orchestra.run.smithi195.stdout: "op": 40, 2024-09-12T03:40:45.337 INFO:teuthology.orchestra.run.smithi195.stdout: "op_r": 0, 2024-09-12T03:40:45.337 INFO:teuthology.orchestra.run.smithi195.stdout: "op_w": 40, 2024-09-12T03:40:45.337 INFO:teuthology.orchestra.run.smithi195.stdout: "op_rmw": 0, 2024-09-12T03:40:45.337 INFO:teuthology.orchestra.run.smithi195.stdout: "op_pg": 0, 2024-09-12T03:40:45.337 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_stat": 0, 2024-09-12T03:40:45.337 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_create": 10, 2024-09-12T03:40:45.338 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_read": 0, 2024-09-12T03:40:45.338 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_write": 3, 2024-09-12T03:40:45.338 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_writefull": 7, 2024-09-12T03:40:45.338 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_writesame": 0, 2024-09-12T03:40:45.338 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_append": 0, 2024-09-12T03:40:45.338 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_zero": 0, 2024-09-12T03:40:45.338 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_truncate": 0, 2024-09-12T03:40:45.338 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_delete": 6, 2024-09-12T03:40:45.338 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_mapext": 0, 2024-09-12T03:40:45.338 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_sparse_read": 0, 2024-09-12T03:40:45.338 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_clonerange": 0, 2024-09-12T03:40:45.338 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_getxattr": 0, 2024-09-12T03:40:45.338 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_setxattr": 20, 2024-09-12T03:40:45.338 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:40:45.339 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_rmxattr": 0, 2024-09-12T03:40:45.339 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:40:45.339 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_call": 0, 2024-09-12T03:40:45.339 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_watch": 0, 2024-09-12T03:40:45.339 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_notify": 0, 2024-09-12T03:40:45.339 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:40:45.339 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_pgls": 0, 2024-09-12T03:40:45.339 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:40:45.339 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_other": 0, 2024-09-12T03:40:45.339 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_active": 0, 2024-09-12T03:40:45.339 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_send": 0, 2024-09-12T03:40:45.339 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_resend": 0, 2024-09-12T03:40:45.339 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_ping": 0, 2024-09-12T03:40:45.339 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_active": 0, 2024-09-12T03:40:45.340 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_send": 0, 2024-09-12T03:40:45.340 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_resend": 0, 2024-09-12T03:40:45.340 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_active": 0, 2024-09-12T03:40:45.340 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_send": 0, 2024-09-12T03:40:45.340 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_resend": 0, 2024-09-12T03:40:45.340 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_active": 0, 2024-09-12T03:40:45.340 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_send": 0, 2024-09-12T03:40:45.340 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_resend": 0, 2024-09-12T03:40:45.340 INFO:teuthology.orchestra.run.smithi195.stdout: "command_active": 0, 2024-09-12T03:40:45.340 INFO:teuthology.orchestra.run.smithi195.stdout: "command_send": 0, 2024-09-12T03:40:45.340 INFO:teuthology.orchestra.run.smithi195.stdout: "command_resend": 0, 2024-09-12T03:40:45.340 INFO:teuthology.orchestra.run.smithi195.stdout: "map_epoch": 103, 2024-09-12T03:40:45.340 INFO:teuthology.orchestra.run.smithi195.stdout: "map_full": 0, 2024-09-12T03:40:45.340 INFO:teuthology.orchestra.run.smithi195.stdout: "map_inc": 15, 2024-09-12T03:40:45.341 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_sessions": 9, 2024-09-12T03:40:45.341 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_session_open": 9, 2024-09-12T03:40:45.341 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_session_close": 0, 2024-09-12T03:40:45.341 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_laggy": 0, 2024-09-12T03:40:45.341 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_wr": 16, 2024-09-12T03:40:45.341 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_rd": 0, 2024-09-12T03:40:45.341 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_del": 1 2024-09-12T03:40:45.341 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.341 INFO:teuthology.orchestra.run.smithi195.stdout: "oft": { 2024-09-12T03:40:45.341 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_objs": 1, 2024-09-12T03:40:45.342 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:40:45.342 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_updates": 0, 2024-09-12T03:40:45.342 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_removes": 0 2024-09-12T03:40:45.342 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.342 INFO:teuthology.orchestra.run.smithi195.stdout: "osd-slow-ops": { 2024-09-12T03:40:45.342 INFO:teuthology.orchestra.run.smithi195.stdout: "slow_ops_count": 0 2024-09-12T03:40:45.342 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.342 INFO:teuthology.orchestra.run.smithi195.stdout: "purge_queue": { 2024-09-12T03:40:45.342 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_ops": 0, 2024-09-12T03:40:45.342 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T03:40:45.343 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing": 0, 2024-09-12T03:40:45.343 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_high_water": 0, 2024-09-12T03:40:45.343 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executed_ops": 0, 2024-09-12T03:40:45.343 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executed": 0, 2024-09-12T03:40:45.343 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_item_in_journal": 0 2024-09-12T03:40:45.343 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.343 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:40:45.343 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-09-12T03:40:45.343 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 104857600, 2024-09-12T03:40:45.343 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-09-12T03:40:45.343 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 1710, 2024-09-12T03:40:45.344 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 1846305, 2024-09-12T03:40:45.344 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-09-12T03:40:45.344 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 1710, 2024-09-12T03:40:45.344 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-09-12T03:40:45.344 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-09-12T03:40:45.344 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 1710, 2024-09-12T03:40:45.344 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 1846305, 2024-09-12T03:40:45.344 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-09-12T03:40:45.344 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:45.344 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:45.344 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:45.345 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-12T03:40:45.345 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.345 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-objecter_bytes": { 2024-09-12T03:40:45.345 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-09-12T03:40:45.345 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 104857600, 2024-09-12T03:40:45.345 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-09-12T03:40:45.345 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-09-12T03:40:45.345 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-09-12T03:40:45.345 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-09-12T03:40:45.345 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-09-12T03:40:45.345 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 40, 2024-09-12T03:40:45.346 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 11937, 2024-09-12T03:40:45.348 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 40, 2024-09-12T03:40:45.348 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 11937, 2024-09-12T03:40:45.348 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-09-12T03:40:45.348 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:45.348 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:45.349 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:45.349 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-12T03:40:45.349 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.349 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-objecter_ops": { 2024-09-12T03:40:45.349 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-09-12T03:40:45.349 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 1024, 2024-09-12T03:40:45.349 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-09-12T03:40:45.349 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-09-12T03:40:45.349 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-09-12T03:40:45.349 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-09-12T03:40:45.350 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-09-12T03:40:45.350 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 40, 2024-09-12T03:40:45.350 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 40, 2024-09-12T03:40:45.350 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 40, 2024-09-12T03:40:45.350 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 40, 2024-09-12T03:40:45.350 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-09-12T03:40:45.350 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:45.350 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:45.350 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:45.350 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-12T03:40:45.351 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.351 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:40:45.351 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-09-12T03:40:45.351 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 3758096384, 2024-09-12T03:40:45.351 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-09-12T03:40:45.351 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-09-12T03:40:45.351 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-09-12T03:40:45.351 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-09-12T03:40:45.351 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-09-12T03:40:45.351 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-09-12T03:40:45.351 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-09-12T03:40:45.351 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 0, 2024-09-12T03:40:45.351 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 0, 2024-09-12T03:40:45.351 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-09-12T03:40:45.351 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:45.352 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:45.352 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:45.352 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-12T03:40:45.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:45.352 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-write_buf_throttle-0x555a9f7e0960": { 2024-09-12T03:40:45.352 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-09-12T03:40:45.352 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 3758096384, 2024-09-12T03:40:45.352 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-09-12T03:40:45.352 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 4, 2024-09-12T03:40:45.353 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 1476, 2024-09-12T03:40:45.353 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-09-12T03:40:45.353 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 4, 2024-09-12T03:40:45.353 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-09-12T03:40:45.353 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-09-12T03:40:45.353 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 3, 2024-09-12T03:40:45.353 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 1476, 2024-09-12T03:40:45.353 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-09-12T03:40:45.353 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:45.353 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:45.354 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:45.354 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-12T03:40:45.354 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-12T03:40:45.354 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-12T03:40:46.050 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-12T03:40:46.050 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-09-12T03:40:46.050 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-09-12T03:40:46.051 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=16 2024-09-12T03:40:46.051 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-09-12T03:40:46.051 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-12T03:40:46.051 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-12T03:40:46.051 DEBUG:tasks.check_counter:Getting stats from l 2024-09-12T03:40:46.051 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-12T03:40:46.298 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:40:46.770 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-12T03:40:46.770 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:40:46.770 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 244, 2024-09-12T03:40:46.770 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 233, 2024-09-12T03:40:46.770 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 1586800, 2024-09-12T03:40:46.770 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 143424, 2024-09-12T03:40:46.771 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 4, 2024-09-12T03:40:46.771 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 0, 2024-09-12T03:40:46.771 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.047110518, 2024-09-12T03:40:46.771 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.016513313, 2024-09-12T03:40:46.771 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.043487564, 2024-09-12T03:40:46.771 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:40:46.771 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:40:46.771 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 232, 2024-09-12T03:40:46.771 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.015218803, 2024-09-12T03:40:46.771 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000065598 2024-09-12T03:40:46.771 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:46.772 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:40:46.772 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:46.772 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:46.772 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:46.772 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:46.772 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 1586800, 2024-09-12T03:40:46.772 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 143424 2024-09-12T03:40:46.772 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:46.772 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:40:46.773 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 2, 2024-09-12T03:40:46.773 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 179, 2024-09-12T03:40:46.773 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 320, 2024-09-12T03:40:46.773 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 188640, 2024-09-12T03:40:46.773 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 5, 2024-09-12T03:40:46.773 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 3, 2024-09-12T03:40:46.773 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.012231761, 2024-09-12T03:40:46.773 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.008352624, 2024-09-12T03:40:46.773 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.001424953, 2024-09-12T03:40:46.774 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:40:46.774 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:40:46.774 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 176, 2024-09-12T03:40:46.774 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.023042857, 2024-09-12T03:40:46.774 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000130925 2024-09-12T03:40:46.774 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:46.774 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:40:46.775 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:46.775 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:46.775 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:46.775 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:46.775 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T03:40:46.775 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 188640 2024-09-12T03:40:46.775 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:46.775 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:40:46.775 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 4, 2024-09-12T03:40:46.775 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 4, 2024-09-12T03:40:46.775 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 2992, 2024-09-12T03:40:46.775 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 448, 2024-09-12T03:40:46.775 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 2, 2024-09-12T03:40:46.775 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 1, 2024-09-12T03:40:46.776 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.002324308, 2024-09-12T03:40:46.776 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.000036850, 2024-09-12T03:40:46.776 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.002298950, 2024-09-12T03:40:46.776 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:40:46.776 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:40:46.776 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 3, 2024-09-12T03:40:46.776 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.004113375, 2024-09-12T03:40:46.776 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.001371125 2024-09-12T03:40:46.776 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:46.776 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:40:46.776 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:46.777 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:46.777 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:46.777 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:46.780 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-12T03:40:46.780 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T03:40:46.780 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:46.781 INFO:teuthology.orchestra.run.smithi195.stdout: "cct": { 2024-09-12T03:40:46.781 INFO:teuthology.orchestra.run.smithi195.stdout: "total_workers": 0, 2024-09-12T03:40:46.781 INFO:teuthology.orchestra.run.smithi195.stdout: "unhealthy_workers": 0 2024-09-12T03:40:46.781 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:46.781 INFO:teuthology.orchestra.run.smithi195.stdout: "mempool": { 2024-09-12T03:40:46.781 INFO:teuthology.orchestra.run.smithi195.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:40:46.781 INFO:teuthology.orchestra.run.smithi195.stdout: "bloom_filter_items": 0, 2024-09-12T03:40:46.781 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:40:46.781 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:40:46.781 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:40:46.781 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:40:46.781 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:40:46.781 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:40:46.781 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:40:46.782 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:40:46.782 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:40:46.782 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:40:46.782 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:40:46.782 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:40:46.782 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:40:46.782 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_extent_items": 0, 2024-09-12T03:40:46.782 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:40:46.782 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_blob_items": 0, 2024-09-12T03:40:46.782 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:40:46.782 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:40:46.782 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:40:46.782 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:40:46.782 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:40:46.783 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:40:46.783 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:40:46.783 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_txc_items": 0, 2024-09-12T03:40:46.783 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:40:46.783 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:40:46.783 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:40:46.783 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_items": 0, 2024-09-12T03:40:46.783 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_bytes": 0, 2024-09-12T03:40:46.783 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_items": 0, 2024-09-12T03:40:46.783 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:40:46.783 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:40:46.783 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:40:46.783 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:40:46.783 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_anon_bytes": 148714, 2024-09-12T03:40:46.783 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_anon_items": 30, 2024-09-12T03:40:46.784 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:40:46.784 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_meta_items": 0, 2024-09-12T03:40:46.784 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_bytes": 0, 2024-09-12T03:40:46.784 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_items": 0, 2024-09-12T03:40:46.784 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:40:46.784 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_mapbl_items": 0, 2024-09-12T03:40:46.784 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:40:46.784 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_pglog_items": 0, 2024-09-12T03:40:46.784 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_bytes": 0, 2024-09-12T03:40:46.784 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_items": 0, 2024-09-12T03:40:46.784 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:40:46.784 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:40:46.784 INFO:teuthology.orchestra.run.smithi195.stdout: "pgmap_bytes": 0, 2024-09-12T03:40:46.784 INFO:teuthology.orchestra.run.smithi195.stdout: "pgmap_items": 0, 2024-09-12T03:40:46.785 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_co_bytes": 928, 2024-09-12T03:40:46.785 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_co_items": 2, 2024-09-12T03:40:46.785 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_1_bytes": 0, 2024-09-12T03:40:46.785 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_1_items": 0, 2024-09-12T03:40:46.785 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_2_bytes": 0, 2024-09-12T03:40:46.785 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_2_items": 0 2024-09-12T03:40:46.785 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-12T03:40:46.785 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:40:46.785 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-09-12T03:40:46.785 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 104857600, 2024-09-12T03:40:46.785 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-09-12T03:40:46.785 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 246, 2024-09-12T03:40:46.785 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 1552385, 2024-09-12T03:40:46.785 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-09-12T03:40:46.786 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 246, 2024-09-12T03:40:46.786 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-09-12T03:40:46.786 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-09-12T03:40:46.786 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 246, 2024-09-12T03:40:46.786 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 1552385, 2024-09-12T03:40:46.786 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-09-12T03:40:46.786 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-09-12T03:40:46.786 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-09-12T03:40:46.786 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-09-12T03:40:46.786 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-12T03:40:46.786 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-12T03:40:46.786 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-12T03:40:46.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:46 smithi195 ceph-mon[34786]: pgmap v358: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.9 KiB/s wr, 5 op/s 2024-09-12T03:40:47.142 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:46 smithi115 ceph-mon[34801]: pgmap v358: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.9 KiB/s wr, 5 op/s 2024-09-12T03:40:47.271 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:46 smithi064 ceph-mon[30952]: pgmap v358: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.9 KiB/s wr, 5 op/s 2024-09-12T03:40:47.446 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-09-12T03:40:47.446 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-09-12T03:40:47.446 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-09-12T03:40:47.446 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-09-12T03:40:47.446 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-09-12T03:40:47.446 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-09-12T03:40:47.446 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-09-12T03:40:47.447 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-12T03:40:47.459 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-12T03:40:47.459 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-12T03:40:47.460 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:40:47.482 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-12T03:40:47.482 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:40:47.542 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-12T03:40:47.851 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:40:48.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.383+0000 7f0b214a0640 1 -- 172.21.15.64:0/3182229700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b1c101420 msgr2=0x7f0b1c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:48.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.383+0000 7f0b214a0640 1 --2- 172.21.15.64:0/3182229700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b1c101420 0x7f0b1c10d950 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f0b14009cd0 tx=0x7f0b1402a8e0 comp rx=0 tx=0).stop 2024-09-12T03:40:48.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.383+0000 7f0b214a0640 1 -- 172.21.15.64:0/3182229700 shutdown_connections 2024-09-12T03:40:48.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.383+0000 7f0b214a0640 1 --2- 172.21.15.64:0/3182229700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b1c101420 0x7f0b1c10d950 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:48.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.383+0000 7f0b214a0640 1 --2- 172.21.15.64:0/3182229700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b1c100a80 0x7f0b1c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:48.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.383+0000 7f0b214a0640 1 --2- 172.21.15.64:0/3182229700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b1c106a80 0x7f0b1c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:48.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.383+0000 7f0b214a0640 1 -- 172.21.15.64:0/3182229700 >> 172.21.15.64:0/3182229700 conn(0x7f0b1c0fc7a0 msgr2=0x7f0b1c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:40:48.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.383+0000 7f0b214a0640 1 -- 172.21.15.64:0/3182229700 shutdown_connections 2024-09-12T03:40:48.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.383+0000 7f0b214a0640 1 -- 172.21.15.64:0/3182229700 wait complete. 2024-09-12T03:40:48.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.384+0000 7f0b214a0640 1 Processor -- start 2024-09-12T03:40:48.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.384+0000 7f0b214a0640 1 -- start start 2024-09-12T03:40:48.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.384+0000 7f0b214a0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b1c100a80 0x7f0b1c0fce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:48.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.384+0000 7f0b214a0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b1c101420 0x7f0b1c0fd390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:48.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.384+0000 7f0b214a0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b1c106a80 0x7f0b1c0fd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:48.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.384+0000 7f0b214a0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1c110d30 con 0x7f0b1c106a80 2024-09-12T03:40:48.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.384+0000 7f0b214a0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1c110bb0 con 0x7f0b1c100a80 2024-09-12T03:40:48.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.384+0000 7f0b214a0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1c110eb0 con 0x7f0b1c101420 2024-09-12T03:40:48.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.384+0000 7f0b1b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b1c106a80 0x7f0b1c0fd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:48.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.384+0000 7f0b1b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b1c106a80 0x7f0b1c0fd8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59498/0 (socket says 172.21.15.64:59498) 2024-09-12T03:40:48.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.384+0000 7f0b1b7fe640 1 -- 172.21.15.64:0/1026210867 learned_addr learned my addr 172.21.15.64:0/1026210867 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:40:48.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.384+0000 7f0b13fff640 1 --2- 172.21.15.64:0/1026210867 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b1c101420 0x7f0b1c0fd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:48.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.384+0000 7f0b1affd640 1 --2- 172.21.15.64:0/1026210867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b1c100a80 0x7f0b1c0fce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:48.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.385+0000 7f0b1b7fe640 1 -- 172.21.15.64:0/1026210867 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b1c101420 msgr2=0x7f0b1c0fd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:48.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.385+0000 7f0b1b7fe640 1 --2- 172.21.15.64:0/1026210867 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b1c101420 0x7f0b1c0fd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:48.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.385+0000 7f0b1b7fe640 1 -- 172.21.15.64:0/1026210867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b1c100a80 msgr2=0x7f0b1c0fce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:48.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.385+0000 7f0b1b7fe640 1 --2- 172.21.15.64:0/1026210867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b1c100a80 0x7f0b1c0fce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:48.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.385+0000 7f0b1b7fe640 1 -- 172.21.15.64:0/1026210867 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b1c1a6730 con 0x7f0b1c106a80 2024-09-12T03:40:48.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.385+0000 7f0b13fff640 1 --2- 172.21.15.64:0/1026210867 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b1c101420 0x7f0b1c0fd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:40:48.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.385+0000 7f0b1b7fe640 1 --2- 172.21.15.64:0/1026210867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b1c106a80 0x7f0b1c0fd8d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f0b14005cc0 tx=0x7f0b14003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:40:48.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.385+0000 7f0b18ff9640 1 -- 172.21.15.64:0/1026210867 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b14004240 con 0x7f0b1c106a80 2024-09-12T03:40:48.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.385+0000 7f0b18ff9640 1 -- 172.21.15.64:0/1026210867 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b140305a0 con 0x7f0b1c106a80 2024-09-12T03:40:48.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.385+0000 7f0b18ff9640 1 -- 172.21.15.64:0/1026210867 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b140466c0 con 0x7f0b1c106a80 2024-09-12T03:40:48.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.385+0000 7f0b214a0640 1 -- 172.21.15.64:0/1026210867 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b1c1a69c0 con 0x7f0b1c106a80 2024-09-12T03:40:48.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.385+0000 7f0b214a0640 1 -- 172.21.15.64:0/1026210867 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b1c1a6f50 con 0x7f0b1c106a80 2024-09-12T03:40:48.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.386+0000 7f0b214a0640 1 -- 172.21.15.64:0/1026210867 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b1c100ee0 con 0x7f0b1c106a80 2024-09-12T03:40:48.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.387+0000 7f0b18ff9640 1 -- 172.21.15.64:0/1026210867 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0b14030ac0 con 0x7f0b1c106a80 2024-09-12T03:40:48.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.388+0000 7f0b18ff9640 1 --2- 172.21.15.64:0/1026210867 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f0b080a2d70 0x7f0b080a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:48.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.388+0000 7f0b1affd640 1 --2- 172.21.15.64:0/1026210867 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f0b080a2d70 0x7f0b080a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:48.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.388+0000 7f0b18ff9640 1 -- 172.21.15.64:0/1026210867 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9583+0+0 (secure 0 0 0) 0x7f0b1402e030 con 0x7f0b1c106a80 2024-09-12T03:40:48.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.389+0000 7f0b1affd640 1 --2- 172.21.15.64:0/1026210867 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f0b080a2d70 0x7f0b080a5230 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c005d90 tx=0x7f0b0c005d20 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:40:48.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.391+0000 7f0b18ff9640 1 -- 172.21.15.64:0/1026210867 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0b1404b060 con 0x7f0b1c106a80 2024-09-12T03:40:48.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.577+0000 7f0b214a0640 1 -- 172.21.15.64:0/1026210867 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0b1c0fe380 con 0x7f0b1c106a80 2024-09-12T03:40:48.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.578+0000 7f0b18ff9640 1 -- 172.21.15.64:0/1026210867 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v103) ==== 81+0+939 (secure 0 0 0) 0x7f0b1c0fe380 con 0x7f0b1c106a80 2024-09-12T03:40:48.578 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1537191801 2024-09-13T03:29:15.597759+0000 2024-09-12T03:40:48.578 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/686137382 2024-09-13T03:19:36.839431+0000 2024-09-12T03:40:48.578 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/3186640984 2024-09-13T03:19:36.839431+0000 2024-09-12T03:40:48.578 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3334629354 2024-09-13T03:20:07.411102+0000 2024-09-12T03:40:48.578 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/759931848 2024-09-13T03:29:15.597759+0000 2024-09-12T03:40:48.578 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3079679964 2024-09-13T03:19:36.839431+0000 2024-09-12T03:40:48.578 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1217136221 2024-09-13T03:20:07.411102+0000 2024-09-12T03:40:48.578 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/68603379 2024-09-13T03:20:07.411102+0000 2024-09-12T03:40:48.578 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/3186640984 2024-09-13T03:19:36.839431+0000 2024-09-12T03:40:48.578 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3827202979 2024-09-13T03:20:07.411102+0000 2024-09-12T03:40:48.579 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/759931848 2024-09-13T03:29:15.597759+0000 2024-09-12T03:40:48.579 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3564418115 2024-09-13T03:19:36.839431+0000 2024-09-12T03:40:48.579 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/68603379 2024-09-13T03:20:07.411102+0000 2024-09-12T03:40:48.579 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3764771090 2024-09-13T03:29:15.597759+0000 2024-09-12T03:40:48.579 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1757703005 2024-09-13T03:29:15.597759+0000 2024-09-12T03:40:48.579 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3775732799 2024-09-13T03:29:15.597759+0000 2024-09-12T03:40:48.579 INFO:teuthology.orchestra.run.smithi064.stderr:listed 16 entries 2024-09-12T03:40:48.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.581+0000 7f0b214a0640 1 -- 172.21.15.64:0/1026210867 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f0b080a2d70 msgr2=0x7f0b080a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:48.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.581+0000 7f0b214a0640 1 --2- 172.21.15.64:0/1026210867 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f0b080a2d70 0x7f0b080a5230 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c005d90 tx=0x7f0b0c005d20 comp rx=0 tx=0).stop 2024-09-12T03:40:48.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.581+0000 7f0b214a0640 1 -- 172.21.15.64:0/1026210867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b1c106a80 msgr2=0x7f0b1c0fd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:48.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.581+0000 7f0b214a0640 1 --2- 172.21.15.64:0/1026210867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b1c106a80 0x7f0b1c0fd8d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f0b14005cc0 tx=0x7f0b14003ba0 comp rx=0 tx=0).stop 2024-09-12T03:40:48.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.581+0000 7f0b214a0640 1 -- 172.21.15.64:0/1026210867 shutdown_connections 2024-09-12T03:40:48.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.581+0000 7f0b214a0640 1 --2- 172.21.15.64:0/1026210867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b1c106a80 0x7f0b1c0fd8d0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:48.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.581+0000 7f0b214a0640 1 --2- 172.21.15.64:0/1026210867 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b1c101420 0x7f0b1c0fd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:48.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.581+0000 7f0b214a0640 1 --2- 172.21.15.64:0/1026210867 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f0b080a2d70 0x7f0b080a5230 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:48.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.581+0000 7f0b214a0640 1 --2- 172.21.15.64:0/1026210867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b1c100a80 0x7f0b1c0fce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:48.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.581+0000 7f0b214a0640 1 -- 172.21.15.64:0/1026210867 >> 172.21.15.64:0/1026210867 conn(0x7f0b1c0fc7a0 msgr2=0x7f0b1c06aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:40:48.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.582+0000 7f0b214a0640 1 -- 172.21.15.64:0/1026210867 shutdown_connections 2024-09-12T03:40:48.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:48.582+0000 7f0b214a0640 1 -- 172.21.15.64:0/1026210867 wait complete. 2024-09-12T03:40:49.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:48 smithi195 ceph-mon[34786]: pgmap v359: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.9 KiB/s wr, 8 op/s 2024-09-12T03:40:49.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:48 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/1026210867' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:40:49.260 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:48 smithi064 ceph-mon[30952]: pgmap v359: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.9 KiB/s wr, 8 op/s 2024-09-12T03:40:49.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:48 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/1026210867' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:40:49.289 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-12T03:40:49.289 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:40:49.307 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-12T03:40:49.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:48 smithi115 ceph-mon[34801]: pgmap v359: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.9 KiB/s wr, 8 op/s 2024-09-12T03:40:49.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:48 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/1026210867' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:40:49.620 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:40:50.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.108+0000 7fbfa801e640 1 -- 172.21.15.64:0/781765296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa0102790 msgr2=0x7fbfa01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:50.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.108+0000 7fbfa801e640 1 --2- 172.21.15.64:0/781765296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa0102790 0x7fbfa01067e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009900 tx=0x7fbf9402a410 comp rx=0 tx=0).stop 2024-09-12T03:40:50.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.108+0000 7fbfa801e640 1 -- 172.21.15.64:0/781765296 shutdown_connections 2024-09-12T03:40:50.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.108+0000 7fbfa801e640 1 --2- 172.21.15.64:0/781765296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfa0106f10 0x7fbfa010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:50.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.108+0000 7fbfa801e640 1 --2- 172.21.15.64:0/781765296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa0102790 0x7fbfa01067e0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:50.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.108+0000 7fbfa801e640 1 --2- 172.21.15.64:0/781765296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfa0101de0 0x7fbfa01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:50.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.108+0000 7fbfa801e640 1 -- 172.21.15.64:0/781765296 >> 172.21.15.64:0/781765296 conn(0x7fbfa00fd8c0 msgr2=0x7fbfa00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:40:50.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.108+0000 7fbfa801e640 1 -- 172.21.15.64:0/781765296 shutdown_connections 2024-09-12T03:40:50.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.108+0000 7fbfa801e640 1 -- 172.21.15.64:0/781765296 wait complete. 2024-09-12T03:40:50.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.108+0000 7fbfa801e640 1 Processor -- start 2024-09-12T03:40:50.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.109+0000 7fbfa801e640 1 -- start start 2024-09-12T03:40:50.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.109+0000 7fbfa801e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfa0101de0 0x7fbfa0198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:50.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.109+0000 7fbfa801e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa0102790 0x7fbfa0198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:50.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.109+0000 7fbfa801e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfa0106f10 0x7fbfa019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:50.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.109+0000 7fbfa801e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa0111e20 con 0x7fbfa0102790 2024-09-12T03:40:50.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.109+0000 7fbfa801e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa0111ca0 con 0x7fbfa0101de0 2024-09-12T03:40:50.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.109+0000 7fbfa801e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa0111fa0 con 0x7fbfa0106f10 2024-09-12T03:40:50.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.109+0000 7fbfa55b9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa0102790 0x7fbfa0198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:50.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.109+0000 7fbfa55b9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa0102790 0x7fbfa0198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59514/0 (socket says 172.21.15.64:59514) 2024-09-12T03:40:50.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.109+0000 7fbfa55b9640 1 -- 172.21.15.64:0/1420559408 learned_addr learned my addr 172.21.15.64:0/1420559408 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:40:50.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.109+0000 7fbfa5dba640 1 --2- 172.21.15.64:0/1420559408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfa0101de0 0x7fbfa0198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:50.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.109+0000 7fbfa65bb640 1 --2- 172.21.15.64:0/1420559408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfa0106f10 0x7fbfa019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:50.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.109+0000 7fbfa55b9640 1 -- 172.21.15.64:0/1420559408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfa0106f10 msgr2=0x7fbfa019cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:50.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.109+0000 7fbfa55b9640 1 --2- 172.21.15.64:0/1420559408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfa0106f10 0x7fbfa019cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:50.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.109+0000 7fbfa55b9640 1 -- 172.21.15.64:0/1420559408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfa0101de0 msgr2=0x7fbfa0198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:50.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.109+0000 7fbfa55b9640 1 --2- 172.21.15.64:0/1420559408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfa0101de0 0x7fbfa0198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:50.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.109+0000 7fbfa55b9640 1 -- 172.21.15.64:0/1420559408 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfa019d2d0 con 0x7fbfa0102790 2024-09-12T03:40:50.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.110+0000 7fbfa55b9640 1 --2- 172.21.15.64:0/1420559408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa0102790 0x7fbfa0198860 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fbf94005940 tx=0x7fbf940042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:40:50.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.110+0000 7fbf9effd640 1 -- 172.21.15.64:0/1420559408 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9403f030 con 0x7fbfa0102790 2024-09-12T03:40:50.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.110+0000 7fbf9effd640 1 -- 172.21.15.64:0/1420559408 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf94003cb0 con 0x7fbfa0102790 2024-09-12T03:40:50.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.110+0000 7fbfa801e640 1 -- 172.21.15.64:0/1420559408 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfa019d560 con 0x7fbfa0102790 2024-09-12T03:40:50.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.110+0000 7fbf9effd640 1 -- 172.21.15.64:0/1420559408 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf94035070 con 0x7fbfa0102790 2024-09-12T03:40:50.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.110+0000 7fbfa801e640 1 -- 172.21.15.64:0/1420559408 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfa01a4ea0 con 0x7fbfa0102790 2024-09-12T03:40:50.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.111+0000 7fbfa801e640 1 -- 172.21.15.64:0/1420559408 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfa01032c0 con 0x7fbfa0102790 2024-09-12T03:40:50.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.112+0000 7fbf9effd640 1 -- 172.21.15.64:0/1420559408 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fbf94042030 con 0x7fbfa0102790 2024-09-12T03:40:50.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.113+0000 7fbf9effd640 1 --2- 172.21.15.64:0/1420559408 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fbf880a2d70 0x7fbf880a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:50.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.113+0000 7fbf9effd640 1 -- 172.21.15.64:0/1420559408 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9583+0+0 (secure 0 0 0) 0x7fbf940f0170 con 0x7fbfa0102790 2024-09-12T03:40:50.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.113+0000 7fbfa5dba640 1 --2- 172.21.15.64:0/1420559408 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fbf880a2d70 0x7fbf880a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:50.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.113+0000 7fbfa5dba640 1 --2- 172.21.15.64:0/1420559408 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fbf880a2d70 0x7fbf880a5230 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbf98009bd0 tx=0x7fbf98005f90 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:40:50.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.116+0000 7fbf9effd640 1 -- 172.21.15.64:0/1420559408 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbf940bc320 con 0x7fbfa0102790 2024-09-12T03:40:50.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.306+0000 7fbfa801e640 1 -- 172.21.15.64:0/1420559408 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbfa01021c0 con 0x7fbfa0102790 2024-09-12T03:40:50.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.306+0000 7fbf9effd640 1 -- 172.21.15.64:0/1420559408 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v103) ==== 81+0+939 (secure 0 0 0) 0x7fbfa01021c0 con 0x7fbfa0102790 2024-09-12T03:40:50.306 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1537191801 2024-09-13T03:29:15.597759+0000 2024-09-12T03:40:50.306 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/686137382 2024-09-13T03:19:36.839431+0000 2024-09-12T03:40:50.306 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/3186640984 2024-09-13T03:19:36.839431+0000 2024-09-12T03:40:50.306 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3334629354 2024-09-13T03:20:07.411102+0000 2024-09-12T03:40:50.306 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/759931848 2024-09-13T03:29:15.597759+0000 2024-09-12T03:40:50.306 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3079679964 2024-09-13T03:19:36.839431+0000 2024-09-12T03:40:50.307 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1217136221 2024-09-13T03:20:07.411102+0000 2024-09-12T03:40:50.307 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/68603379 2024-09-13T03:20:07.411102+0000 2024-09-12T03:40:50.307 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/3186640984 2024-09-13T03:19:36.839431+0000 2024-09-12T03:40:50.307 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3827202979 2024-09-13T03:20:07.411102+0000 2024-09-12T03:40:50.307 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/759931848 2024-09-13T03:29:15.597759+0000 2024-09-12T03:40:50.307 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3564418115 2024-09-13T03:19:36.839431+0000 2024-09-12T03:40:50.307 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/68603379 2024-09-13T03:20:07.411102+0000 2024-09-12T03:40:50.307 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3764771090 2024-09-13T03:29:15.597759+0000 2024-09-12T03:40:50.307 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1757703005 2024-09-13T03:29:15.597759+0000 2024-09-12T03:40:50.307 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3775732799 2024-09-13T03:29:15.597759+0000 2024-09-12T03:40:50.307 INFO:teuthology.orchestra.run.smithi064.stderr:listed 16 entries 2024-09-12T03:40:50.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.309+0000 7fbfa801e640 1 -- 172.21.15.64:0/1420559408 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fbf880a2d70 msgr2=0x7fbf880a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:50.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.309+0000 7fbfa801e640 1 --2- 172.21.15.64:0/1420559408 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fbf880a2d70 0x7fbf880a5230 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbf98009bd0 tx=0x7fbf98005f90 comp rx=0 tx=0).stop 2024-09-12T03:40:50.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.309+0000 7fbfa801e640 1 -- 172.21.15.64:0/1420559408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa0102790 msgr2=0x7fbfa0198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:50.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.309+0000 7fbfa801e640 1 --2- 172.21.15.64:0/1420559408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa0102790 0x7fbfa0198860 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fbf94005940 tx=0x7fbf940042f0 comp rx=0 tx=0).stop 2024-09-12T03:40:50.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.309+0000 7fbfa801e640 1 -- 172.21.15.64:0/1420559408 shutdown_connections 2024-09-12T03:40:50.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.309+0000 7fbfa801e640 1 --2- 172.21.15.64:0/1420559408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfa0106f10 0x7fbfa019cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:50.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.309+0000 7fbfa801e640 1 --2- 172.21.15.64:0/1420559408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfa0102790 0x7fbfa0198860 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:50.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.309+0000 7fbfa801e640 1 --2- 172.21.15.64:0/1420559408 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fbf880a2d70 0x7fbf880a5230 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:50.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.309+0000 7fbfa801e640 1 --2- 172.21.15.64:0/1420559408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfa0101de0 0x7fbfa0198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:50.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.309+0000 7fbfa801e640 1 -- 172.21.15.64:0/1420559408 >> 172.21.15.64:0/1420559408 conn(0x7fbfa00fd8c0 msgr2=0x7fbfa00fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:40:50.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.309+0000 7fbfa801e640 1 -- 172.21.15.64:0/1420559408 shutdown_connections 2024-09-12T03:40:50.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:50.309+0000 7fbfa801e640 1 -- 172.21.15.64:0/1420559408 wait complete. 2024-09-12T03:40:50.985 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi064.front.sepia.ceph.com... 2024-09-12T03:40:50.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:40:50.985 DEBUG:teuthology.orchestra.run.smithi064:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-12T03:40:51.012 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[84997]: fuse finished with error 0 and tester_r 0 2024-09-12T03:40:51.014 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:40:51.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:50 smithi195 ceph-mon[34786]: pgmap v360: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-12T03:40:51.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:50 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/1420559408' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:40:51.271 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:50 smithi064 ceph-mon[30952]: pgmap v360: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-12T03:40:51.272 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:50 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/1420559408' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:40:51.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:50 smithi115 ceph-mon[34801]: pgmap v360: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-12T03:40:51.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:50 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/1420559408' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:40:53.195 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:52 smithi115 ceph-mon[34801]: pgmap v361: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-12T03:40:53.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:52 smithi195 ceph-mon[34786]: pgmap v361: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-12T03:40:53.271 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:52 smithi064 ceph-mon[30952]: pgmap v361: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-12T03:40:55.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:54 smithi195 ceph-mon[34786]: pgmap v362: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-12T03:40:55.271 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:54 smithi064 ceph-mon[30952]: pgmap v362: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-12T03:40:55.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:54 smithi115 ceph-mon[34801]: pgmap v362: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-12T03:40:57.116 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2024-09-12T03:40:57.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:40:57.116 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T03:40:57.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:40:57.137 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2024-09-12T03:40:57.197 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-12T03:40:57.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:40:57.197 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T03:40:57.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:56 smithi195 ceph-mon[34786]: pgmap v363: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-12T03:40:57.263 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:56 smithi064 ceph-mon[30952]: pgmap v363: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-12T03:40:57.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:40:57.268 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link delete ceph-brx 2024-09-12T03:40:57.372 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-12T03:40:57.384 INFO:tasks.cephadm:Teardown begin 2024-09-12T03:40:57.384 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-12T03:40:57.385 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:40:57.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:56 smithi115 ceph-mon[34801]: pgmap v363: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-12T03:40:57.649 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:40:58.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.149+0000 7f0289162640 1 -- 172.21.15.64:0/1856623923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0284100a80 msgr2=0x7f0284100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:58.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.149+0000 7f0289162640 1 --2- 172.21.15.64:0/1856623923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0284100a80 0x7f0284100ee0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f0278009930 tx=0x7f027802a420 comp rx=0 tx=0).stop 2024-09-12T03:40:58.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.149+0000 7f0289162640 1 -- 172.21.15.64:0/1856623923 shutdown_connections 2024-09-12T03:40:58.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.149+0000 7f0289162640 1 --2- 172.21.15.64:0/1856623923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0284101420 0x7f028410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:58.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.149+0000 7f0289162640 1 --2- 172.21.15.64:0/1856623923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0284100a80 0x7f0284100ee0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:58.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.149+0000 7f0289162640 1 --2- 172.21.15.64:0/1856623923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0284106a80 0x7f0284106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:58.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.149+0000 7f0289162640 1 -- 172.21.15.64:0/1856623923 >> 172.21.15.64:0/1856623923 conn(0x7f02840fc7a0 msgr2=0x7f02840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:40:58.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.149+0000 7f0289162640 1 -- 172.21.15.64:0/1856623923 shutdown_connections 2024-09-12T03:40:58.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.149+0000 7f0289162640 1 -- 172.21.15.64:0/1856623923 wait complete. 2024-09-12T03:40:58.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.150+0000 7f0289162640 1 Processor -- start 2024-09-12T03:40:58.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.150+0000 7f0289162640 1 -- start start 2024-09-12T03:40:58.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.150+0000 7f0289162640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0284100a80 0x7f02841983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:58.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.150+0000 7f0289162640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0284101420 0x7f0284198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:58.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.150+0000 7f0289162640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0284106a80 0x7f028419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:58.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.150+0000 7f0289162640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0284110d50 con 0x7f0284100a80 2024-09-12T03:40:58.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.150+0000 7f0289162640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0284110bd0 con 0x7f0284101420 2024-09-12T03:40:58.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.150+0000 7f0289162640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0284110ed0 con 0x7f0284106a80 2024-09-12T03:40:58.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.151+0000 7f0282d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0284100a80 0x7f02841983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:58.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.151+0000 7f0282d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0284100a80 0x7f02841983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38656/0 (socket says 172.21.15.64:38656) 2024-09-12T03:40:58.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.151+0000 7f0282d9d640 1 -- 172.21.15.64:0/4246366185 learned_addr learned my addr 172.21.15.64:0/4246366185 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:40:58.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.151+0000 7f028259c640 1 --2- 172.21.15.64:0/4246366185 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0284101420 0x7f0284198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:58.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.151+0000 7f028359e640 1 --2- 172.21.15.64:0/4246366185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0284106a80 0x7f028419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:58.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.151+0000 7f0282d9d640 1 -- 172.21.15.64:0/4246366185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0284106a80 msgr2=0x7f028419ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:58.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.151+0000 7f0282d9d640 1 --2- 172.21.15.64:0/4246366185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0284106a80 0x7f028419ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:58.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.151+0000 7f0282d9d640 1 -- 172.21.15.64:0/4246366185 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0284101420 msgr2=0x7f0284198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:58.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.151+0000 7f0282d9d640 1 --2- 172.21.15.64:0/4246366185 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0284101420 0x7f0284198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:58.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.151+0000 7f0282d9d640 1 -- 172.21.15.64:0/4246366185 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f028419d390 con 0x7f0284100a80 2024-09-12T03:40:58.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.151+0000 7f028259c640 1 --2- 172.21.15.64:0/4246366185 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0284101420 0x7f0284198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:40:58.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.151+0000 7f028359e640 1 --2- 172.21.15.64:0/4246366185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0284106a80 0x7f028419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:40:58.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.151+0000 7f0282d9d640 1 --2- 172.21.15.64:0/4246366185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0284100a80 0x7f02841983e0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f027400c890 tx=0x7f027400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:40:58.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.151+0000 7f026bfff640 1 -- 172.21.15.64:0/4246366185 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0274004220 con 0x7f0284100a80 2024-09-12T03:40:58.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.151+0000 7f0289162640 1 -- 172.21.15.64:0/4246366185 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f028419d680 con 0x7f0284100a80 2024-09-12T03:40:58.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.151+0000 7f026bfff640 1 -- 172.21.15.64:0/4246366185 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02740043c0 con 0x7f0284100a80 2024-09-12T03:40:58.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.151+0000 7f026bfff640 1 -- 172.21.15.64:0/4246366185 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0274004a00 con 0x7f0284100a80 2024-09-12T03:40:58.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.151+0000 7f0289162640 1 -- 172.21.15.64:0/4246366185 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02841a4f60 con 0x7f0284100a80 2024-09-12T03:40:58.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.152+0000 7f0289162640 1 -- 172.21.15.64:0/4246366185 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0284100ee0 con 0x7f0284100a80 2024-09-12T03:40:58.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.154+0000 7f026bfff640 1 -- 172.21.15.64:0/4246366185 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0274004ba0 con 0x7f0284100a80 2024-09-12T03:40:58.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.154+0000 7f026bfff640 1 --2- 172.21.15.64:0/4246366185 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f026c0a2cf0 0x7f026c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:58.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.154+0000 7f028259c640 1 --2- 172.21.15.64:0/4246366185 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f026c0a2cf0 0x7f026c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:58.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.154+0000 7f026bfff640 1 -- 172.21.15.64:0/4246366185 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9583+0+0 (secure 0 0 0) 0x7f02740cfa00 con 0x7f0284100a80 2024-09-12T03:40:58.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.155+0000 7f028259c640 1 --2- 172.21.15.64:0/4246366185 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f026c0a2cf0 0x7f026c0a51b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0278000c00 tx=0x7f0278031cf0 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:40:58.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.158+0000 7f026bfff640 1 -- 172.21.15.64:0/4246366185 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0274014030 con 0x7f0284100a80 2024-09-12T03:40:58.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.369+0000 7f0289162640 1 -- 172.21.15.64:0/4246366185 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f028402d7c0 con 0x7f0284100a80 2024-09-12T03:40:58.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.370+0000 7f026bfff640 1 -- 172.21.15.64:0/4246366185 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11921 (secure 0 0 0) 0x7f027409c200 con 0x7f0284100a80 2024-09-12T03:40:58.370 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:40:58.370 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":25,"btime":"2024-09-12T03:29:14:375629+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2580078738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2580078738},{"type":"v1","addr":"172.21.15.64:6835","nonce":2580078738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2333415004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2333415004},{"type":"v1","addr":"172.21.15.64:6837","nonce":2333415004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/3030618395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":3030618395},{"type":"v1","addr":"172.21.15.64:6839","nonce":3030618395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3333057625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3333057625},{"type":"v1","addr":"172.21.15.64:6841","nonce":3333057625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2991933266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2991933266},{"type":"v1","addr":"172.21.15.195:6833","nonce":2991933266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1138960346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1138960346},{"type":"v1","addr":"172.21.15.115:6833","nonce":1138960346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:26:59.647657+0000","modified":"2024-09-12T03:29:14.325461+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24481,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/332737916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":332737916},{"type":"v1","addr":"172.21.15.115:6835","nonce":332737916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/2531045583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2531045583},{"type":"v1","addr":"172.21.15.195:6835","nonce":2531045583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/71647401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":71647401},{"type":"v1","addr":"172.21.15.195:6837","nonce":71647401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.115:6837/2915155705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2915155705},{"type":"v1","addr":"172.21.15.115:6837","nonce":2915155705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.195:6839/2574695134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":2574695134},{"type":"v1","addr":"172.21.15.195:6839","nonce":2574695134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6839/2238031796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2238031796},{"type":"v1","addr":"172.21.15.115:6839","nonce":2238031796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24481,24470]},"id":1}]} 2024-09-12T03:40:58.370 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 25 2024-09-12T03:40:58.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.372+0000 7f0289162640 1 -- 172.21.15.64:0/4246366185 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f026c0a2cf0 msgr2=0x7f026c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:58.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.372+0000 7f0289162640 1 --2- 172.21.15.64:0/4246366185 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f026c0a2cf0 0x7f026c0a51b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0278000c00 tx=0x7f0278031cf0 comp rx=0 tx=0).stop 2024-09-12T03:40:58.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.372+0000 7f0289162640 1 -- 172.21.15.64:0/4246366185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0284100a80 msgr2=0x7f02841983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:58.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.372+0000 7f0289162640 1 --2- 172.21.15.64:0/4246366185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0284100a80 0x7f02841983e0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f027400c890 tx=0x7f027400cd50 comp rx=0 tx=0).stop 2024-09-12T03:40:58.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.372+0000 7f0289162640 1 -- 172.21.15.64:0/4246366185 shutdown_connections 2024-09-12T03:40:58.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.372+0000 7f0289162640 1 --2- 172.21.15.64:0/4246366185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0284106a80 0x7f028419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:58.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.372+0000 7f0289162640 1 --2- 172.21.15.64:0/4246366185 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f026c0a2cf0 0x7f026c0a51b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:58.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.372+0000 7f0289162640 1 --2- 172.21.15.64:0/4246366185 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0284101420 0x7f0284198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:58.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.372+0000 7f0289162640 1 --2- 172.21.15.64:0/4246366185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0284100a80 0x7f02841983e0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:58.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.372+0000 7f0289162640 1 -- 172.21.15.64:0/4246366185 >> 172.21.15.64:0/4246366185 conn(0x7f02840fc7a0 msgr2=0x7f028406aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:40:58.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.372+0000 7f0289162640 1 -- 172.21.15.64:0/4246366185 shutdown_connections 2024-09-12T03:40:58.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:58.372+0000 7f0289162640 1 -- 172.21.15.64:0/4246366185 wait complete. 2024-09-12T03:40:59.021 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:58 smithi064 ceph-mon[30952]: pgmap v364: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-09-12T03:40:59.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:40:58 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/4246366185' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:40:59.105 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:40:59.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:58 smithi195 ceph-mon[34786]: pgmap v364: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-09-12T03:40:59.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:40:58 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/4246366185' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:40:59.366 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:40:59.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:58 smithi115 ceph-mon[34801]: pgmap v364: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-09-12T03:40:59.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:40:58 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/4246366185' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:40:59.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:59.892+0000 7fc04c32e640 1 -- 172.21.15.64:0/923797104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0441082a0 msgr2=0x7fc04410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:59.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:59.892+0000 7fc04c32e640 1 --2- 172.21.15.64:0/923797104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0441082a0 0x7fc04410ec40 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc03c009cd0 tx=0x7fc03c02a8e0 comp rx=0 tx=0).stop 2024-09-12T03:40:59.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:59.892+0000 7fc04c32e640 1 -- 172.21.15.64:0/923797104 shutdown_connections 2024-09-12T03:40:59.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:59.892+0000 7fc04c32e640 1 --2- 172.21.15.64:0/923797104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0441082a0 0x7fc04410ec40 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:59.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:59.892+0000 7fc04c32e640 1 --2- 172.21.15.64:0/923797104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0441036e0 0x7fc044107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:59.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:59.892+0000 7fc04c32e640 1 --2- 172.21.15.64:0/923797104 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc044102d30 0x7fc044103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:59.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:59.892+0000 7fc04c32e640 1 -- 172.21.15.64:0/923797104 >> 172.21.15.64:0/923797104 conn(0x7fc0440fc7d0 msgr2=0x7fc0440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:40:59.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:59.892+0000 7fc04c32e640 1 -- 172.21.15.64:0/923797104 shutdown_connections 2024-09-12T03:40:59.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:59.892+0000 7fc04c32e640 1 -- 172.21.15.64:0/923797104 wait complete. 2024-09-12T03:40:59.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:59.893+0000 7fc04c32e640 1 Processor -- start 2024-09-12T03:40:59.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:59.893+0000 7fc04c32e640 1 -- start start 2024-09-12T03:40:59.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:59.893+0000 7fc04c32e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc044102d30 0x7fc044198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:59.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:59.893+0000 7fc04c32e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc0441036e0 0x7fc0441989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:59.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:59.893+0000 7fc04c32e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0441082a0 0x7fc04419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:59.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:59.893+0000 7fc04c32e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc044111fa0 con 0x7fc044102d30 2024-09-12T03:40:59.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:59.893+0000 7fc04c32e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc044111e20 con 0x7fc0441082a0 2024-09-12T03:40:59.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:59.893+0000 7fc04c32e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc044112120 con 0x7fc0441036e0 2024-09-12T03:40:59.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:59.893+0000 7fc04a0ca640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc044102d30 0x7fc044198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:59.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:59.893+0000 7fc04a0ca640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc044102d30 0x7fc044198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38680/0 (socket says 172.21.15.64:38680) 2024-09-12T03:40:59.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:59.893+0000 7fc04a0ca640 1 -- 172.21.15.64:0/2456598146 learned_addr learned my addr 172.21.15.64:0/2456598146 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:40:59.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:59.893+0000 7fc0498c9640 1 --2- 172.21.15.64:0/2456598146 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc0441036e0 0x7fc0441989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:59.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:59.893+0000 7fc04a8cb640 1 --2- 172.21.15.64:0/2456598146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0441082a0 0x7fc04419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:59.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:59.893+0000 7fc04a0ca640 1 -- 172.21.15.64:0/2456598146 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc0441036e0 msgr2=0x7fc0441989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:59.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:59.893+0000 7fc04a0ca640 1 --2- 172.21.15.64:0/2456598146 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc0441036e0 0x7fc0441989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:59.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:59.893+0000 7fc04a0ca640 1 -- 172.21.15.64:0/2456598146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0441082a0 msgr2=0x7fc04419cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:40:59.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:59.893+0000 7fc04a0ca640 1 --2- 172.21.15.64:0/2456598146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0441082a0 0x7fc04419cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:40:59.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:59.893+0000 7fc04a0ca640 1 -- 172.21.15.64:0/2456598146 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc04419d410 con 0x7fc044102d30 2024-09-12T03:40:59.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:59.894+0000 7fc04a0ca640 1 --2- 172.21.15.64:0/2456598146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc044102d30 0x7fc044198460 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fc04000d770 tx=0x7fc04000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:40:59.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:59.894+0000 7fc03b7fe640 1 -- 172.21.15.64:0/2456598146 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc040017070 con 0x7fc044102d30 2024-09-12T03:40:59.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:59.894+0000 7fc04c32e640 1 -- 172.21.15.64:0/2456598146 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc04419d700 con 0x7fc044102d30 2024-09-12T03:40:59.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:59.894+0000 7fc03b7fe640 1 -- 172.21.15.64:0/2456598146 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc0400054d0 con 0x7fc044102d30 2024-09-12T03:40:59.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:59.894+0000 7fc03b7fe640 1 -- 172.21.15.64:0/2456598146 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc040004cf0 con 0x7fc044102d30 2024-09-12T03:40:59.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:59.894+0000 7fc04c32e640 1 -- 172.21.15.64:0/2456598146 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0441a4fe0 con 0x7fc044102d30 2024-09-12T03:40:59.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:59.895+0000 7fc04c32e640 1 -- 172.21.15.64:0/2456598146 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc044103110 con 0x7fc044102d30 2024-09-12T03:40:59.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:59.896+0000 7fc03b7fe640 1 -- 172.21.15.64:0/2456598146 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc040004e90 con 0x7fc044102d30 2024-09-12T03:40:59.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:59.897+0000 7fc03b7fe640 1 --2- 172.21.15.64:0/2456598146 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fc0300a2cf0 0x7fc0300a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:40:59.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:59.897+0000 7fc0498c9640 1 --2- 172.21.15.64:0/2456598146 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fc0300a2cf0 0x7fc0300a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:40:59.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:59.897+0000 7fc03b7fe640 1 -- 172.21.15.64:0/2456598146 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9583+0+0 (secure 0 0 0) 0x7fc0400cf2e0 con 0x7fc044102d30 2024-09-12T03:40:59.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:59.897+0000 7fc0498c9640 1 --2- 172.21.15.64:0/2456598146 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fc0300a2cf0 0x7fc0300a51b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc0440fed90 tx=0x7fc03400a2d0 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:40:59.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:40:59.901+0000 7fc03b7fe640 1 -- 172.21.15.64:0/2456598146 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc04009bae0 con 0x7fc044102d30 2024-09-12T03:41:00.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:00.116+0000 7fc04c32e640 1 -- 172.21.15.64:0/2456598146 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc04410a2b0 con 0x7fc044102d30 2024-09-12T03:41:00.117 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:41:00.117 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":25,"btime":"2024-09-12T03:29:14:375629+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2580078738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2580078738},{"type":"v1","addr":"172.21.15.64:6835","nonce":2580078738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2333415004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2333415004},{"type":"v1","addr":"172.21.15.64:6837","nonce":2333415004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/3030618395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":3030618395},{"type":"v1","addr":"172.21.15.64:6839","nonce":3030618395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/3333057625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":3333057625},{"type":"v1","addr":"172.21.15.64:6841","nonce":3333057625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2991933266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2991933266},{"type":"v1","addr":"172.21.15.195:6833","nonce":2991933266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/1138960346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1138960346},{"type":"v1","addr":"172.21.15.115:6833","nonce":1138960346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:26:59.647657+0000","modified":"2024-09-12T03:29:14.325461+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24481,"mds_2":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/332737916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":332737916},{"type":"v1","addr":"172.21.15.115:6835","nonce":332737916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/2531045583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2531045583},{"type":"v1","addr":"172.21.15.195:6835","nonce":2531045583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/71647401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":71647401},{"type":"v1","addr":"172.21.15.195:6837","nonce":71647401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.115:6837/2915155705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2915155705},{"type":"v1","addr":"172.21.15.115:6837","nonce":2915155705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.195:6839/2574695134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":2574695134},{"type":"v1","addr":"172.21.15.195:6839","nonce":2574695134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6839/2238031796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":2238031796},{"type":"v1","addr":"172.21.15.115:6839","nonce":2238031796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24481,24470]},"id":1}]} 2024-09-12T03:41:00.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:00.117+0000 7fc03b7fe640 1 -- 172.21.15.64:0/2456598146 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11921 (secure 0 0 0) 0x7fc04009b3e0 con 0x7fc044102d30 2024-09-12T03:41:00.117 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 25 2024-09-12T03:41:00.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:00.119+0000 7fc04c32e640 1 -- 172.21.15.64:0/2456598146 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fc0300a2cf0 msgr2=0x7fc0300a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:41:00.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:00.119+0000 7fc04c32e640 1 --2- 172.21.15.64:0/2456598146 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fc0300a2cf0 0x7fc0300a51b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc0440fed90 tx=0x7fc03400a2d0 comp rx=0 tx=0).stop 2024-09-12T03:41:00.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:00.119+0000 7fc04c32e640 1 -- 172.21.15.64:0/2456598146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc044102d30 msgr2=0x7fc044198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:41:00.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:00.119+0000 7fc04c32e640 1 --2- 172.21.15.64:0/2456598146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc044102d30 0x7fc044198460 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fc04000d770 tx=0x7fc04000dc30 comp rx=0 tx=0).stop 2024-09-12T03:41:00.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:00.119+0000 7fc04c32e640 1 -- 172.21.15.64:0/2456598146 shutdown_connections 2024-09-12T03:41:00.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:00.119+0000 7fc04c32e640 1 --2- 172.21.15.64:0/2456598146 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fc0300a2cf0 0x7fc0300a51b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:00.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:00.119+0000 7fc04c32e640 1 --2- 172.21.15.64:0/2456598146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0441082a0 0x7fc04419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:00.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:00.119+0000 7fc04c32e640 1 --2- 172.21.15.64:0/2456598146 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc0441036e0 0x7fc0441989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:00.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:00.119+0000 7fc04c32e640 1 --2- 172.21.15.64:0/2456598146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc044102d30 0x7fc044198460 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:00.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:00.119+0000 7fc04c32e640 1 -- 172.21.15.64:0/2456598146 >> 172.21.15.64:0/2456598146 conn(0x7fc0440fc7d0 msgr2=0x7fc044106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:41:00.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:00.119+0000 7fc04c32e640 1 -- 172.21.15.64:0/2456598146 shutdown_connections 2024-09-12T03:41:00.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:00.119+0000 7fc04c32e640 1 -- 172.21.15.64:0/2456598146 wait complete. 2024-09-12T03:41:00.769 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:41:01.021 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:00 smithi064 ceph-mon[30952]: pgmap v365: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-12T03:41:01.021 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:00 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/2456598146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:41:01.038 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:41:01.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:00 smithi195 ceph-mon[34786]: pgmap v365: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-12T03:41:01.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:00 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/2456598146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:41:01.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:00 smithi115 ceph-mon[34801]: pgmap v365: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-12T03:41:01.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:00 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/2456598146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:41:01.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.536+0000 7f81ef945640 1 -- 172.21.15.64:0/2036716998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81e8102570 msgr2=0x7f81e81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:41:01.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.536+0000 7f81ef945640 1 --2- 172.21.15.64:0/2036716998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81e8102570 0x7f81e81029b0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f81dc009900 tx=0x7f81dc02a480 comp rx=0 tx=0).stop 2024-09-12T03:41:01.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.536+0000 7f81ef945640 1 -- 172.21.15.64:0/2036716998 shutdown_connections 2024-09-12T03:41:01.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.536+0000 7f81ef945640 1 --2- 172.21.15.64:0/2036716998 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81e8102ef0 0x7f81e810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:01.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.536+0000 7f81ef945640 1 --2- 172.21.15.64:0/2036716998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81e8102570 0x7f81e81029b0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:01.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.536+0000 7f81ef945640 1 --2- 172.21.15.64:0/2036716998 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81e8100e20 0x7f81e8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:01.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.536+0000 7f81ef945640 1 -- 172.21.15.64:0/2036716998 >> 172.21.15.64:0/2036716998 conn(0x7f81e80fc7d0 msgr2=0x7f81e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:41:01.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.536+0000 7f81ef945640 1 -- 172.21.15.64:0/2036716998 shutdown_connections 2024-09-12T03:41:01.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.536+0000 7f81ef945640 1 -- 172.21.15.64:0/2036716998 wait complete. 2024-09-12T03:41:01.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.537+0000 7f81ef945640 1 Processor -- start 2024-09-12T03:41:01.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.537+0000 7f81ef945640 1 -- start start 2024-09-12T03:41:01.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.537+0000 7f81ef945640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81e8100e20 0x7f81e8198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:41:01.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.537+0000 7f81ef945640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81e8102570 0x7f81e81989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:41:01.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.537+0000 7f81ef945640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81e8102ef0 0x7f81e819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:41:01.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.537+0000 7f81ef945640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f81e8110e30 con 0x7f81e8102ef0 2024-09-12T03:41:01.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.537+0000 7f81ef945640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f81e8110cb0 con 0x7f81e8102570 2024-09-12T03:41:01.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.537+0000 7f81ef945640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f81e8110fb0 con 0x7f81e8100e20 2024-09-12T03:41:01.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.537+0000 7f81edee2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81e8102ef0 0x7f81e819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:41:01.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.537+0000 7f81edee2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81e8102ef0 0x7f81e819cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38704/0 (socket says 172.21.15.64:38704) 2024-09-12T03:41:01.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.537+0000 7f81edee2640 1 -- 172.21.15.64:0/1753559723 learned_addr learned my addr 172.21.15.64:0/1753559723 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:41:01.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.537+0000 7f81ed6e1640 1 --2- 172.21.15.64:0/1753559723 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81e8100e20 0x7f81e8198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:41:01.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.537+0000 7f81ecee0640 1 --2- 172.21.15.64:0/1753559723 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81e8102570 0x7f81e81989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:41:01.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.537+0000 7f81edee2640 1 -- 172.21.15.64:0/1753559723 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81e8100e20 msgr2=0x7f81e8198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:41:01.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.537+0000 7f81edee2640 1 --2- 172.21.15.64:0/1753559723 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81e8100e20 0x7f81e8198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:01.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.537+0000 7f81edee2640 1 -- 172.21.15.64:0/1753559723 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81e8102570 msgr2=0x7f81e81989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:41:01.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.537+0000 7f81edee2640 1 --2- 172.21.15.64:0/1753559723 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81e8102570 0x7f81e81989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:01.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.537+0000 7f81edee2640 1 -- 172.21.15.64:0/1753559723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81e819d430 con 0x7f81e8102ef0 2024-09-12T03:41:01.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.538+0000 7f81edee2640 1 --2- 172.21.15.64:0/1753559723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81e8102ef0 0x7f81e819cd50 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f81d800eaa0 tx=0x7f81d800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:41:01.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.538+0000 7f81e67fc640 1 -- 172.21.15.64:0/1753559723 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d800cdb0 con 0x7f81e8102ef0 2024-09-12T03:41:01.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.538+0000 7f81e67fc640 1 -- 172.21.15.64:0/1753559723 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81d8005020 con 0x7f81e8102ef0 2024-09-12T03:41:01.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.538+0000 7f81e67fc640 1 -- 172.21.15.64:0/1753559723 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d80116d0 con 0x7f81e8102ef0 2024-09-12T03:41:01.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.538+0000 7f81ef945640 1 -- 172.21.15.64:0/1753559723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81e819d720 con 0x7f81e8102ef0 2024-09-12T03:41:01.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.538+0000 7f81ef945640 1 -- 172.21.15.64:0/1753559723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81e81a5000 con 0x7f81e8102ef0 2024-09-12T03:41:01.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.539+0000 7f81ef945640 1 -- 172.21.15.64:0/1753559723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81e8101200 con 0x7f81e8102ef0 2024-09-12T03:41:01.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.540+0000 7f81e67fc640 1 -- 172.21.15.64:0/1753559723 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f81d80051d0 con 0x7f81e8102ef0 2024-09-12T03:41:01.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.541+0000 7f81e67fc640 1 --2- 172.21.15.64:0/1753559723 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f81d00a2d70 0x7f81d00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:41:01.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.541+0000 7f81ed6e1640 1 --2- 172.21.15.64:0/1753559723 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f81d00a2d70 0x7f81d00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:41:01.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.541+0000 7f81e67fc640 1 -- 172.21.15.64:0/1753559723 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9583+0+0 (secure 0 0 0) 0x7f81d80cf900 con 0x7f81e8102ef0 2024-09-12T03:41:01.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.541+0000 7f81ed6e1640 1 --2- 172.21.15.64:0/1753559723 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f81d00a2d70 0x7f81d00a5230 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f81e0009bd0 tx=0x7f81e0005f90 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:41:01.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.545+0000 7f81e67fc640 1 -- 172.21.15.64:0/1753559723 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f81d8016030 con 0x7f81e8102ef0 2024-09-12T03:41:01.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.722+0000 7f81ef945640 1 -- 172.21.15.64:0/1753559723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f81e810a0a0 con 0x7f81e8102ef0 2024-09-12T03:41:01.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.724+0000 7f81e67fc640 1 -- 172.21.15.64:0/1753559723 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+25533 (secure 0 0 0) 0x7f81d809bab0 con 0x7f81e8102ef0 2024-09-12T03:41:01.726 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-12T03:41:01.726 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":103,"fsid":"b0812460-70b5-11ef-bcea-c7b262605968","created":"2024-09-12T03:19:15.900322+0000","modified":"2024-09-12T03:40:11.148161+0000","last_up_change":"2024-09-12T03:26:06.122971+0000","last_in_change":"2024-09-12T03:25:43.130689+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:22:21.915870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:26:55.826348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:26:57.984319+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T03:27:04.460292+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"326271c9-1f28-4ce6-b3be-fcc57479d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6803","nonce":1307832590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6805","nonce":1307832590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6809","nonce":1307832590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1307832590},{"type":"v1","addr":"172.21.15.64:6807","nonce":1307832590}]},"public_addr":"172.21.15.64:6803/1307832590","cluster_addr":"172.21.15.64:6805/1307832590","heartbeat_back_addr":"172.21.15.64:6809/1307832590","heartbeat_front_addr":"172.21.15.64:6807/1307832590","state":["exists","up"]},{"osd":1,"uuid":"7dca0c28-c234-423c-936e-2351d63f918e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6801","nonce":3123728622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6803","nonce":3123728622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6807","nonce":3123728622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":3123728622},{"type":"v1","addr":"172.21.15.115:6805","nonce":3123728622}]},"public_addr":"172.21.15.115:6801/3123728622","cluster_addr":"172.21.15.115:6803/3123728622","heartbeat_back_addr":"172.21.15.115:6807/3123728622","heartbeat_front_addr":"172.21.15.115:6805/3123728622","state":["exists","up"]},{"osd":2,"uuid":"4b3a3c7a-aec6-455a-9a9a-1ac9cc900f24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6801","nonce":1664125451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6803","nonce":1664125451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6807","nonce":1664125451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1664125451},{"type":"v1","addr":"172.21.15.195:6805","nonce":1664125451}]},"public_addr":"172.21.15.195:6801/1664125451","cluster_addr":"172.21.15.195:6803/1664125451","heartbeat_back_addr":"172.21.15.195:6807/1664125451","heartbeat_front_addr":"172.21.15.195:6805/1664125451","state":["exists","up"]},{"osd":3,"uuid":"8768c339-2c1e-4294-8b78-3c2adab6750f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6811","nonce":3935517185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6813","nonce":3935517185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6817","nonce":3935517185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3935517185},{"type":"v1","addr":"172.21.15.64:6815","nonce":3935517185}]},"public_addr":"172.21.15.64:6811/3935517185","cluster_addr":"172.21.15.64:6813/3935517185","heartbeat_back_addr":"172.21.15.64:6817/3935517185","heartbeat_front_addr":"172.21.15.64:6815/3935517185","state":["exists","up"]},{"osd":4,"uuid":"123aac47-a8ae-47c7-93a7-0dd293356322","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6809","nonce":446665227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6811","nonce":446665227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6815","nonce":446665227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":446665227},{"type":"v1","addr":"172.21.15.115:6813","nonce":446665227}]},"public_addr":"172.21.15.115:6809/446665227","cluster_addr":"172.21.15.115:6811/446665227","heartbeat_back_addr":"172.21.15.115:6815/446665227","heartbeat_front_addr":"172.21.15.115:6813/446665227","state":["exists","up"]},{"osd":5,"uuid":"c90bcaed-e7a8-4852-8677-b01dc73ea44e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6809","nonce":1187293668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6811","nonce":1187293668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6815","nonce":1187293668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":1187293668},{"type":"v1","addr":"172.21.15.195:6813","nonce":1187293668}]},"public_addr":"172.21.15.195:6809/1187293668","cluster_addr":"172.21.15.195:6811/1187293668","heartbeat_back_addr":"172.21.15.195:6815/1187293668","heartbeat_front_addr":"172.21.15.195:6813/1187293668","state":["exists","up"]},{"osd":6,"uuid":"fd292547-6695-4889-8867-e74318bbb00f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6819","nonce":560343274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6821","nonce":560343274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6825","nonce":560343274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":560343274},{"type":"v1","addr":"172.21.15.64:6823","nonce":560343274}]},"public_addr":"172.21.15.64:6819/560343274","cluster_addr":"172.21.15.64:6821/560343274","heartbeat_back_addr":"172.21.15.64:6825/560343274","heartbeat_front_addr":"172.21.15.64:6823/560343274","state":["exists","up"]},{"osd":7,"uuid":"ad7c4151-cd98-4113-879b-ca2b7c83c7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6817","nonce":3075734174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6819","nonce":3075734174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6823","nonce":3075734174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3075734174},{"type":"v1","addr":"172.21.15.115:6821","nonce":3075734174}]},"public_addr":"172.21.15.115:6817/3075734174","cluster_addr":"172.21.15.115:6819/3075734174","heartbeat_back_addr":"172.21.15.115:6823/3075734174","heartbeat_front_addr":"172.21.15.115:6821/3075734174","state":["exists","up"]},{"osd":8,"uuid":"6de50049-c28f-4f21-b123-05e55bf59127","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6817","nonce":2184742494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6819","nonce":2184742494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6823","nonce":2184742494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2184742494},{"type":"v1","addr":"172.21.15.195:6821","nonce":2184742494}]},"public_addr":"172.21.15.195:6817/2184742494","cluster_addr":"172.21.15.195:6819/2184742494","heartbeat_back_addr":"172.21.15.195:6823/2184742494","heartbeat_front_addr":"172.21.15.195:6821/2184742494","state":["exists","up"]},{"osd":9,"uuid":"36ab792c-58f2-4beb-9024-7f3d1e3785dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6827","nonce":561728112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6829","nonce":561728112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6833","nonce":561728112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":561728112},{"type":"v1","addr":"172.21.15.64:6831","nonce":561728112}]},"public_addr":"172.21.15.64:6827/561728112","cluster_addr":"172.21.15.64:6829/561728112","heartbeat_back_addr":"172.21.15.64:6833/561728112","heartbeat_front_addr":"172.21.15.64:6831/561728112","state":["exists","up"]},{"osd":10,"uuid":"d1f7881c-ec3f-466e-9cea-1d5cb441d970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6825","nonce":2977887627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6827","nonce":2977887627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6831","nonce":2977887627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2977887627},{"type":"v1","addr":"172.21.15.115:6829","nonce":2977887627}]},"public_addr":"172.21.15.115:6825/2977887627","cluster_addr":"172.21.15.115:6827/2977887627","heartbeat_back_addr":"172.21.15.115:6831/2977887627","heartbeat_front_addr":"172.21.15.115:6829/2977887627","state":["exists","up"]},{"osd":11,"uuid":"7c46c028-22c7-450f-9927-2144a7a09e9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6825","nonce":591599352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6827","nonce":591599352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6831","nonce":591599352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":591599352},{"type":"v1","addr":"172.21.15.195:6829","nonce":591599352}]},"public_addr":"172.21.15.195:6825/591599352","cluster_addr":"172.21.15.195:6827/591599352","heartbeat_back_addr":"172.21.15.195:6831/591599352","heartbeat_front_addr":"172.21.15.195:6829/591599352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:29.151454+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:21:54.308909+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:18.680995+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:22:44.687548+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:10.166131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:23:35.026132+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:24:00.242966+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:24:24.600848+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:24:49.563060+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:25:15.025730+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:25:39.830995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:26:04.394276+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.14","mappings":[{"from":1,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/1537191801":"2024-09-13T03:29:15.597759+0000","172.21.15.64:0/686137382":"2024-09-13T03:19:36.839431+0000","172.21.15.64:6800/3186640984":"2024-09-13T03:19:36.839431+0000","172.21.15.64:0/3334629354":"2024-09-13T03:20:07.411102+0000","172.21.15.64:6801/759931848":"2024-09-13T03:29:15.597759+0000","172.21.15.64:0/3079679964":"2024-09-13T03:19:36.839431+0000","172.21.15.64:0/1217136221":"2024-09-13T03:20:07.411102+0000","172.21.15.64:6800/68603379":"2024-09-13T03:20:07.411102+0000","172.21.15.64:6801/3186640984":"2024-09-13T03:19:36.839431+0000","172.21.15.64:0/3827202979":"2024-09-13T03:20:07.411102+0000","172.21.15.64:6800/759931848":"2024-09-13T03:29:15.597759+0000","172.21.15.64:0/3564418115":"2024-09-13T03:19:36.839431+0000","172.21.15.64:6801/68603379":"2024-09-13T03:20:07.411102+0000","172.21.15.64:0/3764771090":"2024-09-13T03:29:15.597759+0000","172.21.15.64:0/1757703005":"2024-09-13T03:29:15.597759+0000","172.21.15.64:0/3775732799":"2024-09-13T03:29:15.597759+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:41:01.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.726+0000 7f81ef945640 1 -- 172.21.15.64:0/1753559723 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f81d00a2d70 msgr2=0x7f81d00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:41:01.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.726+0000 7f81ef945640 1 --2- 172.21.15.64:0/1753559723 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f81d00a2d70 0x7f81d00a5230 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f81e0009bd0 tx=0x7f81e0005f90 comp rx=0 tx=0).stop 2024-09-12T03:41:01.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.726+0000 7f81ef945640 1 -- 172.21.15.64:0/1753559723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81e8102ef0 msgr2=0x7f81e819cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:41:01.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.726+0000 7f81ef945640 1 --2- 172.21.15.64:0/1753559723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81e8102ef0 0x7f81e819cd50 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f81d800eaa0 tx=0x7f81d800ef60 comp rx=0 tx=0).stop 2024-09-12T03:41:01.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.726+0000 7f81ef945640 1 -- 172.21.15.64:0/1753559723 shutdown_connections 2024-09-12T03:41:01.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.726+0000 7f81ef945640 1 --2- 172.21.15.64:0/1753559723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81e8102ef0 0x7f81e819cd50 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:01.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.726+0000 7f81ef945640 1 --2- 172.21.15.64:0/1753559723 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f81d00a2d70 0x7f81d00a5230 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:01.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.726+0000 7f81ef945640 1 --2- 172.21.15.64:0/1753559723 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81e8102570 0x7f81e81989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:01.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.726+0000 7f81ef945640 1 --2- 172.21.15.64:0/1753559723 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81e8100e20 0x7f81e8198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:01.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.726+0000 7f81ef945640 1 -- 172.21.15.64:0/1753559723 >> 172.21.15.64:0/1753559723 conn(0x7f81e80fc7d0 msgr2=0x7f81e806a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:41:01.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.726+0000 7f81ef945640 1 -- 172.21.15.64:0/1753559723 shutdown_connections 2024-09-12T03:41:01.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:01.726+0000 7f81ef945640 1 -- 172.21.15.64:0/1753559723 wait complete. 2024-09-12T03:41:02.021 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:01 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/1753559723' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:41:02.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:01 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/1753559723' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:41:02.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:01 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/1753559723' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:41:02.524 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph fs fail cephfs 2024-09-12T03:41:02.783 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:41:03.171 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:02 smithi064 ceph-mon[30952]: pgmap v366: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-12T03:41:03.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:02 smithi195 ceph-mon[34786]: pgmap v366: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-12T03:41:03.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.255+0000 7f8509b64640 1 -- 172.21.15.64:0/3567087449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8504102570 msgr2=0x7f85041029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:41:03.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.255+0000 7f8509b64640 1 --2- 172.21.15.64:0/3567087449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8504102570 0x7f85041029b0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f84f8009900 tx=0x7f84f802a480 comp rx=0 tx=0).stop 2024-09-12T03:41:03.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.255+0000 7f8509b64640 1 -- 172.21.15.64:0/3567087449 shutdown_connections 2024-09-12T03:41:03.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.255+0000 7f8509b64640 1 --2- 172.21.15.64:0/3567087449 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8504102ef0 0x7f850410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:03.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.255+0000 7f8509b64640 1 --2- 172.21.15.64:0/3567087449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8504102570 0x7f85041029b0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:03.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.255+0000 7f8509b64640 1 --2- 172.21.15.64:0/3567087449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8504100e20 0x7f8504101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:03.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.255+0000 7f8509b64640 1 -- 172.21.15.64:0/3567087449 >> 172.21.15.64:0/3567087449 conn(0x7f85040fc7d0 msgr2=0x7f85040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:41:03.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.255+0000 7f8509b64640 1 -- 172.21.15.64:0/3567087449 shutdown_connections 2024-09-12T03:41:03.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.255+0000 7f8509b64640 1 -- 172.21.15.64:0/3567087449 wait complete. 2024-09-12T03:41:03.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.255+0000 7f8509b64640 1 Processor -- start 2024-09-12T03:41:03.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.256+0000 7f8509b64640 1 -- start start 2024-09-12T03:41:03.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.256+0000 7f8509b64640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8504100e20 0x7f8504199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:41:03.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.256+0000 7f8509b64640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8504102570 0x7f85041998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:41:03.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.256+0000 7f8509b64640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8504102ef0 0x7f85041a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:41:03.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.256+0000 7f8509b64640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8504110e50 con 0x7f8504100e20 2024-09-12T03:41:03.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.256+0000 7f8509b64640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8504110cd0 con 0x7f8504102ef0 2024-09-12T03:41:03.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.256+0000 7f8509b64640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8504110fd0 con 0x7f8504102570 2024-09-12T03:41:03.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.256+0000 7f85037fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8504100e20 0x7f8504199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:41:03.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.256+0000 7f85037fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8504100e20 0x7f8504199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38728/0 (socket says 172.21.15.64:38728) 2024-09-12T03:41:03.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.256+0000 7f85037fe640 1 -- 172.21.15.64:0/2323537375 learned_addr learned my addr 172.21.15.64:0/2323537375 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:41:03.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.256+0000 7f8502ffd640 1 --2- 172.21.15.64:0/2323537375 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8504102570 0x7f85041998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:41:03.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.256+0000 7f8503fff640 1 --2- 172.21.15.64:0/2323537375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8504102ef0 0x7f85041a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:41:03.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.256+0000 7f85037fe640 1 -- 172.21.15.64:0/2323537375 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8504102570 msgr2=0x7f85041998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:41:03.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.256+0000 7f85037fe640 1 --2- 172.21.15.64:0/2323537375 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8504102570 0x7f85041998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:03.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.256+0000 7f85037fe640 1 -- 172.21.15.64:0/2323537375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8504102ef0 msgr2=0x7f85041a43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:41:03.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.256+0000 7f85037fe640 1 --2- 172.21.15.64:0/2323537375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8504102ef0 0x7f85041a43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:03.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.256+0000 7f85037fe640 1 -- 172.21.15.64:0/2323537375 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85041a4920 con 0x7f8504100e20 2024-09-12T03:41:03.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.256+0000 7f8503fff640 1 --2- 172.21.15.64:0/2323537375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8504102ef0 0x7f85041a43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:41:03.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.256+0000 7f85037fe640 1 --2- 172.21.15.64:0/2323537375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8504100e20 0x7f8504199360 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f84f400ba50 tx=0x7f84f400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:41:03.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.257+0000 7f8500ff9640 1 -- 172.21.15.64:0/2323537375 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f4019070 con 0x7f8504100e20 2024-09-12T03:41:03.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.257+0000 7f8500ff9640 1 -- 172.21.15.64:0/2323537375 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84f400cbe0 con 0x7f8504100e20 2024-09-12T03:41:03.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.257+0000 7f8509b64640 1 -- 172.21.15.64:0/2323537375 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85041a4af0 con 0x7f8504100e20 2024-09-12T03:41:03.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.257+0000 7f8500ff9640 1 -- 172.21.15.64:0/2323537375 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f40075d0 con 0x7f8504100e20 2024-09-12T03:41:03.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.257+0000 7f8509b64640 1 -- 172.21.15.64:0/2323537375 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85041a5030 con 0x7f8504100e20 2024-09-12T03:41:03.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.258+0000 7f8509b64640 1 -- 172.21.15.64:0/2323537375 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8504101200 con 0x7f8504100e20 2024-09-12T03:41:03.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.259+0000 7f8500ff9640 1 -- 172.21.15.64:0/2323537375 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f84f401c070 con 0x7f8504100e20 2024-09-12T03:41:03.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.260+0000 7f8500ff9640 1 --2- 172.21.15.64:0/2323537375 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f84ec0a2d70 0x7f84ec0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:41:03.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.260+0000 7f8502ffd640 1 --2- 172.21.15.64:0/2323537375 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f84ec0a2d70 0x7f84ec0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:41:03.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.260+0000 7f8500ff9640 1 -- 172.21.15.64:0/2323537375 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9583+0+0 (secure 0 0 0) 0x7f84f40ced60 con 0x7f8504100e20 2024-09-12T03:41:03.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.260+0000 7f8502ffd640 1 --2- 172.21.15.64:0/2323537375 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f84ec0a2d70 0x7f84ec0a5230 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f850406aac0 tx=0x7f84f80095f0 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:41:03.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.264+0000 7f8500ff9640 1 -- 172.21.15.64:0/2323537375 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f84f409b670 con 0x7f8504100e20 2024-09-12T03:41:03.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:02 smithi115 ceph-mon[34801]: pgmap v366: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-12T03:41:03.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.474+0000 7f8509b64640 1 -- 172.21.15.64:0/2323537375 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f8504109090 con 0x7f8504100e20 2024-09-12T03:41:03.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.477+0000 7f8500ff9640 1 -- 172.21.15.64:0/2323537375 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) ==== 161+0+0 (secure 0 0 0) 0x7f84f409af70 con 0x7f8504100e20 2024-09-12T03:41:03.477 INFO:teuthology.orchestra.run.smithi064.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-12T03:41:03.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.480+0000 7f8509b64640 1 -- 172.21.15.64:0/2323537375 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f84ec0a2d70 msgr2=0x7f84ec0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:41:03.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.480+0000 7f8509b64640 1 --2- 172.21.15.64:0/2323537375 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f84ec0a2d70 0x7f84ec0a5230 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f850406aac0 tx=0x7f84f80095f0 comp rx=0 tx=0).stop 2024-09-12T03:41:03.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.480+0000 7f8509b64640 1 -- 172.21.15.64:0/2323537375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8504100e20 msgr2=0x7f8504199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:41:03.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.480+0000 7f8509b64640 1 --2- 172.21.15.64:0/2323537375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8504100e20 0x7f8504199360 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f84f400ba50 tx=0x7f84f400bf10 comp rx=0 tx=0).stop 2024-09-12T03:41:03.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.480+0000 7f8509b64640 1 -- 172.21.15.64:0/2323537375 shutdown_connections 2024-09-12T03:41:03.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.480+0000 7f8509b64640 1 --2- 172.21.15.64:0/2323537375 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f84ec0a2d70 0x7f84ec0a5230 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:03.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.480+0000 7f8509b64640 1 --2- 172.21.15.64:0/2323537375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8504102ef0 0x7f85041a43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:03.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.480+0000 7f8509b64640 1 --2- 172.21.15.64:0/2323537375 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8504102570 0x7f85041998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:03.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.480+0000 7f8509b64640 1 --2- 172.21.15.64:0/2323537375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8504100e20 0x7f8504199360 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:03.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.480+0000 7f8509b64640 1 -- 172.21.15.64:0/2323537375 >> 172.21.15.64:0/2323537375 conn(0x7f85040fc7d0 msgr2=0x7f85040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:41:03.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.480+0000 7f8509b64640 1 -- 172.21.15.64:0/2323537375 shutdown_connections 2024-09-12T03:41:03.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:03.480+0000 7f8509b64640 1 -- 172.21.15.64:0/2323537375 wait complete. 2024-09-12T03:41:03.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:03 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mon-a[30948]: 2024-09-12T03:41:03.475+0000 7fa86f0c5640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:41:03.893 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 12 03:41:03 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-e[63509]: ignoring --setuser ceph since I am not root 2024-09-12T03:41:03.893 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 12 03:41:03 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-e[63509]: ignoring --setgroup ceph since I am not root 2024-09-12T03:41:03.893 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 12 03:41:03 smithi115 ceph-mds[63513]: -- 172.21.15.115:0/1550956999 <== mon.2 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d7a0533860 con 0x55d7a0510800 2024-09-12T03:41:03.893 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 12 03:41:03 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-e[63509]: starting mds.e at 2024-09-12T03:41:03.894 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 12 03:41:03 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-h[63810]: ignoring --setuser ceph since I am not root 2024-09-12T03:41:03.894 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 12 03:41:03 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-h[63810]: ignoring --setgroup ceph since I am not root 2024-09-12T03:41:03.894 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 12 03:41:03 smithi115 ceph-mds[63814]: -- 172.21.15.115:0/1959130736 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559fb5c85860 con 0x559fb5c62800 2024-09-12T03:41:03.894 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 12 03:41:03 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-h[63810]: starting mds.h at 2024-09-12T03:41:03.895 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 12 03:41:03 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-b[64111]: ignoring --setuser ceph since I am not root 2024-09-12T03:41:03.895 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 12 03:41:03 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-b[64111]: ignoring --setgroup ceph since I am not root 2024-09-12T03:41:03.895 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 12 03:41:03 smithi115 ceph-mds[64115]: -- 172.21.15.115:0/3627183230 <== mon.2 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fbaafcb860 con 0x55fbaafa9000 2024-09-12T03:41:03.895 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 12 03:41:03 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-b[64111]: starting mds.b at 2024-09-12T03:41:03.919 INFO:journalctl@ceph.mds.i.smithi195.stdout:Sep 12 03:41:03 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-i[65131]: ignoring --setuser ceph since I am not root 2024-09-12T03:41:03.919 INFO:journalctl@ceph.mds.i.smithi195.stdout:Sep 12 03:41:03 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-i[65131]: ignoring --setgroup ceph since I am not root 2024-09-12T03:41:03.920 INFO:journalctl@ceph.mds.i.smithi195.stdout:Sep 12 03:41:03 smithi195 ceph-mds[65135]: -- 172.21.15.195:0/3937484927 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563100e30680 con 0x563100e05400 2024-09-12T03:41:03.920 INFO:journalctl@ceph.mds.i.smithi195.stdout:Sep 12 03:41:03 smithi195 ceph-mds[65135]: -- 172.21.15.195:0/3937484927 <== mon.0 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563100e27860 con 0x563100e05400 2024-09-12T03:41:03.920 INFO:journalctl@ceph.mds.i.smithi195.stdout:Sep 12 03:41:03 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-i[65131]: starting mds.i at 2024-09-12T03:41:03.920 INFO:journalctl@ceph.mds.c.smithi195.stdout:Sep 12 03:41:03 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-c[64828]: ignoring --setuser ceph since I am not root 2024-09-12T03:41:03.920 INFO:journalctl@ceph.mds.c.smithi195.stdout:Sep 12 03:41:03 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-c[64828]: ignoring --setgroup ceph since I am not root 2024-09-12T03:41:03.920 INFO:journalctl@ceph.mds.c.smithi195.stdout:Sep 12 03:41:03 smithi195 ceph-mds[64832]: -- 172.21.15.195:0/4220240137 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5612605b4680 con 0x561260589400 2024-09-12T03:41:03.920 INFO:journalctl@ceph.mds.c.smithi195.stdout:Sep 12 03:41:03 smithi195 ceph-mds[64832]: -- 172.21.15.195:0/4220240137 <== mon.2 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5612605ab860 con 0x561260589400 2024-09-12T03:41:03.920 INFO:journalctl@ceph.mds.c.smithi195.stdout:Sep 12 03:41:03 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-c[64828]: starting mds.c at 2024-09-12T03:41:03.921 INFO:journalctl@ceph.mds.f.smithi195.stdout:Sep 12 03:41:03 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-f[64528]: ignoring --setuser ceph since I am not root 2024-09-12T03:41:03.921 INFO:journalctl@ceph.mds.f.smithi195.stdout:Sep 12 03:41:03 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-f[64528]: ignoring --setgroup ceph since I am not root 2024-09-12T03:41:03.921 INFO:journalctl@ceph.mds.f.smithi195.stdout:Sep 12 03:41:03 smithi195 ceph-mds[64532]: -- 172.21.15.195:0/821974767 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e91d3bfc20 con 0x55e91d39d000 2024-09-12T03:41:03.921 INFO:journalctl@ceph.mds.f.smithi195.stdout:Sep 12 03:41:03 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-f[64528]: starting mds.f at 2024-09-12T03:41:04.093 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:03 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/2323537375' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T03:41:04.093 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:03 smithi064 ceph-mon[30952]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T03:41:04.093 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:03 smithi064 ceph-mon[30952]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:41:04.093 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:03 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/2323537375' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T03:41:04.093 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:03 smithi064 ceph-mon[30952]: osdmap e104: 12 total, 12 up, 12 in 2024-09-12T03:41:04.093 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:03 smithi064 ceph-mon[30952]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-12T03:41:04.124 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-12T03:41:04.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:03 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/2323537375' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T03:41:04.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:03 smithi195 ceph-mon[34786]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T03:41:04.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:03 smithi195 ceph-mon[34786]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:41:04.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:03 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/2323537375' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T03:41:04.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:03 smithi195 ceph-mon[34786]: osdmap e104: 12 total, 12 up, 12 in 2024-09-12T03:41:04.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:03 smithi195 ceph-mon[34786]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-12T03:41:04.383 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:41:04.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:03 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/2323537375' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T03:41:04.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:03 smithi115 ceph-mon[34801]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T03:41:04.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:03 smithi115 ceph-mon[34801]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:41:04.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:03 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/2323537375' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T03:41:04.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:03 smithi115 ceph-mon[34801]: osdmap e104: 12 total, 12 up, 12 in 2024-09-12T03:41:04.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:03 smithi115 ceph-mon[34801]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-12T03:41:04.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.838+0000 7f1fcefc8640 1 -- 172.21.15.64:0/867195580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fc8102ef0 msgr2=0x7f1fc810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:41:04.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.838+0000 7f1fcefc8640 1 --2- 172.21.15.64:0/867195580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fc8102ef0 0x7f1fc810da40 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0009ca0 tx=0x7f1fc002a950 comp rx=0 tx=0).stop 2024-09-12T03:41:04.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.838+0000 7f1fcefc8640 1 -- 172.21.15.64:0/867195580 shutdown_connections 2024-09-12T03:41:04.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.838+0000 7f1fcefc8640 1 --2- 172.21.15.64:0/867195580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fc8102ef0 0x7f1fc810da40 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:04.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.838+0000 7f1fcefc8640 1 --2- 172.21.15.64:0/867195580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fc8102570 0x7f1fc81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:04.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.838+0000 7f1fcefc8640 1 --2- 172.21.15.64:0/867195580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fc8100e20 0x7f1fc8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:04.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.838+0000 7f1fcefc8640 1 -- 172.21.15.64:0/867195580 >> 172.21.15.64:0/867195580 conn(0x7f1fc80fc7d0 msgr2=0x7f1fc80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:41:04.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.838+0000 7f1fcefc8640 1 -- 172.21.15.64:0/867195580 shutdown_connections 2024-09-12T03:41:04.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.838+0000 7f1fcefc8640 1 -- 172.21.15.64:0/867195580 wait complete. 2024-09-12T03:41:04.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.839+0000 7f1fcefc8640 1 Processor -- start 2024-09-12T03:41:04.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.839+0000 7f1fcefc8640 1 -- start start 2024-09-12T03:41:04.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.839+0000 7f1fcefc8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fc8100e20 0x7f1fc8199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:41:04.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.839+0000 7f1fcefc8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fc8102570 0x7f1fc8199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:41:04.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.839+0000 7f1fcefc8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fc8102ef0 0x7f1fc81a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:41:04.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.839+0000 7f1fcefc8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc8110e50 con 0x7f1fc8102ef0 2024-09-12T03:41:04.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.839+0000 7f1fcefc8640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc8110cd0 con 0x7f1fc8100e20 2024-09-12T03:41:04.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.839+0000 7f1fcefc8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc8110fd0 con 0x7f1fc8102570 2024-09-12T03:41:04.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.839+0000 7f1fcd565640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fc8102ef0 0x7f1fc81a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:41:04.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.839+0000 7f1fcd565640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fc8102ef0 0x7f1fc81a43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38734/0 (socket says 172.21.15.64:38734) 2024-09-12T03:41:04.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.839+0000 7f1fcd565640 1 -- 172.21.15.64:0/2262447485 learned_addr learned my addr 172.21.15.64:0/2262447485 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:41:04.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.839+0000 7f1fc7fff640 1 --2- 172.21.15.64:0/2262447485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fc8102570 0x7f1fc8199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:41:04.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.839+0000 7f1fccd64640 1 --2- 172.21.15.64:0/2262447485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fc8100e20 0x7f1fc8199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:41:04.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.839+0000 7f1fcd565640 1 -- 172.21.15.64:0/2262447485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fc8102570 msgr2=0x7f1fc8199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:41:04.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.839+0000 7f1fcd565640 1 --2- 172.21.15.64:0/2262447485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fc8102570 0x7f1fc8199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:04.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.839+0000 7f1fcd565640 1 -- 172.21.15.64:0/2262447485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fc8100e20 msgr2=0x7f1fc8199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:41:04.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.839+0000 7f1fcd565640 1 --2- 172.21.15.64:0/2262447485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fc8100e20 0x7f1fc8199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:04.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.839+0000 7f1fcd565640 1 -- 172.21.15.64:0/2262447485 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc81a48e0 con 0x7f1fc8102ef0 2024-09-12T03:41:04.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.839+0000 7f1fc7fff640 1 --2- 172.21.15.64:0/2262447485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fc8102570 0x7f1fc8199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:41:04.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.839+0000 7f1fccd64640 1 --2- 172.21.15.64:0/2262447485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fc8100e20 0x7f1fc8199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:41:04.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.840+0000 7f1fcd565640 1 --2- 172.21.15.64:0/2262447485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fc8102ef0 0x7f1fc81a43a0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0006010 tx=0x7f1fc002f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:41:04.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.840+0000 7f1fc5ffb640 1 -- 172.21.15.64:0/2262447485 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc0034040 con 0x7f1fc8102ef0 2024-09-12T03:41:04.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.840+0000 7f1fc5ffb640 1 -- 172.21.15.64:0/2262447485 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fc0003e00 con 0x7f1fc8102ef0 2024-09-12T03:41:04.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.840+0000 7f1fc5ffb640 1 -- 172.21.15.64:0/2262447485 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc0045480 con 0x7f1fc8102ef0 2024-09-12T03:41:04.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.840+0000 7f1fcefc8640 1 -- 172.21.15.64:0/2262447485 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fc81a4ab0 con 0x7f1fc8102ef0 2024-09-12T03:41:04.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.840+0000 7f1fcefc8640 1 -- 172.21.15.64:0/2262447485 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fc81a4f00 con 0x7f1fc8102ef0 2024-09-12T03:41:04.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.841+0000 7f1fcefc8640 1 -- 172.21.15.64:0/2262447485 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fc8101200 con 0x7f1fc8102ef0 2024-09-12T03:41:04.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.842+0000 7f1fc5ffb640 1 -- 172.21.15.64:0/2262447485 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f1fc0037020 con 0x7f1fc8102ef0 2024-09-12T03:41:04.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.843+0000 7f1fc5ffb640 1 --2- 172.21.15.64:0/2262447485 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f1fb00a2cf0 0x7f1fb00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:41:04.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.843+0000 7f1fccd64640 1 --2- 172.21.15.64:0/2262447485 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f1fb00a2cf0 0x7f1fb00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:41:04.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.843+0000 7f1fc5ffb640 1 -- 172.21.15.64:0/2262447485 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9753+0+0 (secure 0 0 0) 0x7f1fc00f0a20 con 0x7f1fc8102ef0 2024-09-12T03:41:04.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.844+0000 7f1fccd64640 1 --2- 172.21.15.64:0/2262447485 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f1fb00a2cf0 0x7f1fb00a51b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8006fd0 tx=0x7f1fb8008040 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:41:04.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:04.848+0000 7f1fc5ffb640 1 -- 172.21.15.64:0/2262447485 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1fc00bd280 con 0x7f1fc8102ef0 2024-09-12T03:41:05.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:05.060+0000 7f1fcefc8640 1 -- 172.21.15.64:0/2262447485 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f1fc8109090 con 0x7f1fc8102ef0 2024-09-12T03:41:05.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:05.490+0000 7f1fc5ffb640 1 -- 172.21.15.64:0/2262447485 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) ==== 104+0+0 (secure 0 0 0) 0x7f1fc00bcb80 con 0x7f1fc8102ef0 2024-09-12T03:41:05.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:05.492+0000 7f1fcefc8640 1 -- 172.21.15.64:0/2262447485 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f1fb00a2cf0 msgr2=0x7f1fb00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:41:05.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:05.492+0000 7f1fcefc8640 1 --2- 172.21.15.64:0/2262447485 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f1fb00a2cf0 0x7f1fb00a51b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8006fd0 tx=0x7f1fb8008040 comp rx=0 tx=0).stop 2024-09-12T03:41:05.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:05.492+0000 7f1fcefc8640 1 -- 172.21.15.64:0/2262447485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fc8102ef0 msgr2=0x7f1fc81a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:41:05.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:05.492+0000 7f1fcefc8640 1 --2- 172.21.15.64:0/2262447485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fc8102ef0 0x7f1fc81a43a0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0006010 tx=0x7f1fc002f710 comp rx=0 tx=0).stop 2024-09-12T03:41:05.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:05.492+0000 7f1fcefc8640 1 -- 172.21.15.64:0/2262447485 shutdown_connections 2024-09-12T03:41:05.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:05.492+0000 7f1fcefc8640 1 --2- 172.21.15.64:0/2262447485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fc8102ef0 0x7f1fc81a43a0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:05.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:05.492+0000 7f1fcefc8640 1 --2- 172.21.15.64:0/2262447485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fc8102570 0x7f1fc8199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:05.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:05.492+0000 7f1fcefc8640 1 --2- 172.21.15.64:0/2262447485 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f1fb00a2cf0 0x7f1fb00a51b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:05.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:05.492+0000 7f1fcefc8640 1 --2- 172.21.15.64:0/2262447485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fc8100e20 0x7f1fc8199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:05.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:05.492+0000 7f1fcefc8640 1 -- 172.21.15.64:0/2262447485 >> 172.21.15.64:0/2262447485 conn(0x7f1fc80fc7d0 msgr2=0x7f1fc80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:41:05.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:05.493+0000 7f1fcefc8640 1 -- 172.21.15.64:0/2262447485 shutdown_connections 2024-09-12T03:41:05.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:05.493+0000 7f1fcefc8640 1 -- 172.21.15.64:0/2262447485 wait complete. 2024-09-12T03:41:05.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:05 smithi064 ceph-mon[30952]: pgmap v368: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-09-12T03:41:05.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:05 smithi064 ceph-mon[30952]: mds.? [v2:172.21.15.195:6834/1748560075,v1:172.21.15.195:6837/1748560075] up:boot 2024-09-12T03:41:05.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:05 smithi064 ceph-mon[30952]: mds.? [v2:172.21.15.195:6835/2580799515,v1:172.21.15.195:6839/2580799515] up:boot 2024-09-12T03:41:05.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:05 smithi064 ceph-mon[30952]: mds.? [v2:172.21.15.195:6836/2360447769,v1:172.21.15.195:6838/2360447769] up:boot 2024-09-12T03:41:05.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:05 smithi064 ceph-mon[30952]: mds.? [v2:172.21.15.115:6834/2452748506,v1:172.21.15.115:6835/2452748506] up:boot 2024-09-12T03:41:05.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:05 smithi064 ceph-mon[30952]: mds.? [v2:172.21.15.115:6837/1301282749,v1:172.21.15.115:6839/1301282749] up:boot 2024-09-12T03:41:05.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:05 smithi064 ceph-mon[30952]: mds.? [v2:172.21.15.115:6836/3095424519,v1:172.21.15.115:6838/3095424519] up:boot 2024-09-12T03:41:05.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:05 smithi064 ceph-mon[30952]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-12T03:41:05.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:05 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:41:05.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:05 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:41:05.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:05 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:41:05.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:05 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:41:05.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:05 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:41:05.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:05 smithi064 ceph-mon[30952]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:41:05.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:05 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/2262447485' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:41:05.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:05 smithi115 ceph-mon[34801]: pgmap v368: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-09-12T03:41:05.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:05 smithi115 ceph-mon[34801]: mds.? [v2:172.21.15.195:6834/1748560075,v1:172.21.15.195:6837/1748560075] up:boot 2024-09-12T03:41:05.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:05 smithi115 ceph-mon[34801]: mds.? [v2:172.21.15.195:6835/2580799515,v1:172.21.15.195:6839/2580799515] up:boot 2024-09-12T03:41:05.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:05 smithi115 ceph-mon[34801]: mds.? [v2:172.21.15.195:6836/2360447769,v1:172.21.15.195:6838/2360447769] up:boot 2024-09-12T03:41:05.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:05 smithi115 ceph-mon[34801]: mds.? [v2:172.21.15.115:6834/2452748506,v1:172.21.15.115:6835/2452748506] up:boot 2024-09-12T03:41:05.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:05 smithi115 ceph-mon[34801]: mds.? [v2:172.21.15.115:6837/1301282749,v1:172.21.15.115:6839/1301282749] up:boot 2024-09-12T03:41:05.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:05 smithi115 ceph-mon[34801]: mds.? [v2:172.21.15.115:6836/3095424519,v1:172.21.15.115:6838/3095424519] up:boot 2024-09-12T03:41:05.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:05 smithi115 ceph-mon[34801]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-12T03:41:05.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:05 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:41:05.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:05 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:41:05.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:05 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:41:05.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:05 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:41:05.894 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:05 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:41:05.894 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:05 smithi115 ceph-mon[34801]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:41:05.894 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:05 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/2262447485' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:41:05.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:05 smithi195 ceph-mon[34786]: pgmap v368: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-09-12T03:41:05.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:05 smithi195 ceph-mon[34786]: mds.? [v2:172.21.15.195:6834/1748560075,v1:172.21.15.195:6837/1748560075] up:boot 2024-09-12T03:41:05.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:05 smithi195 ceph-mon[34786]: mds.? [v2:172.21.15.195:6835/2580799515,v1:172.21.15.195:6839/2580799515] up:boot 2024-09-12T03:41:05.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:05 smithi195 ceph-mon[34786]: mds.? [v2:172.21.15.195:6836/2360447769,v1:172.21.15.195:6838/2360447769] up:boot 2024-09-12T03:41:05.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:05 smithi195 ceph-mon[34786]: mds.? [v2:172.21.15.115:6834/2452748506,v1:172.21.15.115:6835/2452748506] up:boot 2024-09-12T03:41:05.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:05 smithi195 ceph-mon[34786]: mds.? [v2:172.21.15.115:6837/1301282749,v1:172.21.15.115:6839/1301282749] up:boot 2024-09-12T03:41:05.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:05 smithi195 ceph-mon[34786]: mds.? [v2:172.21.15.115:6836/3095424519,v1:172.21.15.115:6838/3095424519] up:boot 2024-09-12T03:41:05.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:05 smithi195 ceph-mon[34786]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-12T03:41:05.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:05 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:41:05.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:05 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:41:05.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:05 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:41:05.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:05 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:41:05.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:05 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:41:05.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:05 smithi195 ceph-mon[34786]: from='mgr.15243 172.21.15.64:0/3129985282' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:41:05.957 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:05 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/2262447485' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:41:06.204 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-12T03:41:06.468 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:41:06.739 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:06 smithi064 ceph-mon[30952]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T03:41:06.739 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:06 smithi064 ceph-mon[30952]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T03:41:06.739 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:06 smithi064 ceph-mon[30952]: Cluster is now healthy 2024-09-12T03:41:06.739 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:06 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/2262447485' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T03:41:06.739 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:06 smithi064 ceph-mon[30952]: fsmap 12 up:standby 2024-09-12T03:41:06.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:06 smithi115 ceph-mon[34801]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T03:41:06.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:06 smithi115 ceph-mon[34801]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T03:41:06.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:06 smithi115 ceph-mon[34801]: Cluster is now healthy 2024-09-12T03:41:06.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:06 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/2262447485' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T03:41:06.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:06 smithi115 ceph-mon[34801]: fsmap 12 up:standby 2024-09-12T03:41:06.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:06 smithi195 ceph-mon[34786]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T03:41:06.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:06 smithi195 ceph-mon[34786]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T03:41:06.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:06 smithi195 ceph-mon[34786]: Cluster is now healthy 2024-09-12T03:41:06.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:06 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/2262447485' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T03:41:06.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:06 smithi195 ceph-mon[34786]: fsmap 12 up:standby 2024-09-12T03:41:06.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:06.983+0000 7f5b0b9d8640 1 -- 172.21.15.64:0/638303611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b041082a0 msgr2=0x7f5b0410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:41:06.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:06.983+0000 7f5b0b9d8640 1 --2- 172.21.15.64:0/638303611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b041082a0 0x7f5b0410ec40 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f5afc009cd0 tx=0x7f5afc02a8e0 comp rx=0 tx=0).stop 2024-09-12T03:41:06.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:06.983+0000 7f5b0b9d8640 1 -- 172.21.15.64:0/638303611 shutdown_connections 2024-09-12T03:41:06.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:06.983+0000 7f5b0b9d8640 1 --2- 172.21.15.64:0/638303611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b041082a0 0x7f5b0410ec40 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:06.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:06.983+0000 7f5b0b9d8640 1 --2- 172.21.15.64:0/638303611 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b041036e0 0x7f5b04107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:06.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:06.983+0000 7f5b0b9d8640 1 --2- 172.21.15.64:0/638303611 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b04102d30 0x7f5b04103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:06.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:06.983+0000 7f5b0b9d8640 1 -- 172.21.15.64:0/638303611 >> 172.21.15.64:0/638303611 conn(0x7f5b040fc7d0 msgr2=0x7f5b040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:41:06.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:06.983+0000 7f5b0b9d8640 1 -- 172.21.15.64:0/638303611 shutdown_connections 2024-09-12T03:41:06.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:06.983+0000 7f5b0b9d8640 1 -- 172.21.15.64:0/638303611 wait complete. 2024-09-12T03:41:06.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:06.983+0000 7f5b0b9d8640 1 Processor -- start 2024-09-12T03:41:06.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:06.983+0000 7f5b0b9d8640 1 -- start start 2024-09-12T03:41:06.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:06.984+0000 7f5b0b9d8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b04102d30 0x7f5b041984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:41:06.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:06.984+0000 7f5b0b9d8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b041036e0 0x7f5b04198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:41:06.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:06.984+0000 7f5b0b9d8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b041082a0 0x7f5b0419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:41:06.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:06.984+0000 7f5b0b9d8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5b04111f50 con 0x7f5b041036e0 2024-09-12T03:41:06.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:06.984+0000 7f5b0b9d8640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5b04111dd0 con 0x7f5b04102d30 2024-09-12T03:41:06.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:06.984+0000 7f5b0b9d8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5b041120d0 con 0x7f5b041082a0 2024-09-12T03:41:06.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:06.984+0000 7f5b08f73640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b041036e0 0x7f5b04198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:41:06.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:06.984+0000 7f5b08f73640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b041036e0 0x7f5b04198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38764/0 (socket says 172.21.15.64:38764) 2024-09-12T03:41:06.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:06.984+0000 7f5b08f73640 1 -- 172.21.15.64:0/3967390695 learned_addr learned my addr 172.21.15.64:0/3967390695 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:41:06.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:06.984+0000 7f5b09f75640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b041082a0 0x7f5b0419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:41:06.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:06.984+0000 7f5b09774640 1 --2- 172.21.15.64:0/3967390695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b04102d30 0x7f5b041984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:41:06.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:06.984+0000 7f5b08f73640 1 -- 172.21.15.64:0/3967390695 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b041082a0 msgr2=0x7f5b0419cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:41:06.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:06.984+0000 7f5b08f73640 1 --2- 172.21.15.64:0/3967390695 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b041082a0 0x7f5b0419cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:06.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:06.984+0000 7f5b08f73640 1 -- 172.21.15.64:0/3967390695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b04102d30 msgr2=0x7f5b041984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:41:06.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:06.984+0000 7f5b08f73640 1 --2- 172.21.15.64:0/3967390695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b04102d30 0x7f5b041984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:06.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:06.984+0000 7f5b08f73640 1 -- 172.21.15.64:0/3967390695 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b0419d480 con 0x7f5b041036e0 2024-09-12T03:41:06.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:06.984+0000 7f5b09774640 1 --2- 172.21.15.64:0/3967390695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b04102d30 0x7f5b041984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:41:06.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:06.984+0000 7f5b08f73640 1 --2- 172.21.15.64:0/3967390695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b041036e0 0x7f5b04198a10 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f5af400c970 tx=0x7f5af400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:41:06.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:06.985+0000 7f5afa7fc640 1 -- 172.21.15.64:0/3967390695 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af4007bb0 con 0x7f5b041036e0 2024-09-12T03:41:06.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:06.985+0000 7f5afa7fc640 1 -- 172.21.15.64:0/3967390695 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5af4007d50 con 0x7f5b041036e0 2024-09-12T03:41:06.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:06.985+0000 7f5b0b9d8640 1 -- 172.21.15.64:0/3967390695 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b0419d770 con 0x7f5b041036e0 2024-09-12T03:41:06.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:06.985+0000 7f5afa7fc640 1 -- 172.21.15.64:0/3967390695 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af400f450 con 0x7f5b041036e0 2024-09-12T03:41:06.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:06.985+0000 7f5b0b9d8640 1 -- 172.21.15.64:0/3967390695 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b041a5050 con 0x7f5b041036e0 2024-09-12T03:41:06.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:06.986+0000 7f5b0b9d8640 1 -- 172.21.15.64:0/3967390695 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b04103110 con 0x7f5b041036e0 2024-09-12T03:41:06.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:06.987+0000 7f5afa7fc640 1 -- 172.21.15.64:0/3967390695 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f5af4004eb0 con 0x7f5b041036e0 2024-09-12T03:41:06.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:06.988+0000 7f5afa7fc640 1 --2- 172.21.15.64:0/3967390695 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f5af00a2d70 0x7f5af00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:41:06.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:06.988+0000 7f5b09774640 1 --2- 172.21.15.64:0/3967390695 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f5af00a2d70 0x7f5af00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:41:06.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:06.988+0000 7f5afa7fc640 1 -- 172.21.15.64:0/3967390695 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9753+0+0 (secure 0 0 0) 0x7f5af40cf560 con 0x7f5b041036e0 2024-09-12T03:41:06.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:06.988+0000 7f5b09774640 1 --2- 172.21.15.64:0/3967390695 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f5af00a2d70 0x7f5af00a5230 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5b000058c0 tx=0x7f5b00005850 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:41:06.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:06.994+0000 7f5afa7fc640 1 -- 172.21.15.64:0/3967390695 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5af4014030 con 0x7f5b041036e0 2024-09-12T03:41:07.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:07.171+0000 7f5b0b9d8640 1 -- 172.21.15.64:0/3967390695 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5b0410a2b0 con 0x7f5b041036e0 2024-09-12T03:41:07.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:07.497+0000 7f5afa7fc640 1 -- 172.21.15.64:0/3967390695 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v105) ==== 181+0+0 (secure 0 0 0) 0x7f5af409bcb0 con 0x7f5b041036e0 2024-09-12T03:41:07.498 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' removed 2024-09-12T03:41:07.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:07.500+0000 7f5b0b9d8640 1 -- 172.21.15.64:0/3967390695 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f5af00a2d70 msgr2=0x7f5af00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:41:07.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:07.500+0000 7f5b0b9d8640 1 --2- 172.21.15.64:0/3967390695 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f5af00a2d70 0x7f5af00a5230 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5b000058c0 tx=0x7f5b00005850 comp rx=0 tx=0).stop 2024-09-12T03:41:07.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:07.500+0000 7f5b0b9d8640 1 -- 172.21.15.64:0/3967390695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b041036e0 msgr2=0x7f5b04198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:41:07.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:07.500+0000 7f5b0b9d8640 1 --2- 172.21.15.64:0/3967390695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b041036e0 0x7f5b04198a10 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f5af400c970 tx=0x7f5af400ce30 comp rx=0 tx=0).stop 2024-09-12T03:41:07.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:07.500+0000 7f5b0b9d8640 1 -- 172.21.15.64:0/3967390695 shutdown_connections 2024-09-12T03:41:07.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:07.500+0000 7f5b0b9d8640 1 --2- 172.21.15.64:0/3967390695 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b041082a0 0x7f5b0419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:07.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:07.500+0000 7f5b0b9d8640 1 --2- 172.21.15.64:0/3967390695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b041036e0 0x7f5b04198a10 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:07.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:07.500+0000 7f5b0b9d8640 1 --2- 172.21.15.64:0/3967390695 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7f5af00a2d70 0x7f5af00a5230 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:07.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:07.500+0000 7f5b0b9d8640 1 --2- 172.21.15.64:0/3967390695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b04102d30 0x7f5b041984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:07.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:07.500+0000 7f5b0b9d8640 1 -- 172.21.15.64:0/3967390695 >> 172.21.15.64:0/3967390695 conn(0x7f5b040fc7d0 msgr2=0x7f5b04105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:41:07.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:07.500+0000 7f5b0b9d8640 1 -- 172.21.15.64:0/3967390695 shutdown_connections 2024-09-12T03:41:07.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:07.500+0000 7f5b0b9d8640 1 -- 172.21.15.64:0/3967390695 wait complete. 2024-09-12T03:41:07.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:07 smithi064 ceph-mon[30952]: pgmap v369: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-09-12T03:41:07.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:07 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/3967390695' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:41:07.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:07 smithi115 ceph-mon[34801]: pgmap v369: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-09-12T03:41:07.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:07 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/3967390695' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:41:07.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:07 smithi195 ceph-mon[34786]: pgmap v369: 129 pgs: 129 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-09-12T03:41:07.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:07 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/3967390695' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:41:08.176 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-12T03:41:08.436 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:41:08.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:08 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/3967390695' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:41:08.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:08 smithi064 ceph-mon[30952]: osdmap e105: 12 total, 12 up, 12 in 2024-09-12T03:41:08.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:08 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/3967390695' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:41:08.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:08 smithi115 ceph-mon[34801]: osdmap e105: 12 total, 12 up, 12 in 2024-09-12T03:41:08.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:08.920+0000 7fb19c918640 1 -- 172.21.15.64:0/2029002932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb198102ef0 msgr2=0x7fb19810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:41:08.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:08.920+0000 7fb19c918640 1 --2- 172.21.15.64:0/2029002932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb198102ef0 0x7fb19810da40 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fb190009ca0 tx=0x7fb19002a950 comp rx=0 tx=0).stop 2024-09-12T03:41:08.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:08.920+0000 7fb19c918640 1 -- 172.21.15.64:0/2029002932 shutdown_connections 2024-09-12T03:41:08.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:08.920+0000 7fb19c918640 1 --2- 172.21.15.64:0/2029002932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb198102ef0 0x7fb19810da40 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:08.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:08.920+0000 7fb19c918640 1 --2- 172.21.15.64:0/2029002932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb198102570 0x7fb1981029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:08.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:08.920+0000 7fb19c918640 1 --2- 172.21.15.64:0/2029002932 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb198100e20 0x7fb198101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:08.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:08.920+0000 7fb19c918640 1 -- 172.21.15.64:0/2029002932 >> 172.21.15.64:0/2029002932 conn(0x7fb1980fc7d0 msgr2=0x7fb1980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:41:08.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:08.920+0000 7fb19c918640 1 -- 172.21.15.64:0/2029002932 shutdown_connections 2024-09-12T03:41:08.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:08.920+0000 7fb19c918640 1 -- 172.21.15.64:0/2029002932 wait complete. 2024-09-12T03:41:08.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:08.920+0000 7fb19c918640 1 Processor -- start 2024-09-12T03:41:08.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:08.921+0000 7fb19c918640 1 -- start start 2024-09-12T03:41:08.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:08.921+0000 7fb19c918640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb198100e20 0x7fb198198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:41:08.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:08.921+0000 7fb19c918640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb198102570 0x7fb1981987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:41:08.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:08.921+0000 7fb19c918640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb198102ef0 0x7fb19819cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:41:08.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:08.921+0000 7fb19c918640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb198110e30 con 0x7fb198102570 2024-09-12T03:41:08.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:08.921+0000 7fb19c918640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb198110cb0 con 0x7fb198102ef0 2024-09-12T03:41:08.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:08.921+0000 7fb19c918640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb198110fb0 con 0x7fb198100e20 2024-09-12T03:41:08.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:08.921+0000 7fb195d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb198102570 0x7fb1981987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:41:08.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:08.921+0000 7fb195d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb198102570 0x7fb1981987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59824/0 (socket says 172.21.15.64:59824) 2024-09-12T03:41:08.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:08.921+0000 7fb195d9b640 1 -- 172.21.15.64:0/1267128282 learned_addr learned my addr 172.21.15.64:0/1267128282 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:41:08.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:08.921+0000 7fb19659c640 1 --2- 172.21.15.64:0/1267128282 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb198100e20 0x7fb198198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:41:08.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:08.921+0000 7fb196d9d640 1 --2- 172.21.15.64:0/1267128282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb198102ef0 0x7fb19819cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:41:08.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:08.921+0000 7fb195d9b640 1 -- 172.21.15.64:0/1267128282 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb198100e20 msgr2=0x7fb198198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:41:08.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:08.921+0000 7fb195d9b640 1 --2- 172.21.15.64:0/1267128282 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb198100e20 0x7fb198198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:08.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:08.921+0000 7fb195d9b640 1 -- 172.21.15.64:0/1267128282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb198102ef0 msgr2=0x7fb19819cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:41:08.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:08.921+0000 7fb195d9b640 1 --2- 172.21.15.64:0/1267128282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb198102ef0 0x7fb19819cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:08.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:08.921+0000 7fb195d9b640 1 -- 172.21.15.64:0/1267128282 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb19819d210 con 0x7fb198102570 2024-09-12T03:41:08.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:08.922+0000 7fb195d9b640 1 --2- 172.21.15.64:0/1267128282 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb198102570 0x7fb1981987a0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fb18c00c920 tx=0x7fb18c00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:41:08.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:08.922+0000 7fb1877fe640 1 -- 172.21.15.64:0/1267128282 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18c007bb0 con 0x7fb198102570 2024-09-12T03:41:08.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:08.922+0000 7fb1877fe640 1 -- 172.21.15.64:0/1267128282 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb18c007d50 con 0x7fb198102570 2024-09-12T03:41:08.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:08.922+0000 7fb19c918640 1 -- 172.21.15.64:0/1267128282 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb19819d500 con 0x7fb198102570 2024-09-12T03:41:08.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:08.922+0000 7fb1877fe640 1 -- 172.21.15.64:0/1267128282 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18c018730 con 0x7fb198102570 2024-09-12T03:41:08.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:08.922+0000 7fb19c918640 1 -- 172.21.15.64:0/1267128282 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1981a4de0 con 0x7fb198102570 2024-09-12T03:41:08.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:08.923+0000 7fb19c918640 1 -- 172.21.15.64:0/1267128282 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb198101200 con 0x7fb198102570 2024-09-12T03:41:08.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:08.924+0000 7fb1877fe640 1 -- 172.21.15.64:0/1267128282 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fb18c00f450 con 0x7fb198102570 2024-09-12T03:41:08.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:08.925+0000 7fb1877fe640 1 --2- 172.21.15.64:0/1267128282 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fb1800a2d70 0x7fb1800a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:41:08.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:08.925+0000 7fb1877fe640 1 -- 172.21.15.64:0/1267128282 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9199+0+0 (secure 0 0 0) 0x7fb18c0cf720 con 0x7fb198102570 2024-09-12T03:41:08.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:08.925+0000 7fb19659c640 1 --2- 172.21.15.64:0/1267128282 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fb1800a2d70 0x7fb1800a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:41:08.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:08.925+0000 7fb19659c640 1 --2- 172.21.15.64:0/1267128282 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fb1800a2d70 0x7fb1800a5230 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb18800abe0 tx=0x7fb18800a2d0 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:41:08.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:08.929+0000 7fb1877fe640 1 -- 172.21.15.64:0/1267128282 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb18c014030 con 0x7fb198102570 2024-09-12T03:41:08.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:08 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/3967390695' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:41:08.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:08 smithi195 ceph-mon[34786]: osdmap e105: 12 total, 12 up, 12 in 2024-09-12T03:41:09.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:09.106+0000 7fb19c918640 1 -- 172.21.15.64:0/1267128282 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb19810a0a0 con 0x7fb198102570 2024-09-12T03:41:09.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:09.514+0000 7fb1877fe640 1 -- 172.21.15.64:0/1267128282 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v106) ==== 169+0+0 (secure 0 0 0) 0x7fb18c09c0a0 con 0x7fb198102570 2024-09-12T03:41:09.515 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' removed 2024-09-12T03:41:09.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:09.517+0000 7fb19c918640 1 -- 172.21.15.64:0/1267128282 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fb1800a2d70 msgr2=0x7fb1800a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:41:09.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:09.517+0000 7fb19c918640 1 --2- 172.21.15.64:0/1267128282 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fb1800a2d70 0x7fb1800a5230 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb18800abe0 tx=0x7fb18800a2d0 comp rx=0 tx=0).stop 2024-09-12T03:41:09.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:09.517+0000 7fb19c918640 1 -- 172.21.15.64:0/1267128282 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb198102570 msgr2=0x7fb1981987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:41:09.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:09.517+0000 7fb19c918640 1 --2- 172.21.15.64:0/1267128282 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb198102570 0x7fb1981987a0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fb18c00c920 tx=0x7fb18c00cd20 comp rx=0 tx=0).stop 2024-09-12T03:41:09.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:09.517+0000 7fb19c918640 1 -- 172.21.15.64:0/1267128282 shutdown_connections 2024-09-12T03:41:09.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:09.517+0000 7fb19c918640 1 --2- 172.21.15.64:0/1267128282 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fb1800a2d70 0x7fb1800a5230 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:09.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:09.517+0000 7fb19c918640 1 --2- 172.21.15.64:0/1267128282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb198102ef0 0x7fb19819cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:09.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:09.517+0000 7fb19c918640 1 --2- 172.21.15.64:0/1267128282 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb198102570 0x7fb1981987a0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:09.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:09.517+0000 7fb19c918640 1 --2- 172.21.15.64:0/1267128282 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb198100e20 0x7fb198198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:09.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:09.517+0000 7fb19c918640 1 -- 172.21.15.64:0/1267128282 >> 172.21.15.64:0/1267128282 conn(0x7fb1980fc7d0 msgr2=0x7fb19806a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:41:09.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:09.517+0000 7fb19c918640 1 -- 172.21.15.64:0/1267128282 shutdown_connections 2024-09-12T03:41:09.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:09.517+0000 7fb19c918640 1 -- 172.21.15.64:0/1267128282 wait complete. 2024-09-12T03:41:09.771 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:09 smithi064 ceph-mon[30952]: pgmap v371: 97 pgs: 97 active+clean; 577 KiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:41:09.772 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:09 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/1267128282' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:41:09.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:09 smithi115 ceph-mon[34801]: pgmap v371: 97 pgs: 97 active+clean; 577 KiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:41:09.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:09 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/1267128282' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:41:09.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:09 smithi195 ceph-mon[34786]: pgmap v371: 97 pgs: 97 active+clean; 577 KiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:41:09.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:09 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/1267128282' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:41:10.230 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b0812460-70b5-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-12T03:41:10.489 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:41:10.783 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:10 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/1267128282' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:41:10.783 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:10 smithi064 ceph-mon[30952]: osdmap e106: 12 total, 12 up, 12 in 2024-09-12T03:41:10.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:10 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/1267128282' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:41:10.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:10 smithi115 ceph-mon[34801]: osdmap e106: 12 total, 12 up, 12 in 2024-09-12T03:41:10.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:10 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/1267128282' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:41:10.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:10 smithi195 ceph-mon[34786]: osdmap e106: 12 total, 12 up, 12 in 2024-09-12T03:41:10.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:10.961+0000 7fac04221640 1 -- 172.21.15.64:0/2379373794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabfc102570 msgr2=0x7fabfc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:41:10.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:10.961+0000 7fac04221640 1 --2- 172.21.15.64:0/2379373794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabfc102570 0x7fabfc1029b0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fabf0009900 tx=0x7fabf002a480 comp rx=0 tx=0).stop 2024-09-12T03:41:10.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:10.961+0000 7fac04221640 1 -- 172.21.15.64:0/2379373794 shutdown_connections 2024-09-12T03:41:10.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:10.961+0000 7fac04221640 1 --2- 172.21.15.64:0/2379373794 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabfc102ef0 0x7fabfc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:10.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:10.961+0000 7fac04221640 1 --2- 172.21.15.64:0/2379373794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabfc102570 0x7fabfc1029b0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:10.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:10.961+0000 7fac04221640 1 --2- 172.21.15.64:0/2379373794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabfc100e20 0x7fabfc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:10.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:10.961+0000 7fac04221640 1 -- 172.21.15.64:0/2379373794 >> 172.21.15.64:0/2379373794 conn(0x7fabfc0fc7d0 msgr2=0x7fabfc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:41:10.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:10.961+0000 7fac04221640 1 -- 172.21.15.64:0/2379373794 shutdown_connections 2024-09-12T03:41:10.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:10.961+0000 7fac04221640 1 -- 172.21.15.64:0/2379373794 wait complete. 2024-09-12T03:41:10.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:10.962+0000 7fac04221640 1 Processor -- start 2024-09-12T03:41:10.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:10.962+0000 7fac04221640 1 -- start start 2024-09-12T03:41:10.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:10.962+0000 7fac04221640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabfc100e20 0x7fabfc1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:41:10.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:10.962+0000 7fac04221640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabfc102570 0x7fabfc198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:41:10.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:10.962+0000 7fac04221640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabfc102ef0 0x7fabfc19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:41:10.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:10.962+0000 7fac04221640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fabfc110e30 con 0x7fabfc102ef0 2024-09-12T03:41:10.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:10.962+0000 7fac04221640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fabfc110cb0 con 0x7fabfc100e20 2024-09-12T03:41:10.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:10.962+0000 7fac04221640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fabfc110fb0 con 0x7fabfc102570 2024-09-12T03:41:10.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:10.962+0000 7fac027be640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabfc102ef0 0x7fabfc19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:41:10.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:10.962+0000 7fac027be640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabfc102ef0 0x7fabfc19cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59832/0 (socket says 172.21.15.64:59832) 2024-09-12T03:41:10.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:10.962+0000 7fac027be640 1 -- 172.21.15.64:0/2166345555 learned_addr learned my addr 172.21.15.64:0/2166345555 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-12T03:41:10.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:10.962+0000 7fac01fbd640 1 --2- 172.21.15.64:0/2166345555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabfc100e20 0x7fabfc1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:41:10.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:10.962+0000 7fac017bc640 1 --2- 172.21.15.64:0/2166345555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabfc102570 0x7fabfc198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:41:10.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:10.963+0000 7fac027be640 1 -- 172.21.15.64:0/2166345555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabfc102570 msgr2=0x7fabfc198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:41:10.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:10.963+0000 7fac027be640 1 --2- 172.21.15.64:0/2166345555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabfc102570 0x7fabfc198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:10.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:10.963+0000 7fac027be640 1 -- 172.21.15.64:0/2166345555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabfc100e20 msgr2=0x7fabfc1984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:41:10.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:10.963+0000 7fac027be640 1 --2- 172.21.15.64:0/2166345555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabfc100e20 0x7fabfc1984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:10.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:10.963+0000 7fac027be640 1 -- 172.21.15.64:0/2166345555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabfc19d480 con 0x7fabfc102ef0 2024-09-12T03:41:10.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:10.963+0000 7fac027be640 1 --2- 172.21.15.64:0/2166345555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabfc102ef0 0x7fabfc19cda0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fabf400eb10 tx=0x7fabf400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:41:10.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:10.963+0000 7fabeeffd640 1 -- 172.21.15.64:0/2166345555 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf4004b20 con 0x7fabfc102ef0 2024-09-12T03:41:10.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:10.963+0000 7fabeeffd640 1 -- 172.21.15.64:0/2166345555 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fabf4004cc0 con 0x7fabfc102ef0 2024-09-12T03:41:10.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:10.963+0000 7fac04221640 1 -- 172.21.15.64:0/2166345555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabfc19d770 con 0x7fabfc102ef0 2024-09-12T03:41:10.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:10.963+0000 7fabeeffd640 1 -- 172.21.15.64:0/2166345555 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf4010690 con 0x7fabfc102ef0 2024-09-12T03:41:10.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:10.963+0000 7fac04221640 1 -- 172.21.15.64:0/2166345555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabfc1a5050 con 0x7fabfc102ef0 2024-09-12T03:41:10.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:10.964+0000 7fac04221640 1 -- 172.21.15.64:0/2166345555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabfc101200 con 0x7fabfc102ef0 2024-09-12T03:41:10.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:10.965+0000 7fabeeffd640 1 -- 172.21.15.64:0/2166345555 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fabf4005000 con 0x7fabfc102ef0 2024-09-12T03:41:10.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:10.966+0000 7fabeeffd640 1 --2- 172.21.15.64:0/2166345555 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fabe80a2d70 0x7fabe80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:41:10.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:10.966+0000 7fac01fbd640 1 --2- 172.21.15.64:0/2166345555 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fabe80a2d70 0x7fabe80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:41:10.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:10.966+0000 7fabeeffd640 1 -- 172.21.15.64:0/2166345555 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8582+0+0 (secure 0 0 0) 0x7fabf40cfe50 con 0x7fabfc102ef0 2024-09-12T03:41:10.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:10.967+0000 7fac01fbd640 1 --2- 172.21.15.64:0/2166345555 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fabe80a2d70 0x7fabe80a5230 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fabf8009bd0 tx=0x7fabf8005f90 comp rx=0 tx=0).ready entity=mgr.15243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:41:10.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:10.969+0000 7fabeeffd640 1 -- 172.21.15.64:0/2166345555 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fabf409ca30 con 0x7fabfc102ef0 2024-09-12T03:41:11.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:11.143+0000 7fac04221640 1 -- 172.21.15.64:0/2166345555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fabfc199830 con 0x7fabfc102ef0 2024-09-12T03:41:11.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:11.525+0000 7fabeeffd640 1 -- 172.21.15.64:0/2166345555 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v107) ==== 178+0+0 (secure 0 0 0) 0x7fabfc199830 con 0x7fabfc102ef0 2024-09-12T03:41:11.526 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data_ec' removed 2024-09-12T03:41:11.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:11.529+0000 7fac04221640 1 -- 172.21.15.64:0/2166345555 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fabe80a2d70 msgr2=0x7fabe80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:41:11.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:11.529+0000 7fac04221640 1 --2- 172.21.15.64:0/2166345555 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fabe80a2d70 0x7fabe80a5230 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fabf8009bd0 tx=0x7fabf8005f90 comp rx=0 tx=0).stop 2024-09-12T03:41:11.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:11.529+0000 7fac04221640 1 -- 172.21.15.64:0/2166345555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabfc102ef0 msgr2=0x7fabfc19cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:41:11.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:11.529+0000 7fac04221640 1 --2- 172.21.15.64:0/2166345555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabfc102ef0 0x7fabfc19cda0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fabf400eb10 tx=0x7fabf400c2d0 comp rx=0 tx=0).stop 2024-09-12T03:41:11.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:11.529+0000 7fac04221640 1 -- 172.21.15.64:0/2166345555 shutdown_connections 2024-09-12T03:41:11.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:11.529+0000 7fac04221640 1 --2- 172.21.15.64:0/2166345555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabfc102ef0 0x7fabfc19cda0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:11.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:11.529+0000 7fac04221640 1 --2- 172.21.15.64:0/2166345555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabfc102570 0x7fabfc198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:11.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:11.529+0000 7fac04221640 1 --2- 172.21.15.64:0/2166345555 >> [v2:172.21.15.64:6800/3960824462,v1:172.21.15.64:6801/3960824462] conn(0x7fabe80a2d70 0x7fabe80a5230 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:11.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:11.529+0000 7fac04221640 1 --2- 172.21.15.64:0/2166345555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabfc100e20 0x7fabfc1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:41:11.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:11.529+0000 7fac04221640 1 -- 172.21.15.64:0/2166345555 >> 172.21.15.64:0/2166345555 conn(0x7fabfc0fc7d0 msgr2=0x7fabfc06a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:41:11.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:11.530+0000 7fac04221640 1 -- 172.21.15.64:0/2166345555 shutdown_connections 2024-09-12T03:41:11.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-12T03:41:11.530+0000 7fac04221640 1 -- 172.21.15.64:0/2166345555 wait complete. 2024-09-12T03:41:11.888 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:11 smithi064 ceph-mon[30952]: pgmap v373: 33 pgs: 33 active+clean; 577 KiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:41:11.888 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:11 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/2166345555' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:41:11.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:11 smithi115 ceph-mon[34801]: pgmap v373: 33 pgs: 33 active+clean; 577 KiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:41:11.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:11 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/2166345555' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:41:11.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:11 smithi195 ceph-mon[34786]: pgmap v373: 33 pgs: 33 active+clean; 577 KiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:41:11.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:11 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/2166345555' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:41:12.249 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:41:12.280 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:41:12.316 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:41:12.349 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-12T03:41:12.349 DEBUG:teuthology.orchestra.run.smithi064:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T03:41:12.366 DEBUG:teuthology.orchestra.run.smithi115:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T03:41:12.385 DEBUG:teuthology.orchestra.run.smithi195:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T03:41:12.407 INFO:tasks.cephadm:Stopping all daemons... 2024-09-12T03:41:12.407 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-12T03:41:12.408 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-b0812460-70b5-11ef-bcea-c7b262605968@mon.a 2024-09-12T03:41:12.521 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:12 smithi064 systemd[1]: Stopping Ceph mon.a for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:41:12.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:12 smithi115 ceph-mon[34801]: from='client.? 172.21.15.64:0/2166345555' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:41:12.642 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:12 smithi115 ceph-mon[34801]: osdmap e107: 12 total, 12 up, 12 in 2024-09-12T03:41:12.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:12 smithi195 ceph-mon[34786]: from='client.? 172.21.15.64:0/2166345555' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:41:12.706 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:12 smithi195 ceph-mon[34786]: osdmap e107: 12 total, 12 up, 12 in 2024-09-12T03:41:13.021 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:12 smithi064 ceph-mon[30952]: from='client.? 172.21.15.64:0/2166345555' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:41:13.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:12 smithi064 ceph-mon[30952]: osdmap e107: 12 total, 12 up, 12 in 2024-09-12T03:41:13.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:12 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mon-a[30948]: 2024-09-12T03:41:12.726+0000 7fa8748d0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:41:13.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:12 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mon-a[30948]: 2024-09-12T03:41:12.726+0000 7fa8748d0640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-12T03:41:13.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:12 smithi064 podman[93802]: 2024-09-12 03:41:12.748399761 +0000 UTC m=+0.138064023 container died 9aa76b73f7af2ea7831e04fc80191f235927cc5c7db6073b4e687510da1b6a80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-12T03:41:13.479 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:13 smithi064 podman[93802]: 2024-09-12 03:41:13.153395588 +0000 UTC m=+0.543059849 container cleanup 9aa76b73f7af2ea7831e04fc80191f235927cc5c7db6073b4e687510da1b6a80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-b53b6d3, GIT_CLEAN=True) 2024-09-12T03:41:13.479 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 12 03:41:13 smithi064 bash[93802]: ceph-b0812460-70b5-11ef-bcea-c7b262605968-mon-a 2024-09-12T03:41:13.642 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@mon.a.service' 2024-09-12T03:41:13.686 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:41:13.686 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-12T03:41:13.687 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-12T03:41:13.687 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-b0812460-70b5-11ef-bcea-c7b262605968@mon.b 2024-09-12T03:41:14.131 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:13 smithi115 systemd[1]: Stopping Ceph mon.b for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:41:14.392 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:14 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mon-b[34797]: 2024-09-12T03:41:14.131+0000 7f14ca8e9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:41:14.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:14 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mon-b[34797]: 2024-09-12T03:41:14.131+0000 7f14ca8e9640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-12T03:41:14.393 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:14 smithi115 podman[69792]: 2024-09-12 03:41:14.284515722 +0000 UTC m=+0.399571693 container died a962e723c8e9def2736f1f9d715da0338454992311a2dc6af048be37822bc316 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mon-b, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-12T03:41:14.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:14 smithi115 podman[69792]: 2024-09-12 03:41:14.73190333 +0000 UTC m=+0.846959302 container cleanup a962e723c8e9def2736f1f9d715da0338454992311a2dc6af048be37822bc316 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mon-b, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-09-12T03:41:14.893 INFO:journalctl@ceph.mon.b.smithi115.stdout:Sep 12 03:41:14 smithi115 bash[69792]: ceph-b0812460-70b5-11ef-bcea-c7b262605968-mon-b 2024-09-12T03:41:15.161 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@mon.b.service' 2024-09-12T03:41:15.209 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:41:15.210 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-12T03:41:15.210 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-12T03:41:15.210 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-b0812460-70b5-11ef-bcea-c7b262605968@mon.c 2024-09-12T03:41:15.535 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:15 smithi195 systemd[1]: Stopping Ceph mon.c for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:41:15.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:15 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mon-c[34782]: 2024-09-12T03:41:15.535+0000 7efecbe4b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:41:15.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:15 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mon-c[34782]: 2024-09-12T03:41:15.535+0000 7efecbe4b640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-12T03:41:15.956 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:15 smithi195 podman[71529]: 2024-09-12 03:41:15.689055048 +0000 UTC m=+0.274297174 container died 7ddb2cba258e6b2e2653a6039f4e1d3e2013edd2e5b8b0d55d89725396940207 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mon-c, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:41:16.395 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:16 smithi195 podman[71529]: 2024-09-12 03:41:16.044674936 +0000 UTC m=+0.629917060 container cleanup 7ddb2cba258e6b2e2653a6039f4e1d3e2013edd2e5b8b0d55d89725396940207 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-12T03:41:16.395 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:16 smithi195 bash[71529]: ceph-b0812460-70b5-11ef-bcea-c7b262605968-mon-c 2024-09-12T03:41:16.629 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@mon.c.service' 2024-09-12T03:41:16.651 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:16 smithi195 podman[71541]: 2024-09-12 03:41:16.396242053 +0000 UTC m=+0.699554826 container remove 7ddb2cba258e6b2e2653a6039f4e1d3e2013edd2e5b8b0d55d89725396940207 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mon-c, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:41:16.651 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:16 smithi195 systemd[1]: ceph-b0812460-70b5-11ef-bcea-c7b262605968@mon.c.service: Deactivated successfully. 2024-09-12T03:41:16.652 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:16 smithi195 systemd[1]: Stopped Ceph mon.c for b0812460-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:41:16.652 INFO:journalctl@ceph.mon.c.smithi195.stdout:Sep 12 03:41:16 smithi195 systemd[1]: ceph-b0812460-70b5-11ef-bcea-c7b262605968@mon.c.service: Consumed 13.905s CPU time. 2024-09-12T03:41:16.701 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:41:16.701 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-12T03:41:16.701 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-12T03:41:16.702 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-b0812460-70b5-11ef-bcea-c7b262605968@mgr.x 2024-09-12T03:41:17.022 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:41:16 smithi064 systemd[1]: Stopping Ceph mgr.x for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:41:17.316 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:41:17 smithi064 podman[93924]: 2024-09-12 03:41:17.04731796 +0000 UTC m=+0.142969132 container died 7310212332e24bfabf38c9bd8bebb447eb8f9c4182ef80b9a580bba080816574 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-12T03:41:17.693 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:41:17 smithi064 podman[93924]: 2024-09-12 03:41:17.417062613 +0000 UTC m=+0.512713783 container cleanup 7310212332e24bfabf38c9bd8bebb447eb8f9c4182ef80b9a580bba080816574 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:41:17.693 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 12 03:41:17 smithi064 bash[93924]: ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-x 2024-09-12T03:41:17.819 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-12T03:41:17.856 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:41:17.856 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-12T03:41:17.856 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-12T03:41:17.856 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-b0812460-70b5-11ef-bcea-c7b262605968@mgr.y 2024-09-12T03:41:18.142 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:41:17 smithi115 systemd[1]: Stopping Ceph mgr.y for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:41:18.503 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:41:18 smithi115 podman[69911]: 2024-09-12 03:41:18.215806654 +0000 UTC m=+0.163592210 container died 4d9efb9334255324927f77dcf5ca7993488fd087ec41564f942947ee751ecdc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-12T03:41:18.879 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:41:18 smithi115 podman[69911]: 2024-09-12 03:41:18.612456952 +0000 UTC m=+0.560242498 container cleanup 4d9efb9334255324927f77dcf5ca7993488fd087ec41564f942947ee751ecdc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-12T03:41:18.880 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:41:18 smithi115 bash[69911]: ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y 2024-09-12T03:41:19.044 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-12T03:41:19.068 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:41:18 smithi115 podman[69925]: 2024-09-12 03:41:18.880503338 +0000 UTC m=+0.657187143 container remove 4d9efb9334255324927f77dcf5ca7993488fd087ec41564f942947ee751ecdc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-y, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-12T03:41:19.068 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:41:18 smithi115 systemd[1]: ceph-b0812460-70b5-11ef-bcea-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-09-12T03:41:19.068 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:41:19 smithi115 systemd[1]: ceph-b0812460-70b5-11ef-bcea-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-09-12T03:41:19.068 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:41:19 smithi115 systemd[1]: Stopped Ceph mgr.y for b0812460-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:41:19.068 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 12 03:41:19 smithi115 systemd[1]: ceph-b0812460-70b5-11ef-bcea-c7b262605968@mgr.y.service: Consumed 11.532s CPU time. 2024-09-12T03:41:19.079 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:41:19.079 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-12T03:41:19.079 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-12T03:41:19.080 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-b0812460-70b5-11ef-bcea-c7b262605968@mgr.z 2024-09-12T03:41:19.439 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:41:19 smithi195 systemd[1]: Stopping Ceph mgr.z for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:41:19.706 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:41:19 smithi195 podman[71654]: 2024-09-12 03:41:19.439489359 +0000 UTC m=+0.160789990 container died b3e44952077302e9879c26179fb2c81b696303eff2d9afa320226ba61037c47d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:41:20.085 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:41:19 smithi195 podman[71654]: 2024-09-12 03:41:19.867536654 +0000 UTC m=+0.588837282 container cleanup b3e44952077302e9879c26179fb2c81b696303eff2d9afa320226ba61037c47d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-12T03:41:20.086 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:41:19 smithi195 bash[71654]: ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z 2024-09-12T03:41:20.360 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:41:20 smithi195 podman[71666]: 2024-09-12 03:41:20.085995031 +0000 UTC m=+0.647481860 container remove b3e44952077302e9879c26179fb2c81b696303eff2d9afa320226ba61037c47d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mgr-z, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-12T03:41:20.361 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Sep 12 03:41:20 smithi195 systemd[1]: ceph-b0812460-70b5-11ef-bcea-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-09-12T03:41:20.371 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@mgr.z.service' 2024-09-12T03:41:20.408 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:41:20.409 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-12T03:41:20.409 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-12T03:41:20.409 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-b0812460-70b5-11ef-bcea-c7b262605968@osd.0 2024-09-12T03:41:20.721 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 12 03:41:20 smithi064 systemd[1]: Stopping Ceph osd.0 for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:41:21.021 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 12 03:41:20 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-0[45113]: 2024-09-12T03:41:20.721+0000 7faa04c3d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:41:21.022 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 12 03:41:20 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-0[45113]: 2024-09-12T03:41:20.721+0000 7faa04c3d640 -1 osd.0 107 *** Got signal Terminated *** 2024-09-12T03:41:21.022 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 12 03:41:20 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-0[45113]: 2024-09-12T03:41:20.721+0000 7faa04c3d640 -1 osd.0 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:41:26.521 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 12 03:41:26 smithi064 podman[94046]: 2024-09-12 03:41:26.167709407 +0000 UTC m=+5.556395730 container died f528e6db87f7c111f0c058969a84cf6aa41bc8f769a0bfea86ed09bb0b7263e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-12T03:41:26.915 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 12 03:41:26 smithi064 podman[94046]: 2024-09-12 03:41:26.688555035 +0000 UTC m=+6.077241348 container cleanup f528e6db87f7c111f0c058969a84cf6aa41bc8f769a0bfea86ed09bb0b7263e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-0, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-12T03:41:26.915 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 12 03:41:26 smithi064 bash[94046]: ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-0 2024-09-12T03:41:27.271 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 12 03:41:26 smithi064 podman[94061]: 2024-09-12 03:41:26.915311787 +0000 UTC m=+0.745105035 container remove f528e6db87f7c111f0c058969a84cf6aa41bc8f769a0bfea86ed09bb0b7263e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-12T03:41:27.735 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 12 03:41:27 smithi064 podman[94152]: 2024-09-12 03:41:27.229147279 +0000 UTC m=+0.016619117 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:41:27.736 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 12 03:41:27 smithi064 podman[94152]: 2024-09-12 03:41:27.358524266 +0000 UTC m=+0.145996143 container create 5de26ab55c70665a90530ba8c09f68f2bcdf158d72497f14f56a4fd71b2b7c96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-0-deactivate, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:41:28.022 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 12 03:41:27 smithi064 podman[94152]: 2024-09-12 03:41:27.736150853 +0000 UTC m=+0.523622689 container init 5de26ab55c70665a90530ba8c09f68f2bcdf158d72497f14f56a4fd71b2b7c96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD) 2024-09-12T03:41:28.022 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 12 03:41:27 smithi064 podman[94152]: 2024-09-12 03:41:27.739801767 +0000 UTC m=+0.527273609 container start 5de26ab55c70665a90530ba8c09f68f2bcdf158d72497f14f56a4fd71b2b7c96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-12T03:41:28.022 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 12 03:41:27 smithi064 podman[94152]: 2024-09-12 03:41:27.811334614 +0000 UTC m=+0.598806457 container attach 5de26ab55c70665a90530ba8c09f68f2bcdf158d72497f14f56a4fd71b2b7c96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-12T03:41:28.023 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 12 03:41:27 smithi064 podman[94152]: 2024-09-12 03:41:27.930937231 +0000 UTC m=+0.718409074 container died 5de26ab55c70665a90530ba8c09f68f2bcdf158d72497f14f56a4fd71b2b7c96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-12T03:41:28.463 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@osd.0.service' 2024-09-12T03:41:28.489 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 12 03:41:28 smithi064 podman[94177]: 2024-09-12 03:41:28.448584784 +0000 UTC m=+0.511790825 container remove 5de26ab55c70665a90530ba8c09f68f2bcdf158d72497f14f56a4fd71b2b7c96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-12T03:41:28.489 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 12 03:41:28 smithi064 systemd[1]: ceph-b0812460-70b5-11ef-bcea-c7b262605968@osd.0.service: Deactivated successfully. 2024-09-12T03:41:28.489 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 12 03:41:28 smithi064 systemd[1]: Stopped Ceph osd.0 for b0812460-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:41:28.489 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 12 03:41:28 smithi064 systemd[1]: ceph-b0812460-70b5-11ef-bcea-c7b262605968@osd.0.service: Consumed 11.858s CPU time. 2024-09-12T03:41:28.506 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:41:28.506 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-12T03:41:28.506 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-12T03:41:28.506 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-b0812460-70b5-11ef-bcea-c7b262605968@osd.1 2024-09-12T03:41:28.892 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 12 03:41:28 smithi115 systemd[1]: Stopping Ceph osd.1 for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:41:29.393 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 12 03:41:29 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-1[39262]: 2024-09-12T03:41:29.048+0000 7f4fc6131640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:41:29.393 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 12 03:41:29 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-1[39262]: 2024-09-12T03:41:29.048+0000 7f4fc6131640 -1 osd.1 107 *** Got signal Terminated *** 2024-09-12T03:41:29.393 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 12 03:41:29 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-1[39262]: 2024-09-12T03:41:29.048+0000 7f4fc6131640 -1 osd.1 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:41:34.643 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 12 03:41:34 smithi115 podman[70045]: 2024-09-12 03:41:34.387259992 +0000 UTC m=+5.689454753 container died e388777c79dce54276e60f06e7326e46669a39241dc8b2779c5c2d41d0324e6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-1, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2) 2024-09-12T03:41:35.066 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 12 03:41:34 smithi115 podman[70045]: 2024-09-12 03:41:34.773967222 +0000 UTC m=+6.076161986 container cleanup e388777c79dce54276e60f06e7326e46669a39241dc8b2779c5c2d41d0324e6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-1, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3) 2024-09-12T03:41:35.066 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 12 03:41:34 smithi115 bash[70045]: ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-1 2024-09-12T03:41:35.393 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 12 03:41:35 smithi115 podman[70060]: 2024-09-12 03:41:35.067344416 +0000 UTC m=+0.675431102 container remove e388777c79dce54276e60f06e7326e46669a39241dc8b2779c5c2d41d0324e6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:41:35.670 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 12 03:41:35 smithi115 podman[70146]: 2024-09-12 03:41:35.302547033 +0000 UTC m=+0.015863064 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:41:35.671 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 12 03:41:35 smithi115 podman[70146]: 2024-09-12 03:41:35.418234911 +0000 UTC m=+0.131550942 container create c293de98db2b58e24c5a99103040052c623601cb3e00b379b327e8f47502c866 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-12T03:41:36.143 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 12 03:41:35 smithi115 podman[70146]: 2024-09-12 03:41:35.671286511 +0000 UTC m=+0.384602543 container init c293de98db2b58e24c5a99103040052c623601cb3e00b379b327e8f47502c866 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-12T03:41:36.143 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 12 03:41:35 smithi115 podman[70146]: 2024-09-12 03:41:35.674808822 +0000 UTC m=+0.388124860 container start c293de98db2b58e24c5a99103040052c623601cb3e00b379b327e8f47502c866 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:41:36.143 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 12 03:41:35 smithi115 podman[70146]: 2024-09-12 03:41:35.779497616 +0000 UTC m=+0.492813669 container attach c293de98db2b58e24c5a99103040052c623601cb3e00b379b327e8f47502c866 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, ceph=True) 2024-09-12T03:41:36.143 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 12 03:41:35 smithi115 podman[70146]: 2024-09-12 03:41:35.865054707 +0000 UTC m=+0.578370748 container died c293de98db2b58e24c5a99103040052c623601cb3e00b379b327e8f47502c866 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:41:36.495 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@osd.1.service' 2024-09-12T03:41:36.535 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:41:36.535 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-12T03:41:36.535 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-12T03:41:36.536 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-b0812460-70b5-11ef-bcea-c7b262605968@osd.2 2024-09-12T03:41:36.855 INFO:journalctl@ceph.osd.2.smithi195.stdout:Sep 12 03:41:36 smithi195 systemd[1]: Stopping Ceph osd.2 for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:41:37.206 INFO:journalctl@ceph.osd.2.smithi195.stdout:Sep 12 03:41:36 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-2[39737]: 2024-09-12T03:41:36.855+0000 7fdc2e9cc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:41:37.206 INFO:journalctl@ceph.osd.2.smithi195.stdout:Sep 12 03:41:36 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-2[39737]: 2024-09-12T03:41:36.855+0000 7fdc2e9cc640 -1 osd.2 107 *** Got signal Terminated *** 2024-09-12T03:41:37.206 INFO:journalctl@ceph.osd.2.smithi195.stdout:Sep 12 03:41:36 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-2[39737]: 2024-09-12T03:41:36.855+0000 7fdc2e9cc640 -1 osd.2 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:41:42.456 INFO:journalctl@ceph.osd.2.smithi195.stdout:Sep 12 03:41:42 smithi195 podman[71786]: 2024-09-12 03:41:42.199304871 +0000 UTC m=+5.462176221 container died 3a77ee82f58c406f93674b63f9bd35a5c2efd25596bc3e5846aada89b773c4bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:41:42.840 INFO:journalctl@ceph.osd.2.smithi195.stdout:Sep 12 03:41:42 smithi195 podman[71786]: 2024-09-12 03:41:42.622399918 +0000 UTC m=+5.885271257 container cleanup 3a77ee82f58c406f93674b63f9bd35a5c2efd25596bc3e5846aada89b773c4bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:41:42.840 INFO:journalctl@ceph.osd.2.smithi195.stdout:Sep 12 03:41:42 smithi195 bash[71786]: ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-2 2024-09-12T03:41:43.206 INFO:journalctl@ceph.osd.2.smithi195.stdout:Sep 12 03:41:42 smithi195 podman[71802]: 2024-09-12 03:41:42.840529852 +0000 UTC m=+0.638606812 container remove 3a77ee82f58c406f93674b63f9bd35a5c2efd25596bc3e5846aada89b773c4bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-12T03:41:43.569 INFO:journalctl@ceph.osd.2.smithi195.stdout:Sep 12 03:41:43 smithi195 podman[71890]: 2024-09-12 03:41:43.184999944 +0000 UTC m=+0.015659726 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:41:43.569 INFO:journalctl@ceph.osd.2.smithi195.stdout:Sep 12 03:41:43 smithi195 podman[71890]: 2024-09-12 03:41:43.316434872 +0000 UTC m=+0.147094655 container create 22d9bef30a3fe2f20592404da5f186948441deaa7ed066fdc7e24f899be0e41b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True) 2024-09-12T03:41:43.569 INFO:journalctl@ceph.osd.2.smithi195.stdout:Sep 12 03:41:43 smithi195 podman[71890]: 2024-09-12 03:41:43.56955264 +0000 UTC m=+0.400212421 container init 22d9bef30a3fe2f20592404da5f186948441deaa7ed066fdc7e24f899be0e41b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-12T03:41:43.956 INFO:journalctl@ceph.osd.2.smithi195.stdout:Sep 12 03:41:43 smithi195 podman[71890]: 2024-09-12 03:41:43.573118297 +0000 UTC m=+0.403778079 container start 22d9bef30a3fe2f20592404da5f186948441deaa7ed066fdc7e24f899be0e41b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-12T03:41:43.956 INFO:journalctl@ceph.osd.2.smithi195.stdout:Sep 12 03:41:43 smithi195 podman[71890]: 2024-09-12 03:41:43.661321066 +0000 UTC m=+0.491980856 container attach 22d9bef30a3fe2f20592404da5f186948441deaa7ed066fdc7e24f899be0e41b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-12T03:41:43.956 INFO:journalctl@ceph.osd.2.smithi195.stdout:Sep 12 03:41:43 smithi195 podman[71890]: 2024-09-12 03:41:43.765146177 +0000 UTC m=+0.595805964 container died 22d9bef30a3fe2f20592404da5f186948441deaa7ed066fdc7e24f899be0e41b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-12T03:41:44.366 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@osd.2.service' 2024-09-12T03:41:44.405 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:41:44.405 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-12T03:41:44.405 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-12T03:41:44.405 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-b0812460-70b5-11ef-bcea-c7b262605968@osd.3 2024-09-12T03:41:44.726 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 12 03:41:44 smithi064 systemd[1]: Stopping Ceph osd.3 for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:41:44.999 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 12 03:41:44 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-3[51715]: 2024-09-12T03:41:44.726+0000 7f177f332640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:41:44.999 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 12 03:41:44 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-3[51715]: 2024-09-12T03:41:44.726+0000 7f177f332640 -1 osd.3 107 *** Got signal Terminated *** 2024-09-12T03:41:44.999 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 12 03:41:44 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-3[51715]: 2024-09-12T03:41:44.726+0000 7f177f332640 -1 osd.3 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:41:47.271 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 12 03:41:46 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-3[51715]: 2024-09-12T03:41:46.984+0000 7f1777940640 -1 osd.3 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:23.185382+0000 front 2024-09-12T03:41:23.185417+0000 (oldest deadline 2024-09-12T03:41:46.685672+0000) 2024-09-12T03:41:47.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:41:47 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:41:47.430+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:41:48.271 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 12 03:41:48 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-3[51715]: 2024-09-12T03:41:48.019+0000 7f1777940640 -1 osd.3 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:23.185382+0000 front 2024-09-12T03:41:23.185417+0000 (oldest deadline 2024-09-12T03:41:46.685672+0000) 2024-09-12T03:41:48.392 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:41:48 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:41:48.021+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:41:48.701 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:41:48 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:41:48.442+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:41:48.956 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:41:48 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5[46203]: 2024-09-12T03:41:48.701+0000 7f6c5ea16640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:25.510799+0000 front 2024-09-12T03:41:25.510774+0000 (oldest deadline 2024-09-12T03:41:48.311345+0000) 2024-09-12T03:41:49.271 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 12 03:41:48 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-3[51715]: 2024-09-12T03:41:48.988+0000 7f1777940640 -1 osd.3 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:23.185382+0000 front 2024-09-12T03:41:23.185417+0000 (oldest deadline 2024-09-12T03:41:46.685672+0000) 2024-09-12T03:41:49.392 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:41:49 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:41:49.033+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:41:49.445 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:41:49 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:41:49.109+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.706298+0000 front 2024-09-12T03:41:24.706335+0000 (oldest deadline 2024-09-12T03:41:48.206454+0000) 2024-09-12T03:41:49.704 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:41:49 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:41:49.444+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:41:49.772 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:41:49 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:41:49.384+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:23.172158+0000 front 2024-09-12T03:41:23.172197+0000 (oldest deadline 2024-09-12T03:41:48.973206+0000) 2024-09-12T03:41:49.956 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:41:49 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5[46203]: 2024-09-12T03:41:49.704+0000 7f6c5ea16640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:25.510799+0000 front 2024-09-12T03:41:25.510774+0000 (oldest deadline 2024-09-12T03:41:48.311345+0000) 2024-09-12T03:41:50.375 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 12 03:41:50 smithi064 podman[94257]: 2024-09-12 03:41:50.075363181 +0000 UTC m=+5.464491910 container died 93bf05989d26a93b2472a2531631f68b2ab60376d573bbea9bedf1e69748d3c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:41:50.392 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:41:50 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:41:50.032+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:41:50.435 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:41:50 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:41:50.088+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.706298+0000 front 2024-09-12T03:41:24.706335+0000 (oldest deadline 2024-09-12T03:41:48.206454+0000) 2024-09-12T03:41:50.687 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:41:50 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:41:50.436+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:41:50.772 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:41:50 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:41:50.374+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:23.172158+0000 front 2024-09-12T03:41:23.172197+0000 (oldest deadline 2024-09-12T03:41:48.973206+0000) 2024-09-12T03:41:50.772 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 12 03:41:50 smithi064 podman[94257]: 2024-09-12 03:41:50.534751391 +0000 UTC m=+5.923880110 container cleanup 93bf05989d26a93b2472a2531631f68b2ab60376d573bbea9bedf1e69748d3c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-3, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-12T03:41:50.772 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 12 03:41:50 smithi064 bash[94257]: ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-3 2024-09-12T03:41:50.956 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:41:50 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5[46203]: 2024-09-12T03:41:50.687+0000 7f6c5ea16640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:25.510799+0000 front 2024-09-12T03:41:25.510774+0000 (oldest deadline 2024-09-12T03:41:48.311345+0000) 2024-09-12T03:41:51.162 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 12 03:41:50 smithi064 podman[94285]: 2024-09-12 03:41:50.827823882 +0000 UTC m=+0.747968005 container remove 93bf05989d26a93b2472a2531631f68b2ab60376d573bbea9bedf1e69748d3c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-3, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3) 2024-09-12T03:41:51.386 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:41:51 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:41:51.094+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.706298+0000 front 2024-09-12T03:41:24.706335+0000 (oldest deadline 2024-09-12T03:41:48.206454+0000) 2024-09-12T03:41:51.392 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:41:50 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:41:50.984+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:41:51.456 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 12 03:41:51 smithi064 podman[94373]: 2024-09-12 03:41:51.066327215 +0000 UTC m=+0.016335351 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:41:51.456 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 12 03:41:51 smithi064 podman[94373]: 2024-09-12 03:41:51.195299355 +0000 UTC m=+0.145307481 container create 05755fb2ad025907bc8c409be59f52c995c4ea970ac9973b68161edeae00479b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3) 2024-09-12T03:41:51.456 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 12 03:41:51 smithi064 podman[94373]: 2024-09-12 03:41:51.45682011 +0000 UTC m=+0.406828235 container init 05755fb2ad025907bc8c409be59f52c995c4ea970ac9973b68161edeae00479b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True) 2024-09-12T03:41:51.457 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:41:51 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:41:51.364+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:23.172158+0000 front 2024-09-12T03:41:23.172197+0000 (oldest deadline 2024-09-12T03:41:48.973206+0000) 2024-09-12T03:41:51.675 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:41:51 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:41:51.386+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:41:51.771 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 12 03:41:51 smithi064 podman[94373]: 2024-09-12 03:41:51.46038426 +0000 UTC m=+0.410392385 container start 05755fb2ad025907bc8c409be59f52c995c4ea970ac9973b68161edeae00479b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3) 2024-09-12T03:41:51.772 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 12 03:41:51 smithi064 podman[94373]: 2024-09-12 03:41:51.548411704 +0000 UTC m=+0.498419829 container attach 05755fb2ad025907bc8c409be59f52c995c4ea970ac9973b68161edeae00479b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-12T03:41:51.772 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 12 03:41:51 smithi064 podman[94373]: 2024-09-12 03:41:51.658095448 +0000 UTC m=+0.608103591 container died 05755fb2ad025907bc8c409be59f52c995c4ea970ac9973b68161edeae00479b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-12T03:41:51.956 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:41:51 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5[46203]: 2024-09-12T03:41:51.675+0000 7f6c5ea16640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:25.510799+0000 front 2024-09-12T03:41:25.510774+0000 (oldest deadline 2024-09-12T03:41:48.311345+0000) 2024-09-12T03:41:52.364 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:41:52 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:41:52.076+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.706298+0000 front 2024-09-12T03:41:24.706335+0000 (oldest deadline 2024-09-12T03:41:48.206454+0000) 2024-09-12T03:41:52.392 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:41:52 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:41:52.017+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:41:52.510 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:41:52 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:41:52.315+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:23.172158+0000 front 2024-09-12T03:41:23.172197+0000 (oldest deadline 2024-09-12T03:41:48.973206+0000) 2024-09-12T03:41:52.538 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@osd.3.service' 2024-09-12T03:41:52.586 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:41:52.586 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-12T03:41:52.587 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-12T03:41:52.587 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-b0812460-70b5-11ef-bcea-c7b262605968@osd.4 2024-09-12T03:41:52.634 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:41:52 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:41:52.364+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:41:52.892 INFO:journalctl@ceph.osd.4.smithi115.stdout:Sep 12 03:41:52 smithi115 systemd[1]: Stopping Ceph osd.4 for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:41:52.956 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:41:52 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5[46203]: 2024-09-12T03:41:52.634+0000 7f6c5ea16640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:25.510799+0000 front 2024-09-12T03:41:25.510774+0000 (oldest deadline 2024-09-12T03:41:48.311345+0000) 2024-09-12T03:41:53.359 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:41:53 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:41:53.047+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.706298+0000 front 2024-09-12T03:41:24.706335+0000 (oldest deadline 2024-09-12T03:41:48.206454+0000) 2024-09-12T03:41:53.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:41:53 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:41:53.047+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:41:53.393 INFO:journalctl@ceph.osd.4.smithi115.stdout:Sep 12 03:41:52 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-4[45965]: 2024-09-12T03:41:52.900+0000 7f3456479640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:41:53.393 INFO:journalctl@ceph.osd.4.smithi115.stdout:Sep 12 03:41:52 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-4[45965]: 2024-09-12T03:41:52.900+0000 7f3456479640 -1 osd.4 107 *** Got signal Terminated *** 2024-09-12T03:41:53.394 INFO:journalctl@ceph.osd.4.smithi115.stdout:Sep 12 03:41:52 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-4[45965]: 2024-09-12T03:41:52.900+0000 7f3456479640 -1 osd.4 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:41:53.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:41:53 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:41:53.359+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:41:53.706 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:41:53 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5[46203]: 2024-09-12T03:41:53.607+0000 7f6c5ea16640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:25.510799+0000 front 2024-09-12T03:41:25.510774+0000 (oldest deadline 2024-09-12T03:41:48.311345+0000) 2024-09-12T03:41:53.771 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:41:53 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:41:53.359+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:23.172158+0000 front 2024-09-12T03:41:23.172197+0000 (oldest deadline 2024-09-12T03:41:48.973206+0000) 2024-09-12T03:41:54.383 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:41:54 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:41:54.039+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.706298+0000 front 2024-09-12T03:41:24.706335+0000 (oldest deadline 2024-09-12T03:41:48.206454+0000) 2024-09-12T03:41:54.392 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:41:54 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:41:54.044+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:41:54.706 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:41:54 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5[46203]: 2024-09-12T03:41:54.591+0000 7f6c5ea16640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:25.510799+0000 front 2024-09-12T03:41:25.510774+0000 (oldest deadline 2024-09-12T03:41:48.311345+0000) 2024-09-12T03:41:54.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:41:54 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:41:54.383+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:41:54.773 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:41:54 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:41:54.407+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:23.172158+0000 front 2024-09-12T03:41:23.172197+0000 (oldest deadline 2024-09-12T03:41:48.973206+0000) 2024-09-12T03:41:55.375 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:41:55 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:41:55.066+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.706298+0000 front 2024-09-12T03:41:24.706335+0000 (oldest deadline 2024-09-12T03:41:48.206454+0000) 2024-09-12T03:41:55.392 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:41:55 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:41:55.056+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:41:55.651 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:41:55 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:41:55.375+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:41:55.652 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:41:55 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5[46203]: 2024-09-12T03:41:55.587+0000 7f6c5ea16640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:25.510799+0000 front 2024-09-12T03:41:25.510774+0000 (oldest deadline 2024-09-12T03:41:48.311345+0000) 2024-09-12T03:41:55.652 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:41:55 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5[46203]: 2024-09-12T03:41:55.587+0000 7f6c5ea16640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:31.212365+0000 front 2024-09-12T03:41:31.212522+0000 (oldest deadline 2024-09-12T03:41:54.712467+0000) 2024-09-12T03:41:55.771 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:41:55 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:41:55.454+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:23.172158+0000 front 2024-09-12T03:41:23.172197+0000 (oldest deadline 2024-09-12T03:41:48.973206+0000) 2024-09-12T03:41:55.892 INFO:journalctl@ceph.osd.4.smithi115.stdout:Sep 12 03:41:55 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-4[45965]: 2024-09-12T03:41:55.557+0000 7f344ea87640 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:31.342027+0000 front 2024-09-12T03:41:31.341997+0000 (oldest deadline 2024-09-12T03:41:55.442127+0000) 2024-09-12T03:41:56.392 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:41:56 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:41:56.026+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:41:56.392 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:41:56 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:41:56.026+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:41:56.393 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:41:55 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:41:55.969+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.424330+0000 front 2024-09-12T03:41:33.424271+0000 (oldest deadline 2024-09-12T03:41:55.124418+0000) 2024-09-12T03:41:56.393 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:41:56 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:41:56.067+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.706298+0000 front 2024-09-12T03:41:24.706335+0000 (oldest deadline 2024-09-12T03:41:48.206454+0000) 2024-09-12T03:41:56.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:41:56 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:41:56.393+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:41:56.706 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:41:56 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5[46203]: 2024-09-12T03:41:56.540+0000 7f6c5ea16640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:25.510799+0000 front 2024-09-12T03:41:25.510774+0000 (oldest deadline 2024-09-12T03:41:48.311345+0000) 2024-09-12T03:41:56.706 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:41:56 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5[46203]: 2024-09-12T03:41:56.540+0000 7f6c5ea16640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:31.212365+0000 front 2024-09-12T03:41:31.212522+0000 (oldest deadline 2024-09-12T03:41:54.712467+0000) 2024-09-12T03:41:56.771 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:41:56 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:41:56.463+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:23.172158+0000 front 2024-09-12T03:41:23.172197+0000 (oldest deadline 2024-09-12T03:41:48.973206+0000) 2024-09-12T03:41:56.772 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:41:56 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:41:56.463+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.375123+0000 front 2024-09-12T03:41:32.375107+0000 (oldest deadline 2024-09-12T03:41:55.874935+0000) 2024-09-12T03:41:56.892 INFO:journalctl@ceph.osd.4.smithi115.stdout:Sep 12 03:41:56 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-4[45965]: 2024-09-12T03:41:56.591+0000 7f344ea87640 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:31.342027+0000 front 2024-09-12T03:41:31.341997+0000 (oldest deadline 2024-09-12T03:41:55.442127+0000) 2024-09-12T03:41:57.392 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:41:56 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:41:56.984+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.424330+0000 front 2024-09-12T03:41:33.424271+0000 (oldest deadline 2024-09-12T03:41:55.124418+0000) 2024-09-12T03:41:57.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:41:57 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:41:57.027+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:41:57.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:41:57 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:41:57.027+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:41:57.442 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:41:57 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:41:57.104+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.706298+0000 front 2024-09-12T03:41:24.706335+0000 (oldest deadline 2024-09-12T03:41:48.206454+0000) 2024-09-12T03:41:57.443 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:41:57 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:41:57.104+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.408037+0000 front 2024-09-12T03:41:33.408013+0000 (oldest deadline 2024-09-12T03:41:56.308104+0000) 2024-09-12T03:41:57.706 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:41:57 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5[46203]: 2024-09-12T03:41:57.558+0000 7f6c5ea16640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:25.510799+0000 front 2024-09-12T03:41:25.510774+0000 (oldest deadline 2024-09-12T03:41:48.311345+0000) 2024-09-12T03:41:57.706 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:41:57 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5[46203]: 2024-09-12T03:41:57.558+0000 7f6c5ea16640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:31.212365+0000 front 2024-09-12T03:41:31.212522+0000 (oldest deadline 2024-09-12T03:41:54.712467+0000) 2024-09-12T03:41:57.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:41:57 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:41:57.442+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:41:57.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:41:57 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:41:57.442+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:41:57.772 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:41:57 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:41:57.437+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:23.172158+0000 front 2024-09-12T03:41:23.172197+0000 (oldest deadline 2024-09-12T03:41:48.973206+0000) 2024-09-12T03:41:57.772 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:41:57 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:41:57.437+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.375123+0000 front 2024-09-12T03:41:32.375107+0000 (oldest deadline 2024-09-12T03:41:55.874935+0000) 2024-09-12T03:41:57.892 INFO:journalctl@ceph.osd.4.smithi115.stdout:Sep 12 03:41:57 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-4[45965]: 2024-09-12T03:41:57.631+0000 7f344ea87640 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:31.342027+0000 front 2024-09-12T03:41:31.341997+0000 (oldest deadline 2024-09-12T03:41:55.442127+0000) 2024-09-12T03:41:58.234 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:41:57 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:41:57.975+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.424330+0000 front 2024-09-12T03:41:33.424271+0000 (oldest deadline 2024-09-12T03:41:55.124418+0000) 2024-09-12T03:41:58.234 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:41:57 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:41:57.983+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:41:58.234 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:41:57 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:41:57.983+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:41:58.271 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:41:57 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:41:57.821+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.723241+0000 front 2024-09-12T03:41:32.723103+0000 (oldest deadline 2024-09-12T03:41:56.822981+0000) 2024-09-12T03:41:58.412 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:41:58 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:41:58.113+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.706298+0000 front 2024-09-12T03:41:24.706335+0000 (oldest deadline 2024-09-12T03:41:48.206454+0000) 2024-09-12T03:41:58.412 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:41:58 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:41:58.113+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.408037+0000 front 2024-09-12T03:41:33.408013+0000 (oldest deadline 2024-09-12T03:41:56.308104+0000) 2024-09-12T03:41:58.510 INFO:journalctl@ceph.osd.4.smithi115.stdout:Sep 12 03:41:58 smithi115 podman[70249]: 2024-09-12 03:41:58.234420173 +0000 UTC m=+5.452061539 container died ff76fa4e1f1fdc7ce34c29a032f1716deeda2620448013d99379a66d7f3bff8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-4, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-12T03:41:58.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:41:58 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:41:58.412+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:41:58.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:41:58 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:41:58.412+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:41:58.706 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:41:58 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5[46203]: 2024-09-12T03:41:58.577+0000 7f6c5ea16640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:25.510799+0000 front 2024-09-12T03:41:25.510774+0000 (oldest deadline 2024-09-12T03:41:48.311345+0000) 2024-09-12T03:41:58.706 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:41:58 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5[46203]: 2024-09-12T03:41:58.577+0000 7f6c5ea16640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:31.212365+0000 front 2024-09-12T03:41:31.212522+0000 (oldest deadline 2024-09-12T03:41:54.712467+0000) 2024-09-12T03:41:58.771 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:41:58 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:41:58.408+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:23.172158+0000 front 2024-09-12T03:41:23.172197+0000 (oldest deadline 2024-09-12T03:41:48.973206+0000) 2024-09-12T03:41:58.771 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:41:58 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:41:58.408+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.375123+0000 front 2024-09-12T03:41:32.375107+0000 (oldest deadline 2024-09-12T03:41:55.874935+0000) 2024-09-12T03:41:58.885 INFO:journalctl@ceph.osd.4.smithi115.stdout:Sep 12 03:41:58 smithi115 podman[70249]: 2024-09-12 03:41:58.609149017 +0000 UTC m=+5.826790353 container cleanup ff76fa4e1f1fdc7ce34c29a032f1716deeda2620448013d99379a66d7f3bff8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-4, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-12T03:41:58.885 INFO:journalctl@ceph.osd.4.smithi115.stdout:Sep 12 03:41:58 smithi115 bash[70249]: ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-4 2024-09-12T03:41:59.142 INFO:journalctl@ceph.osd.4.smithi115.stdout:Sep 12 03:41:58 smithi115 podman[70264]: 2024-09-12 03:41:58.885649365 +0000 UTC m=+0.637482371 container remove ff76fa4e1f1fdc7ce34c29a032f1716deeda2620448013d99379a66d7f3bff8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-4, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-12T03:41:59.143 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:41:59 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:41:59.001+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.424330+0000 front 2024-09-12T03:41:33.424271+0000 (oldest deadline 2024-09-12T03:41:55.124418+0000) 2024-09-12T03:41:59.143 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:41:58 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:41:58.985+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:41:59.143 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:41:58 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:41:58.985+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:41:59.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:41:58 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:41:58.829+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.723241+0000 front 2024-09-12T03:41:32.723103+0000 (oldest deadline 2024-09-12T03:41:56.822981+0000) 2024-09-12T03:41:59.376 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:41:59 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:41:59.099+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.706298+0000 front 2024-09-12T03:41:24.706335+0000 (oldest deadline 2024-09-12T03:41:48.206454+0000) 2024-09-12T03:41:59.376 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:41:59 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:41:59.099+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.408037+0000 front 2024-09-12T03:41:33.408013+0000 (oldest deadline 2024-09-12T03:41:56.308104+0000) 2024-09-12T03:41:59.505 INFO:journalctl@ceph.osd.4.smithi115.stdout:Sep 12 03:41:59 smithi115 podman[70356]: 2024-09-12 03:41:59.129124934 +0000 UTC m=+0.017032621 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:41:59.505 INFO:journalctl@ceph.osd.4.smithi115.stdout:Sep 12 03:41:59 smithi115 podman[70356]: 2024-09-12 03:41:59.253496137 +0000 UTC m=+0.141403875 container create acce30608515fe323998e82a565eb061bf12fdd92e90e4af642467fcfc5b35cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-12T03:41:59.505 INFO:journalctl@ceph.osd.4.smithi115.stdout:Sep 12 03:41:59 smithi115 podman[70356]: 2024-09-12 03:41:59.505828843 +0000 UTC m=+0.393736527 container init acce30608515fe323998e82a565eb061bf12fdd92e90e4af642467fcfc5b35cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-12T03:41:59.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:41:59 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:41:59.376+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:41:59.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:41:59 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:41:59.376+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:41:59.706 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:41:59 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5[46203]: 2024-09-12T03:41:59.564+0000 7f6c5ea16640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:25.510799+0000 front 2024-09-12T03:41:25.510774+0000 (oldest deadline 2024-09-12T03:41:48.311345+0000) 2024-09-12T03:41:59.706 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:41:59 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5[46203]: 2024-09-12T03:41:59.564+0000 7f6c5ea16640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:31.212365+0000 front 2024-09-12T03:41:31.212522+0000 (oldest deadline 2024-09-12T03:41:54.712467+0000) 2024-09-12T03:41:59.771 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:41:59 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:41:59.368+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:23.172158+0000 front 2024-09-12T03:41:23.172197+0000 (oldest deadline 2024-09-12T03:41:48.973206+0000) 2024-09-12T03:41:59.772 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:41:59 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:41:59.368+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.375123+0000 front 2024-09-12T03:41:32.375107+0000 (oldest deadline 2024-09-12T03:41:55.874935+0000) 2024-09-12T03:41:59.892 INFO:journalctl@ceph.osd.4.smithi115.stdout:Sep 12 03:41:59 smithi115 podman[70356]: 2024-09-12 03:41:59.509322878 +0000 UTC m=+0.397230562 container start acce30608515fe323998e82a565eb061bf12fdd92e90e4af642467fcfc5b35cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-4-deactivate, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-12T03:41:59.893 INFO:journalctl@ceph.osd.4.smithi115.stdout:Sep 12 03:41:59 smithi115 podman[70356]: 2024-09-12 03:41:59.6059592 +0000 UTC m=+0.493866907 container attach acce30608515fe323998e82a565eb061bf12fdd92e90e4af642467fcfc5b35cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-4-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-12T03:41:59.893 INFO:journalctl@ceph.osd.4.smithi115.stdout:Sep 12 03:41:59 smithi115 podman[70356]: 2024-09-12 03:41:59.700464947 +0000 UTC m=+0.588372642 container died acce30608515fe323998e82a565eb061bf12fdd92e90e4af642467fcfc5b35cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-09-12T03:42:00.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:41:59 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:41:59.783+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.723241+0000 front 2024-09-12T03:41:32.723103+0000 (oldest deadline 2024-09-12T03:41:56.822981+0000) 2024-09-12T03:42:00.393 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:41:59 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:41:59.998+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.424330+0000 front 2024-09-12T03:41:33.424271+0000 (oldest deadline 2024-09-12T03:41:55.124418+0000) 2024-09-12T03:42:00.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:41:59 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:41:59.995+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:00.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:41:59 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:41:59.995+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:00.456 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:00 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:00.121+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.706298+0000 front 2024-09-12T03:41:24.706335+0000 (oldest deadline 2024-09-12T03:41:48.206454+0000) 2024-09-12T03:42:00.456 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:00 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:00.121+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.408037+0000 front 2024-09-12T03:41:33.408013+0000 (oldest deadline 2024-09-12T03:41:56.308104+0000) 2024-09-12T03:42:00.456 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:00 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:00.360+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:00.456 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:00 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:00.360+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:00.505 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@osd.4.service' 2024-09-12T03:42:00.550 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:42:00.550 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-12T03:42:00.550 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-12T03:42:00.551 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-b0812460-70b5-11ef-bcea-c7b262605968@osd.5 2024-09-12T03:42:00.766 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:00 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:00.340+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:23.172158+0000 front 2024-09-12T03:41:23.172197+0000 (oldest deadline 2024-09-12T03:41:48.973206+0000) 2024-09-12T03:42:00.766 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:00 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:00.340+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.375123+0000 front 2024-09-12T03:41:32.375107+0000 (oldest deadline 2024-09-12T03:41:55.874935+0000) 2024-09-12T03:42:00.868 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:42:00 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5[46203]: 2024-09-12T03:42:00.552+0000 7f6c5ea16640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:25.510799+0000 front 2024-09-12T03:41:25.510774+0000 (oldest deadline 2024-09-12T03:41:48.311345+0000) 2024-09-12T03:42:00.868 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:42:00 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5[46203]: 2024-09-12T03:42:00.552+0000 7f6c5ea16640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:31.212365+0000 front 2024-09-12T03:41:31.212522+0000 (oldest deadline 2024-09-12T03:41:54.712467+0000) 2024-09-12T03:42:00.868 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:42:00 smithi195 systemd[1]: Stopping Ceph osd.5 for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:42:00.868 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:42:00 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5[46203]: 2024-09-12T03:42:00.867+0000 7f6c62400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:42:00.868 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:42:00 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5[46203]: 2024-09-12T03:42:00.867+0000 7f6c62400640 -1 osd.5 107 *** Got signal Terminated *** 2024-09-12T03:42:00.868 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:42:00 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5[46203]: 2024-09-12T03:42:00.867+0000 7f6c62400640 -1 osd.5 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:42:01.021 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:00 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:00.766+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.723241+0000 front 2024-09-12T03:41:32.723103+0000 (oldest deadline 2024-09-12T03:41:56.822981+0000) 2024-09-12T03:42:01.206 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:01 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:01.117+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.706298+0000 front 2024-09-12T03:41:24.706335+0000 (oldest deadline 2024-09-12T03:41:48.206454+0000) 2024-09-12T03:42:01.206 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:01 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:01.117+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.408037+0000 front 2024-09-12T03:41:33.408013+0000 (oldest deadline 2024-09-12T03:41:56.308104+0000) 2024-09-12T03:42:01.392 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:01 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:01.000+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:01.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:01 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:01.000+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:01.393 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:01 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:01.048+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.424330+0000 front 2024-09-12T03:41:33.424271+0000 (oldest deadline 2024-09-12T03:41:55.124418+0000) 2024-09-12T03:42:01.706 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:42:01 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5[46203]: 2024-09-12T03:42:01.562+0000 7f6c5ea16640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:25.510799+0000 front 2024-09-12T03:41:25.510774+0000 (oldest deadline 2024-09-12T03:41:48.311345+0000) 2024-09-12T03:42:01.706 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:42:01 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5[46203]: 2024-09-12T03:42:01.562+0000 7f6c5ea16640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:31.212365+0000 front 2024-09-12T03:41:31.212522+0000 (oldest deadline 2024-09-12T03:41:54.712467+0000) 2024-09-12T03:42:01.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:01 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:01.391+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:01.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:01 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:01.391+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:01.765 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:01 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:01.299+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:23.172158+0000 front 2024-09-12T03:41:23.172197+0000 (oldest deadline 2024-09-12T03:41:48.973206+0000) 2024-09-12T03:42:01.765 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:01 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:01.299+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.375123+0000 front 2024-09-12T03:41:32.375107+0000 (oldest deadline 2024-09-12T03:41:55.874935+0000) 2024-09-12T03:42:02.021 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:01 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:01.765+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.723241+0000 front 2024-09-12T03:41:32.723103+0000 (oldest deadline 2024-09-12T03:41:56.822981+0000) 2024-09-12T03:42:02.392 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:02 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:02.027+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.424330+0000 front 2024-09-12T03:41:33.424271+0000 (oldest deadline 2024-09-12T03:41:55.124418+0000) 2024-09-12T03:42:02.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:02 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:02.033+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:02.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:02 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:02.033+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:02.427 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:02 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:02.130+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.706298+0000 front 2024-09-12T03:41:24.706335+0000 (oldest deadline 2024-09-12T03:41:48.206454+0000) 2024-09-12T03:42:02.427 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:02 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:02.130+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.408037+0000 front 2024-09-12T03:41:33.408013+0000 (oldest deadline 2024-09-12T03:41:56.308104+0000) 2024-09-12T03:42:02.590 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:02 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:02.274+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:23.172158+0000 front 2024-09-12T03:41:23.172197+0000 (oldest deadline 2024-09-12T03:41:48.973206+0000) 2024-09-12T03:42:02.590 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:02 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:02.274+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.375123+0000 front 2024-09-12T03:41:32.375107+0000 (oldest deadline 2024-09-12T03:41:55.874935+0000) 2024-09-12T03:42:02.706 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:42:02 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5[46203]: 2024-09-12T03:42:02.543+0000 7f6c5ea16640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:25.510799+0000 front 2024-09-12T03:41:25.510774+0000 (oldest deadline 2024-09-12T03:41:48.311345+0000) 2024-09-12T03:42:02.706 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:42:02 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5[46203]: 2024-09-12T03:42:02.543+0000 7f6c5ea16640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:31.212365+0000 front 2024-09-12T03:41:31.212522+0000 (oldest deadline 2024-09-12T03:41:54.712467+0000) 2024-09-12T03:42:02.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:02 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:02.427+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:02.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:02 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:02.427+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:02.707 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:02 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:02.427+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:03.022 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:02 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:02.757+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.723241+0000 front 2024-09-12T03:41:32.723103+0000 (oldest deadline 2024-09-12T03:41:56.822981+0000) 2024-09-12T03:42:03.022 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:02 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:02.757+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.224382+0000 front 2024-09-12T03:41:40.224550+0000 (oldest deadline 2024-09-12T03:42:02.524263+0000) 2024-09-12T03:42:03.380 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:03 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:03.095+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.706298+0000 front 2024-09-12T03:41:24.706335+0000 (oldest deadline 2024-09-12T03:41:48.206454+0000) 2024-09-12T03:42:03.380 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:03 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:03.095+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.408037+0000 front 2024-09-12T03:41:33.408013+0000 (oldest deadline 2024-09-12T03:41:56.308104+0000) 2024-09-12T03:42:03.392 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:03 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:03.018+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.424330+0000 front 2024-09-12T03:41:33.424271+0000 (oldest deadline 2024-09-12T03:41:55.124418+0000) 2024-09-12T03:42:03.392 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:03 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:03.056+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:03.392 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:03 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:03.056+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:03.706 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:42:03 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5[46203]: 2024-09-12T03:42:03.509+0000 7f6c5ea16640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:25.510799+0000 front 2024-09-12T03:41:25.510774+0000 (oldest deadline 2024-09-12T03:41:48.311345+0000) 2024-09-12T03:42:03.706 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:42:03 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5[46203]: 2024-09-12T03:42:03.509+0000 7f6c5ea16640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:31.212365+0000 front 2024-09-12T03:41:31.212522+0000 (oldest deadline 2024-09-12T03:41:54.712467+0000) 2024-09-12T03:42:03.706 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:42:03 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5[46203]: 2024-09-12T03:42:03.509+0000 7f6c5ea16640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.013314+0000 front 2024-09-12T03:41:40.013364+0000 (oldest deadline 2024-09-12T03:42:02.913474+0000) 2024-09-12T03:42:03.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:03 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:03.381+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:03.707 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:03 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:03.381+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:03.707 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:03 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:03.381+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:03.771 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:03 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:03.294+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:23.172158+0000 front 2024-09-12T03:41:23.172197+0000 (oldest deadline 2024-09-12T03:41:48.973206+0000) 2024-09-12T03:42:03.772 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:03 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:03.294+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.375123+0000 front 2024-09-12T03:41:32.375107+0000 (oldest deadline 2024-09-12T03:41:55.874935+0000) 2024-09-12T03:42:03.772 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:03 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:03.294+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.575971+0000 front 2024-09-12T03:41:40.576192+0000 (oldest deadline 2024-09-12T03:42:02.876105+0000) 2024-09-12T03:42:04.256 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:03 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:03.802+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.723241+0000 front 2024-09-12T03:41:32.723103+0000 (oldest deadline 2024-09-12T03:41:56.822981+0000) 2024-09-12T03:42:04.256 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:03 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:03.802+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.224382+0000 front 2024-09-12T03:41:40.224550+0000 (oldest deadline 2024-09-12T03:42:02.524263+0000) 2024-09-12T03:42:04.389 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:04 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:04.049+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.706298+0000 front 2024-09-12T03:41:24.706335+0000 (oldest deadline 2024-09-12T03:41:48.206454+0000) 2024-09-12T03:42:04.389 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:04 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:04.049+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.408037+0000 front 2024-09-12T03:41:33.408013+0000 (oldest deadline 2024-09-12T03:41:56.308104+0000) 2024-09-12T03:42:04.392 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:04 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:04.042+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.424330+0000 front 2024-09-12T03:41:33.424271+0000 (oldest deadline 2024-09-12T03:41:55.124418+0000) 2024-09-12T03:42:04.392 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:04 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:04.009+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:04.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:04 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:04.009+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:04.522 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:04 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:04.255+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:23.172158+0000 front 2024-09-12T03:41:23.172197+0000 (oldest deadline 2024-09-12T03:41:48.973206+0000) 2024-09-12T03:42:04.522 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:04 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:04.255+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.375123+0000 front 2024-09-12T03:41:32.375107+0000 (oldest deadline 2024-09-12T03:41:55.874935+0000) 2024-09-12T03:42:04.522 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:04 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:04.255+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.575971+0000 front 2024-09-12T03:41:40.576192+0000 (oldest deadline 2024-09-12T03:42:02.876105+0000) 2024-09-12T03:42:04.706 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:42:04 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5[46203]: 2024-09-12T03:42:04.492+0000 7f6c5ea16640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:25.510799+0000 front 2024-09-12T03:41:25.510774+0000 (oldest deadline 2024-09-12T03:41:48.311345+0000) 2024-09-12T03:42:04.706 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:42:04 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5[46203]: 2024-09-12T03:42:04.492+0000 7f6c5ea16640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:31.212365+0000 front 2024-09-12T03:41:31.212522+0000 (oldest deadline 2024-09-12T03:41:54.712467+0000) 2024-09-12T03:42:04.707 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:42:04 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5[46203]: 2024-09-12T03:42:04.492+0000 7f6c5ea16640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.013314+0000 front 2024-09-12T03:41:40.013364+0000 (oldest deadline 2024-09-12T03:42:02.913474+0000) 2024-09-12T03:42:04.707 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:04 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:04.389+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:04.707 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:04 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:04.389+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:04.707 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:04 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:04.389+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:05.209 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:04 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:04.793+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.723241+0000 front 2024-09-12T03:41:32.723103+0000 (oldest deadline 2024-09-12T03:41:56.822981+0000) 2024-09-12T03:42:05.209 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:04 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:04.793+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.224382+0000 front 2024-09-12T03:41:40.224550+0000 (oldest deadline 2024-09-12T03:42:02.524263+0000) 2024-09-12T03:42:05.354 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:05 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:05.083+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.706298+0000 front 2024-09-12T03:41:24.706335+0000 (oldest deadline 2024-09-12T03:41:48.206454+0000) 2024-09-12T03:42:05.354 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:05 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:05.083+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.408037+0000 front 2024-09-12T03:41:33.408013+0000 (oldest deadline 2024-09-12T03:41:56.308104+0000) 2024-09-12T03:42:05.392 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:05 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:05.021+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.424330+0000 front 2024-09-12T03:41:33.424271+0000 (oldest deadline 2024-09-12T03:41:55.124418+0000) 2024-09-12T03:42:05.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:04 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:04.986+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:05.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:04 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:04.986+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:05.522 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:05 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:05.208+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:23.172158+0000 front 2024-09-12T03:41:23.172197+0000 (oldest deadline 2024-09-12T03:41:48.973206+0000) 2024-09-12T03:42:05.522 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:05 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:05.208+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.375123+0000 front 2024-09-12T03:41:32.375107+0000 (oldest deadline 2024-09-12T03:41:55.874935+0000) 2024-09-12T03:42:05.522 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:05 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:05.208+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.575971+0000 front 2024-09-12T03:41:40.576192+0000 (oldest deadline 2024-09-12T03:42:02.876105+0000) 2024-09-12T03:42:05.706 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:42:05 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5[46203]: 2024-09-12T03:42:05.534+0000 7f6c5ea16640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:25.510799+0000 front 2024-09-12T03:41:25.510774+0000 (oldest deadline 2024-09-12T03:41:48.311345+0000) 2024-09-12T03:42:05.706 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:42:05 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5[46203]: 2024-09-12T03:42:05.534+0000 7f6c5ea16640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:31.212365+0000 front 2024-09-12T03:41:31.212522+0000 (oldest deadline 2024-09-12T03:41:54.712467+0000) 2024-09-12T03:42:05.706 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:42:05 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5[46203]: 2024-09-12T03:42:05.534+0000 7f6c5ea16640 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.013314+0000 front 2024-09-12T03:41:40.013364+0000 (oldest deadline 2024-09-12T03:42:02.913474+0000) 2024-09-12T03:42:05.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:05 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:05.354+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:05.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:05 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:05.354+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:05.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:05 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:05.354+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:06.207 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:05 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:05.790+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.723241+0000 front 2024-09-12T03:41:32.723103+0000 (oldest deadline 2024-09-12T03:41:56.822981+0000) 2024-09-12T03:42:06.208 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:05 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:05.790+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.224382+0000 front 2024-09-12T03:41:40.224550+0000 (oldest deadline 2024-09-12T03:42:02.524263+0000) 2024-09-12T03:42:06.351 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:42:06 smithi195 podman[72007]: 2024-09-12 03:42:06.281101368 +0000 UTC m=+5.531792392 container died 653991c615926255e0d7753301b5aa2fb133ab08ec1c0760e955d28ad1751e27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True) 2024-09-12T03:42:06.351 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:06 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:06.097+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.706298+0000 front 2024-09-12T03:41:24.706335+0000 (oldest deadline 2024-09-12T03:41:48.206454+0000) 2024-09-12T03:42:06.351 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:06 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:06.097+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.408037+0000 front 2024-09-12T03:41:33.408013+0000 (oldest deadline 2024-09-12T03:41:56.308104+0000) 2024-09-12T03:42:06.351 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:06 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:06.097+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:41.509651+0000 front 2024-09-12T03:41:41.509715+0000 (oldest deadline 2024-09-12T03:42:05.609794+0000) 2024-09-12T03:42:06.392 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:06 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:06.023+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.424330+0000 front 2024-09-12T03:41:33.424271+0000 (oldest deadline 2024-09-12T03:41:55.124418+0000) 2024-09-12T03:42:06.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:05 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:05.981+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:06.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:05 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:05.981+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:06.522 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:06 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:06.207+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:23.172158+0000 front 2024-09-12T03:41:23.172197+0000 (oldest deadline 2024-09-12T03:41:48.973206+0000) 2024-09-12T03:42:06.522 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:06 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:06.207+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.375123+0000 front 2024-09-12T03:41:32.375107+0000 (oldest deadline 2024-09-12T03:41:55.874935+0000) 2024-09-12T03:42:06.522 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:06 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:06.207+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.575971+0000 front 2024-09-12T03:41:40.576192+0000 (oldest deadline 2024-09-12T03:42:02.876105+0000) 2024-09-12T03:42:06.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:06 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:06.351+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:06.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:06 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:06.351+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:06.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:06 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:06.351+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:07.088 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:42:06 smithi195 podman[72007]: 2024-09-12 03:42:06.708942169 +0000 UTC m=+5.959633183 container cleanup 653991c615926255e0d7753301b5aa2fb133ab08ec1c0760e955d28ad1751e27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-12T03:42:07.088 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:42:06 smithi195 bash[72007]: ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5 2024-09-12T03:42:07.088 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:42:06 smithi195 podman[72024]: 2024-09-12 03:42:06.886347463 +0000 UTC m=+0.593632320 container remove 653991c615926255e0d7753301b5aa2fb133ab08ec1c0760e955d28ad1751e27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-12T03:42:07.089 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:07 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:07.088+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.706298+0000 front 2024-09-12T03:41:24.706335+0000 (oldest deadline 2024-09-12T03:41:48.206454+0000) 2024-09-12T03:42:07.089 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:07 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:07.088+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.408037+0000 front 2024-09-12T03:41:33.408013+0000 (oldest deadline 2024-09-12T03:41:56.308104+0000) 2024-09-12T03:42:07.089 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:07 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:07.088+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:41.509651+0000 front 2024-09-12T03:41:41.509715+0000 (oldest deadline 2024-09-12T03:42:05.609794+0000) 2024-09-12T03:42:07.193 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:06 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:06.775+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.723241+0000 front 2024-09-12T03:41:32.723103+0000 (oldest deadline 2024-09-12T03:41:56.822981+0000) 2024-09-12T03:42:07.194 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:06 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:06.775+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.224382+0000 front 2024-09-12T03:41:40.224550+0000 (oldest deadline 2024-09-12T03:42:02.524263+0000) 2024-09-12T03:42:07.345 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:42:07 smithi195 podman[72109]: 2024-09-12 03:42:07.221987505 +0000 UTC m=+0.016260343 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:42:07.345 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:42:07 smithi195 podman[72109]: 2024-09-12 03:42:07.345772167 +0000 UTC m=+0.140045032 container create 53f3dbca2815a92eb2a0f3d455c12a1c7e562878cb871167ff7213cedc8805b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3) 2024-09-12T03:42:07.392 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:07 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:07.033+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.424330+0000 front 2024-09-12T03:41:33.424271+0000 (oldest deadline 2024-09-12T03:41:55.124418+0000) 2024-09-12T03:42:07.393 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:07 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:07.033+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:41.025630+0000 front 2024-09-12T03:41:41.025614+0000 (oldest deadline 2024-09-12T03:42:06.925384+0000) 2024-09-12T03:42:07.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:06 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:06.983+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:07.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:06 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:06.983+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:07.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:06 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:06.983+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.525612+0000 front 2024-09-12T03:41:40.525773+0000 (oldest deadline 2024-09-12T03:42:06.425539+0000) 2024-09-12T03:42:07.522 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:07 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:07.194+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:23.172158+0000 front 2024-09-12T03:41:23.172197+0000 (oldest deadline 2024-09-12T03:41:48.973206+0000) 2024-09-12T03:42:07.522 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:07 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:07.194+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.375123+0000 front 2024-09-12T03:41:32.375107+0000 (oldest deadline 2024-09-12T03:41:55.874935+0000) 2024-09-12T03:42:07.522 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:07 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:07.194+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.575971+0000 front 2024-09-12T03:41:40.576192+0000 (oldest deadline 2024-09-12T03:42:02.876105+0000) 2024-09-12T03:42:07.597 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:07 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:07.370+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:07.597 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:07 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:07.370+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:07.597 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:07 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:07.370+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:07.956 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:42:07 smithi195 podman[72109]: 2024-09-12 03:42:07.598339346 +0000 UTC m=+0.392612172 container init 53f3dbca2815a92eb2a0f3d455c12a1c7e562878cb871167ff7213cedc8805b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-12T03:42:07.956 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:42:07 smithi195 podman[72109]: 2024-09-12 03:42:07.601962861 +0000 UTC m=+0.396235686 container start 53f3dbca2815a92eb2a0f3d455c12a1c7e562878cb871167ff7213cedc8805b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5-deactivate, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-12T03:42:07.956 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:42:07 smithi195 podman[72109]: 2024-09-12 03:42:07.681757753 +0000 UTC m=+0.476030594 container attach 53f3dbca2815a92eb2a0f3d455c12a1c7e562878cb871167ff7213cedc8805b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-12T03:42:07.956 INFO:journalctl@ceph.osd.5.smithi195.stdout:Sep 12 03:42:07 smithi195 podman[72109]: 2024-09-12 03:42:07.79104544 +0000 UTC m=+0.585318273 container died 53f3dbca2815a92eb2a0f3d455c12a1c7e562878cb871167ff7213cedc8805b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-12T03:42:08.212 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:07 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:07.792+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.723241+0000 front 2024-09-12T03:41:32.723103+0000 (oldest deadline 2024-09-12T03:41:56.822981+0000) 2024-09-12T03:42:08.212 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:07 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:07.792+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.224382+0000 front 2024-09-12T03:41:40.224550+0000 (oldest deadline 2024-09-12T03:42:02.524263+0000) 2024-09-12T03:42:08.285 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:08 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:08.056+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.706298+0000 front 2024-09-12T03:41:24.706335+0000 (oldest deadline 2024-09-12T03:41:48.206454+0000) 2024-09-12T03:42:08.285 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:08 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:08.056+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.408037+0000 front 2024-09-12T03:41:33.408013+0000 (oldest deadline 2024-09-12T03:41:56.308104+0000) 2024-09-12T03:42:08.285 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:08 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:08.056+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:41.509651+0000 front 2024-09-12T03:41:41.509715+0000 (oldest deadline 2024-09-12T03:42:05.609794+0000) 2024-09-12T03:42:08.315 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@osd.5.service' 2024-09-12T03:42:08.356 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:42:08.356 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-12T03:42:08.356 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-12T03:42:08.357 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-b0812460-70b5-11ef-bcea-c7b262605968@osd.6 2024-09-12T03:42:08.392 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:08 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:08.005+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.424330+0000 front 2024-09-12T03:41:33.424271+0000 (oldest deadline 2024-09-12T03:41:55.124418+0000) 2024-09-12T03:42:08.393 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:08 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:08.005+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:41.025630+0000 front 2024-09-12T03:41:41.025614+0000 (oldest deadline 2024-09-12T03:42:06.925384+0000) 2024-09-12T03:42:08.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:07 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:07.960+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:08.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:07 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:07.960+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:08.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:07 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:07.960+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.525612+0000 front 2024-09-12T03:41:40.525773+0000 (oldest deadline 2024-09-12T03:42:06.425539+0000) 2024-09-12T03:42:08.522 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:08 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:08.212+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:23.172158+0000 front 2024-09-12T03:41:23.172197+0000 (oldest deadline 2024-09-12T03:41:48.973206+0000) 2024-09-12T03:42:08.522 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:08 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:08.212+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.375123+0000 front 2024-09-12T03:41:32.375107+0000 (oldest deadline 2024-09-12T03:41:55.874935+0000) 2024-09-12T03:42:08.522 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:08 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:08.212+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.575971+0000 front 2024-09-12T03:41:40.576192+0000 (oldest deadline 2024-09-12T03:42:02.876105+0000) 2024-09-12T03:42:08.522 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:08 smithi064 systemd[1]: Stopping Ceph osd.6 for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:42:08.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:08 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:08.402+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:08.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:08 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:08.402+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:08.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:08 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:08.402+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:09.021 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:08 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:08.655+0000 7f40c73fc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:42:09.022 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:08 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:08.655+0000 7f40c73fc640 -1 osd.6 107 *** Got signal Terminated *** 2024-09-12T03:42:09.022 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:08 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:08.655+0000 7f40c73fc640 -1 osd.6 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:42:09.022 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:08 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:08.743+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.723241+0000 front 2024-09-12T03:41:32.723103+0000 (oldest deadline 2024-09-12T03:41:56.822981+0000) 2024-09-12T03:42:09.022 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:08 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:08.743+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.224382+0000 front 2024-09-12T03:41:40.224550+0000 (oldest deadline 2024-09-12T03:42:02.524263+0000) 2024-09-12T03:42:09.392 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:08 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:08.994+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.424330+0000 front 2024-09-12T03:41:33.424271+0000 (oldest deadline 2024-09-12T03:41:55.124418+0000) 2024-09-12T03:42:09.393 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:08 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:08.994+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:41.025630+0000 front 2024-09-12T03:41:41.025614+0000 (oldest deadline 2024-09-12T03:42:06.925384+0000) 2024-09-12T03:42:09.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:08 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:08.974+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:09.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:08 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:08.974+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:09.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:08 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:08.974+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.525612+0000 front 2024-09-12T03:41:40.525773+0000 (oldest deadline 2024-09-12T03:42:06.425539+0000) 2024-09-12T03:42:09.417 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:09 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:09.019+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.706298+0000 front 2024-09-12T03:41:24.706335+0000 (oldest deadline 2024-09-12T03:41:48.206454+0000) 2024-09-12T03:42:09.417 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:09 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:09.019+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.408037+0000 front 2024-09-12T03:41:33.408013+0000 (oldest deadline 2024-09-12T03:41:56.308104+0000) 2024-09-12T03:42:09.417 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:09 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:09.019+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:41.509651+0000 front 2024-09-12T03:41:41.509715+0000 (oldest deadline 2024-09-12T03:42:05.609794+0000) 2024-09-12T03:42:09.521 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:09 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:09.237+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:23.172158+0000 front 2024-09-12T03:41:23.172197+0000 (oldest deadline 2024-09-12T03:41:48.973206+0000) 2024-09-12T03:42:09.522 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:09 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:09.237+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.375123+0000 front 2024-09-12T03:41:32.375107+0000 (oldest deadline 2024-09-12T03:41:55.874935+0000) 2024-09-12T03:42:09.524 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:09 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:09.237+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.575971+0000 front 2024-09-12T03:41:40.576192+0000 (oldest deadline 2024-09-12T03:42:02.876105+0000) 2024-09-12T03:42:09.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:09 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:09.416+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:09.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:09 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:09.416+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:09.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:09 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:09.416+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:10.271 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:09 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:09.774+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.723241+0000 front 2024-09-12T03:41:32.723103+0000 (oldest deadline 2024-09-12T03:41:56.822981+0000) 2024-09-12T03:42:10.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:09 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:09.774+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.224382+0000 front 2024-09-12T03:41:40.224550+0000 (oldest deadline 2024-09-12T03:42:02.524263+0000) 2024-09-12T03:42:10.392 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:09 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:09.962+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.424330+0000 front 2024-09-12T03:41:33.424271+0000 (oldest deadline 2024-09-12T03:41:55.124418+0000) 2024-09-12T03:42:10.392 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:09 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:09.962+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:41.025630+0000 front 2024-09-12T03:41:41.025614+0000 (oldest deadline 2024-09-12T03:42:06.925384+0000) 2024-09-12T03:42:10.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:10 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:10.008+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:10.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:10 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:10.008+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:10.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:10 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:10.008+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.525612+0000 front 2024-09-12T03:41:40.525773+0000 (oldest deadline 2024-09-12T03:42:06.425539+0000) 2024-09-12T03:42:10.455 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:10 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:10.063+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.706298+0000 front 2024-09-12T03:41:24.706335+0000 (oldest deadline 2024-09-12T03:41:48.206454+0000) 2024-09-12T03:42:10.455 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:10 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:10.063+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.408037+0000 front 2024-09-12T03:41:33.408013+0000 (oldest deadline 2024-09-12T03:41:56.308104+0000) 2024-09-12T03:42:10.455 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:10 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:10.063+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:41.509651+0000 front 2024-09-12T03:41:41.509715+0000 (oldest deadline 2024-09-12T03:42:05.609794+0000) 2024-09-12T03:42:10.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:10 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:10.455+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:10.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:10 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:10.455+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:10.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:10 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:10.455+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:10.760 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:10 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:10.274+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:23.172158+0000 front 2024-09-12T03:41:23.172197+0000 (oldest deadline 2024-09-12T03:41:48.973206+0000) 2024-09-12T03:42:10.760 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:10 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:10.274+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.375123+0000 front 2024-09-12T03:41:32.375107+0000 (oldest deadline 2024-09-12T03:41:55.874935+0000) 2024-09-12T03:42:10.760 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:10 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:10.274+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.575971+0000 front 2024-09-12T03:41:40.576192+0000 (oldest deadline 2024-09-12T03:42:02.876105+0000) 2024-09-12T03:42:11.021 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:10 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:10.760+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.723241+0000 front 2024-09-12T03:41:32.723103+0000 (oldest deadline 2024-09-12T03:41:56.822981+0000) 2024-09-12T03:42:11.022 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:10 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:10.760+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.224382+0000 front 2024-09-12T03:41:40.224550+0000 (oldest deadline 2024-09-12T03:42:02.524263+0000) 2024-09-12T03:42:11.392 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:11 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:11.027+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:11.392 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:11 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:11.027+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:11.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:11 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:11.027+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.525612+0000 front 2024-09-12T03:41:40.525773+0000 (oldest deadline 2024-09-12T03:42:06.425539+0000) 2024-09-12T03:42:11.393 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:10 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:10.978+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.424330+0000 front 2024-09-12T03:41:33.424271+0000 (oldest deadline 2024-09-12T03:41:55.124418+0000) 2024-09-12T03:42:11.393 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:10 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:10.978+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:41.025630+0000 front 2024-09-12T03:41:41.025614+0000 (oldest deadline 2024-09-12T03:42:06.925384+0000) 2024-09-12T03:42:11.409 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:11 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:11.017+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.706298+0000 front 2024-09-12T03:41:24.706335+0000 (oldest deadline 2024-09-12T03:41:48.206454+0000) 2024-09-12T03:42:11.409 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:11 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:11.017+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.408037+0000 front 2024-09-12T03:41:33.408013+0000 (oldest deadline 2024-09-12T03:41:56.308104+0000) 2024-09-12T03:42:11.410 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:11 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:11.017+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:41.509651+0000 front 2024-09-12T03:41:41.509715+0000 (oldest deadline 2024-09-12T03:42:05.609794+0000) 2024-09-12T03:42:11.521 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:11 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:11.234+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:23.172158+0000 front 2024-09-12T03:41:23.172197+0000 (oldest deadline 2024-09-12T03:41:48.973206+0000) 2024-09-12T03:42:11.522 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:11 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:11.234+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.375123+0000 front 2024-09-12T03:41:32.375107+0000 (oldest deadline 2024-09-12T03:41:55.874935+0000) 2024-09-12T03:42:11.522 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:11 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:11.234+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.575971+0000 front 2024-09-12T03:41:40.576192+0000 (oldest deadline 2024-09-12T03:42:02.876105+0000) 2024-09-12T03:42:11.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:11 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:11.409+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:11.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:11 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:11.409+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:11.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:11 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:11.409+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:12.021 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:11 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:11.764+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.723241+0000 front 2024-09-12T03:41:32.723103+0000 (oldest deadline 2024-09-12T03:41:56.822981+0000) 2024-09-12T03:42:12.022 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:11 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:11.764+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.224382+0000 front 2024-09-12T03:41:40.224550+0000 (oldest deadline 2024-09-12T03:42:02.524263+0000) 2024-09-12T03:42:12.368 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:12 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:12.053+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.706298+0000 front 2024-09-12T03:41:24.706335+0000 (oldest deadline 2024-09-12T03:41:48.206454+0000) 2024-09-12T03:42:12.369 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:12 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:12.053+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.408037+0000 front 2024-09-12T03:41:33.408013+0000 (oldest deadline 2024-09-12T03:41:56.308104+0000) 2024-09-12T03:42:12.369 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:12 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:12.053+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:41.509651+0000 front 2024-09-12T03:41:41.509715+0000 (oldest deadline 2024-09-12T03:42:05.609794+0000) 2024-09-12T03:42:12.392 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:12 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:12.014+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.424330+0000 front 2024-09-12T03:41:33.424271+0000 (oldest deadline 2024-09-12T03:41:55.124418+0000) 2024-09-12T03:42:12.393 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:12 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:12.014+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:41.025630+0000 front 2024-09-12T03:41:41.025614+0000 (oldest deadline 2024-09-12T03:42:06.925384+0000) 2024-09-12T03:42:12.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:12 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:12.004+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:12.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:12 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:12.004+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:12.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:12 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:12.004+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.525612+0000 front 2024-09-12T03:41:40.525773+0000 (oldest deadline 2024-09-12T03:42:06.425539+0000) 2024-09-12T03:42:12.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:12 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:12.004+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:46.425933+0000 front 2024-09-12T03:41:46.426031+0000 (oldest deadline 2024-09-12T03:42:11.725939+0000) 2024-09-12T03:42:12.522 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:12 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:12.235+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:23.172158+0000 front 2024-09-12T03:41:23.172197+0000 (oldest deadline 2024-09-12T03:41:48.973206+0000) 2024-09-12T03:42:12.522 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:12 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:12.235+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.375123+0000 front 2024-09-12T03:41:32.375107+0000 (oldest deadline 2024-09-12T03:41:55.874935+0000) 2024-09-12T03:42:12.522 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:12 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:12.235+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.575971+0000 front 2024-09-12T03:41:40.576192+0000 (oldest deadline 2024-09-12T03:42:02.876105+0000) 2024-09-12T03:42:12.522 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:12 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:12.235+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:48.578045+0000 front 2024-09-12T03:41:48.577902+0000 (oldest deadline 2024-09-12T03:42:12.077940+0000) 2024-09-12T03:42:12.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:12 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:12.369+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:12.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:12 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:12.369+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:12.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:12 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:12.369+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:13.022 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:12 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:12.763+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.723241+0000 front 2024-09-12T03:41:32.723103+0000 (oldest deadline 2024-09-12T03:41:56.822981+0000) 2024-09-12T03:42:13.022 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:12 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:12.763+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.224382+0000 front 2024-09-12T03:41:40.224550+0000 (oldest deadline 2024-09-12T03:42:02.524263+0000) 2024-09-12T03:42:13.022 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:12 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:12.763+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:47.825286+0000 front 2024-09-12T03:41:47.825215+0000 (oldest deadline 2024-09-12T03:42:12.525423+0000) 2024-09-12T03:42:13.356 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:13 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:13.064+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.706298+0000 front 2024-09-12T03:41:24.706335+0000 (oldest deadline 2024-09-12T03:41:48.206454+0000) 2024-09-12T03:42:13.357 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:13 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:13.064+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.408037+0000 front 2024-09-12T03:41:33.408013+0000 (oldest deadline 2024-09-12T03:41:56.308104+0000) 2024-09-12T03:42:13.357 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:13 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:13.064+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:41.509651+0000 front 2024-09-12T03:41:41.509715+0000 (oldest deadline 2024-09-12T03:42:05.609794+0000) 2024-09-12T03:42:13.357 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:13 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:13.064+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:48.712724+0000 front 2024-09-12T03:41:48.712828+0000 (oldest deadline 2024-09-12T03:42:12.212682+0000) 2024-09-12T03:42:13.392 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:13 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:13.032+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:13.392 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:13 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:13.032+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:13.392 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:13 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:13.032+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.525612+0000 front 2024-09-12T03:41:40.525773+0000 (oldest deadline 2024-09-12T03:42:06.425539+0000) 2024-09-12T03:42:13.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:13 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:13.032+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:46.425933+0000 front 2024-09-12T03:41:46.426031+0000 (oldest deadline 2024-09-12T03:42:11.725939+0000) 2024-09-12T03:42:13.393 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:13 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:13.018+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.424330+0000 front 2024-09-12T03:41:33.424271+0000 (oldest deadline 2024-09-12T03:41:55.124418+0000) 2024-09-12T03:42:13.393 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:13 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:13.018+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:41.025630+0000 front 2024-09-12T03:41:41.025614+0000 (oldest deadline 2024-09-12T03:42:06.925384+0000) 2024-09-12T03:42:13.393 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:13 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:13.018+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:48.026179+0000 front 2024-09-12T03:41:48.026274+0000 (oldest deadline 2024-09-12T03:42:12.126214+0000) 2024-09-12T03:42:13.521 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:13 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:13.202+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:23.172158+0000 front 2024-09-12T03:41:23.172197+0000 (oldest deadline 2024-09-12T03:41:48.973206+0000) 2024-09-12T03:42:13.522 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:13 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:13.202+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.375123+0000 front 2024-09-12T03:41:32.375107+0000 (oldest deadline 2024-09-12T03:41:55.874935+0000) 2024-09-12T03:42:13.522 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:13 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:13.202+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.575971+0000 front 2024-09-12T03:41:40.576192+0000 (oldest deadline 2024-09-12T03:42:02.876105+0000) 2024-09-12T03:42:13.522 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:13 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6[58343]: 2024-09-12T03:42:13.202+0000 7f40bfa0a640 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:48.578045+0000 front 2024-09-12T03:41:48.577902+0000 (oldest deadline 2024-09-12T03:42:12.077940+0000) 2024-09-12T03:42:13.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:13 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:13.357+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:13.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:13 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:13.357+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:13.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:13 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:13.357+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:14.272 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:13 smithi064 podman[94478]: 2024-09-12 03:42:13.997474798 +0000 UTC m=+5.440900676 container died cc985890eb3eddb4b9ec63b20069161bebd913a043f253b61c3149f3042ccf18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-12T03:42:14.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:13 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:13.776+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.723241+0000 front 2024-09-12T03:41:32.723103+0000 (oldest deadline 2024-09-12T03:41:56.822981+0000) 2024-09-12T03:42:14.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:13 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:13.776+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.224382+0000 front 2024-09-12T03:41:40.224550+0000 (oldest deadline 2024-09-12T03:42:02.524263+0000) 2024-09-12T03:42:14.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:13 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:13.776+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:47.825286+0000 front 2024-09-12T03:41:47.825215+0000 (oldest deadline 2024-09-12T03:42:12.525423+0000) 2024-09-12T03:42:14.356 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:14 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:14.050+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.706298+0000 front 2024-09-12T03:41:24.706335+0000 (oldest deadline 2024-09-12T03:41:48.206454+0000) 2024-09-12T03:42:14.356 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:14 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:14.050+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.408037+0000 front 2024-09-12T03:41:33.408013+0000 (oldest deadline 2024-09-12T03:41:56.308104+0000) 2024-09-12T03:42:14.356 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:14 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:14.050+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:41.509651+0000 front 2024-09-12T03:41:41.509715+0000 (oldest deadline 2024-09-12T03:42:05.609794+0000) 2024-09-12T03:42:14.356 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:14 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:14.050+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:48.712724+0000 front 2024-09-12T03:41:48.712828+0000 (oldest deadline 2024-09-12T03:42:12.212682+0000) 2024-09-12T03:42:14.393 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:14 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:14.023+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.424330+0000 front 2024-09-12T03:41:33.424271+0000 (oldest deadline 2024-09-12T03:41:55.124418+0000) 2024-09-12T03:42:14.393 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:14 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:14.023+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:41.025630+0000 front 2024-09-12T03:41:41.025614+0000 (oldest deadline 2024-09-12T03:42:06.925384+0000) 2024-09-12T03:42:14.393 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:14 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:14.023+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:48.026179+0000 front 2024-09-12T03:41:48.026274+0000 (oldest deadline 2024-09-12T03:42:12.126214+0000) 2024-09-12T03:42:14.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:14 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:14.056+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:14.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:14 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:14.056+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:14.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:14 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:14.056+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.525612+0000 front 2024-09-12T03:41:40.525773+0000 (oldest deadline 2024-09-12T03:42:06.425539+0000) 2024-09-12T03:42:14.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:14 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:14.056+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:46.425933+0000 front 2024-09-12T03:41:46.426031+0000 (oldest deadline 2024-09-12T03:42:11.725939+0000) 2024-09-12T03:42:14.607 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:14 smithi064 podman[94478]: 2024-09-12 03:42:14.497264797 +0000 UTC m=+5.940690664 container cleanup cc985890eb3eddb4b9ec63b20069161bebd913a043f253b61c3149f3042ccf18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:42:14.608 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:14 smithi064 bash[94478]: ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6 2024-09-12T03:42:14.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:14 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:14.356+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:14.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:14 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:14.356+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:14.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:14 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:14.356+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:15.021 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:14 smithi064 podman[94494]: 2024-09-12 03:42:14.773992565 +0000 UTC m=+0.775244506 container remove cc985890eb3eddb4b9ec63b20069161bebd913a043f253b61c3149f3042ccf18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:42:15.022 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:14 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:14.820+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.723241+0000 front 2024-09-12T03:41:32.723103+0000 (oldest deadline 2024-09-12T03:41:56.822981+0000) 2024-09-12T03:42:15.022 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:14 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:14.820+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.224382+0000 front 2024-09-12T03:41:40.224550+0000 (oldest deadline 2024-09-12T03:42:02.524263+0000) 2024-09-12T03:42:15.022 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:14 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:14.820+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:47.825286+0000 front 2024-09-12T03:41:47.825215+0000 (oldest deadline 2024-09-12T03:42:12.525423+0000) 2024-09-12T03:42:15.372 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:15 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:15.085+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.706298+0000 front 2024-09-12T03:41:24.706335+0000 (oldest deadline 2024-09-12T03:41:48.206454+0000) 2024-09-12T03:42:15.372 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:15 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:15.085+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.408037+0000 front 2024-09-12T03:41:33.408013+0000 (oldest deadline 2024-09-12T03:41:56.308104+0000) 2024-09-12T03:42:15.372 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:15 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:15.085+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:41.509651+0000 front 2024-09-12T03:41:41.509715+0000 (oldest deadline 2024-09-12T03:42:05.609794+0000) 2024-09-12T03:42:15.372 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:15 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:15.085+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:48.712724+0000 front 2024-09-12T03:41:48.712828+0000 (oldest deadline 2024-09-12T03:42:12.212682+0000) 2024-09-12T03:42:15.392 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:15 smithi064 podman[94590]: 2024-09-12 03:42:15.044450887 +0000 UTC m=+0.017606133 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:42:15.392 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:15 smithi064 podman[94590]: 2024-09-12 03:42:15.277482926 +0000 UTC m=+0.250638170 container create fbfc965d2a179f256f70cc52e770c2220d9949ba119f49d8c5b5f18c0ba7daa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD) 2024-09-12T03:42:15.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:15 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:15.084+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:15.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:15 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:15.084+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:15.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:15 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:15.084+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.525612+0000 front 2024-09-12T03:41:40.525773+0000 (oldest deadline 2024-09-12T03:42:06.425539+0000) 2024-09-12T03:42:15.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:15 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:15.084+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:46.425933+0000 front 2024-09-12T03:41:46.426031+0000 (oldest deadline 2024-09-12T03:42:11.725939+0000) 2024-09-12T03:42:15.394 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:15 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:15.054+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.424330+0000 front 2024-09-12T03:41:33.424271+0000 (oldest deadline 2024-09-12T03:41:55.124418+0000) 2024-09-12T03:42:15.394 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:15 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:15.054+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:41.025630+0000 front 2024-09-12T03:41:41.025614+0000 (oldest deadline 2024-09-12T03:42:06.925384+0000) 2024-09-12T03:42:15.394 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:15 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:15.054+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:48.026179+0000 front 2024-09-12T03:41:48.026274+0000 (oldest deadline 2024-09-12T03:42:12.126214+0000) 2024-09-12T03:42:15.644 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:15 smithi064 podman[94590]: 2024-09-12 03:42:15.527668658 +0000 UTC m=+0.500823906 container init fbfc965d2a179f256f70cc52e770c2220d9949ba119f49d8c5b5f18c0ba7daa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-12T03:42:15.644 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:15 smithi064 podman[94590]: 2024-09-12 03:42:15.531398359 +0000 UTC m=+0.504553607 container start fbfc965d2a179f256f70cc52e770c2220d9949ba119f49d8c5b5f18c0ba7daa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-12T03:42:15.644 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:15 smithi064 podman[94590]: 2024-09-12 03:42:15.602700343 +0000 UTC m=+0.575855595 container attach fbfc965d2a179f256f70cc52e770c2220d9949ba119f49d8c5b5f18c0ba7daa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:42:15.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:15 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:15.371+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:15.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:15 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:15.371+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:15.707 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:15 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:15.371+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:16.013 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 12 03:42:15 smithi064 podman[94590]: 2024-09-12 03:42:15.730417098 +0000 UTC m=+0.703572352 container died fbfc965d2a179f256f70cc52e770c2220d9949ba119f49d8c5b5f18c0ba7daa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-b53b6d3) 2024-09-12T03:42:16.013 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:15 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:15.814+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.723241+0000 front 2024-09-12T03:41:32.723103+0000 (oldest deadline 2024-09-12T03:41:56.822981+0000) 2024-09-12T03:42:16.013 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:15 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:15.814+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.224382+0000 front 2024-09-12T03:41:40.224550+0000 (oldest deadline 2024-09-12T03:42:02.524263+0000) 2024-09-12T03:42:16.014 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:15 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:15.814+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:47.825286+0000 front 2024-09-12T03:41:47.825215+0000 (oldest deadline 2024-09-12T03:42:12.525423+0000) 2024-09-12T03:42:16.301 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@osd.6.service' 2024-09-12T03:42:16.342 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:42:16.343 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-12T03:42:16.343 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-12T03:42:16.343 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-b0812460-70b5-11ef-bcea-c7b262605968@osd.7 2024-09-12T03:42:16.370 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:16 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:16.006+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.424330+0000 front 2024-09-12T03:41:33.424271+0000 (oldest deadline 2024-09-12T03:41:55.124418+0000) 2024-09-12T03:42:16.370 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:16 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:16.006+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:41.025630+0000 front 2024-09-12T03:41:41.025614+0000 (oldest deadline 2024-09-12T03:42:06.925384+0000) 2024-09-12T03:42:16.370 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:16 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:16.006+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:48.026179+0000 front 2024-09-12T03:41:48.026274+0000 (oldest deadline 2024-09-12T03:42:12.126214+0000) 2024-09-12T03:42:16.371 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:16 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:16.077+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:16.371 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:16 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:16.077+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:16.371 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:16 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:16.077+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.525612+0000 front 2024-09-12T03:41:40.525773+0000 (oldest deadline 2024-09-12T03:42:06.425539+0000) 2024-09-12T03:42:16.371 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:16 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:16.077+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:46.425933+0000 front 2024-09-12T03:41:46.426031+0000 (oldest deadline 2024-09-12T03:42:11.725939+0000) 2024-09-12T03:42:16.394 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:16 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:16.112+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.706298+0000 front 2024-09-12T03:41:24.706335+0000 (oldest deadline 2024-09-12T03:41:48.206454+0000) 2024-09-12T03:42:16.394 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:16 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:16.112+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.408037+0000 front 2024-09-12T03:41:33.408013+0000 (oldest deadline 2024-09-12T03:41:56.308104+0000) 2024-09-12T03:42:16.394 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:16 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:16.112+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:41.509651+0000 front 2024-09-12T03:41:41.509715+0000 (oldest deadline 2024-09-12T03:42:05.609794+0000) 2024-09-12T03:42:16.394 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:16 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:16.112+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:48.712724+0000 front 2024-09-12T03:41:48.712828+0000 (oldest deadline 2024-09-12T03:42:12.212682+0000) 2024-09-12T03:42:16.643 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:16 smithi115 systemd[1]: Stopping Ceph osd.7 for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:42:16.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:16 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:16.394+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:16.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:16 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:16.394+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:16.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:16 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:16.394+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:17.008 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:16 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:16.669+0000 7f15a5384640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:42:17.008 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:16 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:16.669+0000 7f15a5384640 -1 osd.7 107 *** Got signal Terminated *** 2024-09-12T03:42:17.008 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:16 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:16.669+0000 7f15a5384640 -1 osd.7 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:42:17.008 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:17 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:17.007+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.424330+0000 front 2024-09-12T03:41:33.424271+0000 (oldest deadline 2024-09-12T03:41:55.124418+0000) 2024-09-12T03:42:17.008 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:17 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:17.007+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:41.025630+0000 front 2024-09-12T03:41:41.025614+0000 (oldest deadline 2024-09-12T03:42:06.925384+0000) 2024-09-12T03:42:17.008 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:17 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:17.007+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:48.026179+0000 front 2024-09-12T03:41:48.026274+0000 (oldest deadline 2024-09-12T03:42:12.126214+0000) 2024-09-12T03:42:17.271 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:16 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:16.819+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.723241+0000 front 2024-09-12T03:41:32.723103+0000 (oldest deadline 2024-09-12T03:41:56.822981+0000) 2024-09-12T03:42:17.271 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:16 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:16.819+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.224382+0000 front 2024-09-12T03:41:40.224550+0000 (oldest deadline 2024-09-12T03:42:02.524263+0000) 2024-09-12T03:42:17.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:16 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:16.819+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:47.825286+0000 front 2024-09-12T03:41:47.825215+0000 (oldest deadline 2024-09-12T03:42:12.525423+0000) 2024-09-12T03:42:17.392 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:17 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:17.094+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:17.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:17 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:17.094+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:17.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:17 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:17.094+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.525612+0000 front 2024-09-12T03:41:40.525773+0000 (oldest deadline 2024-09-12T03:42:06.425539+0000) 2024-09-12T03:42:17.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:17 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:17.094+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:46.425933+0000 front 2024-09-12T03:41:46.426031+0000 (oldest deadline 2024-09-12T03:42:11.725939+0000) 2024-09-12T03:42:17.440 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:17 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:17.148+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.706298+0000 front 2024-09-12T03:41:24.706335+0000 (oldest deadline 2024-09-12T03:41:48.206454+0000) 2024-09-12T03:42:17.441 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:17 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:17.148+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.408037+0000 front 2024-09-12T03:41:33.408013+0000 (oldest deadline 2024-09-12T03:41:56.308104+0000) 2024-09-12T03:42:17.441 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:17 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:17.148+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:41.509651+0000 front 2024-09-12T03:41:41.509715+0000 (oldest deadline 2024-09-12T03:42:05.609794+0000) 2024-09-12T03:42:17.441 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:17 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:17.148+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:48.712724+0000 front 2024-09-12T03:41:48.712828+0000 (oldest deadline 2024-09-12T03:42:12.212682+0000) 2024-09-12T03:42:17.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:17 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:17.440+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:17.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:17 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:17.440+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:17.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:17 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:17.440+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:18.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:17 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:17.803+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.723241+0000 front 2024-09-12T03:41:32.723103+0000 (oldest deadline 2024-09-12T03:41:56.822981+0000) 2024-09-12T03:42:18.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:17 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:17.803+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.224382+0000 front 2024-09-12T03:41:40.224550+0000 (oldest deadline 2024-09-12T03:42:02.524263+0000) 2024-09-12T03:42:18.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:17 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:17.803+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:47.825286+0000 front 2024-09-12T03:41:47.825215+0000 (oldest deadline 2024-09-12T03:42:12.525423+0000) 2024-09-12T03:42:18.392 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:18 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:18.011+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.424330+0000 front 2024-09-12T03:41:33.424271+0000 (oldest deadline 2024-09-12T03:41:55.124418+0000) 2024-09-12T03:42:18.393 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:18 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:18.011+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:41.025630+0000 front 2024-09-12T03:41:41.025614+0000 (oldest deadline 2024-09-12T03:42:06.925384+0000) 2024-09-12T03:42:18.393 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:18 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:18.011+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:48.026179+0000 front 2024-09-12T03:41:48.026274+0000 (oldest deadline 2024-09-12T03:42:12.126214+0000) 2024-09-12T03:42:18.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:18 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:18.080+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:18.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:18 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:18.080+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:18.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:18 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:18.080+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.525612+0000 front 2024-09-12T03:41:40.525773+0000 (oldest deadline 2024-09-12T03:42:06.425539+0000) 2024-09-12T03:42:18.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:18 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:18.080+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:46.425933+0000 front 2024-09-12T03:41:46.426031+0000 (oldest deadline 2024-09-12T03:42:11.725939+0000) 2024-09-12T03:42:18.410 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:18 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:18.113+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.706298+0000 front 2024-09-12T03:41:24.706335+0000 (oldest deadline 2024-09-12T03:41:48.206454+0000) 2024-09-12T03:42:18.410 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:18 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:18.113+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.408037+0000 front 2024-09-12T03:41:33.408013+0000 (oldest deadline 2024-09-12T03:41:56.308104+0000) 2024-09-12T03:42:18.410 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:18 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:18.113+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:41.509651+0000 front 2024-09-12T03:41:41.509715+0000 (oldest deadline 2024-09-12T03:42:05.609794+0000) 2024-09-12T03:42:18.410 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:18 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:18.113+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:48.712724+0000 front 2024-09-12T03:41:48.712828+0000 (oldest deadline 2024-09-12T03:42:12.212682+0000) 2024-09-12T03:42:18.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:18 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:18.409+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:18.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:18 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:18.409+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:18.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:18 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:18.409+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:19.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:18 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:18.804+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.723241+0000 front 2024-09-12T03:41:32.723103+0000 (oldest deadline 2024-09-12T03:41:56.822981+0000) 2024-09-12T03:42:19.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:18 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:18.804+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.224382+0000 front 2024-09-12T03:41:40.224550+0000 (oldest deadline 2024-09-12T03:42:02.524263+0000) 2024-09-12T03:42:19.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:18 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:18.804+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:47.825286+0000 front 2024-09-12T03:41:47.825215+0000 (oldest deadline 2024-09-12T03:42:12.525423+0000) 2024-09-12T03:42:19.362 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:19 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:19.074+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.706298+0000 front 2024-09-12T03:41:24.706335+0000 (oldest deadline 2024-09-12T03:41:48.206454+0000) 2024-09-12T03:42:19.362 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:19 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:19.074+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.408037+0000 front 2024-09-12T03:41:33.408013+0000 (oldest deadline 2024-09-12T03:41:56.308104+0000) 2024-09-12T03:42:19.363 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:19 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:19.074+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:41.509651+0000 front 2024-09-12T03:41:41.509715+0000 (oldest deadline 2024-09-12T03:42:05.609794+0000) 2024-09-12T03:42:19.363 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:19 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:19.074+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:48.712724+0000 front 2024-09-12T03:41:48.712828+0000 (oldest deadline 2024-09-12T03:42:12.212682+0000) 2024-09-12T03:42:19.393 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:18 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:18.982+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.424330+0000 front 2024-09-12T03:41:33.424271+0000 (oldest deadline 2024-09-12T03:41:55.124418+0000) 2024-09-12T03:42:19.393 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:18 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:18.982+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:41.025630+0000 front 2024-09-12T03:41:41.025614+0000 (oldest deadline 2024-09-12T03:42:06.925384+0000) 2024-09-12T03:42:19.393 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:18 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:18.982+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:48.026179+0000 front 2024-09-12T03:41:48.026274+0000 (oldest deadline 2024-09-12T03:42:12.126214+0000) 2024-09-12T03:42:19.393 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:18 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:18.982+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:54.427014+0000 front 2024-09-12T03:41:54.427030+0000 (oldest deadline 2024-09-12T03:42:18.527186+0000) 2024-09-12T03:42:19.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:19 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:19.066+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:19.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:19 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:19.066+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:19.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:19 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:19.066+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.525612+0000 front 2024-09-12T03:41:40.525773+0000 (oldest deadline 2024-09-12T03:42:06.425539+0000) 2024-09-12T03:42:19.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:19 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:19.066+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:46.425933+0000 front 2024-09-12T03:41:46.426031+0000 (oldest deadline 2024-09-12T03:42:11.725939+0000) 2024-09-12T03:42:19.636 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:19 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:19.362+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:19.636 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:19 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:19.362+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:19.636 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:19 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:19.362+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:19.636 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:19 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:19.362+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:56.172333+0000 front 2024-09-12T03:41:56.172398+0000 (oldest deadline 2024-09-12T03:42:18.472329+0000) 2024-09-12T03:42:20.271 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:19 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:19.841+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.723241+0000 front 2024-09-12T03:41:32.723103+0000 (oldest deadline 2024-09-12T03:41:56.822981+0000) 2024-09-12T03:42:20.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:19 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:19.841+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.224382+0000 front 2024-09-12T03:41:40.224550+0000 (oldest deadline 2024-09-12T03:42:02.524263+0000) 2024-09-12T03:42:20.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:19 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:19.841+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:47.825286+0000 front 2024-09-12T03:41:47.825215+0000 (oldest deadline 2024-09-12T03:42:12.525423+0000) 2024-09-12T03:42:20.346 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:20 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:20.051+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.706298+0000 front 2024-09-12T03:41:24.706335+0000 (oldest deadline 2024-09-12T03:41:48.206454+0000) 2024-09-12T03:42:20.346 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:20 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:20.051+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.408037+0000 front 2024-09-12T03:41:33.408013+0000 (oldest deadline 2024-09-12T03:41:56.308104+0000) 2024-09-12T03:42:20.346 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:20 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:20.051+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:41.509651+0000 front 2024-09-12T03:41:41.509715+0000 (oldest deadline 2024-09-12T03:42:05.609794+0000) 2024-09-12T03:42:20.346 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:20 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:20.051+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:48.712724+0000 front 2024-09-12T03:41:48.712828+0000 (oldest deadline 2024-09-12T03:42:12.212682+0000) 2024-09-12T03:42:20.392 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:19 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:19.976+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.424330+0000 front 2024-09-12T03:41:33.424271+0000 (oldest deadline 2024-09-12T03:41:55.124418+0000) 2024-09-12T03:42:20.393 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:19 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:19.976+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:41.025630+0000 front 2024-09-12T03:41:41.025614+0000 (oldest deadline 2024-09-12T03:42:06.925384+0000) 2024-09-12T03:42:20.393 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:19 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:19.976+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:48.026179+0000 front 2024-09-12T03:41:48.026274+0000 (oldest deadline 2024-09-12T03:42:12.126214+0000) 2024-09-12T03:42:20.393 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:19 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:19.976+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:54.427014+0000 front 2024-09-12T03:41:54.427030+0000 (oldest deadline 2024-09-12T03:42:18.527186+0000) 2024-09-12T03:42:20.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:20 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:20.031+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:20.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:20 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:20.031+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:20.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:20 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:20.031+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.525612+0000 front 2024-09-12T03:41:40.525773+0000 (oldest deadline 2024-09-12T03:42:06.425539+0000) 2024-09-12T03:42:20.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:20 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:20.031+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:46.425933+0000 front 2024-09-12T03:41:46.426031+0000 (oldest deadline 2024-09-12T03:42:11.725939+0000) 2024-09-12T03:42:20.599 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:20 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:20.345+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:20.599 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:20 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:20.345+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:20.599 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:20 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:20.345+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:20.599 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:20 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:20.345+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:56.172333+0000 front 2024-09-12T03:41:56.172398+0000 (oldest deadline 2024-09-12T03:42:18.472329+0000) 2024-09-12T03:42:21.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:20 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:20.820+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.723241+0000 front 2024-09-12T03:41:32.723103+0000 (oldest deadline 2024-09-12T03:41:56.822981+0000) 2024-09-12T03:42:21.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:20 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:20.820+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.224382+0000 front 2024-09-12T03:41:40.224550+0000 (oldest deadline 2024-09-12T03:42:02.524263+0000) 2024-09-12T03:42:21.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:20 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:20.820+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:47.825286+0000 front 2024-09-12T03:41:47.825215+0000 (oldest deadline 2024-09-12T03:42:12.525423+0000) 2024-09-12T03:42:21.302 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:21 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:21.030+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.706298+0000 front 2024-09-12T03:41:24.706335+0000 (oldest deadline 2024-09-12T03:41:48.206454+0000) 2024-09-12T03:42:21.302 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:21 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:21.030+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.408037+0000 front 2024-09-12T03:41:33.408013+0000 (oldest deadline 2024-09-12T03:41:56.308104+0000) 2024-09-12T03:42:21.302 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:21 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:21.030+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:41.509651+0000 front 2024-09-12T03:41:41.509715+0000 (oldest deadline 2024-09-12T03:42:05.609794+0000) 2024-09-12T03:42:21.302 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:21 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:21.030+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:48.712724+0000 front 2024-09-12T03:41:48.712828+0000 (oldest deadline 2024-09-12T03:42:12.212682+0000) 2024-09-12T03:42:21.392 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:20 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:20.937+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.424330+0000 front 2024-09-12T03:41:33.424271+0000 (oldest deadline 2024-09-12T03:41:55.124418+0000) 2024-09-12T03:42:21.393 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:20 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:20.937+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:41.025630+0000 front 2024-09-12T03:41:41.025614+0000 (oldest deadline 2024-09-12T03:42:06.925384+0000) 2024-09-12T03:42:21.393 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:20 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:20.937+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:48.026179+0000 front 2024-09-12T03:41:48.026274+0000 (oldest deadline 2024-09-12T03:42:12.126214+0000) 2024-09-12T03:42:21.393 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:20 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7[52433]: 2024-09-12T03:42:20.937+0000 7f15a199a640 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:54.427014+0000 front 2024-09-12T03:41:54.427030+0000 (oldest deadline 2024-09-12T03:42:18.527186+0000) 2024-09-12T03:42:21.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:21 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:21.076+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:21.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:21 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:21.076+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:21.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:21 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:21.076+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.525612+0000 front 2024-09-12T03:41:40.525773+0000 (oldest deadline 2024-09-12T03:42:06.425539+0000) 2024-09-12T03:42:21.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:21 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:21.076+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:46.425933+0000 front 2024-09-12T03:41:46.426031+0000 (oldest deadline 2024-09-12T03:42:11.725939+0000) 2024-09-12T03:42:21.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:21 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:21.302+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:21.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:21 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:21.302+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:21.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:21 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:21.302+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:21.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:21 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:21.302+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:56.172333+0000 front 2024-09-12T03:41:56.172398+0000 (oldest deadline 2024-09-12T03:42:18.472329+0000) 2024-09-12T03:42:22.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:21 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:21.798+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.723241+0000 front 2024-09-12T03:41:32.723103+0000 (oldest deadline 2024-09-12T03:41:56.822981+0000) 2024-09-12T03:42:22.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:21 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:21.798+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.224382+0000 front 2024-09-12T03:41:40.224550+0000 (oldest deadline 2024-09-12T03:42:02.524263+0000) 2024-09-12T03:42:22.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:21 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:21.798+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:47.825286+0000 front 2024-09-12T03:41:47.825215+0000 (oldest deadline 2024-09-12T03:42:12.525423+0000) 2024-09-12T03:42:22.299 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:21 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:21.990+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.706298+0000 front 2024-09-12T03:41:24.706335+0000 (oldest deadline 2024-09-12T03:41:48.206454+0000) 2024-09-12T03:42:22.299 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:21 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:21.990+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.408037+0000 front 2024-09-12T03:41:33.408013+0000 (oldest deadline 2024-09-12T03:41:56.308104+0000) 2024-09-12T03:42:22.299 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:21 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:21.990+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:41.509651+0000 front 2024-09-12T03:41:41.509715+0000 (oldest deadline 2024-09-12T03:42:05.609794+0000) 2024-09-12T03:42:22.299 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:21 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:21.990+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:48.712724+0000 front 2024-09-12T03:41:48.712828+0000 (oldest deadline 2024-09-12T03:42:12.212682+0000) 2024-09-12T03:42:22.385 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:22 smithi115 podman[70458]: 2024-09-12 03:42:22.07989137 +0000 UTC m=+5.544432031 container died 827da419ab6789012629998dd5af40c9994b6f217542ac5e8e487bfa2fe49d8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-12T03:42:22.386 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:22 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:22.098+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:22.386 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:22 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:22.098+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:22.386 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:22 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:22.098+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.525612+0000 front 2024-09-12T03:41:40.525773+0000 (oldest deadline 2024-09-12T03:42:06.425539+0000) 2024-09-12T03:42:22.386 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:22 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:22.098+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:46.425933+0000 front 2024-09-12T03:41:46.426031+0000 (oldest deadline 2024-09-12T03:42:11.725939+0000) 2024-09-12T03:42:22.644 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:22 smithi115 podman[70458]: 2024-09-12 03:42:22.451735648 +0000 UTC m=+5.916276228 container cleanup 827da419ab6789012629998dd5af40c9994b6f217542ac5e8e487bfa2fe49d8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-12T03:42:22.644 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:22 smithi115 bash[70458]: ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7 2024-09-12T03:42:22.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:22 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:22.298+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:22.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:22 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:22.298+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:22.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:22 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:22.298+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:22.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:22 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:22.298+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:56.172333+0000 front 2024-09-12T03:41:56.172398+0000 (oldest deadline 2024-09-12T03:42:18.472329+0000) 2024-09-12T03:42:23.019 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:22 smithi115 podman[70473]: 2024-09-12 03:42:22.711127057 +0000 UTC m=+0.623668869 container remove 827da419ab6789012629998dd5af40c9994b6f217542ac5e8e487bfa2fe49d8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True) 2024-09-12T03:42:23.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:22 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:22.799+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.723241+0000 front 2024-09-12T03:41:32.723103+0000 (oldest deadline 2024-09-12T03:41:56.822981+0000) 2024-09-12T03:42:23.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:22 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:22.799+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.224382+0000 front 2024-09-12T03:41:40.224550+0000 (oldest deadline 2024-09-12T03:42:02.524263+0000) 2024-09-12T03:42:23.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:22 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:22.799+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:47.825286+0000 front 2024-09-12T03:41:47.825215+0000 (oldest deadline 2024-09-12T03:42:12.525423+0000) 2024-09-12T03:42:23.293 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:22 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:22.964+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.706298+0000 front 2024-09-12T03:41:24.706335+0000 (oldest deadline 2024-09-12T03:41:48.206454+0000) 2024-09-12T03:42:23.293 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:22 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:22.964+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.408037+0000 front 2024-09-12T03:41:33.408013+0000 (oldest deadline 2024-09-12T03:41:56.308104+0000) 2024-09-12T03:42:23.293 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:22 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:22.964+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:41.509651+0000 front 2024-09-12T03:41:41.509715+0000 (oldest deadline 2024-09-12T03:42:05.609794+0000) 2024-09-12T03:42:23.293 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:22 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:22.964+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:48.712724+0000 front 2024-09-12T03:41:48.712828+0000 (oldest deadline 2024-09-12T03:42:12.212682+0000) 2024-09-12T03:42:23.364 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:23 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:23.094+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:23.364 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:23 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:23.094+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:23.364 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:23 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:23.094+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.525612+0000 front 2024-09-12T03:41:40.525773+0000 (oldest deadline 2024-09-12T03:42:06.425539+0000) 2024-09-12T03:42:23.364 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:23 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:23.094+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:46.425933+0000 front 2024-09-12T03:41:46.426031+0000 (oldest deadline 2024-09-12T03:42:11.725939+0000) 2024-09-12T03:42:23.364 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:23 smithi115 podman[70564]: 2024-09-12 03:42:23.020302675 +0000 UTC m=+0.106331764 container create 9900a4baadae093c91be3084871ade9e950c80bbcb898597876ecc2b79e774f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-b53b6d3) 2024-09-12T03:42:23.364 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:23 smithi115 podman[70564]: 2024-09-12 03:42:22.930079228 +0000 UTC m=+0.016108282 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:42:23.364 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:23 smithi115 podman[70564]: 2024-09-12 03:42:23.264201657 +0000 UTC m=+0.350230713 container init 9900a4baadae093c91be3084871ade9e950c80bbcb898597876ecc2b79e774f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:42:23.364 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:23 smithi115 podman[70564]: 2024-09-12 03:42:23.267420719 +0000 UTC m=+0.353449773 container start 9900a4baadae093c91be3084871ade9e950c80bbcb898597876ecc2b79e774f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-12T03:42:23.365 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:23 smithi115 podman[70564]: 2024-09-12 03:42:23.364194077 +0000 UTC m=+0.450223137 container attach 9900a4baadae093c91be3084871ade9e950c80bbcb898597876ecc2b79e774f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:42:23.642 INFO:journalctl@ceph.osd.7.smithi115.stdout:Sep 12 03:42:23 smithi115 podman[70564]: 2024-09-12 03:42:23.459438012 +0000 UTC m=+0.545467068 container died 9900a4baadae093c91be3084871ade9e950c80bbcb898597876ecc2b79e774f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:42:23.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:23 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:23.293+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:23.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:23 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:23.293+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:23.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:23 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:23.293+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:23.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:23 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:23.293+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:56.172333+0000 front 2024-09-12T03:41:56.172398+0000 (oldest deadline 2024-09-12T03:42:18.472329+0000) 2024-09-12T03:42:24.101 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@osd.7.service' 2024-09-12T03:42:24.180 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:42:24.180 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-12T03:42:24.181 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-12T03:42:24.181 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-b0812460-70b5-11ef-bcea-c7b262605968@osd.8 2024-09-12T03:42:24.206 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:23 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:23.942+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.706298+0000 front 2024-09-12T03:41:24.706335+0000 (oldest deadline 2024-09-12T03:41:48.206454+0000) 2024-09-12T03:42:24.206 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:23 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:23.942+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.408037+0000 front 2024-09-12T03:41:33.408013+0000 (oldest deadline 2024-09-12T03:41:56.308104+0000) 2024-09-12T03:42:24.206 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:23 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:23.942+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:41.509651+0000 front 2024-09-12T03:41:41.509715+0000 (oldest deadline 2024-09-12T03:42:05.609794+0000) 2024-09-12T03:42:24.206 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:23 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:23.942+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:48.712724+0000 front 2024-09-12T03:41:48.712828+0000 (oldest deadline 2024-09-12T03:42:12.212682+0000) 2024-09-12T03:42:24.271 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:23 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:23.823+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.723241+0000 front 2024-09-12T03:41:32.723103+0000 (oldest deadline 2024-09-12T03:41:56.822981+0000) 2024-09-12T03:42:24.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:23 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:23.823+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.224382+0000 front 2024-09-12T03:41:40.224550+0000 (oldest deadline 2024-09-12T03:42:02.524263+0000) 2024-09-12T03:42:24.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:23 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:23.823+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:47.825286+0000 front 2024-09-12T03:41:47.825215+0000 (oldest deadline 2024-09-12T03:42:12.525423+0000) 2024-09-12T03:42:24.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:23 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:23.823+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:57.726913+0000 front 2024-09-12T03:41:57.727091+0000 (oldest deadline 2024-09-12T03:42:23.026695+0000) 2024-09-12T03:42:24.354 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:24 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:24.080+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:24.354 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:24 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:24.080+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:24.354 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:24 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:24.080+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.525612+0000 front 2024-09-12T03:41:40.525773+0000 (oldest deadline 2024-09-12T03:42:06.425539+0000) 2024-09-12T03:42:24.354 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:24 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:24.080+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:46.425933+0000 front 2024-09-12T03:41:46.426031+0000 (oldest deadline 2024-09-12T03:42:11.725939+0000) 2024-09-12T03:42:24.354 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:24 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:24.080+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:55.226745+0000 front 2024-09-12T03:41:55.226726+0000 (oldest deadline 2024-09-12T03:42:24.027136+0000) 2024-09-12T03:42:24.496 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:24 smithi195 systemd[1]: Stopping Ceph osd.8 for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:42:24.496 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:24 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:24.247+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:24.496 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:24 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:24.247+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:24.496 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:24 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:24.247+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:24.496 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:24 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:24.247+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:56.172333+0000 front 2024-09-12T03:41:56.172398+0000 (oldest deadline 2024-09-12T03:42:18.472329+0000) 2024-09-12T03:42:24.956 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:24 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:24.496+0000 7fa76bb6b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:42:24.956 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:24 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:24.496+0000 7fa76bb6b640 -1 osd.8 107 *** Got signal Terminated *** 2024-09-12T03:42:24.956 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:24 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:24.496+0000 7fa76bb6b640 -1 osd.8 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:42:25.131 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:24 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:24.839+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.723241+0000 front 2024-09-12T03:41:32.723103+0000 (oldest deadline 2024-09-12T03:41:56.822981+0000) 2024-09-12T03:42:25.131 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:24 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:24.839+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.224382+0000 front 2024-09-12T03:41:40.224550+0000 (oldest deadline 2024-09-12T03:42:02.524263+0000) 2024-09-12T03:42:25.131 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:24 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:24.839+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:47.825286+0000 front 2024-09-12T03:41:47.825215+0000 (oldest deadline 2024-09-12T03:42:12.525423+0000) 2024-09-12T03:42:25.131 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:24 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:24.839+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:57.726913+0000 front 2024-09-12T03:41:57.727091+0000 (oldest deadline 2024-09-12T03:42:23.026695+0000) 2024-09-12T03:42:25.237 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:24 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:24.979+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.706298+0000 front 2024-09-12T03:41:24.706335+0000 (oldest deadline 2024-09-12T03:41:48.206454+0000) 2024-09-12T03:42:25.237 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:24 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:24.979+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.408037+0000 front 2024-09-12T03:41:33.408013+0000 (oldest deadline 2024-09-12T03:41:56.308104+0000) 2024-09-12T03:42:25.237 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:24 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:24.979+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:41.509651+0000 front 2024-09-12T03:41:41.509715+0000 (oldest deadline 2024-09-12T03:42:05.609794+0000) 2024-09-12T03:42:25.237 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:24 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:24.979+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:48.712724+0000 front 2024-09-12T03:41:48.712828+0000 (oldest deadline 2024-09-12T03:42:12.212682+0000) 2024-09-12T03:42:25.304 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:25 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:25.047+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:25.304 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:25 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:25.047+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:25.304 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:25 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:25.047+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.525612+0000 front 2024-09-12T03:41:40.525773+0000 (oldest deadline 2024-09-12T03:42:06.425539+0000) 2024-09-12T03:42:25.304 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:25 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:25.047+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:46.425933+0000 front 2024-09-12T03:41:46.426031+0000 (oldest deadline 2024-09-12T03:42:11.725939+0000) 2024-09-12T03:42:25.304 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:25 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:25.047+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:55.226745+0000 front 2024-09-12T03:41:55.226726+0000 (oldest deadline 2024-09-12T03:42:24.027136+0000) 2024-09-12T03:42:25.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:25 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:25.236+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:25.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:25 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:25.236+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:25.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:25 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:25.236+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:25.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:25 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:25.236+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:56.172333+0000 front 2024-09-12T03:41:56.172398+0000 (oldest deadline 2024-09-12T03:42:18.472329+0000) 2024-09-12T03:42:26.022 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:25 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:25.878+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.723241+0000 front 2024-09-12T03:41:32.723103+0000 (oldest deadline 2024-09-12T03:41:56.822981+0000) 2024-09-12T03:42:26.022 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:25 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:25.878+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.224382+0000 front 2024-09-12T03:41:40.224550+0000 (oldest deadline 2024-09-12T03:42:02.524263+0000) 2024-09-12T03:42:26.022 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:25 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:25.878+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:47.825286+0000 front 2024-09-12T03:41:47.825215+0000 (oldest deadline 2024-09-12T03:42:12.525423+0000) 2024-09-12T03:42:26.022 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:25 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:25.878+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:57.726913+0000 front 2024-09-12T03:41:57.727091+0000 (oldest deadline 2024-09-12T03:42:23.026695+0000) 2024-09-12T03:42:26.280 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:26 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:26.013+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.706298+0000 front 2024-09-12T03:41:24.706335+0000 (oldest deadline 2024-09-12T03:41:48.206454+0000) 2024-09-12T03:42:26.280 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:26 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:26.013+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.408037+0000 front 2024-09-12T03:41:33.408013+0000 (oldest deadline 2024-09-12T03:41:56.308104+0000) 2024-09-12T03:42:26.280 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:26 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:26.013+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:41.509651+0000 front 2024-09-12T03:41:41.509715+0000 (oldest deadline 2024-09-12T03:42:05.609794+0000) 2024-09-12T03:42:26.281 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:26 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:26.013+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:48.712724+0000 front 2024-09-12T03:41:48.712828+0000 (oldest deadline 2024-09-12T03:42:12.212682+0000) 2024-09-12T03:42:26.392 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:26 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:26.049+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:26.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:26 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:26.049+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:26.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:26 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:26.049+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.525612+0000 front 2024-09-12T03:41:40.525773+0000 (oldest deadline 2024-09-12T03:42:06.425539+0000) 2024-09-12T03:42:26.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:26 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:26.049+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:46.425933+0000 front 2024-09-12T03:41:46.426031+0000 (oldest deadline 2024-09-12T03:42:11.725939+0000) 2024-09-12T03:42:26.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:26 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:26.049+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:55.226745+0000 front 2024-09-12T03:41:55.226726+0000 (oldest deadline 2024-09-12T03:42:24.027136+0000) 2024-09-12T03:42:26.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:26 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:26.280+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:26.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:26 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:26.280+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:26.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:26 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:26.280+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:26.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:26 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:26.280+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:56.172333+0000 front 2024-09-12T03:41:56.172398+0000 (oldest deadline 2024-09-12T03:42:18.472329+0000) 2024-09-12T03:42:27.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:26 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:26.902+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.723241+0000 front 2024-09-12T03:41:32.723103+0000 (oldest deadline 2024-09-12T03:41:56.822981+0000) 2024-09-12T03:42:27.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:26 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:26.902+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.224382+0000 front 2024-09-12T03:41:40.224550+0000 (oldest deadline 2024-09-12T03:42:02.524263+0000) 2024-09-12T03:42:27.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:26 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:26.902+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:47.825286+0000 front 2024-09-12T03:41:47.825215+0000 (oldest deadline 2024-09-12T03:42:12.525423+0000) 2024-09-12T03:42:27.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:26 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:26.902+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:57.726913+0000 front 2024-09-12T03:41:57.727091+0000 (oldest deadline 2024-09-12T03:42:23.026695+0000) 2024-09-12T03:42:27.313 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:26 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:26.969+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.706298+0000 front 2024-09-12T03:41:24.706335+0000 (oldest deadline 2024-09-12T03:41:48.206454+0000) 2024-09-12T03:42:27.313 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:26 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:26.969+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.408037+0000 front 2024-09-12T03:41:33.408013+0000 (oldest deadline 2024-09-12T03:41:56.308104+0000) 2024-09-12T03:42:27.313 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:26 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:26.969+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:41.509651+0000 front 2024-09-12T03:41:41.509715+0000 (oldest deadline 2024-09-12T03:42:05.609794+0000) 2024-09-12T03:42:27.313 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:26 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:26.969+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:48.712724+0000 front 2024-09-12T03:41:48.712828+0000 (oldest deadline 2024-09-12T03:42:12.212682+0000) 2024-09-12T03:42:27.392 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:27 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:27.090+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:27.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:27 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:27.090+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:27.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:27 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:27.090+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.525612+0000 front 2024-09-12T03:41:40.525773+0000 (oldest deadline 2024-09-12T03:42:06.425539+0000) 2024-09-12T03:42:27.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:27 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:27.090+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:46.425933+0000 front 2024-09-12T03:41:46.426031+0000 (oldest deadline 2024-09-12T03:42:11.725939+0000) 2024-09-12T03:42:27.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:27 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:27.090+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:55.226745+0000 front 2024-09-12T03:41:55.226726+0000 (oldest deadline 2024-09-12T03:42:24.027136+0000) 2024-09-12T03:42:27.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:27 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:27.313+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:27.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:27 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:27.313+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:27.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:27 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:27.313+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:27.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:27 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:27.313+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:56.172333+0000 front 2024-09-12T03:41:56.172398+0000 (oldest deadline 2024-09-12T03:42:18.472329+0000) 2024-09-12T03:42:28.271 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:27 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:27.898+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.723241+0000 front 2024-09-12T03:41:32.723103+0000 (oldest deadline 2024-09-12T03:41:56.822981+0000) 2024-09-12T03:42:28.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:27 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:27.898+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.224382+0000 front 2024-09-12T03:41:40.224550+0000 (oldest deadline 2024-09-12T03:42:02.524263+0000) 2024-09-12T03:42:28.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:27 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:27.898+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:47.825286+0000 front 2024-09-12T03:41:47.825215+0000 (oldest deadline 2024-09-12T03:42:12.525423+0000) 2024-09-12T03:42:28.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:27 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:27.898+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:57.726913+0000 front 2024-09-12T03:41:57.727091+0000 (oldest deadline 2024-09-12T03:42:23.026695+0000) 2024-09-12T03:42:28.306 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:28 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:28.040+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:28.306 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:28 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:28.040+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:28.306 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:28 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:28.040+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.525612+0000 front 2024-09-12T03:41:40.525773+0000 (oldest deadline 2024-09-12T03:42:06.425539+0000) 2024-09-12T03:42:28.306 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:28 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:28.040+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:46.425933+0000 front 2024-09-12T03:41:46.426031+0000 (oldest deadline 2024-09-12T03:42:11.725939+0000) 2024-09-12T03:42:28.306 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:28 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:28.040+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:55.226745+0000 front 2024-09-12T03:41:55.226726+0000 (oldest deadline 2024-09-12T03:42:24.027136+0000) 2024-09-12T03:42:28.306 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:28 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:28.040+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:04.027660+0000 front 2024-09-12T03:42:04.027609+0000 (oldest deadline 2024-09-12T03:42:27.527567+0000) 2024-09-12T03:42:28.345 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:27 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:27.996+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.706298+0000 front 2024-09-12T03:41:24.706335+0000 (oldest deadline 2024-09-12T03:41:48.206454+0000) 2024-09-12T03:42:28.345 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:27 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:27.996+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.408037+0000 front 2024-09-12T03:41:33.408013+0000 (oldest deadline 2024-09-12T03:41:56.308104+0000) 2024-09-12T03:42:28.345 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:27 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:27.996+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:41.509651+0000 front 2024-09-12T03:41:41.509715+0000 (oldest deadline 2024-09-12T03:42:05.609794+0000) 2024-09-12T03:42:28.345 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:27 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:27.996+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:48.712724+0000 front 2024-09-12T03:41:48.712828+0000 (oldest deadline 2024-09-12T03:42:12.212682+0000) 2024-09-12T03:42:28.345 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:27 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:27.996+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:01.514470+0000 front 2024-09-12T03:42:01.514488+0000 (oldest deadline 2024-09-12T03:42:27.414487+0000) 2024-09-12T03:42:28.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:28 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:28.344+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:28.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:28 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:28.345+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:28.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:28 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:28.345+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:28.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:28 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:28.345+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:56.172333+0000 front 2024-09-12T03:41:56.172398+0000 (oldest deadline 2024-09-12T03:42:18.472329+0000) 2024-09-12T03:42:29.271 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:28 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:28.947+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.723241+0000 front 2024-09-12T03:41:32.723103+0000 (oldest deadline 2024-09-12T03:41:56.822981+0000) 2024-09-12T03:42:29.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:28 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:28.947+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.224382+0000 front 2024-09-12T03:41:40.224550+0000 (oldest deadline 2024-09-12T03:42:02.524263+0000) 2024-09-12T03:42:29.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:28 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:28.947+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:47.825286+0000 front 2024-09-12T03:41:47.825215+0000 (oldest deadline 2024-09-12T03:42:12.525423+0000) 2024-09-12T03:42:29.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:28 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:28.947+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:57.726913+0000 front 2024-09-12T03:41:57.727091+0000 (oldest deadline 2024-09-12T03:42:23.026695+0000) 2024-09-12T03:42:29.330 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:28 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:28.981+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.706298+0000 front 2024-09-12T03:41:24.706335+0000 (oldest deadline 2024-09-12T03:41:48.206454+0000) 2024-09-12T03:42:29.330 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:28 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:28.981+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.408037+0000 front 2024-09-12T03:41:33.408013+0000 (oldest deadline 2024-09-12T03:41:56.308104+0000) 2024-09-12T03:42:29.330 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:28 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:28.981+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:41.509651+0000 front 2024-09-12T03:41:41.509715+0000 (oldest deadline 2024-09-12T03:42:05.609794+0000) 2024-09-12T03:42:29.330 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:28 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:28.981+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:48.712724+0000 front 2024-09-12T03:41:48.712828+0000 (oldest deadline 2024-09-12T03:42:12.212682+0000) 2024-09-12T03:42:29.330 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:28 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8[52650]: 2024-09-12T03:42:28.981+0000 7fa768181640 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:01.514470+0000 front 2024-09-12T03:42:01.514488+0000 (oldest deadline 2024-09-12T03:42:27.414487+0000) 2024-09-12T03:42:29.392 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:29 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:29.001+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:29.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:29 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:29.001+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:29.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:29 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:29.001+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.525612+0000 front 2024-09-12T03:41:40.525773+0000 (oldest deadline 2024-09-12T03:42:06.425539+0000) 2024-09-12T03:42:29.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:29 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:29.001+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:46.425933+0000 front 2024-09-12T03:41:46.426031+0000 (oldest deadline 2024-09-12T03:42:11.725939+0000) 2024-09-12T03:42:29.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:29 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:29.001+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:55.226745+0000 front 2024-09-12T03:41:55.226726+0000 (oldest deadline 2024-09-12T03:42:24.027136+0000) 2024-09-12T03:42:29.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:29 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:29.001+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:04.027660+0000 front 2024-09-12T03:42:04.027609+0000 (oldest deadline 2024-09-12T03:42:27.527567+0000) 2024-09-12T03:42:29.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:29 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:29.330+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:29.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:29 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:29.330+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:29.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:29 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:29.330+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:29.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:29 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:29.330+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:56.172333+0000 front 2024-09-12T03:41:56.172398+0000 (oldest deadline 2024-09-12T03:42:18.472329+0000) 2024-09-12T03:42:29.707 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:29 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:29.330+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:04.873875+0000 front 2024-09-12T03:42:04.873833+0000 (oldest deadline 2024-09-12T03:42:28.973822+0000) 2024-09-12T03:42:30.119 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:29 smithi195 podman[72224]: 2024-09-12 03:42:29.857204813 +0000 UTC m=+5.483397027 container died 260c29ce5ea8bd163b5ab97bf544985fadbbdba22a7994b14e94f7bf1347842a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:42:30.271 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:29 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:29.972+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.723241+0000 front 2024-09-12T03:41:32.723103+0000 (oldest deadline 2024-09-12T03:41:56.822981+0000) 2024-09-12T03:42:30.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:29 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:29.972+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.224382+0000 front 2024-09-12T03:41:40.224550+0000 (oldest deadline 2024-09-12T03:42:02.524263+0000) 2024-09-12T03:42:30.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:29 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:29.972+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:47.825286+0000 front 2024-09-12T03:41:47.825215+0000 (oldest deadline 2024-09-12T03:42:12.525423+0000) 2024-09-12T03:42:30.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:29 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:29.972+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:57.726913+0000 front 2024-09-12T03:41:57.727091+0000 (oldest deadline 2024-09-12T03:42:23.026695+0000) 2024-09-12T03:42:30.392 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:29 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:29.963+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:30.392 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:29 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:29.963+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:30.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:29 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:29.963+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.525612+0000 front 2024-09-12T03:41:40.525773+0000 (oldest deadline 2024-09-12T03:42:06.425539+0000) 2024-09-12T03:42:30.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:29 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:29.963+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:46.425933+0000 front 2024-09-12T03:41:46.426031+0000 (oldest deadline 2024-09-12T03:42:11.725939+0000) 2024-09-12T03:42:30.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:29 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:29.963+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:55.226745+0000 front 2024-09-12T03:41:55.226726+0000 (oldest deadline 2024-09-12T03:42:24.027136+0000) 2024-09-12T03:42:30.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:29 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:29.963+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:04.027660+0000 front 2024-09-12T03:42:04.027609+0000 (oldest deadline 2024-09-12T03:42:27.527567+0000) 2024-09-12T03:42:30.456 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:30 smithi195 podman[72224]: 2024-09-12 03:42:30.229691127 +0000 UTC m=+5.855883333 container cleanup 260c29ce5ea8bd163b5ab97bf544985fadbbdba22a7994b14e94f7bf1347842a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-12T03:42:30.457 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:30 smithi195 bash[72224]: ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8 2024-09-12T03:42:30.457 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:30 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:30.367+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:30.457 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:30 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:30.367+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:30.457 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:30 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:30.367+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:30.458 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:30 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:30.367+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:56.172333+0000 front 2024-09-12T03:41:56.172398+0000 (oldest deadline 2024-09-12T03:42:18.472329+0000) 2024-09-12T03:42:30.458 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:30 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:30.367+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:04.873875+0000 front 2024-09-12T03:42:04.873833+0000 (oldest deadline 2024-09-12T03:42:28.973822+0000) 2024-09-12T03:42:30.838 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:30 smithi195 podman[72239]: 2024-09-12 03:42:30.489497295 +0000 UTC m=+0.629118403 container remove 260c29ce5ea8bd163b5ab97bf544985fadbbdba22a7994b14e94f7bf1347842a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731) 2024-09-12T03:42:31.092 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:30 smithi195 podman[72327]: 2024-09-12 03:42:30.74150676 +0000 UTC m=+0.015628979 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:42:31.092 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:30 smithi195 podman[72327]: 2024-09-12 03:42:30.865496716 +0000 UTC m=+0.139618997 container create 37669d426e1f19c69d593cec79a57eecef684ae201b54e7fa1f739b0e6cc1d87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2) 2024-09-12T03:42:31.271 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:30 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:30.944+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.723241+0000 front 2024-09-12T03:41:32.723103+0000 (oldest deadline 2024-09-12T03:41:56.822981+0000) 2024-09-12T03:42:31.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:30 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:30.944+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.224382+0000 front 2024-09-12T03:41:40.224550+0000 (oldest deadline 2024-09-12T03:42:02.524263+0000) 2024-09-12T03:42:31.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:30 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:30.944+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:47.825286+0000 front 2024-09-12T03:41:47.825215+0000 (oldest deadline 2024-09-12T03:42:12.525423+0000) 2024-09-12T03:42:31.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:30 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:30.944+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:57.726913+0000 front 2024-09-12T03:41:57.727091+0000 (oldest deadline 2024-09-12T03:42:23.026695+0000) 2024-09-12T03:42:31.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:30 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:30.944+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:03.027227+0000 front 2024-09-12T03:42:03.027291+0000 (oldest deadline 2024-09-12T03:42:30.627717+0000) 2024-09-12T03:42:31.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:31 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:31.007+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:31.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:31 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:31.007+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:31.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:31 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:31.007+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.525612+0000 front 2024-09-12T03:41:40.525773+0000 (oldest deadline 2024-09-12T03:42:06.425539+0000) 2024-09-12T03:42:31.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:31 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:31.007+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:46.425933+0000 front 2024-09-12T03:41:46.426031+0000 (oldest deadline 2024-09-12T03:42:11.725939+0000) 2024-09-12T03:42:31.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:31 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:31.007+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:55.226745+0000 front 2024-09-12T03:41:55.226726+0000 (oldest deadline 2024-09-12T03:42:24.027136+0000) 2024-09-12T03:42:31.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:31 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:31.007+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:04.027660+0000 front 2024-09-12T03:42:04.027609+0000 (oldest deadline 2024-09-12T03:42:27.527567+0000) 2024-09-12T03:42:31.418 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:31 smithi195 podman[72327]: 2024-09-12 03:42:31.092660115 +0000 UTC m=+0.366782342 container init 37669d426e1f19c69d593cec79a57eecef684ae201b54e7fa1f739b0e6cc1d87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2) 2024-09-12T03:42:31.418 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:31 smithi195 podman[72327]: 2024-09-12 03:42:31.09609901 +0000 UTC m=+0.370221227 container start 37669d426e1f19c69d593cec79a57eecef684ae201b54e7fa1f739b0e6cc1d87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-12T03:42:31.418 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:31 smithi195 podman[72327]: 2024-09-12 03:42:31.184375395 +0000 UTC m=+0.458497616 container attach 37669d426e1f19c69d593cec79a57eecef684ae201b54e7fa1f739b0e6cc1d87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3) 2024-09-12T03:42:31.418 INFO:journalctl@ceph.osd.8.smithi195.stdout:Sep 12 03:42:31 smithi195 podman[72327]: 2024-09-12 03:42:31.288673727 +0000 UTC m=+0.562795943 container died 37669d426e1f19c69d593cec79a57eecef684ae201b54e7fa1f739b0e6cc1d87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=) 2024-09-12T03:42:31.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:31 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:31.417+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:31.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:31 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:31.417+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:31.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:31 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:31.417+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:31.707 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:31 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:31.417+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:56.172333+0000 front 2024-09-12T03:41:56.172398+0000 (oldest deadline 2024-09-12T03:42:18.472329+0000) 2024-09-12T03:42:31.707 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:31 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:31.417+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:04.873875+0000 front 2024-09-12T03:42:04.873833+0000 (oldest deadline 2024-09-12T03:42:28.973822+0000) 2024-09-12T03:42:31.949 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@osd.8.service' 2024-09-12T03:42:32.028 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:42:32.028 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-12T03:42:32.029 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-12T03:42:32.029 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-b0812460-70b5-11ef-bcea-c7b262605968@osd.9 2024-09-12T03:42:32.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:31 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:31.973+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.723241+0000 front 2024-09-12T03:41:32.723103+0000 (oldest deadline 2024-09-12T03:41:56.822981+0000) 2024-09-12T03:42:32.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:31 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:31.973+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.224382+0000 front 2024-09-12T03:41:40.224550+0000 (oldest deadline 2024-09-12T03:42:02.524263+0000) 2024-09-12T03:42:32.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:31 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:31.973+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:47.825286+0000 front 2024-09-12T03:41:47.825215+0000 (oldest deadline 2024-09-12T03:42:12.525423+0000) 2024-09-12T03:42:32.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:31 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:31.973+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:57.726913+0000 front 2024-09-12T03:41:57.727091+0000 (oldest deadline 2024-09-12T03:42:23.026695+0000) 2024-09-12T03:42:32.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:31 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:31.973+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:03.027227+0000 front 2024-09-12T03:42:03.027291+0000 (oldest deadline 2024-09-12T03:42:30.627717+0000) 2024-09-12T03:42:32.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:32 smithi064 systemd[1]: Stopping Ceph osd.9 for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:42:32.392 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:32 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:32.012+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:32.392 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:32 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:32.012+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:32.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:32 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:32.012+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.525612+0000 front 2024-09-12T03:41:40.525773+0000 (oldest deadline 2024-09-12T03:42:06.425539+0000) 2024-09-12T03:42:32.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:32 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:32.012+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:46.425933+0000 front 2024-09-12T03:41:46.426031+0000 (oldest deadline 2024-09-12T03:42:11.725939+0000) 2024-09-12T03:42:32.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:32 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:32.012+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:55.226745+0000 front 2024-09-12T03:41:55.226726+0000 (oldest deadline 2024-09-12T03:42:24.027136+0000) 2024-09-12T03:42:32.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:32 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:32.012+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:04.027660+0000 front 2024-09-12T03:42:04.027609+0000 (oldest deadline 2024-09-12T03:42:27.527567+0000) 2024-09-12T03:42:32.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:32 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:32.433+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:32.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:32 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:32.433+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:32.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:32 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:32.433+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:32.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:32 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:32.433+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:56.172333+0000 front 2024-09-12T03:41:56.172398+0000 (oldest deadline 2024-09-12T03:42:18.472329+0000) 2024-09-12T03:42:32.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:32 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:32.433+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:04.873875+0000 front 2024-09-12T03:42:04.873833+0000 (oldest deadline 2024-09-12T03:42:28.973822+0000) 2024-09-12T03:42:32.771 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:32 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:32.358+0000 7fe5af535640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:42:32.772 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:32 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:32.358+0000 7fe5af535640 -1 osd.9 107 *** Got signal Terminated *** 2024-09-12T03:42:32.772 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:32 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:32.358+0000 7fe5af535640 -1 osd.9 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:42:33.271 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:32 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:32.930+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.723241+0000 front 2024-09-12T03:41:32.723103+0000 (oldest deadline 2024-09-12T03:41:56.822981+0000) 2024-09-12T03:42:33.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:32 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:32.930+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.224382+0000 front 2024-09-12T03:41:40.224550+0000 (oldest deadline 2024-09-12T03:42:02.524263+0000) 2024-09-12T03:42:33.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:32 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:32.930+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:47.825286+0000 front 2024-09-12T03:41:47.825215+0000 (oldest deadline 2024-09-12T03:42:12.525423+0000) 2024-09-12T03:42:33.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:32 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:32.930+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:57.726913+0000 front 2024-09-12T03:41:57.727091+0000 (oldest deadline 2024-09-12T03:42:23.026695+0000) 2024-09-12T03:42:33.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:32 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:32.930+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:03.027227+0000 front 2024-09-12T03:42:03.027291+0000 (oldest deadline 2024-09-12T03:42:30.627717+0000) 2024-09-12T03:42:33.392 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:33 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:33.031+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:33.392 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:33 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:33.031+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:33.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:33 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:33.031+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.525612+0000 front 2024-09-12T03:41:40.525773+0000 (oldest deadline 2024-09-12T03:42:06.425539+0000) 2024-09-12T03:42:33.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:33 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:33.031+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:46.425933+0000 front 2024-09-12T03:41:46.426031+0000 (oldest deadline 2024-09-12T03:42:11.725939+0000) 2024-09-12T03:42:33.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:33 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:33.031+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:55.226745+0000 front 2024-09-12T03:41:55.226726+0000 (oldest deadline 2024-09-12T03:42:24.027136+0000) 2024-09-12T03:42:33.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:33 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:33.031+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:04.027660+0000 front 2024-09-12T03:42:04.027609+0000 (oldest deadline 2024-09-12T03:42:27.527567+0000) 2024-09-12T03:42:33.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:33 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:33.457+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:33.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:33 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:33.457+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:33.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:33 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:33.457+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:33.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:33 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:33.457+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:56.172333+0000 front 2024-09-12T03:41:56.172398+0000 (oldest deadline 2024-09-12T03:42:18.472329+0000) 2024-09-12T03:42:33.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:33 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:33.457+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:04.873875+0000 front 2024-09-12T03:42:04.873833+0000 (oldest deadline 2024-09-12T03:42:28.973822+0000) 2024-09-12T03:42:34.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:33 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:33.941+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.723241+0000 front 2024-09-12T03:41:32.723103+0000 (oldest deadline 2024-09-12T03:41:56.822981+0000) 2024-09-12T03:42:34.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:33 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:33.941+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.224382+0000 front 2024-09-12T03:41:40.224550+0000 (oldest deadline 2024-09-12T03:42:02.524263+0000) 2024-09-12T03:42:34.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:33 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:33.941+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:47.825286+0000 front 2024-09-12T03:41:47.825215+0000 (oldest deadline 2024-09-12T03:42:12.525423+0000) 2024-09-12T03:42:34.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:33 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:33.941+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:57.726913+0000 front 2024-09-12T03:41:57.727091+0000 (oldest deadline 2024-09-12T03:42:23.026695+0000) 2024-09-12T03:42:34.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:33 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:33.941+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:03.027227+0000 front 2024-09-12T03:42:03.027291+0000 (oldest deadline 2024-09-12T03:42:30.627717+0000) 2024-09-12T03:42:34.392 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:34 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:34.076+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:34.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:34 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:34.076+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:34.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:34 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:34.076+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.525612+0000 front 2024-09-12T03:41:40.525773+0000 (oldest deadline 2024-09-12T03:42:06.425539+0000) 2024-09-12T03:42:34.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:34 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:34.076+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:46.425933+0000 front 2024-09-12T03:41:46.426031+0000 (oldest deadline 2024-09-12T03:42:11.725939+0000) 2024-09-12T03:42:34.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:34 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:34.076+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:55.226745+0000 front 2024-09-12T03:41:55.226726+0000 (oldest deadline 2024-09-12T03:42:24.027136+0000) 2024-09-12T03:42:34.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:34 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:34.076+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:04.027660+0000 front 2024-09-12T03:42:04.027609+0000 (oldest deadline 2024-09-12T03:42:27.527567+0000) 2024-09-12T03:42:34.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:34 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:34.437+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:34.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:34 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:34.437+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:34.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:34 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:34.437+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:34.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:34 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:34.437+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:56.172333+0000 front 2024-09-12T03:41:56.172398+0000 (oldest deadline 2024-09-12T03:42:18.472329+0000) 2024-09-12T03:42:34.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:34 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:34.437+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:04.873875+0000 front 2024-09-12T03:42:04.873833+0000 (oldest deadline 2024-09-12T03:42:28.973822+0000) 2024-09-12T03:42:35.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:34 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:34.914+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.723241+0000 front 2024-09-12T03:41:32.723103+0000 (oldest deadline 2024-09-12T03:41:56.822981+0000) 2024-09-12T03:42:35.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:34 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:34.914+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.224382+0000 front 2024-09-12T03:41:40.224550+0000 (oldest deadline 2024-09-12T03:42:02.524263+0000) 2024-09-12T03:42:35.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:34 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:34.914+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:47.825286+0000 front 2024-09-12T03:41:47.825215+0000 (oldest deadline 2024-09-12T03:42:12.525423+0000) 2024-09-12T03:42:35.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:34 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:34.914+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:57.726913+0000 front 2024-09-12T03:41:57.727091+0000 (oldest deadline 2024-09-12T03:42:23.026695+0000) 2024-09-12T03:42:35.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:34 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:34.914+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:03.027227+0000 front 2024-09-12T03:42:03.027291+0000 (oldest deadline 2024-09-12T03:42:30.627717+0000) 2024-09-12T03:42:35.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:35 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:35.039+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:35.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:35 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:35.039+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:35.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:35 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:35.039+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.525612+0000 front 2024-09-12T03:41:40.525773+0000 (oldest deadline 2024-09-12T03:42:06.425539+0000) 2024-09-12T03:42:35.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:35 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:35.039+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:46.425933+0000 front 2024-09-12T03:41:46.426031+0000 (oldest deadline 2024-09-12T03:42:11.725939+0000) 2024-09-12T03:42:35.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:35 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:35.039+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:55.226745+0000 front 2024-09-12T03:41:55.226726+0000 (oldest deadline 2024-09-12T03:42:24.027136+0000) 2024-09-12T03:42:35.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:35 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:35.039+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:04.027660+0000 front 2024-09-12T03:42:04.027609+0000 (oldest deadline 2024-09-12T03:42:27.527567+0000) 2024-09-12T03:42:35.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:35 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:35.470+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:35.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:35 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:35.470+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:35.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:35 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:35.470+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:35.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:35 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:35.470+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:56.172333+0000 front 2024-09-12T03:41:56.172398+0000 (oldest deadline 2024-09-12T03:42:18.472329+0000) 2024-09-12T03:42:35.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:35 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:35.470+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:04.873875+0000 front 2024-09-12T03:42:04.873833+0000 (oldest deadline 2024-09-12T03:42:28.973822+0000) 2024-09-12T03:42:35.957 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:35 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:35.470+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-12T03:42:13.575044+0000 front 2024-09-12T03:42:13.575001+0000 (oldest deadline 2024-09-12T03:42:34.674959+0000) 2024-09-12T03:42:36.271 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:35 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:35.899+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.723241+0000 front 2024-09-12T03:41:32.723103+0000 (oldest deadline 2024-09-12T03:41:56.822981+0000) 2024-09-12T03:42:36.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:35 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:35.899+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.224382+0000 front 2024-09-12T03:41:40.224550+0000 (oldest deadline 2024-09-12T03:42:02.524263+0000) 2024-09-12T03:42:36.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:35 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:35.899+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:47.825286+0000 front 2024-09-12T03:41:47.825215+0000 (oldest deadline 2024-09-12T03:42:12.525423+0000) 2024-09-12T03:42:36.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:35 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:35.899+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:57.726913+0000 front 2024-09-12T03:41:57.727091+0000 (oldest deadline 2024-09-12T03:42:23.026695+0000) 2024-09-12T03:42:36.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:35 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:35.899+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:03.027227+0000 front 2024-09-12T03:42:03.027291+0000 (oldest deadline 2024-09-12T03:42:30.627717+0000) 2024-09-12T03:42:36.392 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:36 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:36.060+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:36.392 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:36 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:36.060+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:36.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:36 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:36.060+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.525612+0000 front 2024-09-12T03:41:40.525773+0000 (oldest deadline 2024-09-12T03:42:06.425539+0000) 2024-09-12T03:42:36.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:36 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:36.060+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:46.425933+0000 front 2024-09-12T03:41:46.426031+0000 (oldest deadline 2024-09-12T03:42:11.725939+0000) 2024-09-12T03:42:36.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:36 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:36.060+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:55.226745+0000 front 2024-09-12T03:41:55.226726+0000 (oldest deadline 2024-09-12T03:42:24.027136+0000) 2024-09-12T03:42:36.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:36 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:36.060+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:04.027660+0000 front 2024-09-12T03:42:04.027609+0000 (oldest deadline 2024-09-12T03:42:27.527567+0000) 2024-09-12T03:42:36.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:36 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:36.469+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:36.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:36 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:36.469+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:36.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:36 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:36.469+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:36.957 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:36 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:36.469+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:56.172333+0000 front 2024-09-12T03:41:56.172398+0000 (oldest deadline 2024-09-12T03:42:18.472329+0000) 2024-09-12T03:42:36.957 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:36 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:36.469+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:04.873875+0000 front 2024-09-12T03:42:04.873833+0000 (oldest deadline 2024-09-12T03:42:28.973822+0000) 2024-09-12T03:42:36.957 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:36 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:36.469+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-12T03:42:13.575044+0000 front 2024-09-12T03:42:13.575001+0000 (oldest deadline 2024-09-12T03:42:34.674959+0000) 2024-09-12T03:42:37.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:36 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:36.908+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:32.723241+0000 front 2024-09-12T03:41:32.723103+0000 (oldest deadline 2024-09-12T03:41:56.822981+0000) 2024-09-12T03:42:37.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:36 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:36.908+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.224382+0000 front 2024-09-12T03:41:40.224550+0000 (oldest deadline 2024-09-12T03:42:02.524263+0000) 2024-09-12T03:42:37.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:36 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:36.908+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:47.825286+0000 front 2024-09-12T03:41:47.825215+0000 (oldest deadline 2024-09-12T03:42:12.525423+0000) 2024-09-12T03:42:37.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:36 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:36.908+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:57.726913+0000 front 2024-09-12T03:41:57.727091+0000 (oldest deadline 2024-09-12T03:42:23.026695+0000) 2024-09-12T03:42:37.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:36 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9[64906]: 2024-09-12T03:42:36.908+0000 7fe5abb4b640 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:03.027227+0000 front 2024-09-12T03:42:03.027291+0000 (oldest deadline 2024-09-12T03:42:30.627717+0000) 2024-09-12T03:42:37.392 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:37 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:37.075+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:37.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:37 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:37.075+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:37.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:37 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:37.075+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.525612+0000 front 2024-09-12T03:41:40.525773+0000 (oldest deadline 2024-09-12T03:42:06.425539+0000) 2024-09-12T03:42:37.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:37 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:37.075+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:46.425933+0000 front 2024-09-12T03:41:46.426031+0000 (oldest deadline 2024-09-12T03:42:11.725939+0000) 2024-09-12T03:42:37.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:37 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:37.075+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:55.226745+0000 front 2024-09-12T03:41:55.226726+0000 (oldest deadline 2024-09-12T03:42:24.027136+0000) 2024-09-12T03:42:37.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:37 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:37.075+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:04.027660+0000 front 2024-09-12T03:42:04.027609+0000 (oldest deadline 2024-09-12T03:42:27.527567+0000) 2024-09-12T03:42:37.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:37 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:37.443+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:37.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:37 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:37.443+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:37.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:37 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:37.443+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:37.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:37 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:37.443+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:56.172333+0000 front 2024-09-12T03:41:56.172398+0000 (oldest deadline 2024-09-12T03:42:18.472329+0000) 2024-09-12T03:42:37.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:37 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:37.443+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:04.873875+0000 front 2024-09-12T03:42:04.873833+0000 (oldest deadline 2024-09-12T03:42:28.973822+0000) 2024-09-12T03:42:37.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:37 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:37.443+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-12T03:42:13.575044+0000 front 2024-09-12T03:42:13.575001+0000 (oldest deadline 2024-09-12T03:42:34.674959+0000) 2024-09-12T03:42:37.967 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:37 smithi064 podman[94695]: 2024-09-12 03:42:37.700589348 +0000 UTC m=+5.466078166 container died fd367fb13c75c5d0b2d86df8943511bea1cb88ec6c3e611b5864812a15a78a40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-12T03:42:38.271 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:38 smithi064 podman[94695]: 2024-09-12 03:42:38.058604687 +0000 UTC m=+5.824093504 container cleanup fd367fb13c75c5d0b2d86df8943511bea1cb88ec6c3e611b5864812a15a78a40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-12T03:42:38.272 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:38 smithi064 bash[94695]: ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9 2024-09-12T03:42:38.380 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:38 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:38.124+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:38.380 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:38 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:38.124+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:38.380 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:38 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:38.124+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.525612+0000 front 2024-09-12T03:41:40.525773+0000 (oldest deadline 2024-09-12T03:42:06.425539+0000) 2024-09-12T03:42:38.380 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:38 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:38.124+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:46.425933+0000 front 2024-09-12T03:41:46.426031+0000 (oldest deadline 2024-09-12T03:42:11.725939+0000) 2024-09-12T03:42:38.380 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:38 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:38.124+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:55.226745+0000 front 2024-09-12T03:41:55.226726+0000 (oldest deadline 2024-09-12T03:42:24.027136+0000) 2024-09-12T03:42:38.380 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:38 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:38.124+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:04.027660+0000 front 2024-09-12T03:42:04.027609+0000 (oldest deadline 2024-09-12T03:42:27.527567+0000) 2024-09-12T03:42:38.380 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:38 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:38.124+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-12T03:42:11.429415+0000 front 2024-09-12T03:42:11.429513+0000 (oldest deadline 2024-09-12T03:42:37.329281+0000) 2024-09-12T03:42:38.685 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:38 smithi064 podman[94710]: 2024-09-12 03:42:38.318385967 +0000 UTC m=+0.614336066 container remove fd367fb13c75c5d0b2d86df8943511bea1cb88ec6c3e611b5864812a15a78a40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-12T03:42:38.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:38 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:38.398+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:38.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:38 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:38.398+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:38.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:38 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:38.398+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:38.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:38 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:38.398+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:56.172333+0000 front 2024-09-12T03:41:56.172398+0000 (oldest deadline 2024-09-12T03:42:18.472329+0000) 2024-09-12T03:42:38.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:38 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:38.398+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:04.873875+0000 front 2024-09-12T03:42:04.873833+0000 (oldest deadline 2024-09-12T03:42:28.973822+0000) 2024-09-12T03:42:38.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:38 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:38.398+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-12T03:42:13.575044+0000 front 2024-09-12T03:42:13.575001+0000 (oldest deadline 2024-09-12T03:42:34.674959+0000) 2024-09-12T03:42:38.970 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:38 smithi064 podman[94798]: 2024-09-12 03:42:38.685811653 +0000 UTC m=+0.110246670 container create a0cba877e9db72f3819b7e062373fa53b9b9b05f84aa30f2c6330b62962a90ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-12T03:42:38.971 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:38 smithi064 podman[94798]: 2024-09-12 03:42:38.59191363 +0000 UTC m=+0.016348661 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:42:38.971 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:38 smithi064 podman[94798]: 2024-09-12 03:42:38.89665168 +0000 UTC m=+0.321086707 container init a0cba877e9db72f3819b7e062373fa53b9b9b05f84aa30f2c6330b62962a90ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2) 2024-09-12T03:42:38.971 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:38 smithi064 podman[94798]: 2024-09-12 03:42:38.900152192 +0000 UTC m=+0.324587218 container start a0cba877e9db72f3819b7e062373fa53b9b9b05f84aa30f2c6330b62962a90ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-12T03:42:38.971 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:38 smithi064 podman[94798]: 2024-09-12 03:42:38.971422429 +0000 UTC m=+0.395857470 container attach a0cba877e9db72f3819b7e062373fa53b9b9b05f84aa30f2c6330b62962a90ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3) 2024-09-12T03:42:39.271 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:39 smithi064 podman[94798]: 2024-09-12 03:42:39.092438158 +0000 UTC m=+0.516873182 container died a0cba877e9db72f3819b7e062373fa53b9b9b05f84aa30f2c6330b62962a90ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-12T03:42:39.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:39 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:39.129+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:39.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:39 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:39.129+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:39.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:39 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:39.129+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.525612+0000 front 2024-09-12T03:41:40.525773+0000 (oldest deadline 2024-09-12T03:42:06.425539+0000) 2024-09-12T03:42:39.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:39 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:39.129+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:46.425933+0000 front 2024-09-12T03:41:46.426031+0000 (oldest deadline 2024-09-12T03:42:11.725939+0000) 2024-09-12T03:42:39.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:39 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:39.129+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:55.226745+0000 front 2024-09-12T03:41:55.226726+0000 (oldest deadline 2024-09-12T03:42:24.027136+0000) 2024-09-12T03:42:39.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:39 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:39.129+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:04.027660+0000 front 2024-09-12T03:42:04.027609+0000 (oldest deadline 2024-09-12T03:42:27.527567+0000) 2024-09-12T03:42:39.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:39 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:39.129+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-12T03:42:11.429415+0000 front 2024-09-12T03:42:11.429513+0000 (oldest deadline 2024-09-12T03:42:37.329281+0000) 2024-09-12T03:42:39.610 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 12 03:42:39 smithi064 podman[94821]: 2024-09-12 03:42:39.608313426 +0000 UTC m=+0.506344940 container remove a0cba877e9db72f3819b7e062373fa53b9b9b05f84aa30f2c6330b62962a90ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:42:39.631 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@osd.9.service' 2024-09-12T03:42:39.671 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:42:39.671 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-12T03:42:39.672 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-12T03:42:39.672 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-b0812460-70b5-11ef-bcea-c7b262605968@osd.10 2024-09-12T03:42:39.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:39 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:39.398+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:39.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:39 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:39.398+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:39.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:39 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:39.398+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:39.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:39 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:39.398+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:56.172333+0000 front 2024-09-12T03:41:56.172398+0000 (oldest deadline 2024-09-12T03:42:18.472329+0000) 2024-09-12T03:42:39.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:39 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:39.398+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:04.873875+0000 front 2024-09-12T03:42:04.873833+0000 (oldest deadline 2024-09-12T03:42:28.973822+0000) 2024-09-12T03:42:39.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:39 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:39.398+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-12T03:42:13.575044+0000 front 2024-09-12T03:42:13.575001+0000 (oldest deadline 2024-09-12T03:42:34.674959+0000) 2024-09-12T03:42:39.969 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:39 smithi115 systemd[1]: Stopping Ceph osd.10 for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:42:39.969 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:39 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:39.967+0000 7f6f16f2e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:42:39.969 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:39 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:39.967+0000 7f6f16f2e640 -1 osd.10 107 *** Got signal Terminated *** 2024-09-12T03:42:39.969 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:39 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:39.967+0000 7f6f16f2e640 -1 osd.10 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:42:40.392 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:40 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:40.147+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:40.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:40 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:40.147+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:40.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:40 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:40.147+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.525612+0000 front 2024-09-12T03:41:40.525773+0000 (oldest deadline 2024-09-12T03:42:06.425539+0000) 2024-09-12T03:42:40.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:40 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:40.147+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:46.425933+0000 front 2024-09-12T03:41:46.426031+0000 (oldest deadline 2024-09-12T03:42:11.725939+0000) 2024-09-12T03:42:40.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:40 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:40.147+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:55.226745+0000 front 2024-09-12T03:41:55.226726+0000 (oldest deadline 2024-09-12T03:42:24.027136+0000) 2024-09-12T03:42:40.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:40 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:40.147+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:04.027660+0000 front 2024-09-12T03:42:04.027609+0000 (oldest deadline 2024-09-12T03:42:27.527567+0000) 2024-09-12T03:42:40.393 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:40 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:40.147+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-12T03:42:11.429415+0000 front 2024-09-12T03:42:11.429513+0000 (oldest deadline 2024-09-12T03:42:37.329281+0000) 2024-09-12T03:42:40.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:40 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:40.434+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:40.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:40 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:40.434+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:40.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:40 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:40.434+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:40.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:40 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:40.434+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:56.172333+0000 front 2024-09-12T03:41:56.172398+0000 (oldest deadline 2024-09-12T03:42:18.472329+0000) 2024-09-12T03:42:40.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:40 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:40.434+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:04.873875+0000 front 2024-09-12T03:42:04.873833+0000 (oldest deadline 2024-09-12T03:42:28.973822+0000) 2024-09-12T03:42:40.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:40 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:40.434+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-12T03:42:13.575044+0000 front 2024-09-12T03:42:13.575001+0000 (oldest deadline 2024-09-12T03:42:34.674959+0000) 2024-09-12T03:42:41.642 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:41 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:41.189+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:41.643 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:41 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:41.189+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:41.643 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:41 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:41.189+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.525612+0000 front 2024-09-12T03:41:40.525773+0000 (oldest deadline 2024-09-12T03:42:06.425539+0000) 2024-09-12T03:42:41.643 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:41 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:41.189+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:46.425933+0000 front 2024-09-12T03:41:46.426031+0000 (oldest deadline 2024-09-12T03:42:11.725939+0000) 2024-09-12T03:42:41.643 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:41 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:41.189+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:55.226745+0000 front 2024-09-12T03:41:55.226726+0000 (oldest deadline 2024-09-12T03:42:24.027136+0000) 2024-09-12T03:42:41.643 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:41 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:41.189+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:04.027660+0000 front 2024-09-12T03:42:04.027609+0000 (oldest deadline 2024-09-12T03:42:27.527567+0000) 2024-09-12T03:42:41.643 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:41 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:41.189+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-12T03:42:11.429415+0000 front 2024-09-12T03:42:11.429513+0000 (oldest deadline 2024-09-12T03:42:37.329281+0000) 2024-09-12T03:42:41.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:41 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:41.461+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:41.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:41 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:41.461+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:41.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:41 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:41.461+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:41.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:41 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:41.461+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:56.172333+0000 front 2024-09-12T03:41:56.172398+0000 (oldest deadline 2024-09-12T03:42:18.472329+0000) 2024-09-12T03:42:41.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:41 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:41.461+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:04.873875+0000 front 2024-09-12T03:42:04.873833+0000 (oldest deadline 2024-09-12T03:42:28.973822+0000) 2024-09-12T03:42:41.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:41 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:41.461+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-12T03:42:13.575044+0000 front 2024-09-12T03:42:13.575001+0000 (oldest deadline 2024-09-12T03:42:34.674959+0000) 2024-09-12T03:42:42.642 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:42 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:42.191+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:42.643 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:42 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:42.191+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:42.643 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:42 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:42.191+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.525612+0000 front 2024-09-12T03:41:40.525773+0000 (oldest deadline 2024-09-12T03:42:06.425539+0000) 2024-09-12T03:42:42.643 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:42 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:42.191+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:46.425933+0000 front 2024-09-12T03:41:46.426031+0000 (oldest deadline 2024-09-12T03:42:11.725939+0000) 2024-09-12T03:42:42.643 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:42 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:42.191+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:55.226745+0000 front 2024-09-12T03:41:55.226726+0000 (oldest deadline 2024-09-12T03:42:24.027136+0000) 2024-09-12T03:42:42.643 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:42 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:42.191+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:04.027660+0000 front 2024-09-12T03:42:04.027609+0000 (oldest deadline 2024-09-12T03:42:27.527567+0000) 2024-09-12T03:42:42.643 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:42 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:42.192+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-12T03:42:11.429415+0000 front 2024-09-12T03:42:11.429513+0000 (oldest deadline 2024-09-12T03:42:37.329281+0000) 2024-09-12T03:42:42.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:42 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:42.469+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:42.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:42 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:42.469+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:42.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:42 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:42.469+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:42.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:42 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:42.469+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:56.172333+0000 front 2024-09-12T03:41:56.172398+0000 (oldest deadline 2024-09-12T03:42:18.472329+0000) 2024-09-12T03:42:42.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:42 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:42.469+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:04.873875+0000 front 2024-09-12T03:42:04.873833+0000 (oldest deadline 2024-09-12T03:42:28.973822+0000) 2024-09-12T03:42:42.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:42 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:42.469+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-12T03:42:13.575044+0000 front 2024-09-12T03:42:13.575001+0000 (oldest deadline 2024-09-12T03:42:34.674959+0000) 2024-09-12T03:42:43.642 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:43 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:43.213+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:43.642 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:43 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:43.213+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:43.643 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:43 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:43.213+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.525612+0000 front 2024-09-12T03:41:40.525773+0000 (oldest deadline 2024-09-12T03:42:06.425539+0000) 2024-09-12T03:42:43.643 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:43 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:43.213+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:46.425933+0000 front 2024-09-12T03:41:46.426031+0000 (oldest deadline 2024-09-12T03:42:11.725939+0000) 2024-09-12T03:42:43.643 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:43 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:43.213+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:55.226745+0000 front 2024-09-12T03:41:55.226726+0000 (oldest deadline 2024-09-12T03:42:24.027136+0000) 2024-09-12T03:42:43.643 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:43 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:43.213+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:04.027660+0000 front 2024-09-12T03:42:04.027609+0000 (oldest deadline 2024-09-12T03:42:27.527567+0000) 2024-09-12T03:42:43.643 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:43 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:43.213+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-12T03:42:11.429415+0000 front 2024-09-12T03:42:11.429513+0000 (oldest deadline 2024-09-12T03:42:37.329281+0000) 2024-09-12T03:42:43.961 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:43 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:43.513+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:43.961 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:43 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:43.513+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:43.961 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:43 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:43.513+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:43.961 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:43 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:43.513+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:56.172333+0000 front 2024-09-12T03:41:56.172398+0000 (oldest deadline 2024-09-12T03:42:18.472329+0000) 2024-09-12T03:42:43.961 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:43 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:43.513+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:04.873875+0000 front 2024-09-12T03:42:04.873833+0000 (oldest deadline 2024-09-12T03:42:28.973822+0000) 2024-09-12T03:42:43.962 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:43 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:43.513+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-12T03:42:13.575044+0000 front 2024-09-12T03:42:13.575001+0000 (oldest deadline 2024-09-12T03:42:34.674959+0000) 2024-09-12T03:42:43.962 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:43 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:43.513+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6820 osd.7 since back 2024-09-12T03:42:19.375888+0000 front 2024-09-12T03:42:19.375966+0000 (oldest deadline 2024-09-12T03:42:43.376381+0000) 2024-09-12T03:42:44.642 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:44 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:44.172+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:24.823070+0000 front 2024-09-12T03:41:24.823101+0000 (oldest deadline 2024-09-12T03:41:47.122812+0000) 2024-09-12T03:42:44.643 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:44 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:44.173+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.023913+0000 front 2024-09-12T03:41:33.023948+0000 (oldest deadline 2024-09-12T03:41:55.324040+0000) 2024-09-12T03:42:44.643 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:44 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:44.173+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:40.525612+0000 front 2024-09-12T03:41:40.525773+0000 (oldest deadline 2024-09-12T03:42:06.425539+0000) 2024-09-12T03:42:44.643 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:44 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:44.173+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-12T03:41:46.425933+0000 front 2024-09-12T03:41:46.426031+0000 (oldest deadline 2024-09-12T03:42:11.725939+0000) 2024-09-12T03:42:44.643 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:44 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:44.173+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:55.226745+0000 front 2024-09-12T03:41:55.226726+0000 (oldest deadline 2024-09-12T03:42:24.027136+0000) 2024-09-12T03:42:44.643 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:44 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:44.173+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:04.027660+0000 front 2024-09-12T03:42:04.027609+0000 (oldest deadline 2024-09-12T03:42:27.527567+0000) 2024-09-12T03:42:44.643 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:44 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10[58941]: 2024-09-12T03:42:44.173+0000 7f6f0f53c640 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-12T03:42:11.429415+0000 front 2024-09-12T03:42:11.429513+0000 (oldest deadline 2024-09-12T03:42:37.329281+0000) 2024-09-12T03:42:44.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:44 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:44.509+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:44.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:44 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:44.509+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:44.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:44 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:44.509+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:44.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:44 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:44.509+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:56.172333+0000 front 2024-09-12T03:41:56.172398+0000 (oldest deadline 2024-09-12T03:42:18.472329+0000) 2024-09-12T03:42:44.957 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:44 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:44.509+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:04.873875+0000 front 2024-09-12T03:42:04.873833+0000 (oldest deadline 2024-09-12T03:42:28.973822+0000) 2024-09-12T03:42:44.957 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:44 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:44.509+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-12T03:42:13.575044+0000 front 2024-09-12T03:42:13.575001+0000 (oldest deadline 2024-09-12T03:42:34.674959+0000) 2024-09-12T03:42:44.957 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:44 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:44.509+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6820 osd.7 since back 2024-09-12T03:42:19.375888+0000 front 2024-09-12T03:42:19.375966+0000 (oldest deadline 2024-09-12T03:42:43.376381+0000) 2024-09-12T03:42:45.642 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:45 smithi115 podman[70678]: 2024-09-12 03:42:45.307345232 +0000 UTC m=+5.432351761 container died c411a2636075ff2666f57261e2f4dd65e08d5da9d07b5d3759ad1d5cdd293623 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True) 2024-09-12T03:42:45.892 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:45 smithi115 podman[70678]: 2024-09-12 03:42:45.642628084 +0000 UTC m=+5.767634615 container cleanup c411a2636075ff2666f57261e2f4dd65e08d5da9d07b5d3759ad1d5cdd293623 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-12T03:42:45.893 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:45 smithi115 bash[70678]: ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10 2024-09-12T03:42:45.893 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:45 smithi115 podman[70692]: 2024-09-12 03:42:45.844043289 +0000 UTC m=+0.529146326 container remove c411a2636075ff2666f57261e2f4dd65e08d5da9d07b5d3759ad1d5cdd293623 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-12T03:42:45.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:45 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:45.524+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:45.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:45 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:45.524+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:45.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:45 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:45.524+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:45.957 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:45 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:45.524+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:56.172333+0000 front 2024-09-12T03:41:56.172398+0000 (oldest deadline 2024-09-12T03:42:18.472329+0000) 2024-09-12T03:42:45.957 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:45 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:45.524+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:04.873875+0000 front 2024-09-12T03:42:04.873833+0000 (oldest deadline 2024-09-12T03:42:28.973822+0000) 2024-09-12T03:42:45.957 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:45 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:45.524+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-12T03:42:13.575044+0000 front 2024-09-12T03:42:13.575001+0000 (oldest deadline 2024-09-12T03:42:34.674959+0000) 2024-09-12T03:42:45.957 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:45 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:45.524+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6820 osd.7 since back 2024-09-12T03:42:19.375888+0000 front 2024-09-12T03:42:19.375966+0000 (oldest deadline 2024-09-12T03:42:43.376381+0000) 2024-09-12T03:42:46.513 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:46 smithi115 podman[70782]: 2024-09-12 03:42:46.244751443 +0000 UTC m=+0.104592510 container create 9a6ee83b5b96a0e3a4c362fb4c42c8881bbe77a011ea56d16d7cbd48a6a07393 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:42:46.513 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:46 smithi115 podman[70782]: 2024-09-12 03:42:46.156158201 +0000 UTC m=+0.015999221 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:42:46.513 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:46 smithi115 podman[70782]: 2024-09-12 03:42:46.44710896 +0000 UTC m=+0.306949981 container init 9a6ee83b5b96a0e3a4c362fb4c42c8881bbe77a011ea56d16d7cbd48a6a07393 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-12T03:42:46.513 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:46 smithi115 podman[70782]: 2024-09-12 03:42:46.450517775 +0000 UTC m=+0.310358797 container start 9a6ee83b5b96a0e3a4c362fb4c42c8881bbe77a011ea56d16d7cbd48a6a07393 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-12T03:42:46.513 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:46 smithi115 podman[70782]: 2024-09-12 03:42:46.513403687 +0000 UTC m=+0.373244713 container attach 9a6ee83b5b96a0e3a4c362fb4c42c8881bbe77a011ea56d16d7cbd48a6a07393 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:42:46.892 INFO:journalctl@ceph.osd.10.smithi115.stdout:Sep 12 03:42:46 smithi115 podman[70782]: 2024-09-12 03:42:46.639157352 +0000 UTC m=+0.498998376 container died 9a6ee83b5b96a0e3a4c362fb4c42c8881bbe77a011ea56d16d7cbd48a6a07393 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:42:46.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:46 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:46.486+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:46.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:46 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:46.486+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:46.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:46 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:46.486+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:46.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:46 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:46.486+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:56.172333+0000 front 2024-09-12T03:41:56.172398+0000 (oldest deadline 2024-09-12T03:42:18.472329+0000) 2024-09-12T03:42:46.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:46 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:46.486+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:04.873875+0000 front 2024-09-12T03:42:04.873833+0000 (oldest deadline 2024-09-12T03:42:28.973822+0000) 2024-09-12T03:42:46.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:46 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:46.486+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-12T03:42:13.575044+0000 front 2024-09-12T03:42:13.575001+0000 (oldest deadline 2024-09-12T03:42:34.674959+0000) 2024-09-12T03:42:46.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:46 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:46.486+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6820 osd.7 since back 2024-09-12T03:42:19.375888+0000 front 2024-09-12T03:42:19.375966+0000 (oldest deadline 2024-09-12T03:42:43.376381+0000) 2024-09-12T03:42:47.369 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@osd.10.service' 2024-09-12T03:42:47.409 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:42:47.409 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-12T03:42:47.409 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-12T03:42:47.409 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-b0812460-70b5-11ef-bcea-c7b262605968@osd.11 2024-09-12T03:42:47.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:47 smithi195 systemd[1]: Stopping Ceph osd.11 for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:42:47.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:47 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:47.533+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:47.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:47 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:47.533+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:47.706 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:47 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:47.533+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:47.707 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:47 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:47.533+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:56.172333+0000 front 2024-09-12T03:41:56.172398+0000 (oldest deadline 2024-09-12T03:42:18.472329+0000) 2024-09-12T03:42:47.707 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:47 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:47.533+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:04.873875+0000 front 2024-09-12T03:42:04.873833+0000 (oldest deadline 2024-09-12T03:42:28.973822+0000) 2024-09-12T03:42:47.707 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:47 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:47.533+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-12T03:42:13.575044+0000 front 2024-09-12T03:42:13.575001+0000 (oldest deadline 2024-09-12T03:42:34.674959+0000) 2024-09-12T03:42:47.707 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:47 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:47.533+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6820 osd.7 since back 2024-09-12T03:42:19.375888+0000 front 2024-09-12T03:42:19.375966+0000 (oldest deadline 2024-09-12T03:42:43.376381+0000) 2024-09-12T03:42:48.206 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:47 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:47.714+0000 7f97d0b07640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:42:48.206 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:47 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:47.714+0000 7f97d0b07640 -1 osd.11 107 *** Got signal Terminated *** 2024-09-12T03:42:48.206 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:47 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:47.714+0000 7f97d0b07640 -1 osd.11 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:42:48.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:48 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:48.549+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:48.957 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:48 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:48.549+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:48.957 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:48 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:48.549+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:48.957 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:48 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:48.549+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:56.172333+0000 front 2024-09-12T03:41:56.172398+0000 (oldest deadline 2024-09-12T03:42:18.472329+0000) 2024-09-12T03:42:48.957 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:48 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:48.549+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:04.873875+0000 front 2024-09-12T03:42:04.873833+0000 (oldest deadline 2024-09-12T03:42:28.973822+0000) 2024-09-12T03:42:48.957 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:48 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:48.549+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-12T03:42:13.575044+0000 front 2024-09-12T03:42:13.575001+0000 (oldest deadline 2024-09-12T03:42:34.674959+0000) 2024-09-12T03:42:48.957 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:48 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:48.549+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6820 osd.7 since back 2024-09-12T03:42:19.375888+0000 front 2024-09-12T03:42:19.375966+0000 (oldest deadline 2024-09-12T03:42:43.376381+0000) 2024-09-12T03:42:49.858 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:49 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:49.514+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:49.859 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:49 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:49.514+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:49.859 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:49 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:49.514+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:49.859 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:49 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:49.514+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:56.172333+0000 front 2024-09-12T03:41:56.172398+0000 (oldest deadline 2024-09-12T03:42:18.472329+0000) 2024-09-12T03:42:49.859 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:49 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:49.514+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:04.873875+0000 front 2024-09-12T03:42:04.873833+0000 (oldest deadline 2024-09-12T03:42:28.973822+0000) 2024-09-12T03:42:49.859 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:49 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:49.514+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-12T03:42:13.575044+0000 front 2024-09-12T03:42:13.575001+0000 (oldest deadline 2024-09-12T03:42:34.674959+0000) 2024-09-12T03:42:49.859 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:49 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:49.514+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6820 osd.7 since back 2024-09-12T03:42:19.375888+0000 front 2024-09-12T03:42:19.375966+0000 (oldest deadline 2024-09-12T03:42:43.376381+0000) 2024-09-12T03:42:50.879 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:50 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:50.510+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:50.879 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:50 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:50.510+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:50.879 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:50 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:50.510+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:50.879 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:50 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:50.510+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:56.172333+0000 front 2024-09-12T03:41:56.172398+0000 (oldest deadline 2024-09-12T03:42:18.472329+0000) 2024-09-12T03:42:50.879 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:50 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:50.510+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:04.873875+0000 front 2024-09-12T03:42:04.873833+0000 (oldest deadline 2024-09-12T03:42:28.973822+0000) 2024-09-12T03:42:50.879 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:50 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:50.510+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-12T03:42:13.575044+0000 front 2024-09-12T03:42:13.575001+0000 (oldest deadline 2024-09-12T03:42:34.674959+0000) 2024-09-12T03:42:50.880 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:50 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:50.510+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6820 osd.7 since back 2024-09-12T03:42:19.375888+0000 front 2024-09-12T03:42:19.375966+0000 (oldest deadline 2024-09-12T03:42:43.376381+0000) 2024-09-12T03:42:50.880 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:50 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:50.510+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6820 osd.8 since back 2024-09-12T03:42:27.377980+0000 front 2024-09-12T03:42:27.378025+0000 (oldest deadline 2024-09-12T03:42:50.178359+0000) 2024-09-12T03:42:51.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:51 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:51.470+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:51.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:51 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:51.470+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:51.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:51 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:51.470+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:51.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:51 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:51.471+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:56.172333+0000 front 2024-09-12T03:41:56.172398+0000 (oldest deadline 2024-09-12T03:42:18.472329+0000) 2024-09-12T03:42:51.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:51 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:51.471+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:04.873875+0000 front 2024-09-12T03:42:04.873833+0000 (oldest deadline 2024-09-12T03:42:28.973822+0000) 2024-09-12T03:42:51.956 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:51 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:51.471+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-12T03:42:13.575044+0000 front 2024-09-12T03:42:13.575001+0000 (oldest deadline 2024-09-12T03:42:34.674959+0000) 2024-09-12T03:42:51.957 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:51 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:51.471+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6820 osd.7 since back 2024-09-12T03:42:19.375888+0000 front 2024-09-12T03:42:19.375966+0000 (oldest deadline 2024-09-12T03:42:43.376381+0000) 2024-09-12T03:42:51.957 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:51 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:51.471+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6820 osd.8 since back 2024-09-12T03:42:27.377980+0000 front 2024-09-12T03:42:27.378025+0000 (oldest deadline 2024-09-12T03:42:50.178359+0000) 2024-09-12T03:42:52.957 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:52 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:52.458+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-12T03:41:21.966432+0000 front 2024-09-12T03:41:21.966329+0000 (oldest deadline 2024-09-12T03:41:47.266267+0000) 2024-09-12T03:42:52.957 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:52 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:52.458+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-09-12T03:41:33.568451+0000 front 2024-09-12T03:41:33.568466+0000 (oldest deadline 2024-09-12T03:41:57.068420+0000) 2024-09-12T03:42:52.957 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:52 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:52.458+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-09-12T03:41:37.068731+0000 front 2024-09-12T03:41:37.068758+0000 (oldest deadline 2024-09-12T03:42:02.368936+0000) 2024-09-12T03:42:52.957 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:52 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:52.458+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-09-12T03:41:56.172333+0000 front 2024-09-12T03:41:56.172398+0000 (oldest deadline 2024-09-12T03:42:18.472329+0000) 2024-09-12T03:42:52.957 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:52 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:52.458+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-09-12T03:42:04.873875+0000 front 2024-09-12T03:42:04.873833+0000 (oldest deadline 2024-09-12T03:42:28.973822+0000) 2024-09-12T03:42:52.957 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:52 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:52.458+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-12T03:42:13.575044+0000 front 2024-09-12T03:42:13.575001+0000 (oldest deadline 2024-09-12T03:42:34.674959+0000) 2024-09-12T03:42:52.957 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:52 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:52.458+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.115:6820 osd.7 since back 2024-09-12T03:42:19.375888+0000 front 2024-09-12T03:42:19.375966+0000 (oldest deadline 2024-09-12T03:42:43.376381+0000) 2024-09-12T03:42:52.957 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:52 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11[59623]: 2024-09-12T03:42:52.458+0000 7f97cd11d640 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.195:6820 osd.8 since back 2024-09-12T03:42:27.377980+0000 front 2024-09-12T03:42:27.378025+0000 (oldest deadline 2024-09-12T03:42:50.178359+0000) 2024-09-12T03:42:53.515 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:53 smithi195 podman[72428]: 2024-09-12 03:42:53.255354218 +0000 UTC m=+5.649479568 container died 60de4099a24048eb0d40ac588669807c68a0e31f83e8b39c659f8a47a511432c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:42:53.865 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:53 smithi195 podman[72428]: 2024-09-12 03:42:53.614897465 +0000 UTC m=+6.009022819 container cleanup 60de4099a24048eb0d40ac588669807c68a0e31f83e8b39c659f8a47a511432c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-12T03:42:53.865 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:53 smithi195 bash[72428]: ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11 2024-09-12T03:42:54.206 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:53 smithi195 podman[72458]: 2024-09-12 03:42:53.866123506 +0000 UTC m=+0.607878618 container remove 60de4099a24048eb0d40ac588669807c68a0e31f83e8b39c659f8a47a511432c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3) 2024-09-12T03:42:54.485 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:54 smithi195 podman[72546]: 2024-09-12 03:42:54.217006051 +0000 UTC m=+0.105575274 container create a03c2f2b82973e9ef24c9027163b7e007d3bfc42cf56ac9c6c2db7a6c508aaa6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11-deactivate, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:42:54.485 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:54 smithi195 podman[72546]: 2024-09-12 03:42:54.12741413 +0000 UTC m=+0.015983321 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:42:54.485 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:54 smithi195 podman[72546]: 2024-09-12 03:42:54.411607221 +0000 UTC m=+0.300176408 container init a03c2f2b82973e9ef24c9027163b7e007d3bfc42cf56ac9c6c2db7a6c508aaa6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-12T03:42:54.485 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:54 smithi195 podman[72546]: 2024-09-12 03:42:54.415419284 +0000 UTC m=+0.303988465 container start a03c2f2b82973e9ef24c9027163b7e007d3bfc42cf56ac9c6c2db7a6c508aaa6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:42:54.486 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:54 smithi195 podman[72546]: 2024-09-12 03:42:54.485979469 +0000 UTC m=+0.374548651 container attach a03c2f2b82973e9ef24c9027163b7e007d3bfc42cf56ac9c6c2db7a6c508aaa6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11-deactivate, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-12T03:42:54.863 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:54 smithi195 podman[72546]: 2024-09-12 03:42:54.601316565 +0000 UTC m=+0.489885755 container died a03c2f2b82973e9ef24c9027163b7e007d3bfc42cf56ac9c6c2db7a6c508aaa6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-12T03:42:55.123 INFO:journalctl@ceph.osd.11.smithi195.stdout:Sep 12 03:42:55 smithi195 podman[72569]: 2024-09-12 03:42:55.105733123 +0000 UTC m=+0.498775087 container remove a03c2f2b82973e9ef24c9027163b7e007d3bfc42cf56ac9c6c2db7a6c508aaa6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD) 2024-09-12T03:42:55.128 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@osd.11.service' 2024-09-12T03:42:55.166 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:42:55.166 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-12T03:42:55.166 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-12T03:42:55.166 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-b0812460-70b5-11ef-bcea-c7b262605968@mds.a 2024-09-12T03:42:55.486 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 12 03:42:55 smithi064 systemd[1]: Stopping Ceph mds.a for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:42:55.771 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 12 03:42:55 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-a[72157]: 2024-09-12T03:42:55.484+0000 7f90bdea2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:42:55.771 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 12 03:42:55 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-a[72157]: 2024-09-12T03:42:55.484+0000 7f90bdea2640 -1 mds.a *** got signal Terminated *** 2024-09-12T03:42:57.844 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 12 03:42:57 smithi064 podman[94916]: 2024-09-12 03:42:57.557251412 +0000 UTC m=+2.189414532 container died 74a579f1e1e3f7b42dd170ff33c1d2f8c4dd5c8fb7b33939a5b2a19ea66da36a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-12T03:42:58.168 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 12 03:42:57 smithi064 podman[94916]: 2024-09-12 03:42:57.918316909 +0000 UTC m=+2.550480020 container cleanup 74a579f1e1e3f7b42dd170ff33c1d2f8c4dd5c8fb7b33939a5b2a19ea66da36a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-12T03:42:58.169 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 12 03:42:57 smithi064 bash[94916]: ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-a 2024-09-12T03:42:58.318 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@mds.a.service' 2024-09-12T03:42:58.354 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:42:58.355 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-12T03:42:58.355 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-12T03:42:58.355 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-b0812460-70b5-11ef-bcea-c7b262605968@mds.d 2024-09-12T03:42:58.680 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 12 03:42:58 smithi064 systemd[1]: Stopping Ceph mds.d for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:42:59.021 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 12 03:42:58 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-d[70862]: 2024-09-12T03:42:58.679+0000 7f72505fb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:42:59.021 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 12 03:42:58 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-d[70862]: 2024-09-12T03:42:58.679+0000 7f72505fb640 -1 mds.d *** got signal Terminated *** 2024-09-12T03:43:01.257 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 12 03:43:00 smithi064 podman[95036]: 2024-09-12 03:43:00.989817721 +0000 UTC m=+2.405168243 container died 379a334559f53db66547b4c713e13426a992120fdfb8aa9605c19d65ecb01d3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True) 2024-09-12T03:43:01.521 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 12 03:43:01 smithi064 podman[95036]: 2024-09-12 03:43:01.35806444 +0000 UTC m=+2.773414969 container cleanup 379a334559f53db66547b4c713e13426a992120fdfb8aa9605c19d65ecb01d3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-d, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-12T03:43:01.522 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 12 03:43:01 smithi064 bash[95036]: ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-d 2024-09-12T03:43:01.796 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@mds.d.service' 2024-09-12T03:43:01.830 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:43:01.830 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-12T03:43:01.831 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-12T03:43:01.831 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-b0812460-70b5-11ef-bcea-c7b262605968@mds.g 2024-09-12T03:43:02.153 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 12 03:43:01 smithi064 systemd[1]: Stopping Ceph mds.g for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:43:02.521 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 12 03:43:02 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-g[71741]: 2024-09-12T03:43:02.153+0000 7fd226290640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:43:02.522 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 12 03:43:02 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-g[71741]: 2024-09-12T03:43:02.153+0000 7fd226290640 -1 mds.g *** got signal Terminated *** 2024-09-12T03:43:06.271 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 12 03:43:05 smithi064 podman[95158]: 2024-09-12 03:43:05.959471883 +0000 UTC m=+3.899460928 container died 30355fb6af3a02f81de8046bc7af0f96c5f5b3958722b1560692fd41224b39ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-12T03:43:06.578 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 12 03:43:06 smithi064 podman[95158]: 2024-09-12 03:43:06.386552783 +0000 UTC m=+4.326541819 container cleanup 30355fb6af3a02f81de8046bc7af0f96c5f5b3958722b1560692fd41224b39ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-g, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-12T03:43:06.579 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 12 03:43:06 smithi064 bash[95158]: ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-g 2024-09-12T03:43:06.829 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 12 03:43:06 smithi064 podman[95169]: 2024-09-12 03:43:06.579512934 +0000 UTC m=+0.621424103 container remove 30355fb6af3a02f81de8046bc7af0f96c5f5b3958722b1560692fd41224b39ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-g, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-12T03:43:06.935 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@mds.g.service' 2024-09-12T03:43:06.969 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:43:06.969 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-12T03:43:06.970 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-12T03:43:06.970 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-b0812460-70b5-11ef-bcea-c7b262605968@mds.j 2024-09-12T03:43:07.271 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 12 03:43:07 smithi064 systemd[1]: Stopping Ceph mds.j for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:43:07.772 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 12 03:43:07 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-j[70443]: 2024-09-12T03:43:07.417+0000 7ff072173640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:43:07.772 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 12 03:43:07 smithi064 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-j[70443]: 2024-09-12T03:43:07.417+0000 7ff072173640 -1 mds.j *** got signal Terminated *** 2024-09-12T03:43:09.691 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 12 03:43:09 smithi064 podman[95274]: 2024-09-12 03:43:09.43008191 +0000 UTC m=+2.243774827 container died 38ce48cd536e5a8a60c76775c13186460a659a13751e4b5136abfa9566a3669e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-12T03:43:10.021 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 12 03:43:09 smithi064 podman[95274]: 2024-09-12 03:43:09.898936741 +0000 UTC m=+2.712629647 container cleanup 38ce48cd536e5a8a60c76775c13186460a659a13751e4b5136abfa9566a3669e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-j, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-12T03:43:10.022 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 12 03:43:09 smithi064 bash[95274]: ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-j 2024-09-12T03:43:10.290 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@mds.j.service' 2024-09-12T03:43:10.322 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:43:10.322 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-12T03:43:10.322 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-12T03:43:10.322 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-b0812460-70b5-11ef-bcea-c7b262605968@mds.b 2024-09-12T03:43:10.642 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 12 03:43:10 smithi115 systemd[1]: Stopping Ceph mds.b for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:43:11.142 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 12 03:43:10 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-b[64111]: 2024-09-12T03:43:10.681+0000 7f242b324640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:43:11.142 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 12 03:43:10 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-b[64111]: 2024-09-12T03:43:10.681+0000 7f242b324640 -1 mds.b *** got signal Terminated *** 2024-09-12T03:43:15.056 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 12 03:43:14 smithi115 podman[70884]: 2024-09-12 03:43:14.695357597 +0000 UTC m=+4.183415337 container died 66e746a883690558d4d72d2fb1dcc9cc5545824a5c263c8363f530a150594b4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-b, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:43:15.392 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 12 03:43:15 smithi115 podman[70884]: 2024-09-12 03:43:15.139366664 +0000 UTC m=+4.627424396 container cleanup 66e746a883690558d4d72d2fb1dcc9cc5545824a5c263c8363f530a150594b4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:43:15.392 INFO:journalctl@ceph.mds.b.smithi115.stdout:Sep 12 03:43:15 smithi115 bash[70884]: ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-b 2024-09-12T03:43:15.594 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@mds.b.service' 2024-09-12T03:43:15.629 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:43:15.630 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-12T03:43:15.630 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-12T03:43:15.630 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-b0812460-70b5-11ef-bcea-c7b262605968@mds.e 2024-09-12T03:43:16.142 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 12 03:43:15 smithi115 systemd[1]: Stopping Ceph mds.e for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:43:16.143 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 12 03:43:15 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-e[63509]: 2024-09-12T03:43:15.934+0000 7f30d2a7d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:43:16.143 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 12 03:43:15 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-e[63509]: 2024-09-12T03:43:15.934+0000 7f30d2a7d640 -1 mds.e *** got signal Terminated *** 2024-09-12T03:43:19.801 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 12 03:43:19 smithi115 podman[71016]: 2024-09-12 03:43:19.550357035 +0000 UTC m=+3.701756583 container died e75ad4f3457546d974c8bdcfba8dcbbfac057c087cf2e3ca5c4d7fecf7a1a378 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-e, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-12T03:43:20.067 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 12 03:43:19 smithi115 podman[71016]: 2024-09-12 03:43:19.875493074 +0000 UTC m=+4.026892615 container cleanup e75ad4f3457546d974c8bdcfba8dcbbfac057c087cf2e3ca5c4d7fecf7a1a378 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-09-12T03:43:20.067 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 12 03:43:19 smithi115 bash[71016]: ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-e 2024-09-12T03:43:20.300 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@mds.e.service' 2024-09-12T03:43:20.321 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 12 03:43:20 smithi115 podman[71030]: 2024-09-12 03:43:20.068172465 +0000 UTC m=+0.510265999 container remove e75ad4f3457546d974c8bdcfba8dcbbfac057c087cf2e3ca5c4d7fecf7a1a378 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2) 2024-09-12T03:43:20.321 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 12 03:43:20 smithi115 systemd[1]: ceph-b0812460-70b5-11ef-bcea-c7b262605968@mds.e.service: Deactivated successfully. 2024-09-12T03:43:20.321 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 12 03:43:20 smithi115 systemd[1]: Stopped Ceph mds.e for b0812460-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:43:20.321 INFO:journalctl@ceph.mds.e.smithi115.stdout:Sep 12 03:43:20 smithi115 systemd[1]: ceph-b0812460-70b5-11ef-bcea-c7b262605968@mds.e.service: Consumed 2.266s CPU time. 2024-09-12T03:43:20.332 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:43:20.333 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-12T03:43:20.333 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-12T03:43:20.333 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-b0812460-70b5-11ef-bcea-c7b262605968@mds.h 2024-09-12T03:43:20.642 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 12 03:43:20 smithi115 systemd[1]: Stopping Ceph mds.h for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:43:21.142 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 12 03:43:20 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-h[63810]: 2024-09-12T03:43:20.670+0000 7f7200f5f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:43:21.142 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 12 03:43:20 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-h[63810]: 2024-09-12T03:43:20.670+0000 7f7200f5f640 -1 mds.h *** got signal Terminated *** 2024-09-12T03:43:23.878 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 12 03:43:23 smithi115 podman[71138]: 2024-09-12 03:43:23.547032793 +0000 UTC m=+2.999023172 container died d0ac6da922dcbe63728c1d2bdbc27d07befd2d84980bc17af774f35fbfcff5d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-h, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-12T03:43:24.142 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 12 03:43:23 smithi115 podman[71138]: 2024-09-12 03:43:23.978810168 +0000 UTC m=+3.430800547 container cleanup d0ac6da922dcbe63728c1d2bdbc27d07befd2d84980bc17af774f35fbfcff5d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-12T03:43:24.143 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 12 03:43:23 smithi115 bash[71138]: ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-h 2024-09-12T03:43:24.402 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@mds.h.service' 2024-09-12T03:43:24.423 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 12 03:43:24 smithi115 podman[71153]: 2024-09-12 03:43:24.163187489 +0000 UTC m=+0.606702336 container remove d0ac6da922dcbe63728c1d2bdbc27d07befd2d84980bc17af774f35fbfcff5d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-h, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-12T03:43:24.423 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 12 03:43:24 smithi115 systemd[1]: ceph-b0812460-70b5-11ef-bcea-c7b262605968@mds.h.service: Deactivated successfully. 2024-09-12T03:43:24.423 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 12 03:43:24 smithi115 systemd[1]: Stopped Ceph mds.h for b0812460-70b5-11ef-bcea-c7b262605968. 2024-09-12T03:43:24.423 INFO:journalctl@ceph.mds.h.smithi115.stdout:Sep 12 03:43:24 smithi115 systemd[1]: ceph-b0812460-70b5-11ef-bcea-c7b262605968@mds.h.service: Consumed 2.047s CPU time. 2024-09-12T03:43:24.434 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:43:24.434 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-12T03:43:24.434 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-12T03:43:24.435 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-b0812460-70b5-11ef-bcea-c7b262605968@mds.k 2024-09-12T03:43:24.798 INFO:journalctl@ceph.mds.k.smithi115.stdout:Sep 12 03:43:24 smithi115 systemd[1]: Stopping Ceph mds.k for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:43:25.142 INFO:journalctl@ceph.mds.k.smithi115.stdout:Sep 12 03:43:24 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-k[63196]: 2024-09-12T03:43:24.798+0000 7fbc60d96640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:43:25.142 INFO:journalctl@ceph.mds.k.smithi115.stdout:Sep 12 03:43:24 smithi115 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-k[63196]: 2024-09-12T03:43:24.798+0000 7fbc60d96640 -1 mds.k *** got signal Terminated *** 2024-09-12T03:43:26.607 INFO:journalctl@ceph.mds.k.smithi115.stdout:Sep 12 03:43:26 smithi115 podman[71261]: 2024-09-12 03:43:26.334450284 +0000 UTC m=+1.684690615 container died cf7a19fc47bbeb338ffeeeade29b9fdc3ea8a3dafb33b591bce89c85fc1e2199 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-k, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-12T03:43:26.865 INFO:journalctl@ceph.mds.k.smithi115.stdout:Sep 12 03:43:26 smithi115 podman[71261]: 2024-09-12 03:43:26.706802439 +0000 UTC m=+2.057042757 container cleanup cf7a19fc47bbeb338ffeeeade29b9fdc3ea8a3dafb33b591bce89c85fc1e2199 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-k, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0) 2024-09-12T03:43:26.865 INFO:journalctl@ceph.mds.k.smithi115.stdout:Sep 12 03:43:26 smithi115 bash[71261]: ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-k 2024-09-12T03:43:27.073 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@mds.k.service' 2024-09-12T03:43:27.105 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:43:27.105 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-12T03:43:27.105 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-12T03:43:27.105 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-b0812460-70b5-11ef-bcea-c7b262605968@mds.c 2024-09-12T03:43:27.405 INFO:journalctl@ceph.mds.c.smithi195.stdout:Sep 12 03:43:27 smithi195 systemd[1]: Stopping Ceph mds.c for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:43:27.659 INFO:journalctl@ceph.mds.c.smithi195.stdout:Sep 12 03:43:27 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-c[64828]: 2024-09-12T03:43:27.404+0000 7fb67a69b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:43:27.659 INFO:journalctl@ceph.mds.c.smithi195.stdout:Sep 12 03:43:27 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-c[64828]: 2024-09-12T03:43:27.404+0000 7fb67a69b640 -1 mds.c *** got signal Terminated *** 2024-09-12T03:43:31.706 INFO:journalctl@ceph.mds.c.smithi195.stdout:Sep 12 03:43:31 smithi195 podman[72648]: 2024-09-12 03:43:31.418202975 +0000 UTC m=+4.112200545 container died 1c5adae5c26de23ebc8ca5c3fbcb3410b5e82f95d73cc28190cd84db98806983 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-c, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3) 2024-09-12T03:43:32.191 INFO:journalctl@ceph.mds.c.smithi195.stdout:Sep 12 03:43:31 smithi195 podman[72648]: 2024-09-12 03:43:31.780108007 +0000 UTC m=+4.474105589 container cleanup 1c5adae5c26de23ebc8ca5c3fbcb3410b5e82f95d73cc28190cd84db98806983 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-c, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-12T03:43:32.191 INFO:journalctl@ceph.mds.c.smithi195.stdout:Sep 12 03:43:31 smithi195 bash[72648]: ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-c 2024-09-12T03:43:32.191 INFO:journalctl@ceph.mds.c.smithi195.stdout:Sep 12 03:43:31 smithi195 podman[72672]: 2024-09-12 03:43:31.956226442 +0000 UTC m=+0.534365361 container remove 1c5adae5c26de23ebc8ca5c3fbcb3410b5e82f95d73cc28190cd84db98806983 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-c, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-12T03:43:32.212 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@mds.c.service' 2024-09-12T03:43:32.247 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:43:32.247 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-12T03:43:32.247 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-12T03:43:32.248 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-b0812460-70b5-11ef-bcea-c7b262605968@mds.f 2024-09-12T03:43:32.559 INFO:journalctl@ceph.mds.f.smithi195.stdout:Sep 12 03:43:32 smithi195 systemd[1]: Stopping Ceph mds.f for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:43:32.956 INFO:journalctl@ceph.mds.f.smithi195.stdout:Sep 12 03:43:32 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-f[64528]: 2024-09-12T03:43:32.559+0000 7f69dd5d7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:43:32.956 INFO:journalctl@ceph.mds.f.smithi195.stdout:Sep 12 03:43:32 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-f[64528]: 2024-09-12T03:43:32.559+0000 7f69dd5d7640 -1 mds.f *** got signal Terminated *** 2024-09-12T03:43:35.824 INFO:journalctl@ceph.mds.f.smithi195.stdout:Sep 12 03:43:35 smithi195 podman[72780]: 2024-09-12 03:43:35.538430124 +0000 UTC m=+3.069638565 container died ccaea7d954ed2cec35084927ab1806dc44b6fcdbe0aebf1253005ebad4f86cef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-f, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-12T03:43:35.824 INFO:journalctl@ceph.mds.f.smithi195.stdout:Sep 12 03:43:35 smithi195 podman[72780]: 2024-09-12 03:43:35.825284045 +0000 UTC m=+3.356492489 container cleanup ccaea7d954ed2cec35084927ab1806dc44b6fcdbe0aebf1253005ebad4f86cef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-f, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-12T03:43:36.182 INFO:journalctl@ceph.mds.f.smithi195.stdout:Sep 12 03:43:35 smithi195 bash[72780]: ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-f 2024-09-12T03:43:36.182 INFO:journalctl@ceph.mds.f.smithi195.stdout:Sep 12 03:43:36 smithi195 podman[72793]: 2024-09-12 03:43:36.067975946 +0000 UTC m=+0.523671081 container remove ccaea7d954ed2cec35084927ab1806dc44b6fcdbe0aebf1253005ebad4f86cef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-f, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:43:36.210 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@mds.f.service' 2024-09-12T03:43:36.243 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:43:36.243 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-12T03:43:36.243 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-12T03:43:36.243 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-b0812460-70b5-11ef-bcea-c7b262605968@mds.i 2024-09-12T03:43:36.553 INFO:journalctl@ceph.mds.i.smithi195.stdout:Sep 12 03:43:36 smithi195 systemd[1]: Stopping Ceph mds.i for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:43:36.956 INFO:journalctl@ceph.mds.i.smithi195.stdout:Sep 12 03:43:36 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-i[65131]: 2024-09-12T03:43:36.553+0000 7f2860f94640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:43:36.956 INFO:journalctl@ceph.mds.i.smithi195.stdout:Sep 12 03:43:36 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-i[65131]: 2024-09-12T03:43:36.553+0000 7f2860f94640 -1 mds.i *** got signal Terminated *** 2024-09-12T03:43:39.812 INFO:journalctl@ceph.mds.i.smithi195.stdout:Sep 12 03:43:39 smithi195 podman[72901]: 2024-09-12 03:43:39.539005438 +0000 UTC m=+3.081135384 container died f47840627fc84e43c4f48380f9eb982bb04de86a97db8e0352d4d7aca2249d57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-i, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-12T03:43:40.120 INFO:journalctl@ceph.mds.i.smithi195.stdout:Sep 12 03:43:39 smithi195 podman[72901]: 2024-09-12 03:43:39.912020178 +0000 UTC m=+3.454150113 container cleanup f47840627fc84e43c4f48380f9eb982bb04de86a97db8e0352d4d7aca2249d57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-i, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-12T03:43:40.121 INFO:journalctl@ceph.mds.i.smithi195.stdout:Sep 12 03:43:39 smithi195 bash[72901]: ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-i 2024-09-12T03:43:40.256 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@mds.i.service' 2024-09-12T03:43:40.288 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:43:40.289 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-12T03:43:40.289 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-12T03:43:40.289 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-b0812460-70b5-11ef-bcea-c7b262605968@mds.l 2024-09-12T03:43:40.456 INFO:journalctl@ceph.mds.l.smithi195.stdout:Sep 12 03:43:40 smithi195 systemd[1]: Stopping Ceph mds.l for b0812460-70b5-11ef-bcea-c7b262605968... 2024-09-12T03:43:40.956 INFO:journalctl@ceph.mds.l.smithi195.stdout:Sep 12 03:43:40 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-l[64228]: 2024-09-12T03:43:40.607+0000 7eff7a2ab640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:43:40.956 INFO:journalctl@ceph.mds.l.smithi195.stdout:Sep 12 03:43:40 smithi195 ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-l[64228]: 2024-09-12T03:43:40.607+0000 7eff7a2ab640 -1 mds.l *** got signal Terminated *** 2024-09-12T03:43:44.083 INFO:journalctl@ceph.mds.l.smithi195.stdout:Sep 12 03:43:43 smithi195 podman[73023]: 2024-09-12 03:43:43.752210104 +0000 UTC m=+3.250144149 container died bdd5cfba81e90cc0dfcbe2fed1c1cd4bc6dbb1ec97aa24225af47e28b5e4ecac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:43:44.365 INFO:journalctl@ceph.mds.l.smithi195.stdout:Sep 12 03:43:44 smithi195 podman[73023]: 2024-09-12 03:43:44.157100268 +0000 UTC m=+3.655034314 container cleanup bdd5cfba81e90cc0dfcbe2fed1c1cd4bc6dbb1ec97aa24225af47e28b5e4ecac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-l, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux ) 2024-09-12T03:43:44.366 INFO:journalctl@ceph.mds.l.smithi195.stdout:Sep 12 03:43:44 smithi195 bash[73023]: ceph-b0812460-70b5-11ef-bcea-c7b262605968-mds-l 2024-09-12T03:43:44.500 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0812460-70b5-11ef-bcea-c7b262605968@mds.l.service' 2024-09-12T03:43:44.532 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:43:44.532 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-12T03:43:44.532 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b0812460-70b5-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T03:43:44.746 INFO:teuthology.orchestra.run.smithi064.stdout:Deleting cluster with fsid: b0812460-70b5-11ef-bcea-c7b262605968 2024-09-12T03:43:47.247 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b0812460-70b5-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T03:43:47.449 INFO:teuthology.orchestra.run.smithi115.stdout:Deleting cluster with fsid: b0812460-70b5-11ef-bcea-c7b262605968 2024-09-12T03:43:49.898 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b0812460-70b5-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T03:43:50.100 INFO:teuthology.orchestra.run.smithi195.stdout:Deleting cluster with fsid: b0812460-70b5-11ef-bcea-c7b262605968 2024-09-12T03:43:52.436 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:43:52.469 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:43:52.503 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:43:52.533 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-12T03:43:52.535 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901679/remote/smithi064/crash 2024-09-12T03:43:52.536 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/crash -- . 2024-09-12T03:43:52.577 INFO:teuthology.orchestra.run.smithi064.stderr:tar: /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T03:43:52.577 INFO:teuthology.orchestra.run.smithi064.stderr:tar: Error is not recoverable: exiting now 2024-09-12T03:43:52.579 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901679/remote/smithi115/crash 2024-09-12T03:43:52.580 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/crash -- . 2024-09-12T03:43:52.681 INFO:teuthology.orchestra.run.smithi115.stderr:tar: /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T03:43:52.681 INFO:teuthology.orchestra.run.smithi115.stderr:tar: Error is not recoverable: exiting now 2024-09-12T03:43:52.683 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901679/remote/smithi195/crash 2024-09-12T03:43:52.684 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/crash -- . 2024-09-12T03:43:52.721 INFO:teuthology.orchestra.run.smithi195.stderr:tar: /var/lib/ceph/b0812460-70b5-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T03:43:52.721 INFO:teuthology.orchestra.run.smithi195.stderr:tar: Error is not recoverable: exiting now 2024-09-12T03:43:52.722 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-12T03:43:52.723 DEBUG:teuthology.orchestra.run.smithi064:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-12T03:43:52.761 INFO:tasks.cephadm:Compressing logs... 2024-09-12T03:43:52.762 DEBUG:teuthology.orchestra.run.smithi064:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T03:43:52.804 DEBUG:teuthology.orchestra.run.smithi115:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T03:43:52.808 DEBUG:teuthology.orchestra.run.smithi195:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T03:43:52.831 INFO:teuthology.orchestra.run.smithi064.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-09-12T03:43:52.831 INFO:teuthology.orchestra.run.smithi064.stderr:: No such file or directory 2024-09-12T03:43:52.831 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-12T03:43:52.833 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-osd.9.log 2024-09-12T03:43:52.833 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T03:43:52.833 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T03:43:52.833 INFO:teuthology.orchestra.run.smithi064.stderr: 90.6%/var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-volume.log: -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-09-12T03:43:52.833 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mds.a.log 2024-09-12T03:43:52.833 INFO:teuthology.orchestra.run.smithi064.stderr:gzip/var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph.audit.log: -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T03:43:52.839 INFO:teuthology.orchestra.run.smithi115.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-osd.10.log 2024-09-12T03:43:52.839 INFO:teuthology.orchestra.run.smithi115.stderr:: No such file or directory 2024-09-12T03:43:52.839 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-12T03:43:52.839 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-12T03:43:52.839 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T03:43:52.839 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mds.b.log 2024-09-12T03:43:52.839 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mds.k.log 2024-09-12T03:43:52.839 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-12T03:43:52.839 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T03:43:52.839 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T03:43:52.850 INFO:teuthology.orchestra.run.smithi195.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T03:43:52.851 INFO:teuthology.orchestra.run.smithi195.stderr:: No such file or directory 2024-09-12T03:43:52.851 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T03:43:52.851 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mds.i.log 2024-09-12T03:43:52.851 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-12T03:43:52.851 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T03:43:52.851 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-osd.11.log 2024-09-12T03:43:52.851 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph.audit.log: /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mds.i.log: /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mon.c.log: /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-12T03:43:52.851 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-12T03:43:52.851 INFO:teuthology.orchestra.run.smithi195.stderr: 82.4% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T03:43:52.851 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph.log 2024-09-12T03:43:52.852 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mds.l.log 2024-09-12T03:43:52.852 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-osd.5.log: /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph.log: 90.4% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T03:43:52.852 INFO:teuthology.orchestra.run.smithi195.stderr: 87.4% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T03:43:52.852 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-osd.8.log 2024-09-12T03:43:52.852 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mgr.z.log 2024-09-12T03:43:52.852 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-osd.8.log: 94.7% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mds.l.log.gz 2024-09-12T03:43:52.852 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mds.c.log 2024-09-12T03:43:52.852 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mgr.z.log: 90.9% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T03:43:52.853 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph.audit.log: 90.4% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T03:43:52.853 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-12T03:43:52.853 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph.cephadm.log: 94.5% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mds.k.log.gz 2024-09-12T03:43:52.854 INFO:teuthology.orchestra.run.smithi115.stderr: 82.4% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T03:43:52.854 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph.log 2024-09-12T03:43:52.854 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mds.h.log 2024-09-12T03:43:52.854 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph.log: 87.4% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T03:43:52.854 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-12T03:43:52.854 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mds.h.log: 90.9% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T03:43:52.854 INFO:teuthology.orchestra.run.smithi115.stderr: 92.0% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-12T03:43:52.855 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mds.f.log 2024-09-12T03:43:52.856 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-12T03:43:52.856 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mds.g.log 2024-09-12T03:43:52.856 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mon.a.log: 84.8% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T03:43:52.856 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-12T03:43:52.856 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph.log 2024-09-12T03:43:52.856 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mds.g.log: /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-osd.6.log: 90.3% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T03:43:52.856 INFO:teuthology.orchestra.run.smithi064.stderr: 94.6% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mds.a.log.gz 2024-09-12T03:43:52.856 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mds.d.log 2024-09-12T03:43:52.856 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-12T03:43:52.856 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mds.g.log.gz 2024-09-12T03:43:52.857 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mds.j.log 2024-09-12T03:43:52.857 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mgr.x.log: 87.4% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T03:43:52.857 INFO:teuthology.orchestra.run.smithi064.stderr: 94.6% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mds.d.log.gz 2024-09-12T03:43:52.857 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-12T03:43:52.857 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-09-12T03:43:52.857 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-osd.3.log: 91.1% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T03:43:52.857 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T03:43:52.857 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.0.log: 94.7% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mds.j.log.gz 2024-09-12T03:43:52.858 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mds.c.log: 92.0% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mgr.z.log.gz 2024-09-12T03:43:52.858 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T03:43:52.859 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mds.e.log 2024-09-12T03:43:52.859 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T03:43:52.871 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T03:43:52.872 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T03:43:52.873 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/cephadm.log: 93.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T03:43:52.927 INFO:teuthology.orchestra.run.smithi195.stderr: 92.7% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mds.i.log.gz 2024-09-12T03:43:52.928 INFO:teuthology.orchestra.run.smithi195.stderr: 94.6% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mds.f.log.gz 2024-09-12T03:43:52.928 INFO:teuthology.orchestra.run.smithi115.stderr: 94.4% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mds.e.log.gz 2024-09-12T03:43:52.935 INFO:teuthology.orchestra.run.smithi115.stderr: 92.7% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mds.h.log.gz 2024-09-12T03:43:53.140 INFO:teuthology.orchestra.run.smithi195.stderr: 94.3% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mds.c.log.gz 2024-09-12T03:43:53.224 INFO:teuthology.orchestra.run.smithi064.stderr: 90.2% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-12T03:43:53.233 INFO:teuthology.orchestra.run.smithi195.stderr: 91.7% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-12T03:43:53.276 INFO:teuthology.orchestra.run.smithi064.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-09-12T03:43:53.287 INFO:teuthology.orchestra.run.smithi115.stderr: 92.0% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-12T03:43:53.811 INFO:teuthology.orchestra.run.smithi115.stderr: 92.2% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-12T03:43:53.829 INFO:teuthology.orchestra.run.smithi064.stderr: 90.6% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-12T03:43:54.062 INFO:teuthology.orchestra.run.smithi195.stderr: 91.9% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-osd.11.log.gz 2024-09-12T03:43:54.089 INFO:teuthology.orchestra.run.smithi064.stderr: 92.0% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-osd.9.log.gz 2024-09-12T03:43:54.155 INFO:teuthology.orchestra.run.smithi064.stderr: 93.1% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-12T03:43:54.181 INFO:teuthology.orchestra.run.smithi195.stderr: 91.9% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-osd.8.log.gz 2024-09-12T03:43:54.414 INFO:teuthology.orchestra.run.smithi115.stderr: 91.8% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-12T03:43:54.521 INFO:teuthology.orchestra.run.smithi115.stderr: 92.9% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-12T03:43:54.543 INFO:teuthology.orchestra.run.smithi064.stderr: 92.8% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-12T03:43:54.601 INFO:teuthology.orchestra.run.smithi064.stderr: 93.0% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-12T03:43:54.603 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-09-12T03:43:54.603 INFO:teuthology.orchestra.run.smithi064.stderr:real 0m1.784s 2024-09-12T03:43:54.603 INFO:teuthology.orchestra.run.smithi064.stderr:user 0m7.644s 2024-09-12T03:43:54.604 INFO:teuthology.orchestra.run.smithi064.stderr:sys 0m0.277s 2024-09-12T03:43:54.645 INFO:teuthology.orchestra.run.smithi195.stderr: 91.7% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-12T03:43:54.888 INFO:teuthology.orchestra.run.smithi115.stderr: 92.9% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-mds.b.log.gz 2024-09-12T03:43:54.960 INFO:teuthology.orchestra.run.smithi115.stderr: 92.1% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-osd.10.log.gz 2024-09-12T03:43:54.962 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-09-12T03:43:54.962 INFO:teuthology.orchestra.run.smithi115.stderr:real 0m2.139s 2024-09-12T03:43:54.962 INFO:teuthology.orchestra.run.smithi115.stderr:user 0m8.734s 2024-09-12T03:43:54.962 INFO:teuthology.orchestra.run.smithi115.stderr:sys 0m0.323s 2024-09-12T03:43:55.448 INFO:teuthology.orchestra.run.smithi195.stderr: 92.0% -- replaced with /var/log/ceph/b0812460-70b5-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-12T03:43:55.450 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-09-12T03:43:55.450 INFO:teuthology.orchestra.run.smithi195.stderr:real 0m2.625s 2024-09-12T03:43:55.451 INFO:teuthology.orchestra.run.smithi195.stderr:user 0m7.587s 2024-09-12T03:43:55.451 INFO:teuthology.orchestra.run.smithi195.stderr:sys 0m0.286s 2024-09-12T03:43:55.451 INFO:tasks.cephadm:Archiving logs... 2024-09-12T03:43:55.451 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901679/remote/smithi064/log 2024-09-12T03:43:55.452 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T03:43:56.595 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901679/remote/smithi115/log 2024-09-12T03:43:56.596 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T03:43:57.927 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901679/remote/smithi195/log 2024-09-12T03:43:57.928 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T03:43:59.211 INFO:tasks.cephadm:Removing cluster... 2024-09-12T03:43:59.211 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b0812460-70b5-11ef-bcea-c7b262605968 --force 2024-09-12T03:43:59.422 INFO:teuthology.orchestra.run.smithi064.stdout:Deleting cluster with fsid: b0812460-70b5-11ef-bcea-c7b262605968 2024-09-12T03:43:59.684 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b0812460-70b5-11ef-bcea-c7b262605968 --force 2024-09-12T03:43:59.875 INFO:teuthology.orchestra.run.smithi115.stdout:Deleting cluster with fsid: b0812460-70b5-11ef-bcea-c7b262605968 2024-09-12T03:44:00.142 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b0812460-70b5-11ef-bcea-c7b262605968 --force 2024-09-12T03:44:00.333 INFO:teuthology.orchestra.run.smithi195.stdout:Deleting cluster with fsid: b0812460-70b5-11ef-bcea-c7b262605968 2024-09-12T03:44:00.622 INFO:tasks.cephadm:Removing cephadm ... 2024-09-12T03:44:00.623 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T03:44:00.641 DEBUG:teuthology.orchestra.run.smithi115:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T03:44:00.657 DEBUG:teuthology.orchestra.run.smithi195:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T03:44:00.673 INFO:tasks.cephadm:Teardown complete 2024-09-12T03:44:00.673 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-12T03:44:00.684 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-12T03:44:00.685 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T03:44:00.687 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T03:44:00.700 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T03:44:00.840 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-12T03:44:01.065 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-12T03:44:01.065 DEBUG:teuthology.orchestra.run.smithi064:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T03:44:01.069 DEBUG:teuthology.orchestra.run.smithi115:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T03:44:01.072 DEBUG:teuthology.orchestra.run.smithi195:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T03:44:01.084 INFO:teuthology.orchestra.run.smithi115.stderr:bash: line 1: ntpq: command not found 2024-09-12T03:44:01.085 INFO:teuthology.orchestra.run.smithi064.stderr:bash: line 1: ntpq: command not found 2024-09-12T03:44:01.086 INFO:teuthology.orchestra.run.smithi195.stderr:bash: line 1: ntpq: command not found 2024-09-12T03:44:01.090 INFO:teuthology.orchestra.run.smithi115.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T03:44:01.090 INFO:teuthology.orchestra.run.smithi115.stdout:=============================================================================== 2024-09-12T03:44:01.090 INFO:teuthology.orchestra.run.smithi115.stdout:^* hv01.front.sepia.ceph.com 3 6 377 18 -2877us[-2891us] +/- 63ms 2024-09-12T03:44:01.090 INFO:teuthology.orchestra.run.smithi115.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 18 +1595us[+1581us] +/- 63ms 2024-09-12T03:44:01.090 INFO:teuthology.orchestra.run.smithi115.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 16 +2232us[+2232us] +/- 88ms 2024-09-12T03:44:01.090 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:44:01.092 INFO:teuthology.orchestra.run.smithi195.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T03:44:01.092 INFO:teuthology.orchestra.run.smithi195.stdout:=============================================================================== 2024-09-12T03:44:01.092 INFO:teuthology.orchestra.run.smithi195.stdout:^* hv01.front.sepia.ceph.com 3 6 377 15 -3012us[-2995us] +/- 63ms 2024-09-12T03:44:01.092 INFO:teuthology.orchestra.run.smithi195.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 18 +1462us[+1480us] +/- 63ms 2024-09-12T03:44:01.092 INFO:teuthology.orchestra.run.smithi195.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 15 +2084us[+2102us] +/- 88ms 2024-09-12T03:44:01.092 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:44:01.093 INFO:teuthology.orchestra.run.smithi064.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T03:44:01.093 INFO:teuthology.orchestra.run.smithi064.stdout:=============================================================================== 2024-09-12T03:44:01.093 INFO:teuthology.orchestra.run.smithi064.stdout:^* hv01.front.sepia.ceph.com 3 6 377 16 -2935us[-2932us] +/- 63ms 2024-09-12T03:44:01.093 INFO:teuthology.orchestra.run.smithi064.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 14 +1544us[+1544us] +/- 63ms 2024-09-12T03:44:01.093 INFO:teuthology.orchestra.run.smithi064.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 16 +2190us[+2193us] +/- 88ms 2024-09-12T03:44:01.093 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:44:01.094 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-12T03:44:01.777 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-12T03:44:01.779 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-12T03:44:01.799 DEBUG:teuthology.orchestra.run.smithi064:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T03:44:01.886 DEBUG:teuthology.orchestra.run.smithi115:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T03:44:01.961 DEBUG:teuthology.orchestra.run.smithi195:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T03:44:02.036 DEBUG:teuthology.orchestra.run.smithi064:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T03:44:02.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:44:02.070 DEBUG:teuthology.orchestra.run.smithi115:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T03:44:02.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:44:02.101 DEBUG:teuthology.orchestra.run.smithi195:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T03:44:02.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:44:02.131 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-12T03:44:02.141 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-12T03:44:02.151 INFO:teuthology.task.internal:Duration was 1870.432662 seconds 2024-09-12T03:44:02.151 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-12T03:44:02.197 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-12T03:44:02.197 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T03:44:02.200 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T03:44:02.202 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T03:44:02.241 INFO:teuthology.orchestra.run.smithi115.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T03:44:02.243 INFO:teuthology.orchestra.run.smithi195.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T03:44:02.243 INFO:teuthology.orchestra.run.smithi064.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T03:44:02.696 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-12T03:44:02.696 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi064.front.sepia.ceph.com 2024-09-12T03:44:02.696 DEBUG:teuthology.orchestra.run.smithi064:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T03:44:02.721 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi115.front.sepia.ceph.com 2024-09-12T03:44:02.722 DEBUG:teuthology.orchestra.run.smithi115:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T03:44:02.759 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi195.front.sepia.ceph.com 2024-09-12T03:44:02.760 DEBUG:teuthology.orchestra.run.smithi195:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T03:44:02.783 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-12T03:44:02.783 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T03:44:02.786 DEBUG:teuthology.orchestra.run.smithi115:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T03:44:02.802 DEBUG:teuthology.orchestra.run.smithi195:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T03:44:02.851 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-12T03:44:02.851 DEBUG:teuthology.orchestra.run.smithi064:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T03:44:02.858 DEBUG:teuthology.orchestra.run.smithi115:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T03:44:02.870 DEBUG:teuthology.orchestra.run.smithi195:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T03:44:03.382 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-12T03:44:03.394 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-12T03:44:03.394 DEBUG:teuthology.orchestra.run.smithi064:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T03:44:03.420 DEBUG:teuthology.orchestra.run.smithi115:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T03:44:03.445 DEBUG:teuthology.orchestra.run.smithi195:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T03:44:03.471 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-12T03:44:03.482 DEBUG:teuthology.orchestra.run.smithi064:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:44:03.484 DEBUG:teuthology.orchestra.run.smithi115:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:44:03.488 DEBUG:teuthology.orchestra.run.smithi195:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:44:03.506 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = core 2024-09-12T03:44:03.509 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern = core 2024-09-12T03:44:03.534 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern = core 2024-09-12T03:44:03.557 DEBUG:teuthology.orchestra.run.smithi064:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:44:03.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:44:03.576 DEBUG:teuthology.orchestra.run.smithi115:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:44:03.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:44:03.613 DEBUG:teuthology.orchestra.run.smithi195:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:44:03.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:44:03.627 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-12T03:44:03.639 INFO:teuthology.task.internal:Transferring archived files... 2024-09-12T03:44:03.640 DEBUG:teuthology.misc:Transferring archived files from smithi064:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901679/remote/smithi064 2024-09-12T03:44:03.640 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T03:44:03.686 DEBUG:teuthology.misc:Transferring archived files from smithi115:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901679/remote/smithi115 2024-09-12T03:44:03.686 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T03:44:03.722 DEBUG:teuthology.misc:Transferring archived files from smithi195:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901679/remote/smithi195 2024-09-12T03:44:03.723 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T03:44:03.759 INFO:teuthology.task.internal:Removing archive directory... 2024-09-12T03:44:03.759 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T03:44:03.762 DEBUG:teuthology.orchestra.run.smithi115:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T03:44:03.764 DEBUG:teuthology.orchestra.run.smithi195:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T03:44:03.808 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-12T03:44:03.819 INFO:teuthology.task.internal:Not uploading archives. 2024-09-12T03:44:03.819 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-12T03:44:03.830 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-12T03:44:03.830 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T03:44:03.833 DEBUG:teuthology.orchestra.run.smithi115:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T03:44:03.835 DEBUG:teuthology.orchestra.run.smithi195:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T03:44:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 03:44 /home/ubuntu/cephtest 2024-09-12T03:44:03.847 INFO:teuthology.orchestra.run.smithi115.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 03:44 /home/ubuntu/cephtest 2024-09-12T03:44:03.862 INFO:teuthology.orchestra.run.smithi195.stdout: 266064 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 03:44 /home/ubuntu/cephtest 2024-09-12T03:44:03.864 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-12T03:44:03.876 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-12T03:44:03.903 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/fs/test_o_trunc}} duration: 1870.4326617717743 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-12T03:44:03.903 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T03:44:04.016 INFO:teuthology.run:pass